Distributed Multiagent for NAO Robot Joint Position Control Based on Echo State Network

Abstract

Based on echo state networks, the joints position control of NAO robot is studied in this paper. The process to control the robot position can be divided into two phases. The senor parameters are released during the first phase. Depending on the dynamic coupling effect between the angle acceleration of passive joint and the torque of active joint, passive joint can be controlled indirectly to the desired position along the desired trajectory. The ESN control rules during the first phase are described and ESN controller is designed to control the motion of passive joint. The brake is locked during the second phase; then active joint is controlled to the desired position. The experimental control system based on PMAC controller is designed and developed. Finally, the joint position control of the NAO robot is achieved successfully by experiments. Echo state networks utilized incremental updates driven by new sensor readings and massive short memory with history inputs; thus varying communication rates can help imitate human upper limb motion based on wearable sensors to obtain human joint angles.

Ling Qin 1, 2 and Bo Lei 3

Academic Editor:Wei Zhang

1, School of Automation, Huazhong University of Technology and Science, Wuhan 430074, China
2, Hubei Key Laboratory of Advanced Technology for Automotive Components, Wuhan University of Technology, Wuhan 430070, China
3, School of Mechanical and Electronic Information, University of Geosciences, Wuhan 430074, China

Received 25 July 2014; Accepted 29 October 2014; 1 September 2015

This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

1. Introduction

Humanoid robot is very common in our daily life. It can communicate with humans, respond to the surrounding environment, and complete kinds of work instead of humans. In addition, it plays an important role in exploring human interaction and cognitive processes. In robotics technology, there is much space for improvement, because it is hard to keep balance and stability of the robot at any moment, especially during movement. Path planning is one of difficult and hot spots in biped robot research. The researchers also devoted themselves to studying it.

Park and Rhee [1] observed that the zero moment point (ZMP) forward constantly on human movement; on this basis, according to the hip position and the free leg configuration, they implemented a fuzzy-logic to generate the ZMP trajectory. But the scheme was just simulated on a 7-degree-of-freedom biped robot. Lee et al. [2] simplified the dominant dynamics into a simply linear inverted pendulum model (LIPM). Under the designed conditions, the choice of ZMP function was determined with five parameters, and then the walking pattern was formed; finally the goal to perform complex tasks was achieved. In order to reduce several difficulties brought about by the uncertainty world model and unknown environment, Garcia et al. [3] presented an estimator that depends on sensor fusion strategy; it was able to filter out low frequency and high frequency.

Based on the NAO robot platform, Gouaillier et al. [4] proposed a new walking method. They added trapezoid function to joint trajectory calculation, compensated joints that are not up to the specified location, and introduced trunk error feedback to form the close-loop achieved NAO robot dynamic humanoid walk. Strom et al. [5] proposed an omnidirectional walk for NAO based on ZMP and preview controller. They firstly confirmed the following parameters: leg trajectories in initial frame, foot frame, and center of mass (CoM) frame, and then, combining with a series of steps, specified CoM trajectory. Next they adopted preview control for walking, which suits fast-moving environment. Besides, Kulk and Welsh [6] achieved move smoothly by setting the NAO robot motor in a low stiffness.

With the upgrading of the study level, researchers are constantly expanding their research directions. To imitate human movement seems to become a new content.

As early as in 1999, Dasgupta and Nakamura [7] had already begun to study driving robot by human motion data. They attached markers at the limbs and joints of human, using the VICOT-370 system from Oxford metrics to obtain the human motion data. In 2003, a system that imitates human dance motions [8] was raised. According to motion features which can be obtained from analyzing limbs motion track, then calculated joint angle sequence of the robot. At the same time, based on ZMP, it moved the waist trajectory for keeping the balance and stability of the robot. Suleiman et al. [9] also focused their energies on studying imitation of human captured motions. They used a virtual actor to provide the joint value as motions, took the physical limitations of the humanoid robot as an optimization problem, and finally based on an optimization framework to realize the upper body imitate motion of humanoid.

Infrared camera has been used to obtain the location of human [10] successfully. Firstly it used the motion information in the virtual human model and calculated the gait information with the aid of related professional software and then derived the NAO robot. Ude et al. [11] suggested transforming human motion information obtained from an optical tracking device into a high-dimensional trajectory for the humanoid robot; then b-spline wavelet analysis is used to remove noise. But the method based on static motion capture device is difficult to realize when used in the real-time control in a lab environment. The most direct way to get the joint information is to adopt ewearable sensor devices [12]. ENC-03J sensor is produced by Japan Murata, which is composed of a gyroscope and an accelerometer. Gyroscope and accelerometer signals are amplified and filtered before output to remove electrical noise.

In this paper, our main contribution is to realize the humanoid robot to imitate human upper limb motion in real-time based on ESN (Figure 3). We tie the wearable sensors to the upper limb to get motion information. And then with the help of specialized software platform, such as Choregraphe developed by Aldebaran-Robotics Company and Visual Studio, the corresponding data is relayed to the humanoid robot. Ultimately we complete the real-time trajectory imitation of the humanoid robot.

2. Method

This section provides the basic concepts and notations of joint motion of NAO robot based echo state network. Section 2.1 gives a general description of NAO robot and how to control the robot by traditional ways. Then, the basic architecture of echo state network is introduced in the paper in Section 2.2. Finally, the joint motion control based echo state network is provided thereafter in Section 2.3.

2.1. NAO Robot

NAO robot is more popular humanoid robot nowadays. It stands 57 cm high and weighs 4.5 kg and has 25 degrees of freedom. NAO was developed by Aldebaran-Robotics Company with the latest technology and a variety of sensors, so it can ensure the fluency of motion. The more important feature of NAO is the embedded software. With the software, NAO can do voice synthesis, acoustic positioning, detect the visual image and shape with color, and detect obstacles based on dual-channel ultrasonic system. Human body can be simplified as a model that can be described as in Figure 1, which contains 12 main body joints.

Figure 1: Human bones simplified model.

[figure omitted; refer to PDF]

The purpose of this paper is to drive the robot arm joint, mapped to the human body, corresponding to the upper arm joint of the human body. So we tied the MTi sensors on human upper limb to obtain joints movement data. And we make sure the joint of human and NAO robot is the same.

As for the wearable sensors, we adopt the Xsens sensors-MTi. It is a miniature gyroscope enhanced heading measurement system that integrates MEMS inertial measurement sensors. The internal low-power signal processor provides three-dimensional orientations of no drift and corrected 3D acceleration, 3D angular velocity, and 3D magnetic field data. Now it is broadly used in robot fields.

MTi's coordinate system is shown in Figure 2 and aligns with the MTi sensor equipment enclosure. When starting collecting data, the initial position of the MTi sensor is marked as the coordinate direction of the current experiment. We can use the MTi sensor directly with its function of RS232, which can meet the real-time requirements of our experiment. Then we can obtain the data from the serial port with the MTi binary communication protocol using flow (free-running) operation or polling (request) mode. The initialization of the MTi sensor devices is done on the computer programming. It is quite easy to achieve with the Xsens code examples. The MTi sensors have a variety of output modes; researchers can choose the mode according to their needs. In this paper, we choose the Euler Angle output mode. The output format is roll, pitch, and yaw; all data elements are four-byte floating point.

Figure 2: MTi with sensor-fixed coordinate system overlaid (S).

[figure omitted; refer to PDF]

Figure 3: Basic ESN architecture.

[figure omitted; refer to PDF]

Because human is more flexible, the activity scope is relatively larger than robot; when the human body joint movement value exceeds the limit value of the NAO robot, the NAO robot will take the angle value as the maximum. In order to achieve data-driven robot, the joint data obtained by the MTi sensors is assigned to a target angle, with the NAO robot's angle function method; the target angle values will be sent to the NAO robot. We can complete the programming of the NAO robot motion control based on VS platform and implement this program under the chorographer. Then call the program with port and IP address by the DOS command.

The dynamic equation of NAT robot is given as [figure omitted; refer to PDF] where [figure omitted; refer to PDF] is the mass and inertia matrix, [figure omitted; refer to PDF] is the stiffness matrix, [figure omitted; refer to PDF] is the damping matrix, and [figure omitted; refer to PDF] is the NAO robot joint matrix. And the dynamic equation is described as matrix as [figure omitted; refer to PDF]

2.2. Brief Formal Description on ESN

More formally, an ESN consisted of [figure omitted; refer to PDF] input units, [figure omitted; refer to PDF] internal units, and [figure omitted; refer to PDF] output units. Then, activation of input, internal, and output units at time step [figure omitted; refer to PDF] is [figure omitted; refer to PDF] , [figure omitted; refer to PDF] , and [figure omitted; refer to PDF] , respectively. Connection weights between units are kept in four connection matrices. There are [figure omitted; refer to PDF] weights in the input weight matrix [figure omitted; refer to PDF] , [figure omitted; refer to PDF] weights in the internal weight matrix [figure omitted; refer to PDF] , [figure omitted; refer to PDF] weights in the output weight matrix, and [figure omitted; refer to PDF] weights in a matrix [figure omitted; refer to PDF] for connection projecting back from the output to internal units.

The activation of internal units was calculated as [figure omitted; refer to PDF] with [figure omitted; refer to PDF] being the output functions of the internal units, a sigmoid function for the experiments in this paper. Similarly, the output was computed as [figure omitted; refer to PDF] with [figure omitted; refer to PDF] , the output functions of the output units and [figure omitted; refer to PDF] the concatenation of input, internal, and previous output activation vector.

2.3. Motivation and Simulation on ESN

In order to get motion flatness and velocity continuity of NAO robot's joint, human body can be simplified as a model that contained 12 main body joints (Figure 3). And the inputs of ESN were the 12 main body joints, described as [figure omitted; refer to PDF] . The outputs of ESN were 3 main parameters, described as [figure omitted; refer to PDF] .

First we created a random RNN with 800 neurons (called the "reservoir") and one output neuron. Importantly, the output neuron was equipped with random connections that project back into the reservoir. A 2500-step teacher sequence [figure omitted; refer to PDF] was generated from the MGS equation and fed into the output neuron. This excited the internal neurons through the output feedback connections. After an initial transient, they started to exhibit systematic individual variations of the teacher sequence.

The fact that the internal neurons display systematic variants of the exciting external signal is constitutional for ESN: the internal neurons must work as "echo functions" for the driving signal. Not every randomly generated RNN has this property, but it can effectively be built into a reservoir. Figure 4 was the simulation for trace of NAO robot's legs. In the paper, the parameter settings for the robot's joint of NAO robot are shown in Table 1.

Table 1: Parameter settings for the reservoir of ESN.

Reservoir parameter	Values
State dimensionality	100
Sparseness	5%
Radius [figure omitted; refer to PDF]	0.8
Input weight [figure omitted; refer to PDF]	[figure omitted; refer to PDF] , even distribution
Back weight [figure omitted; refer to PDF]	[figure omitted; refer to PDF] , even distribution

Figure 4: Trace of NAO's legs.

[figure omitted; refer to PDF]

Figure 4 shows that the parameters of single joint are set. Taking the single joint of NAO robot as an example, the dynamic trace for passive joint is as shown in [figure omitted; refer to PDF] And for function (5), in this paper, the states for shift, velocity, and acceleration in the moving joint of NAO robot are set in [figure omitted; refer to PDF]

3. Experiment

In this section, brief review the steps for collecting human pose data is given in Section 3.1. Then, Section 3.2 provides the sequential update scheme for the algorithm of multiple sensors.

3.1. Collecting Human Motion Data

Follow the steps below; our experiment of collecting human motion data goes on well.

[figure omitted; refer to PDF] The initial position of the human body and NAO robot: to guarantee that the NAO robot can imitate human body movement accurately, we first make sure that the initial position of human body coordinates and the NAO should keep in the same position. This can be achieved through Choregraphe. In Choregraphe, processing the initialization instruction, the NAO robot will be at the zero position, as shown in Figure 5.

Figure 5: The zero position of the NAO robot.

[figure omitted; refer to PDF]

[figure omitted; refer to PDF] Signal acquisition depended on MTi sensor: this paper puts forward a humanoid robot control system based on a wearable sensor. This system is not strictly limited by fixed motion analysis equipment. Wear the MTi sensor in the Shoulder joint, and as shown in Figure 6, just like the NAO robot. With the Visual Studio platform, we connect the computer and NAO robot. When ready, with the input validation rules, the MTi sensor begins to collect the shoulder joint data.

Figure 6: The initial position of human and NAO.

[figure omitted; refer to PDF]

[figure omitted; refer to PDF] Joint angle calculation and processing: the angle value of human shoulder joint obtained by the MTi sensor should be transformation and processing that will meet the NAO robot's requirements. On the right-handed Cartesian coordinate system, rotation about [figure omitted; refer to PDF] , [figure omitted; refer to PDF] , and [figure omitted; refer to PDF] axis, respectively, called roll, pitch and yaw rotation. It is easy to express when the rotation is expressed as on a rotating shaft. Combined with the above formulas, it is easy to calculate the required angle value.

[figure omitted; refer to PDF] Data driven NAO robot: with the existing NAO robot joint function, ready processed joint angle value is assigned to the NAO robot. This process needs using two software systems-Choregraphe and Visual Studio. The NAO robot can be connected by its IP address.

3.2. Algorithms

Inputs for ESN networks controller are joint sensors' information or datum of NAO robot. The difference of the different components of the range can reduce the studying accuracy and speed for networks. In order to improve better studying accuracy and speed for ESN, inputting datum for sensors must be normalized firstly with function (6) and then be linear transformation: [figure omitted; refer to PDF] In training, the inner neurons update as follow: [figure omitted; refer to PDF] where [figure omitted; refer to PDF] is used as sigmoid function. When [figure omitted; refer to PDF] , the network begins to run and the state [figure omitted; refer to PDF] is initialized randomly (here, [figure omitted; refer to PDF] ), [figure omitted; refer to PDF] . The training I/O date series is inputted in the network. Then DR is encouraged fully, which shows system dynamic characteristics (see Algorithm 1).

Algorithm 1: Control law of joint motion.

Input: [figure omitted; refer to PDF]

(1) Initialize networks

(2) Linear transform inputs: [figure omitted; refer to PDF]

(2) Setting parameter for ESN: input weight [figure omitted; refer to PDF] , back weight [figure omitted; refer to PDF] , radius [figure omitted; refer to PDF]

(3) Compute predicted estimation: sparseness

(4) Update: [figure omitted; refer to PDF]

(5) Training

(6) end for

(7) for [figure omitted; refer to PDF] do

(8) Getting Training MSE: [figure omitted; refer to PDF]

(9) Getting Testing MSE: [figure omitted; refer to PDF]

(10) for each [figure omitted; refer to PDF] do

(11) While ( [figure omitted; refer to PDF] ) then to Step 7

(12) compute output value

(13) end for

Output: [figure omitted; refer to PDF]

4. Analysis of Simulation

In order to demonstrate the performance of proposed multiagent control approach for joint motion of NAO robot, numerical results are presented in the paper for a planar multitarget tracking scenario in multiagent system where three controllable moving observers, two equipped with range-only sensors and one with a bearing-only sensor, are placed in a specified surveillance area to estimate the number of targets as well as their positions. Each joint shares its current locations, sensor type, and observations with the other agents. There are an unknown and time-varying number of targets observed in clutter for each agent.

A series of experiments are done to prove the reliability of this point proposed in this paper. The motion trace of NAO robot joint is shown in Figure 7. During the experiments, we constantly change the stop position of human right arm; the robot can imitate the human right arm movement accurately. Figure 8 shows that the passive robot's joints could be controlled more accurately along expecting trace with human pose data.

Figure 7: (a) The results of the experiment. (b) The motion trace of NAO joint.

(a) [figure omitted; refer to PDF]

(b) [figure omitted; refer to PDF]

Figure 8: (a) Joint motion trace of NAO robot. (b) Joint motion simulation of NAO robot.

(a) [figure omitted; refer to PDF]

(b) [figure omitted; refer to PDF]

The simulation result is shown that the joint motion control based on ESN can get better motion flatness and velocity continuity.

5. Conclusions and Future Work

In this paper, we propose a novel joint motion control of NAO robot approach based on echo state network. Simulation results demonstrate that the proposed approach is capable of joint motion control via effective sensor control. Unfortunately, during the experiments, we cannot achieve the man-robot synchronization. There could be network reason for this. Or there may be a better way for data optimization. How to optimize the time question is the focus of our efforts in the future.

Conflict of Interests

The authors declare that there is no conflict of interests regarding the publication of this paper.

References

[1] J. H. Park, Y. K. Rhee, "ZMP trajectory generation for reduced trunk motions of biped robots," in Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 90-95, October 1998.

[2] B.-J. Lee, D. Stonier, Y.-D. Kim, J.-K. Yoo, J.-H. Kim, "Modifiable walking pattern of a humanoid robot by using allowable ZMP variation," IEEE Transactions on Robotics , vol. 24, no. 4, pp. 917-925, 2008.

[3] J. G. Garcia, A. Robertsson, J. G. Ortega, R. Johansson, "Sensor fusion for compliant robot motion control," IEEE Transactions on Robotics , vol. 24, no. 2, pp. 430-441, 2008.

[4] D. Gouaillier, C. Collette, C. Kilner, "Omni-directional closed-loop walk for NAO," in Proceedings of the 10th IEEE-RAS International Conference on Humanoid Robots, pp. 448-454, 2010.

[5] J. Strom, G. Slavov, E. Chown, "Omnidirectional walking using ZMP and preview control for the NAO humanoid robot," RoboCup 2009: Robot Soccer World Cup XIII , vol. 5949, of Lecture Notes in Computer Science, pp. 378-389, Springer, Berlin, Germany, 2010.

[6] J. Kulk, J. Welsh, J. Kim, R. Mahony, "A low power walk for the NAO robot," in Proceedings of the Australasian Conference on Robotics and Automation (ACRA '08), December 2008.

[7] A. Dasgupta, Y. Nakamura, "Making feasible walking motion of humanoid robots from human motion capture data," in Proceedings of the IEEE International Conference on Robotics and Automation (ICRA '99), pp. 1044-1049, May 1999.

[8] S. Nakaoka, A. Nakazawa, K. Yokoi, H. Hirukawa, K. Ikeuchi, "Generating whole body motions for a biped humanoid robot from captured human dances," in Proceedings of the IEEE International Conference on Robotics and Automation (ICRA '03), vol. 3, pp. 3905-3910, September 2003.

[9] W. Suleiman, E. Yoshida, F. Kanehiro, J.-P. Laumond, A. Monin, "On human motion imitation by humanoid robot," in Proceedings of the IEEE International Conference on Robotics and Automation (ICRA '08), pp. 2697-2704, Pasadena, Calif, USA, May 2008.

[10] P. L. Jean-Christophe, B. Olivier, F. Jean-Guy, "From human motion captures to humanoid spatial coordination," International Journal of Humanoid Robotics , vol. 9, no. 3, 2012.

[11] A. Ude, C. G. Atkeson, M. Riley, "Programming full-body movements for humanoid robots by observation," Robotics and Autonomous Systems , vol. 47, no. 2-3, pp. 93-108, 2004.

[12] D. Roetenberg, H. Luinge, P. Slycke, "Xsens MVN: full 6DOF human motion tracking using miniature inertial sensors,", Sens Technologies, 2009.