In the future, human-like robots will live among people to provide company and help carrying out tasks in cooperation with humans. These interactions require that robots understand not only human actions, but also the way in which we perceive the world. Human perception heavily relies on the time dimension, especially when it comes to processing visual motion. Critically, human time perception for dynamic events is often inaccurate. Robots interacting with humans may want to see the world and tell time the way humans do: if so, they must incorporate human-like fallacy. Observers asked to judge the duration of brief scenes are prone to errors: perceived duration often does not match the physical duration of the event. Several kinds of temporal distortions have been described in the specialized literature. Here we review the topic with a special emphasis on our work dealing with time perception of animate actors versus inanimate actors. This work shows the existence of specialized time bases for different categories of targets. The time base used by the human brain to process visual motion appears to be calibrated against the specific predictions regarding the motion of human figures in case of animate motion, while it can be calibrated against the predictions of motion of passive objects in case of inanimate motion. Human perception of time appears to be strictly linked with the mechanisms used to control movements. Thus, neural time can be entrained by external cues in a similar manner for both perceptual judgments of elapsed time and in motor control tasks. One possible strategy could be to implement in humanoids a unique architecture for dealing with time, which would apply the same specialized mechanisms to both perception and action, similarly to humans. This shared implementation might render the humanoids more acceptable to humans, thus facilitating reciprocal interactions.
visual motion; biological motion; animate; inanimate; time perception; humanoid
Muscle activities underlying many motor behaviors can be generated by a small number of basic activation patterns with specific features shared across movement conditions. Such low-dimensionality suggests that the central nervous system (CNS) relies on a modular organization to simplify control. However, the relationship between the dimensionality of muscle patterns and that of joint torques is not fixed, because of redundancy and non-linearity in mapping the former into the latter, and needs to be investigated. We compared the torques acting at four arm joints during fast reaching movements in different directions in the frontal and sagittal planes and the underlying muscle patterns. The dimensionality of the non-gravitational components of torques and muscle patterns in the spatial, temporal, and spatiotemporal domains was estimated by multidimensional decomposition techniques. The spatial organization of torques was captured by two or three generators, indicating that not all the available coordination patterns are employed by the CNS. A single temporal generator with a biphasic profile was identified, generalizing previous observations on a single plane. The number of spatiotemporal generators was equal to the product of the spatial and temporal dimensionalities and their organization was essentially synchronous. Muscle pattern dimensionalities were higher than torques dimensionalities but also higher than the minimum imposed by the inherent non-negativity of muscle activations. The spatiotemporal dimensionality of the muscle patterns was lower than the product of their spatial and temporal dimensionality, indicating the existence of specific asynchronous coordination patterns. Thus, the larger dimensionalities of the muscle patterns may be required for CNS to overcome the non-linearities of the musculoskeletal system and to flexibly generate endpoint trajectories with simple kinematic features using a limited number of building blocks.
modularity; reaching movements; human subjects; inverse dynamics; EMGs; muscle synergies; temporal components; joint torques
Electrical microstimulation studies provide some of the most direct evidence for the neural representation of muscle synergies. These synergies, i.e., coordinated activations of groups of muscles, have been proposed as building blocks for the construction of motor behaviors by the nervous system. Intraspinal or intracortical microstimulation (ICMS) has been shown to evoke muscle patterns that can be resolved into a small set of synergies similar to those seen in natural behavior. However, questions remain about the validity of microstimulation as a probe of neural function, particularly given the relatively long trains of supratheshold stimuli used in these studies. Here, we examined whether muscle synergies evoked during ICMS in two rhesus macaques were similarly encoded by nearby motor cortical units during a purely voluntary behavior involving object reach, grasp, and carry movements. At each microstimulation site we identified the synergy most strongly evoked among those extracted from muscle patterns evoked over all microstimulation sites. For each cortical unit recorded at the same microstimulation site, we then identified the synergy most strongly encoded among those extracted from muscle patterns recorded during the voluntary behavior. We found that the synergy most strongly evoked at an ICMS site matched the synergy most strongly encoded by proximal units more often than expected by chance. These results suggest a common neural substrate for microstimulation-evoked motor responses and for the generation of muscle patterns during natural behaviors.
motor; movement; muscle; synergy; hand; macaque; grasping; cortex
Muscle synergies have been proposed as a way for the central nervous system (CNS) to simplify the generation of motor commands and they have been shown to explain a large fraction of the variation in the muscle patterns across a variety of conditions. However, whether human subjects are able to control forces and movements effectively with a small set of synergies has not been tested directly. Here we show that muscle synergies can be used to generate target forces in multiple directions with the same accuracy achieved using individual muscles. We recorded electromyographic (EMG) activity from 13 arm muscles and isometric hand forces during a force reaching task in a virtual environment. From these data we estimated the force associated to each muscle by linear regression and we identified muscle synergies by non-negative matrix factorization. We compared trajectories of a virtual mass displaced by the force estimated using the entire set of recorded EMGs to trajectories obtained using 4–5 muscle synergies. While trajectories were similar, when feedback was provided according to force estimated from recorded EMGs (EMG-control) on average trajectories generated with the synergies were less accurate. However, when feedback was provided according to recorded force (force-control) we did not find significant differences in initial angle error and endpoint error. We then tested whether synergies could be used as effectively as individual muscles to control cursor movement in the force reaching task by providing feedback according to force estimated from the projection of the recorded EMGs into synergy space (synergy-control). Human subjects were able to perform the task immediately after switching from force-control to EMG-control and synergy-control and we found no differences between initial movement direction errors and endpoint errors in all control modes. These results indicate that muscle synergies provide an effective strategy for motor coordination.
non-negative matrix factorization; isometric force; reaching movements; myoelectric control; modularity; electromyography
Neuroprosthetic technology and robotic exoskeletons are being developed to facilitate stepping, reduce muscle efforts, and promote motor recovery. Nevertheless, the guidance forces of an exoskeleton may influence the sensory inputs, sensorimotor interactions and resulting muscle activity patterns during stepping. The aim of this study was to report the muscle activation patterns in a sample of intact and injured subjects while walking with a robotic exoskeleton and, in particular, to quantify the level of muscle activity during assisted gait. We recorded electromyographic (EMG) activity of different leg and arm muscles during overground walking in an exoskeleton in six healthy individuals and four spinal cord injury (SCI) participants. In SCI patients, EMG activity of the upper limb muscles was augmented while activation of leg muscles was typically small. Contrary to our expectations, however, in neurologically intact subjects, EMG activity of leg muscles was similar or even larger during exoskeleton-assisted walking compared to normal overground walking. In addition, significant variations in the EMG waveforms were found across different walking conditions. The most variable pattern was observed in the hamstring muscles. Overall, the results are consistent with a non-linear reorganization of the locomotor output when using the robotic stepping devices. The findings may contribute to our understanding of human-machine interactions and adaptation of locomotor activity patterns.
robotic exoskeleton; assisted gait; EMG patterns; spinal cord injury; neuroprosthetic technology
Muscle synergies have been proposed as a mechanism to simplify movement control. Whether these coactivation patterns have any physiological reality within the nervous system remains unknown. Here we applied electrical microstimulation to motor cortical areas of rhesus macaques to evoke hand movements. Movements tended to converge towards particular postures, driven by synchronous bursts of muscle activity. Across stimulation sites, the muscle activations were reducible to linear sums of a few basic patterns—each corresponding to a muscle synergy evident in voluntary reach, grasp, and transport movements made by the animal. These synergies were represented non-uniformly over the cortical surface. We argue that the brain exploits these properties of synergies—postural equivalence, low dimensionality, and topographical representation—to simplify motor planning, even for complex hand movements.
Controlling the movement of the arm to achieve a goal, such as reaching for an object, is challenging because it requires coordinating many muscles acting on many joints. The central nervous system (CNS) might simplify the control of reaching by directly mapping initial states and goals into muscle activations through the combination of muscle synergies, coordinated recruitment of groups of muscles with specific activation profiles. Here we review recent results from the analysis of reaching muscle patterns supporting such a control strategy. Muscle patterns for point-to-point movements can be reconstructed by the combination of a small number of time-varying muscle synergies, modulated in amplitude and timing according to movement directions and speeds. Moreover, the modulation and superposition of the synergies identified from point-to-point movements captures the muscle patterns underlying multi-phasic movements, such as reaching through a via-point or to a target whose location changes after movement initiation. Thus, the sequencing of time-varying muscle synergies might implement an intermittent controller which would allow the construction of complex movements from simple building blocks.
muscle synergies; reaching movements; human; motor control; intermittency; EMG
The identification of biological modules at the systems level often follows top-down decomposition of a task goal, or bottom-up decomposition of multidimensional data arrays into basic elements or patterns representing shared features. These approaches traditionally have been applied to mature, fully developed systems. Here we review some results from two other perspectives on modularity, namely the developmental and evolutionary perspective. There is growing evidence that modular units of development were highly preserved and recombined during evolution. We first consider a few examples of modules well identifiable from morphology. Next we consider the more difficult issue of identifying functional developmental modules. We dwell especially on modular control of locomotion to argue that the building blocks used to construct different locomotor behaviors are similar across several animal species, presumably related to ancestral neural networks of command. A recurrent theme from comparative studies is that the developmental addition of new premotor modules underlies the postnatal acquisition and refinement of several different motor behaviors in vertebrates.
activation pattern; CPG; gene expression; interneurons; locomotion
During visuomotor adaptation a novel mapping between visual targets and motor commands is gradually acquired. How muscle activation patterns are affected by this process is an open question. We tested whether the structure of muscle synergies is preserved during adaptation to a visuomotor rotation. Eight subjects applied targeted isometric forces on a handle instrumented with a force transducer while electromyographic (EMG) activity was recorded from 13 shoulder and elbow muscles. The recorded forces were mapped into horizontal displacements of a virtual sphere with simulated mass, elasticity, and damping. The task consisted of moving the sphere to a target at one of eight equally spaced directions. Subjects performed three baseline blocks of 32 trials, followed by six blocks with a 45° CW rotation applied to the planar force, and finally three wash-out blocks without the perturbation. The sphere position at 100 ms after movement onset revealed significant directional error at the beginning of the rotation, a gradual learning in subsequent blocks, and aftereffects at the beginning of the wash-out. The change in initial force direction was closely related to the change in directional tuning of the initial EMG activity of most muscles. Throughout the experiment muscle synergies extracted using a non-negative matrix factorization algorithm from the muscle patterns recorded during the baseline blocks could reconstruct the muscle patterns of all other blocks with an accuracy significantly higher than chance indicating structural robustness. In addition, the synergies extracted from individual blocks remained similar to the baseline synergies throughout the experiment. Thus synergy structure is robust during visuomotor adaptation suggesting that changes in muscle patterns are obtained by rotating the directional tuning of the synergy recruitment.
muscle synergies; visuomotor rotation; motor adaptation; isometric force; EMG; directional tuning
A salient feature of human motor skill learning is the ability to exploit similarities across related tasks. In biological motor control, it has been hypothesized that muscle synergies, coherent activations of groups of muscles, allow for exploiting shared knowledge. Recent studies have shown that a rich set of complex motor skills can be generated by a combination of a small number of muscle synergies. In robotics, dynamic movement primitives are commonly used for motor skill learning. This machine learning approach implements a stable attractor system that facilitates learning and it can be used in high-dimensional continuous spaces. However, it does not allow for reusing shared knowledge, i.e., for each task an individual set of parameters has to be learned. We propose a novel movement primitive representation that employs parametrized basis functions, which combines the benefits of muscle synergies and dynamic movement primitives. For each task a superposition of synergies modulates a stable attractor system. This approach leads to a compact representation of multiple motor skills and at the same time enables efficient learning in high-dimensional continuous systems. The movement representation supports discrete and rhythmic movements and in particular includes the dynamic movement primitive approach as a special case. We demonstrate the feasibility of the movement representation in three multi-task learning simulated scenarios. First, the characteristics of the proposed representation are illustrated in a point-mass task. Second, in complex humanoid walking experiments, multiple walking patterns with different step heights are learned robustly and efficiently. Finally, in a multi-directional reaching task simulated with a musculoskeletal model of the human arm, we show how the proposed movement primitives can be used to learn appropriate muscle excitation patterns and to generalize effectively to new reaching skills.
dynamic movement primitives; muscle synergies; reinforcement learning; motor control; musculoskeletal model
To generate a force at the hand in a given spatial direction and with a given magnitude the central nervous system (CNS) has to coordinate the recruitment of many muscles. Because of the redundancy in the musculoskeletal system, the CNS can choose one of infinitely many possible muscle activation patterns which generate the same force. What strategies and constraints underlie such selection is an open issue. The CNS might optimize a performance criterion, such as accuracy or effort. Moreover, the CNS might simplify the solution by constraining it to be a combination of a few muscle synergies, coordinated recruitment of groups of muscles. We tested whether the CNS generates forces by minimum effort recruitment of either individual muscles or muscle synergies. We compared the activation of arm muscles observed during the generation of isometric forces at the hand across multiple three-dimensional force targets with the activation predicted by either minimizing the sum of squared muscle activations or the sum of squared synergy activations. Muscle synergies were identified from the recorded muscle pattern using non-negative matrix factorization. To perform both optimizations we assumed a linear relationship between rectified and filtered electromyographic (EMG) signal which we estimated using multiple linear regressions. We found that the minimum effort recruitment of synergies predicted the observed muscle patterns better than the minimum effort recruitment of individual muscles. However, both predictions had errors much larger than the reconstruction error obtained by the synergies, suggesting that the CNS generates three-dimensional forces by sub-optimal recruitment of muscle synergies.
muscle synergies; isometric force; directional tuning; effort minimization; non-negative matrix factorization
Analyses of experimental data acquired from humans and other vertebrates have suggested that motor commands may emerge from the combination of a limited set of modules. While many studies have focused on physiological aspects of this modularity, in this paper we propose an investigation of its theoretical foundations. We consider the problem of controlling a planar kinematic chain, and we restrict the admissible actuations to linear combinations of a small set of torque profiles (i.e., motor synergies). This scheme is equivalent to the time-varying synergy model, and it is formalized by means of the dynamic response decomposition (DRD). DRD is a general method to generate open-loop controllers for a dynamical system to solve desired tasks, and it can also be used to synthesize effective motor synergies. We show that a control architecture based on synergies can greatly reduce the dimensionality of the control problem, while keeping a good performance level. Our results suggest that in order to realize an effective and low-dimensional controller, synergies should embed features of both the desired tasks and the system dynamics. These characteristics can be achieved by defining synergies as solutions to a representative set of task instances. The required number of synergies increases with the complexity of the desired tasks. However, a possible strategy to keep the number of synergies low is to construct solutions to complex tasks by concatenating synergy-based actuations associated to simple point-to-point movements, with a limited loss of performance. Ultimately, this work supports the feasibility of controlling a non-linear dynamical systems by linear combinations of basic actuations, and illustrates the fundamental relationship between synergies, desired tasks and system dynamics.
muscle synergies; number of synergies; system dynamics; kinematic strokes; kinematic chain
Intercepting a moving object requires accurate spatio-temporal control. Several studies have investigated how the CNS copes with such a challenging task, focusing on the nature of the information used to extract target motion parameters and on the identification of general control strategies. In the present study we provide evidence that the right time and place of the collision is not univocally specified by the CNS for a given target motion; instead, different but equally successful solutions can be adopted by different subjects when task constraints are loose. We characterized arm kinematics of fourteen subjects and performed a detailed analysis on a subset of six subjects who showed comparable success rates when asked to catch a flying ball in three dimensional space. Balls were projected by an actuated launching apparatus in order to obtain different arrival flight time and height conditions. Inter-individual variability was observed in several kinematic parameters, such as wrist trajectory, wrist velocity profile, timing and spatial distribution of the impact point, upper limb posture, trunk motion, and submovement decomposition. Individual idiosyncratic behaviors were consistent across different ball flight time conditions and across two experimental sessions carried out at one year distance. These results highlight the importance of a systematic characterization of individual factors in the study of interceptive tasks.
The Cymanus is a novel flex sensor glove for measuring hand kinematics in primates. It was used to monitor 9 joints of a rhesus macaque performing a grasping task with 25 objects. Over 6 days, the monkey tolerated the glove and showed no significant impairment in performance. The sensors linearly tracked joint angles, with joint trajectories preserved over days. Angular positions discriminated objects as accurately as electromyograms recorded simultaneously from 24 arm and hand muscles, and were maximally informative of object identity at the end of reach-to-grasp. In a further final validation of the glove, muscle activity controlling a joint was correlated with the joint’s angular acceleration 70 ms later.
motor; hand; finger; kinematics; monkey; grasping; muscle
The diagnosis of arrhythmogenic right ventricular cardiomyopathy can be challenging. Disease-causing mutations in desmosomal genes have been identified. A novel diagnostic feature, loss of immunoreactivity for plakoglobin from the intercalated disks, recently was proposed.
The purpose of this study was to identify two novel mutations in the intracellular cadherin segment of desmoglein-2 (G812S and C813R in exon 15). Co-segregation of the G812S mutation with disease expression was established in a large Caucasian family. Endomyocardial biopsies of two individuals showed reduced plakoglobin signal at the intercalated disk.
To understand the pathologic changes occurring in the diseased myocardium, functional studies on three mutations in exon 15 of desmoglein-2 (G812C, G812S, C813R) were performed.
Localization studies failed to detect any differences in targeting or stability of the mutant proteins, suggesting that they act via a dominant negative mechanism. Binding assays were performed to probe for altered binding affinities toward other desmosomal proteins, such as plakoglobin and plakophilin-2. Although no differences were observed for the mutated proteins in comparison to wild-type desmoglein-2, binding to plakophilin-2 depended on the expression system (i.e., bacterial vs mammalian protein expression). In addition, abnormal migration of the C813R mutant protein was observed in gel electrophoresis.
Loss of plakoglobin immunoreactivity from the intercalated disks appears to be the endpoint of complex pathologic changes, and our functional data suggest that yet unknown posttranslational modifications of desmoglein-2 might be involved.
Arrhythmogenic right ventricular cardiomyopathy; Desmoglein-2; Desmosome; Genetics; Missense mutation; Plakophilin-2; ARVC, arrhythmogenic right ventricular cardiomyopathy; Cx43, connexin43; DSC2, desmocollin-2; DSG2, desmoglein-2; DSP, desmoplakin; GFP, green fluorescent protein; GST, glutathione-S-transferase; ICS, intracellular cadherin segment; PG, plakoglobin; PKP2, plakophilin-2; RV, right ventricle