Walking sub-optimally is the way forward

Today we’re going to do something a little different. I’ve been posting a lot about reaching movements, because that’s what I’m most interested in, but it may surprise you to learn that humans do actually have the capacity to move other parts of their bodies as well. I know, I’m as shocked as you are… so! The paper I’m going to cover is about the regulation of step variability in walking. It’s a little longer and more complex than normal, so strap yourselves in.

Walking is a hard problem, and we’re not really sure how we do it. Like reaching, there are many muscles to coordinate in order to make a step forward. Unlike in arm reaching, these coordinated steps need to follow one another cyclically in such a way as to keep the body stable and upright while simultaneously moving it over terrain that might well be rough and uneven. Just think for a moment about how difficult that is, and what different processes might be involved in the control of such movements.

One question that remains unanswered is how we control variability in walking. It’s a simple matter to control average position or velocity, but the variation in these parameters between steps is still unexplained. It is pretty well established that over the long-term people tend to try to minimize energy costs while walking – hence the gait we learn to adopt over the first few years of life. But there’s evidence that such a seemingly “optimal” strategy is not the whole story.

Consider walking on a treadmill. What’s the primary goal of continuous treadmill walking? Well, it’s to not fall off. The researchers in the article took that idea and reasoned that because the treadmill is moving at a constant speed, the best way not to fall off is to move at a constant speed yourself. That’s not the only strategy of course – you could also do something a little more complicated like make some short, quick steps followed by some long, slow ones in sequence, which would also keep you on the treadmill.

To test how the parameters varied, the researchers used five different walking speeds. You can see this in the figure below (Figure 3 in the paper):

Human treadmill walking data with speed as percentage of preferred walking speed (PWS)

L is stride length, T is stride time and S is stride speed. So A-C in the figure show how these values change with the five different treadmill speeds – length increases, time decreases and speed increases. D-F show the variability (σ) in these different parameters. G-I show something slightly more complex: a value called α that is defined as a measure of persistence, i.e. how much or little the parameters were corrected on subsequent strides. Values of α > ½ mean that there was less correction, whereas values < ½ mean that there was more correction. So panels G-I show that variability in stride length and time were not generally corrected quickly, but that variations in stride speed were.

Read that last paragraph through again to make sure you get it. It will be important shortly!

So: now we have a measure of human walking parameters. The question is, how are these parameters produced by the motor control system? That is, what does the system care about when it initiates and monitors walking? Well, one thing we can get from the data here is that the system seems to care about stride speed, but doesn’t care about stride time and stride length individually. And if that’s the case, then as long as the coupled length and time lie on a line that defines the speed, the system should be happy. A line a bit like this (figure 2B in the paper):

Human stride parameters lie along line of constant speed

The figure shows the GEM (which stands for Goal Equivalent Manifold, essentially the line of constant speed) plotted against stride time and stride length. The red dots show some data. Right away you can see that the dots generally lie along the line. Ignore the green arrows, but do take note of the blue ones – they’re showing a measure of deviations tangent to (δ_T) and perpendicular to (δ_P) the line. Why is δ_T so much bigger than δ_P? Because perpendicular variations push you off the line and thus interfere with the goal, whereas tangential variations don’t. So the system is either not stepping off the line much in the first place or correcting heavily when it does.

Here’s one more figure (Figure 5C and D in the paper) showing the variability (σ) and persistence (α) for δ_T and δ_P :

Variability and persistence of deviations

You can see that δ_T is much more variable than δ_P, as you might expect from the shape of the data shown in the second figure. You can also see something else, however: the persistence for δ_P is less than ½, whereas the persistence for δ_T is greater than ½. Thus, the system cares very much about correcting not just stride speed but the combination of stride time and stride length that take the stride speed away from the goal speed.

Great, you may think, a lot of funny numbers to tell us that the system cares about maintaining a constant speed when it’s trying to maintain a constant speed! What do you scientists get paid for anyway? The cool thing about this paper is that the researchers are trying to figure out precisely how the brain produces these numbers. It turns out that if you just use an ‘optimal’ model that corrects for δ_P while ignoring δ_T, you don’t get the same numbers. So that can’t be it. How about if you specify in your model that you have to keep at a certain speed – say the same average speed as in the human data? That doesn’t work either. The numbers are better, but they’re not right.

The solution that seems to work best is when the deviations off the GEM line (i.e. δ_P) are overcorrected for. This controller is sub-optimal, so basically efficiency is being sacrificed for tight control over this parameter. Thus, humans don’t appear to simply minimize energy loss – they also perform more complex corrections depending on the task goal.

I’ve covered in a previous post the inkling that this might be the case; while we do tend to minimize energy over the long term, in the short term the optimization process is much more centred around the particular goal, and people are very good at exploiting the inherent variability in the motor system to perform the task more easily. This paper does a great job of testing these hypotheses and providing models to explain how this might happen. What I’d be interested to see in the future is an explanation of why the system is set up to overcorrect like that in the first place – is it overall a more efficient way of producing movement than just a standard optimization over all parameters? Time, perhaps, will tell.

--

Dingwell JB, John J, & Cusumano JP (2010). Do humans optimally exploit redundancy to control step variability in walking? PLoS computational biology, 6 (7) PMID: 20657664

Images copyright © 2010 Dingwell, John & Cusumano

Errors and use both contribute to learning

Learning how to make a reaching movement is, as I’ve said before, a very hard problem. There are so many muscles in the arm and so many ways we can get from one point to another that there are for all intents and purposes an infinite set of ways the brain could choose to send motor commands to achieve the same goal. And yet what we see consistently from people is a very stereotyped kind of movement.

How do we learn to make reaching movements in the presence of destabilizing perturbations? The standard way of thinking about this assumes that if you misreach, your motor system will notice the error and get better next time, whether it’s through recalibration of the sensory system or through a new cognitive strategy to better achieve the goal. But this paper from Diedrichsen et al. (2010) postulates another learning mechanism than error-based learning: something they call use-dependent learning.

The basic idea is that if you’re performing a task, like reaching to an object straight ahead, and you’re constantly getting pushed off to the side, you’ll correct for these sideways perturbations using error-based learning. But you’re also learning to make movements in the non-perturbed direction, and the more you make these movements the more experience you have with making these kinds of movements, so each movement becomes more similar to the last.

The authors demonstrate this with some nice experiments using a redundant movement task – rather than moving a cursor to a target as in standard motor control tasks, participants had to move a horizontal bar up the screen to a horizontal bar target. The key thing is that it was only the vertical movement that made the bar move; horizontal movements had no effect. In the first experiment, participants initially reached to the bar before being passively moved by a robotic system in one of two directional tilts (left or right) and were then allowed to move by themselves again. The results are below (Figure 1 in the paper):

Redundant reaching task

You can see that after the passive movement was applied, the overall angle changed depending on whether it was to the left (blue) or right (red). Remember that the tilt was across the task-redundant (horizontal) dimension, so it didn’t cause errors in the task at all! Despite this, participants continued to reach in the way that they’d been forced to do after the passive movement was finished – demonstrating use-dependent learning.

To follow this up, the authors did two more experiments. The first showed that error-based and use-dependent learning are separate processes and occur at the same time. They used a similar task but this time rather than a passive movement participants made active reaches in a left- or right-tilting ‘force channel’. This time the initial angle results showed motor aftereffects that reflected error-based learning, while the overall angle showed similar use-dependent effects as in the first experiment.

Finally they investigated use-dependent learning in a perturbation study. As participants moved the bar toward the target they had to fight against a horizontal force that was proportional to their velocity (i.e. it got bigger as they went faster). Compared to a ‘standard’ perturbation study (a reach to a target, where participants could see their horizontal error) the horizontal errors weren’t corrected after learning. However, the initial movement directions in the redundant task were in the direction of the force field – meaning that as participants learnt the task the planned movement direction changed through use-dependent learning.

I think this is a really cool idea. Most studies focus on error as the sole basis for driving motor learning, but thinking about use-dependent learning makes sense because of what we know about how the brain makes connections through something called Hebbian learning. Basically, though an oversimplification: ‘what fires together, wires together’, which means that connections tend to strengthen if they are used a lot and weaken if they are not. So it seems reasonable (to me at least!) that if you make a movement, you’re more likely to make another one like it than come up with a new solution.

It also might explain something about optimal feedback control that I’ve been thinking about for a while since seeing some work from Paul Gribble’s lab: we often talk about the motor system minimizing the energy required to perform a reach, but their work has shown pretty conclusively that the motor system prefers straight reaches even if the minimum energy path is decidedly not straight. There must therefore be some top-down mechanism that prioritises ‘straightness’ in the motor system, even if it’s not the most ‘optimal’ strategy for the task at hand.

Lots to chew over and think about here. I haven’t even covered the modelling work the authors did, but it’s pretty nice.

---

Diedrichsen J, White O, Newman D, & Lally N (2010). Use-dependent and error-based learning of motor behaviors. Journal of Neuroscience, 30 (15), 5159-66 PMID: 20392938

Image copyright © 2010 Diedrichsen, White, Newman & Lally

The cost of uncertainty

Back from my girlfriend-induced hiatus and onto a really interesting paper published ahead of print in the Journal of Neurophysiology. This work asks some questions, and postulates some answers, very similar to the line of thinking I’ve been going down recently – which is, of course, the main reason I find it interesting! (The other reason is that they used parabolic flights. Very cool.)

One theory of how the brain performs complex movements in a dynamical environment – like, say, lifting objects – is known as optimal feedback control (OFC). The basic idea is that the brain makes movements that are optimized to the task constraints. For example, to lift an object, the control system might want to minimize the amount of energy used* and at the same time lift the object to a particular position. In OFC we combine these constraints into something called a cost function: how much the action ‘costs’ the system to perform. To optimize the movement, the system simply works to reduce the total cost.

But where does the system get information about the limb and the task from in the first place so as to optimize its control? There are two sources for knowledge about limb dynamics. The most obvious is reactive: feedback from the senses, from both vision and proprioception (the sense of where the arm is in space). But feedback takes a while to travel to the brain and so another source is needed: a predictive source of knowledge, an internal model of the task and limb dynamics. The predictive and reactive components can be combined in an optimal fashion to form an estimate of the state of the limb (i.e. where it is and how fast it’s going). This ‘state estimate’ can then be used to calculate the overall cost of the movement.

In today’s paper the authors argue that at the start of a new task, a new internal model has to be learnt, or an old one modified, to deal with the new task demands. So far so uncontroversial. What’s new here is the claim that the cost function being optimized for actually changes when dealing with a new task – because there is higher uncertainty in the internal prediction so the system is temporarily more reliant on feedback. They have some nice data and models to back up their conclusion.

The task was simple: participants had to grip a block and move it up or down from a central position while their position and grip force was recorded. After they’d learnt the task at normal gravity, they had to perform it in microgravity during a parabolic flight, which essentially made their arm and the object weightless. Their grip force increased markedly even though they now had a weightless object, and kinematic (e.g. position, velocity) measures changed too; movements took more time, and the peak acceleration was lower. Over the course of several trials the grip force decreased again as participants learnt the task. You can see some representative kinematic data in the figure below (Figure 4 in the paper):

Kinematic data from a single participant

Panels A-D show the average movement trace of one participant in normal (1 g) and microgravity (0 g) conditions, while panels E and F show the changes in acceleration and movement time respectively. The authors argue that the grip force changes at the beginning of the first few trials point towards uncertainty in the internal prediction, which results in the altered kinematics.

To test this idea, they ran a simulation based on a single-joint model of the limb using OFC and the optimal combination of information from the predictive system and sensory feedback. What they varied in this model was the noise, and thus the reliability, in the predictive system. The idea was that as the prediction became less reliable, the kinematics should change to reflect more dependence on the sensory feedback. But that's not quite what happened, as you can see from the figure below (Figure 8 in the paper):

Data and simulation results

Here the graphs show various kinematic parameters. In black and grey are the mean data points from all the participants for the upward and downward movements. The red squares show the parameters the simulation came up with when noise was injected into the prediction. As you can see, they're pretty far off! So what was the problem? Well, it seems that you need to change not only the uncertainty of the prediction but also the cost function that is being optimized. The blue diamonds show what happens when you manipulate the cost function (by increasing the parameter shown as alpha); suddenly the kinematics are much closer to the way people actually perform.

Thus, the conclusion is that when you have uncertainty in your predictive system, you actually change your cost function while you're learning a new internal model. I find this really interesting because it's a good piece of evidence that uncertainty in the predictive system feeds into the selection of a new cost function for a movement, rather than the motor system just sticking with the old cost function and continuing to bash away.

It's a nice paper but I do wonder, why did the authors go to all the trouble of using parabolic flights to get the data here? If what they're saying is true and any uncertainty in the internal model/predictive system is enough to make you change your cost function, this experiment could have been done much more simply – and for much longer than the 30 trials they were able to do under microgravity – by just using a robotic system. Perhaps they didn't have access to one, but even so it seems a bit of overkill to spend money on parabolic flights which are so limited in duration.

Overall though it's a really fun paper with some interesting and thought-provoking conclusions.

*To be precise there is some evidence that it's not the amount of energy used that gets minimized, but the size of the motor command itself (because a bigger command has more variability due to something called signal-dependent noise... I'm not going to go into that though!).

---

Crevecoeur, F., McIntyre, J., Thonnard, J., & Lefevre, P. (2010). Movement Stability under Uncertain Internal Models of Dynamics Journal of Neurophysiology DOI: 10.1152/jn.00315.2010

Images copyright © 2010 The American Physiological Society