Motion Capturing with Stretch Sensor

I have not posted in a while here... anyway ... I wrote a report to wrap up everything I have done so far before I go off to Canada... I'll upload it somewhere... this here is the part that is probably most interesting for most people... (its not 1 to 1 the way it'll be in my final report, but close enough...)

Stretch Sensors

The stretch sensors I use are made of conductive plastic material which changes its resistance depending on the amount it is stretched. This material is produced by the American company Images and can be bought directly at their webshop or  at shops such as the RobotShop. The material is approximately 2mm in diameter and can be stretched up to 175%. 

Basically I want to capture body movement, which is induced by muscles. Muscles are linear actuators; the only movement which they can perform is expanding or contracting. This implies, that all movements of the body are measurable in terms of contracting (flexing) or expanding (relaxing) of the muscle by stretch sensors.

Angular Movement (Elbow and Shoulder)

           The axis of rotation is inaccessible which would be a problem for most sensor types. However, the fact that the attachment points vary in distance  to each other (when measured around the elbow as seen in the following picture) is actually of benefit for this sensor. In the elbow the Triceps Brachii is the main muscle controlling the elbows motion and takes advantage of this very principle. The stretch sensor basically copies the Triceps motion. The following two graphs compare the stretch sensor with the flex sensor. You can see that the flex sensor requires a sliding attachment while the stretch sensor can be permanently attached due to its elasticity.

Capturing angular movement with flex and stretch sensor (sharp angle, Tricaps Brachii relaxed)

Capturing angular movement with flex and stretch sensor (blunt angle, Triceps Brachii flexed)

            Rotational Movement (Rotation of upper and lower arm)

The fact that the rotational axis is not accessible is of no relevance to the stretch sensor as we are not interested in measuring the actual rotation. What is measured is the distance between the two attachment points. From this one can then make inferences on the rotation of the arm. This method works analogue to the Pronator Teres in the lower arm, which by flexing or relaxing dictates the rotation of the wrist. The following two graphs demonstrate how rotating the arm is merely a change in distance between two attachment points.

 Measuring rotation with stretch sensor (Pronator Teres flexed)

 Measuring rotation with stretch sensor (Pronator Teres relaxed)

Working model of ShadowCoat using stretch sensors sewn to long sleeved t-shirt

Signal Processing

One of the drawbacks of using stretch sensors as opposed to using potentiometers is that stretch sensors have an ill defined output. The output is neither logarithmic nor exponential nor straight and is additionally dependent not only on the sensors current state, but also on its previous state. The next image displays the relationship between stretch and signal level. Note how the output is almost linear between 125% stretch and 156.25% stretch. Also note how sensitive the sensor is below 112.5% stretch (sorry for the hard-to-read graph, ill get a nicer version online eventually).

 Change of signal strength depending on stretch of sensor

The next image displays sensor output over time. The x axis is the magnitude of the signal and the y axis displays its change over time. What is displayed is a sensor in relaxed state which is then abruptly pulled to 175% of its original length and the abruptly released again. Note the spikes after stretching and while releasing the tension of the sensor. Interesting is also the gradual decline of the signal after relaxation of the sensor. In this example the sensor has been previously already stretched several times. Therefore the initial signal is already higher than the initial signal at the start of experimentation. I did not establish the time it takes for the sensor to return to baseline state, but it takes quite some time.

 Change of signal over time when stretched to 175% and released again

While the sensor is mechanically ideal for the application, the problems just discussed show that there is much need for signal optimization. This can be done on the software and hardware end of things.

Hardware aspects of signal optimizing:

The signal can be optimized by using two counteracting sensors. These should be arranged in such a form, that they are always in opposite phase to each other. The following graphs demonstrate how this can be achieved.

 Measuring rotation with two counteracting stretch sensors (Pronator Teres stretched)

Measuring rotation with two counteracting stretch sensors (Pronator Teres relaxed)

Capturing angular movement with two counteracting stretch sensors (sharp angle)

Capturing angular movement with two counteracting stretch sensors (blunt angle)

In all examples, one sensor is stretched while the other sensor is relaxes. The two sensor can never both be relaxed or stretched at the same time; they counteract eachother. By adding the two signals together it is possible to improve the sensor.

Software aspects of Signal Optimizing

Output Function Optimizing

The signal of one of the sensors needs to be inverted, allowing for both readings to be used together. If averaged there are several benefits. First of all, the new output is now closer to a linear function – in practice this deviation from linear relationship has proven to be neglectable; the output can be considered approximately linear.

Change of signal strength of  two counteracting stretch sensors and the average signal depending on stretch

If one is mainly interested in amplitude of the signal, two counteracting sensors can be used in a way to maximize the output at the cost of linearity. The fact that signal A is more sensitive below 137.5% and signal B is more sensitive above 137.5% can be taking advantage of by simply creating a threshold value for assigning which sensors readings will be used. One could also use a soft threshold, starting out by using 100% of sensor A’s signal and then gradually adding sensor B’s signal while decreasing sensor A’s contribution to the final signal. If there is little stretch, one would rely mainly on sensor A, while with strong stretch sensor B would be the dominant. This takes advantage of the most sensitive data-range of both sensors.

Noise Reduction

Using two discreet signals and taking the average can also be beneficial in reducing the noise level of the signal. However the noise does not necessarily cancel out, which requires for additional noise reduction measures. One way of achieving this is simply averaging over time, one could for example wait till 10 readings have accumulated. Each outputted value from then on is the average of its 10 predecessors. This method however has the drawback that there is a noticeable time delay and that the response is sluggish.

            The time delay and the sluggish response can be reduced be dynamic averaging, for example by using an algorithm which outputs the average of the last 5 outputs together with the newest 5 inputs. This dynamic averaging could be considered as a very crude low pass filtering method. A low pass filter does exactly what it says: It allows low frequencies to pass, while blocking high frequency signals. Here are algorithm of two variants of a low pass filter which have proven useful:

float alpha = 0.8;

float rawSignal = sensorInput;

float oldCeanSignal;

float cleanSignal;

void loop()
{
oldCleanSignal = cleanSignal;

cleanSignal = alpha * oldRawSignal + (1 - alpha) * rawSignal;
}

(Rumour has it, that this is what apple uses for cleaning up the output of the gyros in its iPhones etc.)

OR

float alpha = 0.35;

float rawSignal = sensorInput;

float oldCleanSignal;

float cleanSignal;

void loop()
{

oldRawSignal = cleanSignal;

cleanSignal = oldCleanSignal + (alpha * (rawSignal - oldCleanSignal));

}

(I got this code snippet from Jeff Rowberg who is creating another awesome input tool)

Alpha is a value which decides upon how strong the filter should work on the signal, it has slightly different functions in both algorithms. sensorInput is the voltage reading of the stretch sensor. rawSignal is a variables for storing these voltage readings and cleanSignal is the filtered signal. oldCleanSignal is a variable for storing the cleanSignal of the last iteration.

Artifact Reduction.

As can be seen in figure 12 the release of the stretch sensors signal is very slow and only reaches zero after an extended period of time. Therefore the output at zero stretch (sensor stretched to 100%) can change over time. This effect is reduced because the opposing sensor at 175% stretch has a constent output, however it is only reduced by a factor of 0.5, in other words it still represents a major problem.

            This problem can be solved by regular auto-calibrations. The measured voltage needs to be scaled to a sensible output (ideally to degrees which correspond with the actual degrees of movement.) During this process the program can constantly update maximum and minimum output values, thus allowing for a signal minimum and maximum which is not fixed. To avoid glitches in this autocalibration (such as a spike changing the maximum to something unnaturally high, thus distorting the scale) it may make sense to periodically reset the minimum and maximum output to its average value. A possible algorithm for doing this in Processing can be found below

//calibrating & converting to correct values
   //setting maximas (RAW values are the input signal, HIGH and LOW values are //maximas and minimas used to outocalibrate

    if (wristRotateARAW > wristRotateAHIGH){
      wristRotateAHIGH = wristRotateARAW;
    }
    if (wristRotateBRAW > wristRotateBHIGH){
      wristRotateBHIGH = wristRotateBRAW;
    }
    if (elbowAngleRAW > elbowAngleHIGH){
      elbowAngleHIGH = elbowAngleRAW;
    }
    //setting minimas
    if (wristRotateARAW < wristRotateALOW){
      wristRotateALOW = wristRotateARAW;
    }
    if (wristRotateBRAW < wristRotateBLOW){
      wristRotateBLOW = wristRotateBRAW;
    }
    if (elbowAngleRAW < elbowAngleLOW){
      elbowAngleLOW = elbowAngleRAW;
    }
  
// remapping to degrees
    wristRotateA = map(x, wristRotateALOW, wristRotateAHIGH, 0, 180);
    wristRotateB = map(y, wristRotateBLOW, wristRotateBHIGH, 0, 180);
    elbowAngle   = map(z, elbowAngleLOW, elbowAngleHIGH, 0, 180);
   

//creating compound variables
   
    wristRotate = (950 - wristRotateA +  wristRotateB )/ 2;

//reset scaling at mouseclick (this could also be done automatically at descreet time //intervalls, but then one would have to be more carefull with the values you assign //then I am in this peace of code. I still need to work on this.

if (mousePressed == true) {
 
   wristRotateALOW = 10000;        // minimum measured value
   wristRotateBLOW = 10000;        // minimum measured value
   elbowAngleLOW = 10000;          // minimum measured value

   wristRotateAHIGH = -10000;       // maximum measured value
   wristRotateBHIGH = -10000;       // maximum measured value
   elbowAngleHIGH = -10000;         // maximum measured value;

  }

-----------------------------------------------------------

 Anyway - thats it for now. I'll upload my full (preliminary) report soon-ish

p.