RaibowCoat - ShadowCoat with some Color :-)

Take a look :-D



And some stills:

And my arduino and breadbord, which powers all of this:

ShadowCoat v 0.02

Before I start rambling, take a look at this video. It’s a demonstration of how the prototype I built can be used as an interface device. I am here demonstrating how it could be used as a mouse. It’s sort of cool, though it sort of takes something really cool and turns it into something banal. While this works, and I know I can get it to work a lot better still, I am hoping to find more intuitive and natural ways of interfacing with computers using this type of technology. (ah, I have already started rambling. Whatever. Watch the video-clip.)(and hear some more rambling)

While my original demo worked quite well, it didn’t have any cable management, so you can spend quite some time figuring out which cable leads where etc. Also the material was quite flimsy and I mounted the Velcro upside down which isn’t helping the material either... also, the pressure fitting was sort of annoying etc. etc. Lots of little things which could be improved.

So on my week off of university, I decided to build a new prototype. Things I wanted to include/change where

- Feedback
- Cable Management
- Tilt of wrist
- Smarter positioning of Velcro
- Sturdier material 


Halfway through my work my sewing machine died on me. Well. It didn’t really die. It just sort of protested against sewing cables and electrical equipment to clothing. Guess this isn’t the future it envisioned. But it still is pretty, have a look:

Also I realized that by sheer luck I did a really good job on my first prototype. I had to figure out, often by trial and error, lots of little things which I, by pure coincidence, got right the first time I did it.

But I prevailed :-D. And while I got about half of what I wanted done, it’s a good start and I can definitely build on it, it is much more expandable/upgradeable than my first attempt.

The white thing you se protruding from my arm is an LED strip, so I can have color feedback to my movements.

One of the things which really baffled me was the voltage divider. First time round it just worked, so I didn’t spend much thought or time on it. This time I was not so lucky. So, here is how it should be done:

It is my experience, that the resolution is best when the resistance of R2 is as close as possible to the nominal resistance of the corresponding sensor. I think it might help this project to include opamps for better resolution. I hope to be testing that soon.

I also rewrote all of my code. For the first time in my life I found myself actually appreciating how powerful object oriented programming is. It improves the simplicity of my code drastically, makes it much easier to add sensors, to scale and map them and to just generally have some useful signal flow.

I also figured out that timing is really crucial. I have never really worried about how long it takes to execute a function, but I realized that this is the reason to 90% of errors and bugs I had in my programming, as it can go haywire when the timing between serial and java as well as the communication between objects is out of sync.

Anyway, for the first time I have some code which actually works well enough, that I feel like sharing it. I will figure out the proper format of doing so as soon as I get around to it. In the meantime feel free to contact me at paul dot strohmeier at gmail.com if you would like a look at it.

*


OK, some more random pictures, just because I took them :-D

 

4 Voltage dividers (schematic of individual one can be found in a pic further up)

 

Stretch Sensor material from Images

I did this at my grandparents place. This is my gradfather workshop, where I did some soldering.

It’s hard to see, but in case you are interested in how I connected the sensor material with the wired, this sort of gives a clue. I basically just wrap the wire ends around the sensor. If I had a working sewing machine I would pin this to the canvas using criss-cross stitches. As my sewing machine preferred not to support this line of work, I just put electrical tape around it, before sewing it to the shirt by hand.

Anyway - that’s it for now. I really hope I will have time to incorporate the leds and other stuff, but I am afraid it might take a while before I find time to continue work again...

Motion Capturing with Stretch Sensor

I have not posted in a while here... anyway ... I wrote a report to wrap up everything I have done so far before I go off to Canada... I'll upload it somewhere... this here is the part that is probably most interesting for most people... (its not 1 to 1 the way it'll be in my final report, but close enough...)

Stretch Sensors

The stretch sensors I use are made of conductive plastic material which changes its resistance depending on the amount it is stretched. This material is produced by the American company Images and can be bought directly at their webshop or  at shops such as the RobotShop. The material is approximately 2mm in diameter and can be stretched up to 175%. 

Basically I want to capture body movement, which is induced by muscles. Muscles are linear actuators; the only movement which they can perform is expanding or contracting. This implies, that all movements of the body are measurable in terms of contracting (flexing) or expanding (relaxing) of the muscle by stretch sensors.

Angular Movement (Elbow and Shoulder)

           The axis of rotation is inaccessible which would be a problem for most sensor types. However, the fact that the attachment points vary in distance  to each other (when measured around the elbow as seen in the following picture) is actually of benefit for this sensor. In the elbow the Triceps Brachii is the main muscle controlling the elbows motion and takes advantage of this very principle. The stretch sensor basically copies the Triceps motion. The following two graphs compare the stretch sensor with the flex sensor. You can see that the flex sensor requires a sliding attachment while the stretch sensor can be permanently attached due to its elasticity.

Capturing angular movement with flex and stretch sensor (sharp angle, Tricaps Brachii relaxed)

Capturing angular movement with flex and stretch sensor (blunt angle, Triceps Brachii flexed)

            Rotational Movement (Rotation of upper and lower arm)

The fact that the rotational axis is not accessible is of no relevance to the stretch sensor as we are not interested in measuring the actual rotation. What is measured is the distance between the two attachment points. From this one can then make inferences on the rotation of the arm. This method works analogue to the Pronator Teres in the lower arm, which by flexing or relaxing dictates the rotation of the wrist. The following two graphs demonstrate how rotating the arm is merely a change in distance between two attachment points.

 Measuring rotation with stretch sensor (Pronator Teres flexed)

 Measuring rotation with stretch sensor (Pronator Teres relaxed)

Working model of ShadowCoat using stretch sensors sewn to long sleeved t-shirt

Signal Processing

One of the drawbacks of using stretch sensors as opposed to using potentiometers is that stretch sensors have an ill defined output. The output is neither logarithmic nor exponential nor straight and is additionally dependent not only on the sensors current state, but also on its previous state. The next image displays the relationship between stretch and signal level. Note how the output is almost linear between 125% stretch and 156.25% stretch. Also note how sensitive the sensor is below 112.5% stretch (sorry for the hard-to-read graph, ill get a nicer version online eventually).

 Change of signal strength depending on stretch of sensor

The next image displays sensor output over time. The x axis is the magnitude of the signal and the y axis displays its change over time. What is displayed is a sensor in relaxed state which is then abruptly pulled to 175% of its original length and the abruptly released again. Note the spikes after stretching and while releasing the tension of the sensor. Interesting is also the gradual decline of the signal after relaxation of the sensor. In this example the sensor has been previously already stretched several times. Therefore the initial signal is already higher than the initial signal at the start of experimentation. I did not establish the time it takes for the sensor to return to baseline state, but it takes quite some time.

 Change of signal over time when stretched to 175% and released again

While the sensor is mechanically ideal for the application, the problems just discussed show that there is much need for signal optimization. This can be done on the software and hardware end of things.

Hardware aspects of signal optimizing:

The signal can be optimized by using two counteracting sensors. These should be arranged in such a form, that they are always in opposite phase to each other. The following graphs demonstrate how this can be achieved.

 Measuring rotation with two counteracting stretch sensors (Pronator Teres stretched)

Measuring rotation with two counteracting stretch sensors (Pronator Teres relaxed)

Capturing angular movement with two counteracting stretch sensors (sharp angle)

Capturing angular movement with two counteracting stretch sensors (blunt angle)

In all examples, one sensor is stretched while the other sensor is relaxes. The two sensor can never both be relaxed or stretched at the same time; they counteract eachother. By adding the two signals together it is possible to improve the sensor.

Software aspects of Signal Optimizing

Output Function Optimizing

The signal of one of the sensors needs to be inverted, allowing for both readings to be used together. If averaged there are several benefits. First of all, the new output is now closer to a linear function – in practice this deviation from linear relationship has proven to be neglectable; the output can be considered approximately linear.

Change of signal strength of  two counteracting stretch sensors and the average signal depending on stretch

If one is mainly interested in amplitude of the signal, two counteracting sensors can be used in a way to maximize the output at the cost of linearity. The fact that signal A is more sensitive below 137.5% and signal B is more sensitive above 137.5% can be taking advantage of by simply creating a threshold value for assigning which sensors readings will be used. One could also use a soft threshold, starting out by using 100% of sensor A’s signal and then gradually adding sensor B’s signal while decreasing sensor A’s contribution to the final signal. If there is little stretch, one would rely mainly on sensor A, while with strong stretch sensor B would be the dominant. This takes advantage of the most sensitive data-range of both sensors.

Noise Reduction

Using two discreet signals and taking the average can also be beneficial in reducing the noise level of the signal. However the noise does not necessarily cancel out, which requires for additional noise reduction measures. One way of achieving this is simply averaging over time, one could for example wait till 10 readings have accumulated. Each outputted value from then on is the average of its 10 predecessors. This method however has the drawback that there is a noticeable time delay and that the response is sluggish.

            The time delay and the sluggish response can be reduced be dynamic averaging, for example by using an algorithm which outputs the average of the last 5 outputs together with the newest 5 inputs. This dynamic averaging could be considered as a very crude low pass filtering method. A low pass filter does exactly what it says: It allows low frequencies to pass, while blocking high frequency signals. Here are algorithm of two variants of a low pass filter which have proven useful:

float alpha = 0.8;

float rawSignal = sensorInput;

float oldCeanSignal;

float cleanSignal;

void loop()
{
oldCleanSignal = cleanSignal;

cleanSignal = alpha * oldRawSignal + (1 - alpha) * rawSignal;
}

(Rumour has it, that this is what apple uses for cleaning up the output of the gyros in its iPhones etc.)

OR

float alpha = 0.35;

float rawSignal = sensorInput;

float oldCleanSignal;

float cleanSignal;

void loop()
{

oldRawSignal = cleanSignal;

cleanSignal = oldCleanSignal + (alpha * (rawSignal - oldCleanSignal));

}

(I got this code snippet from Jeff Rowberg who is creating another awesome input tool)

Alpha is a value which decides upon how strong the filter should work on the signal, it has slightly different functions in both algorithms. sensorInput is the voltage reading of the stretch sensor. rawSignal is a variables for storing these voltage readings and cleanSignal is the filtered signal. oldCleanSignal is a variable for storing the cleanSignal of the last iteration.

Artifact Reduction.

As can be seen in figure 12 the release of the stretch sensors signal is very slow and only reaches zero after an extended period of time. Therefore the output at zero stretch (sensor stretched to 100%) can change over time. This effect is reduced because the opposing sensor at 175% stretch has a constent output, however it is only reduced by a factor of 0.5, in other words it still represents a major problem.

            This problem can be solved by regular auto-calibrations. The measured voltage needs to be scaled to a sensible output (ideally to degrees which correspond with the actual degrees of movement.) During this process the program can constantly update maximum and minimum output values, thus allowing for a signal minimum and maximum which is not fixed. To avoid glitches in this autocalibration (such as a spike changing the maximum to something unnaturally high, thus distorting the scale) it may make sense to periodically reset the minimum and maximum output to its average value. A possible algorithm for doing this in Processing can be found below

//calibrating & converting to correct values
   //setting maximas (RAW values are the input signal, HIGH and LOW values are //maximas and minimas used to outocalibrate

    if (wristRotateARAW > wristRotateAHIGH){
      wristRotateAHIGH = wristRotateARAW;
    }
    if (wristRotateBRAW > wristRotateBHIGH){
      wristRotateBHIGH = wristRotateBRAW;
    }
    if (elbowAngleRAW > elbowAngleHIGH){
      elbowAngleHIGH = elbowAngleRAW;
    }
    //setting minimas
    if (wristRotateARAW < wristRotateALOW){
      wristRotateALOW = wristRotateARAW;
    }
    if (wristRotateBRAW < wristRotateBLOW){
      wristRotateBLOW = wristRotateBRAW;
    }
    if (elbowAngleRAW < elbowAngleLOW){
      elbowAngleLOW = elbowAngleRAW;
    }
  
// remapping to degrees
    wristRotateA = map(x, wristRotateALOW, wristRotateAHIGH, 0, 180);
    wristRotateB = map(y, wristRotateBLOW, wristRotateBHIGH, 0, 180);
    elbowAngle   = map(z, elbowAngleLOW, elbowAngleHIGH, 0, 180);
   

//creating compound variables
   
    wristRotate = (950 - wristRotateA +  wristRotateB )/ 2;

//reset scaling at mouseclick (this could also be done automatically at descreet time //intervalls, but then one would have to be more carefull with the values you assign //then I am in this peace of code. I still need to work on this.

if (mousePressed == true) {
 
   wristRotateALOW = 10000;        // minimum measured value
   wristRotateBLOW = 10000;        // minimum measured value
   elbowAngleLOW = 10000;          // minimum measured value

   wristRotateAHIGH = -10000;       // maximum measured value
   wristRotateBHIGH = -10000;       // maximum measured value
   elbowAngleHIGH = -10000;         // maximum measured value;

  }

-----------------------------------------------------------

 Anyway - thats it for now. I'll upload my full (preliminary) report soon-ish

p.

Introducing: The ShadowCoat (Part 2)

As mentioned before,  I have changed my approach a bit, dropping the thermoplastics and going more towards smart clothing direction. I suggest you just take a look at the video. I'll post some pictures and explanations below.

What is most noteworthy in the video is that the rotation of the arm does the two sensor thing I've been talking about (cant wait to see that work on the elbow and shoulder as well). This greatly increased the accuracy of the reading. Also, the software now out-calibrates. This makes the suit ultra-usable. A friend just tried it on and it worked as effortlessly with her as it did with me.





I am still using the stretch sensor by Images. As you can't soldier plastic, I had to find other means of connecting them to cables:

  

I use multi-core cables (ahh, I know thats not the correct term, but its late...), so sewing over/through/on them is no problem

I am sewing them to the construction I prepared yesterday, you can see them in this post.

I was fooling around with the readings from the stretch sensors earlier and realized, that using two sensors which counteract each other would greatly benefit accuracy and amplitude of the signal. I tried it, and yep, it works :-)

From my ShadowCoat :-) the sensors connect to voltage dividers which in turn connect to the Arduino Mega...

Here is what the whole thing (and me) look  like at the moment. I'm quite happy, cause this worked better than expected :-)

Sending Data over the Network --> First Success

So I figured out that Processing has a network library. Its a step in the right direction. Raises the question weather this can be used for sending data via the Internet. Probably not. Actually, yes it can :-) But hey. One step at a time

Enjoy the clip:

University is catching up on me - any help?

I was hoping to get the communication with the Nao (or at least with choregraphe) up and running by Monday...

Well, looks like I have a ton of other things to do for university and that's not gonna happen :-(

I have not come that far yet. This is the way I understand it: Choregraphe is just a programme for stringing together individual movements. These movements are programmed in python.
So what I need to do is figure out how to read serial import with python. 

Only problem. Today is the first time I even opened a python manual. However, it seems simple enough and writing interpreted code is sort of fun. (Its like you have a super smart text editor, which talks back to you while you write...  I like it :-)..

So this is how far I got, up until now.

Physical set up:

A potentiometer attached to my Arduino on analogue pin 4.

Arduino Code:

Python Code:
(using pySerial: http://pyserial.sourceforge.net/)

What needs to happen now?

a) (easy) The '1023\r\n' needs to be translated into an integer (I guess there are some string operators, which can allow me to do it. need to look that up again.)

b) (no idea how to do that as of yet.) Within choregraphe I need to write some code which will enable me to use the integer to define the angle of the servo in the elbow of the NAO.

So.

Who can help me?
Please?

As I said, I need to focus on other university things, and if anyone could give me some suggestions on how to go about this, it would be very much appreciated.