RaibowCoat - ShadowCoat with some Color :-)

Take a look :-D



And some stills:

And my arduino and breadbord, which powers all of this:

More Experiments with Nitinol Paper

We are accustomed to objects behaving the same way not matter what our mood or how we treat an object. A book behaves the same to someone who loves it and treats it as gentle as possible as it would to another person who hates the book, almost tearing out the pages while reading it. What if that where to be changed?

Take a look at what we are trying to do with paper:

We want to create paper, which moves depending on inputs it receives. It may shy away or close up altogether if you reach for it to fast. It may open and revile its message if left alone, or if soothed. The content of the paper itself will be projected on the inside of it. The projection will be of very graphic nature, playing with the phenomenon that one often is drawn to gruesome content and cannot look away. Here, one may be forced to look away because the paper my physically prevent you from seeing its content.


Animating the Paper

We animate the paper using Nitinol Memory wire. Memory wire has two phases, the Martensite phase at room temperature and the Austenite phase at temperatures greater than 70° Celsius. When in the Martensite phase the shape of the wire is freely deformable, when however heated, in the Austenite phase, it returns to its original shape. The shape it returns to in its Austenite phase can be set by heating the Memory Wire to over 540° Celsius.

When heated the memory wire has enough force to bend a second memory wire. This enables counteracting actuators and is the basic principle of our animated paper. We heat individual strands of memory wire by connecting them to a high powered battery for a brief moment. As we do not add resistors to this circuit, the power spike and, more specifically, the heat which is created in the process induces the phase shift in the memory wire. This setup can easily fold and unfold a piece of paper as can be seen in the above video or in the following picture: 

This mechanism can be used to create paper which can autonomously fold and unfold into simple shapes, reminiscent of folding origami.

Input for interaction

The folding and unfolding will be initiated by various different inputs. We have a variety of sensors available, microphones for measuring sound intensity and frequency, photo resistors which would react to shadows, Sharp IR range-sensors (as seen in the Theremin demo which we brought to class) or kinect for proximity sensing.  Using a combination of these sensors we will attempt to create an intuitive way of interacting with the paper. Proximity and sound amplitude may set off a folding motion when a certain threshold is reached, however it may be interesting to have interaction between the dominant frequency of the sound and the threshold levels. This may enable ‘talking’ to the paper to change its reaction to proximity, volume levels or light levels. If threshold levels have not been surpassed over a certain amount of time, the paper will unfold again. The sensors will not be physically placed on the paper, as the paper needs to be as light as possible and will most likely be prone to interfere with readings due to the temperature changes and the high amount of mA which will be sent through the circuitry on the paper.

A demo of IR range-sensors as Input can be found here:

http://fkeel.blogspot.com/2011/01/maybe-max-isnt-as-terrible-as-i-first.html

Software

 All sensors (except, if we use it, the kinect) will be connected to an arduino which in turn will communicate to Max/MSP/Jitter via serial connection.  The computer-vision aspects will also be handled in Max using cv.jitt.

Power supply

As the paper will be animated using a very high power circuit, we do not want to directly connect any microcontroller to it. All sensor readings will be done using an arduino, powered by a secondary electrical circuit.  The memory wire will then be activated by transistors or possibly even by relays. The primary circuit for animating the Paper is currently using a 9.6v DC power supply intended for RC cars; however for the final installation we will use a laptop power supply.  The sensors and Arduino will be powered by USB.

Tracking Projection Area

For projecting onto the paper we will use an active approach. Surface mount infrared LEDs combined with an IR webcam will be used to track the projection area. Surface mount LEDs are optimal due to their small form factor and can be placed directly onto the paper.  This is similar to the setup used by David Holman on his paper computer project. (http://www.organicui.org/?page_id=5)

Set up & Miscellaneous

As the memory wire can only support paper up to a certain size, we are limited in how large our animated paper can be. It may be interesting creating a whole array or series of small objects instead of one large object. For displaying it most effectively a high contrast setting will be essential. Also, the input which activates the motion of the paper should also activate sounds, creating an ambient atmosphere.

ShadowCoat v 0.02

Before I start rambling, take a look at this video. It’s a demonstration of how the prototype I built can be used as an interface device. I am here demonstrating how it could be used as a mouse. It’s sort of cool, though it sort of takes something really cool and turns it into something banal. While this works, and I know I can get it to work a lot better still, I am hoping to find more intuitive and natural ways of interfacing with computers using this type of technology. (ah, I have already started rambling. Whatever. Watch the video-clip.)(and hear some more rambling)

While my original demo worked quite well, it didn’t have any cable management, so you can spend quite some time figuring out which cable leads where etc. Also the material was quite flimsy and I mounted the Velcro upside down which isn’t helping the material either... also, the pressure fitting was sort of annoying etc. etc. Lots of little things which could be improved.

So on my week off of university, I decided to build a new prototype. Things I wanted to include/change where

- Feedback
- Cable Management
- Tilt of wrist
- Smarter positioning of Velcro
- Sturdier material 


Halfway through my work my sewing machine died on me. Well. It didn’t really die. It just sort of protested against sewing cables and electrical equipment to clothing. Guess this isn’t the future it envisioned. But it still is pretty, have a look:

Also I realized that by sheer luck I did a really good job on my first prototype. I had to figure out, often by trial and error, lots of little things which I, by pure coincidence, got right the first time I did it.

But I prevailed :-D. And while I got about half of what I wanted done, it’s a good start and I can definitely build on it, it is much more expandable/upgradeable than my first attempt.

The white thing you se protruding from my arm is an LED strip, so I can have color feedback to my movements.

One of the things which really baffled me was the voltage divider. First time round it just worked, so I didn’t spend much thought or time on it. This time I was not so lucky. So, here is how it should be done:

It is my experience, that the resolution is best when the resistance of R2 is as close as possible to the nominal resistance of the corresponding sensor. I think it might help this project to include opamps for better resolution. I hope to be testing that soon.

I also rewrote all of my code. For the first time in my life I found myself actually appreciating how powerful object oriented programming is. It improves the simplicity of my code drastically, makes it much easier to add sensors, to scale and map them and to just generally have some useful signal flow.

I also figured out that timing is really crucial. I have never really worried about how long it takes to execute a function, but I realized that this is the reason to 90% of errors and bugs I had in my programming, as it can go haywire when the timing between serial and java as well as the communication between objects is out of sync.

Anyway, for the first time I have some code which actually works well enough, that I feel like sharing it. I will figure out the proper format of doing so as soon as I get around to it. In the meantime feel free to contact me at paul dot strohmeier at gmail.com if you would like a look at it.

*


OK, some more random pictures, just because I took them :-D

 

4 Voltage dividers (schematic of individual one can be found in a pic further up)

 

Stretch Sensor material from Images

I did this at my grandparents place. This is my gradfather workshop, where I did some soldering.

It’s hard to see, but in case you are interested in how I connected the sensor material with the wired, this sort of gives a clue. I basically just wrap the wire ends around the sensor. If I had a working sewing machine I would pin this to the canvas using criss-cross stitches. As my sewing machine preferred not to support this line of work, I just put electrical tape around it, before sewing it to the shirt by hand.

Anyway - that’s it for now. I really hope I will have time to incorporate the leds and other stuff, but I am afraid it might take a while before I find time to continue work again...

Motion Capturing with Stretch Sensor

I have not posted in a while here... anyway ... I wrote a report to wrap up everything I have done so far before I go off to Canada... I'll upload it somewhere... this here is the part that is probably most interesting for most people... (its not 1 to 1 the way it'll be in my final report, but close enough...)

Stretch Sensors

The stretch sensors I use are made of conductive plastic material which changes its resistance depending on the amount it is stretched. This material is produced by the American company Images and can be bought directly at their webshop or  at shops such as the RobotShop. The material is approximately 2mm in diameter and can be stretched up to 175%. 

Basically I want to capture body movement, which is induced by muscles. Muscles are linear actuators; the only movement which they can perform is expanding or contracting. This implies, that all movements of the body are measurable in terms of contracting (flexing) or expanding (relaxing) of the muscle by stretch sensors.

Angular Movement (Elbow and Shoulder)

           The axis of rotation is inaccessible which would be a problem for most sensor types. However, the fact that the attachment points vary in distance  to each other (when measured around the elbow as seen in the following picture) is actually of benefit for this sensor. In the elbow the Triceps Brachii is the main muscle controlling the elbows motion and takes advantage of this very principle. The stretch sensor basically copies the Triceps motion. The following two graphs compare the stretch sensor with the flex sensor. You can see that the flex sensor requires a sliding attachment while the stretch sensor can be permanently attached due to its elasticity.

Capturing angular movement with flex and stretch sensor (sharp angle, Tricaps Brachii relaxed)

Capturing angular movement with flex and stretch sensor (blunt angle, Triceps Brachii flexed)

            Rotational Movement (Rotation of upper and lower arm)

The fact that the rotational axis is not accessible is of no relevance to the stretch sensor as we are not interested in measuring the actual rotation. What is measured is the distance between the two attachment points. From this one can then make inferences on the rotation of the arm. This method works analogue to the Pronator Teres in the lower arm, which by flexing or relaxing dictates the rotation of the wrist. The following two graphs demonstrate how rotating the arm is merely a change in distance between two attachment points.

 Measuring rotation with stretch sensor (Pronator Teres flexed)

 Measuring rotation with stretch sensor (Pronator Teres relaxed)

Working model of ShadowCoat using stretch sensors sewn to long sleeved t-shirt

Signal Processing

One of the drawbacks of using stretch sensors as opposed to using potentiometers is that stretch sensors have an ill defined output. The output is neither logarithmic nor exponential nor straight and is additionally dependent not only on the sensors current state, but also on its previous state. The next image displays the relationship between stretch and signal level. Note how the output is almost linear between 125% stretch and 156.25% stretch. Also note how sensitive the sensor is below 112.5% stretch (sorry for the hard-to-read graph, ill get a nicer version online eventually).

 Change of signal strength depending on stretch of sensor

The next image displays sensor output over time. The x axis is the magnitude of the signal and the y axis displays its change over time. What is displayed is a sensor in relaxed state which is then abruptly pulled to 175% of its original length and the abruptly released again. Note the spikes after stretching and while releasing the tension of the sensor. Interesting is also the gradual decline of the signal after relaxation of the sensor. In this example the sensor has been previously already stretched several times. Therefore the initial signal is already higher than the initial signal at the start of experimentation. I did not establish the time it takes for the sensor to return to baseline state, but it takes quite some time.

 Change of signal over time when stretched to 175% and released again

While the sensor is mechanically ideal for the application, the problems just discussed show that there is much need for signal optimization. This can be done on the software and hardware end of things.

Hardware aspects of signal optimizing:

The signal can be optimized by using two counteracting sensors. These should be arranged in such a form, that they are always in opposite phase to each other. The following graphs demonstrate how this can be achieved.

 Measuring rotation with two counteracting stretch sensors (Pronator Teres stretched)

Measuring rotation with two counteracting stretch sensors (Pronator Teres relaxed)

Capturing angular movement with two counteracting stretch sensors (sharp angle)

Capturing angular movement with two counteracting stretch sensors (blunt angle)

In all examples, one sensor is stretched while the other sensor is relaxes. The two sensor can never both be relaxed or stretched at the same time; they counteract eachother. By adding the two signals together it is possible to improve the sensor.

Software aspects of Signal Optimizing

Output Function Optimizing

The signal of one of the sensors needs to be inverted, allowing for both readings to be used together. If averaged there are several benefits. First of all, the new output is now closer to a linear function – in practice this deviation from linear relationship has proven to be neglectable; the output can be considered approximately linear.

Change of signal strength of  two counteracting stretch sensors and the average signal depending on stretch

If one is mainly interested in amplitude of the signal, two counteracting sensors can be used in a way to maximize the output at the cost of linearity. The fact that signal A is more sensitive below 137.5% and signal B is more sensitive above 137.5% can be taking advantage of by simply creating a threshold value for assigning which sensors readings will be used. One could also use a soft threshold, starting out by using 100% of sensor A’s signal and then gradually adding sensor B’s signal while decreasing sensor A’s contribution to the final signal. If there is little stretch, one would rely mainly on sensor A, while with strong stretch sensor B would be the dominant. This takes advantage of the most sensitive data-range of both sensors.

Noise Reduction

Using two discreet signals and taking the average can also be beneficial in reducing the noise level of the signal. However the noise does not necessarily cancel out, which requires for additional noise reduction measures. One way of achieving this is simply averaging over time, one could for example wait till 10 readings have accumulated. Each outputted value from then on is the average of its 10 predecessors. This method however has the drawback that there is a noticeable time delay and that the response is sluggish.

            The time delay and the sluggish response can be reduced be dynamic averaging, for example by using an algorithm which outputs the average of the last 5 outputs together with the newest 5 inputs. This dynamic averaging could be considered as a very crude low pass filtering method. A low pass filter does exactly what it says: It allows low frequencies to pass, while blocking high frequency signals. Here are algorithm of two variants of a low pass filter which have proven useful:

float alpha = 0.8;

float rawSignal = sensorInput;

float oldCeanSignal;

float cleanSignal;

void loop()
{
oldCleanSignal = cleanSignal;

cleanSignal = alpha * oldRawSignal + (1 - alpha) * rawSignal;
}

(Rumour has it, that this is what apple uses for cleaning up the output of the gyros in its iPhones etc.)

OR

float alpha = 0.35;

float rawSignal = sensorInput;

float oldCleanSignal;

float cleanSignal;

void loop()
{

oldRawSignal = cleanSignal;

cleanSignal = oldCleanSignal + (alpha * (rawSignal - oldCleanSignal));

}

(I got this code snippet from Jeff Rowberg who is creating another awesome input tool)

Alpha is a value which decides upon how strong the filter should work on the signal, it has slightly different functions in both algorithms. sensorInput is the voltage reading of the stretch sensor. rawSignal is a variables for storing these voltage readings and cleanSignal is the filtered signal. oldCleanSignal is a variable for storing the cleanSignal of the last iteration.

Artifact Reduction.

As can be seen in figure 12 the release of the stretch sensors signal is very slow and only reaches zero after an extended period of time. Therefore the output at zero stretch (sensor stretched to 100%) can change over time. This effect is reduced because the opposing sensor at 175% stretch has a constent output, however it is only reduced by a factor of 0.5, in other words it still represents a major problem.

            This problem can be solved by regular auto-calibrations. The measured voltage needs to be scaled to a sensible output (ideally to degrees which correspond with the actual degrees of movement.) During this process the program can constantly update maximum and minimum output values, thus allowing for a signal minimum and maximum which is not fixed. To avoid glitches in this autocalibration (such as a spike changing the maximum to something unnaturally high, thus distorting the scale) it may make sense to periodically reset the minimum and maximum output to its average value. A possible algorithm for doing this in Processing can be found below

//calibrating & converting to correct values
   //setting maximas (RAW values are the input signal, HIGH and LOW values are //maximas and minimas used to outocalibrate

    if (wristRotateARAW > wristRotateAHIGH){
      wristRotateAHIGH = wristRotateARAW;
    }
    if (wristRotateBRAW > wristRotateBHIGH){
      wristRotateBHIGH = wristRotateBRAW;
    }
    if (elbowAngleRAW > elbowAngleHIGH){
      elbowAngleHIGH = elbowAngleRAW;
    }
    //setting minimas
    if (wristRotateARAW < wristRotateALOW){
      wristRotateALOW = wristRotateARAW;
    }
    if (wristRotateBRAW < wristRotateBLOW){
      wristRotateBLOW = wristRotateBRAW;
    }
    if (elbowAngleRAW < elbowAngleLOW){
      elbowAngleLOW = elbowAngleRAW;
    }
  
// remapping to degrees
    wristRotateA = map(x, wristRotateALOW, wristRotateAHIGH, 0, 180);
    wristRotateB = map(y, wristRotateBLOW, wristRotateBHIGH, 0, 180);
    elbowAngle   = map(z, elbowAngleLOW, elbowAngleHIGH, 0, 180);
   

//creating compound variables
   
    wristRotate = (950 - wristRotateA +  wristRotateB )/ 2;

//reset scaling at mouseclick (this could also be done automatically at descreet time //intervalls, but then one would have to be more carefull with the values you assign //then I am in this peace of code. I still need to work on this.

if (mousePressed == true) {
 
   wristRotateALOW = 10000;        // minimum measured value
   wristRotateBLOW = 10000;        // minimum measured value
   elbowAngleLOW = 10000;          // minimum measured value

   wristRotateAHIGH = -10000;       // maximum measured value
   wristRotateBHIGH = -10000;       // maximum measured value
   elbowAngleHIGH = -10000;         // maximum measured value;

  }

-----------------------------------------------------------

 Anyway - thats it for now. I'll upload my full (preliminary) report soon-ish

p.

Sewing the Stretch Sensor

I have been trying to figure out how to best connect the stretch sensors to fabric. What I did before was simply using relatively thick multi-strand cables and attaching the stretch sensor using pressure fitting. I would then just sew the multi-strand cable to the fabric.


However, the rigid plastic parts required for the pressure fitting do not especially appeal to me, they are clumsy and don’t work as well as I was hoping they would.
I was hoping to find a method of making the actual sensor more stable and making it easier to attach the sensor to clothing. I ended up sewing the sensor and cable directly to elastic bands. This works fairly well, especially as the maximum stretch of the elastic is less than that of the sensor, protecting the sensor from being overstretched. Well … enough talk, take a look at the pictures to see how this looks in practice

Ahh… and ignore all the extra threads and sloppy craftmanship. Next-time round this will look a lot prettyer. My sewing machine and me *are* (slowly) becoming friends…

ShadowCoat Mouse

I procrastinate productivly: I should have been doing other university stuff, instead I thought... "can I use my set-up as a mouse?"

Well, I gave it a shot. I'll coment more on it later.

Introducing: The ShadowCoat (Part 2)

As mentioned before,  I have changed my approach a bit, dropping the thermoplastics and going more towards smart clothing direction. I suggest you just take a look at the video. I'll post some pictures and explanations below.

What is most noteworthy in the video is that the rotation of the arm does the two sensor thing I've been talking about (cant wait to see that work on the elbow and shoulder as well). This greatly increased the accuracy of the reading. Also, the software now out-calibrates. This makes the suit ultra-usable. A friend just tried it on and it worked as effortlessly with her as it did with me.





I am still using the stretch sensor by Images. As you can't soldier plastic, I had to find other means of connecting them to cables:

  

I use multi-core cables (ahh, I know thats not the correct term, but its late...), so sewing over/through/on them is no problem

I am sewing them to the construction I prepared yesterday, you can see them in this post.

I was fooling around with the readings from the stretch sensors earlier and realized, that using two sensors which counteract each other would greatly benefit accuracy and amplitude of the signal. I tried it, and yep, it works :-)

From my ShadowCoat :-) the sensors connect to voltage dividers which in turn connect to the Arduino Mega...

Here is what the whole thing (and me) look  like at the moment. I'm quite happy, cause this worked better than expected :-)

Sending Data over the Network --> First Success

So I figured out that Processing has a network library. Its a step in the right direction. Raises the question weather this can be used for sending data via the Internet. Probably not. Actually, yes it can :-) But hey. One step at a time

Enjoy the clip:

Getting better aquainted with the Stretch Sensors

As mentioned in a previous post, I now have a stretch sensor by Images in my possession. Though I was first a bit skeptical I am beginning to see more and more ways of using this. I sort of had a hunch that the relationship between stretch and resistance might not be linear (yeah admitted, nothing strange in that, I would be more surprised *where* there a linear relationship.)

I am a visual person and I need to see things to understand them, so I made these graphs:

Stretched to 125 % of original length.

Stretched to 150 %

Stretched to 175 %

Stretched to 112.5 %, 125 %, 137.5 %, 150 %, 162,5 %, and 175%

Stretched with incrementions of 6.25 %

 

Due to the fact that I am lazy the graphs might be a bit confusing to read. The x-axis represents time and the white vertical lines represent seconds (I just double checked my math and the white lines are more like half seconds - maybe I will redo these if I get around to it. However you still get the generally picture). The y-axis measure the change in resistance. The light gray area is the resistance of the sensor. I did not add units, as the interesting part is the relative change and that is all I measured. 

The fact that the signal is non-linear can, with some tinkering, be used as a bonus. The resolution in the first couple percent of stretch is quite high - so if I where to use two stretch sensors simultaneously but in opposite phase (if one is loose, the other one is stretched) I should be able to maximize the resolution available :-)

Images Stretch & Flex sensor

(scroll down a bit to go directly to the videos.)

As I said, I received two sensors by images the other day. A flex sensors and a stretch sensor (find documentation here and here. Unlike a potentiometer these do not come with a voltage divider to simply measure the resistance, so I had to go about constructing my own. Basically what I did was put a resister between ground and the sensor and attached a third cable to the sensor right before where I attached the resister. (The same way as you would do it for a photo-resistor or any other resistive sensor i guess...)

Here are some pictures of the first readings I took with the flex sensor:

straight:

and bent:

I then went about to see if I could actually use them as I hoped. (The flex sensor for measuring the hands clasp and the stretch sensor for measuring wrist rotation)

Here's an image of my flex-sensor-glove setup:

And here are videos of both flex and stretch sensors in action:






I am a bit worried that the signal from the stretch sensor might be too weak, but I believe I can improve this when I actually go about constructing the whole thing (I might not be using the optimal resistor in the voltage divider and have to experiment with location of the sensor etc.)

The flex sensor/glove combo works so well that I've had everyone who comes close to my room wear it and try it out for themselves :-D