Electronics Team #12 Week April 20-26

The video says it all.

Android Dev Team #12 Week April 20-26

The progress:
So far the android app has a new gesture comparison algorithm similar to the one dollar gesture recognizer algorithm. The alteration is simpler in that it doesn't have rotation and the number of points for resampling and the bound box is a much smaller scale. The recognition is based on the point by point difference between the gesture inputted via the android app to a new gesture inputted by the fabric device. While the fabric device is configured on a separate app designed by Matthew Moss, I will port the blue tooth communication over to the main android app I am working on.
The app also has Text To Speech implemented. The application will now respond to input by sound, prompting the user with the gesture it is comparing the input gesture to. It is meant to completely replace the screen by earpiece sound and fabric input on bluetooth. However I do not know if adding the bluetooth headset for outbound sound will be within the scope of this project.
I had to change the way I had input the buttons by the device earlier this week as it had been causing my algorithm to not work correctly. The way I was doing this before was by grabbing each button top to bottom and right to left, causing an incorrect path. This has modified to input the button being pressed in series by the sequence in which they are being pressed. Also, I had to modify the gesture table in the sqlite database to contain not just numbers, but points sorted by which the order they are being pressed in the template x,y; where the colon is the delimiter that another point is in the gesture and a null character terminator. e.g. 5,8;5,7;6,7;\0

Things to do:
I need to make my android app work with the fabric device. I will do this by copying the code from Matt's testing app. Once this is in place, I will need to alter my input algorithm to match the fabric instead of the previous way of using the touch screen on the phone to input gestures. This may prove to be challenging. I plan on having the user make a single gesture, after the finger has removed pressure from all the buttons on the fabric interface, after a second the gesture with all the buttons that had pressure applied will be submitted for testing.
The professor had also mentioned changing the gesture database to be input straight from the fabric device itself so that it will have the same variations the fabric outputs have, as opposed to the phone touch interface, as it differs quite a bit. The fabric seems to apply pressure to multiple buttons surrounding the button the user wants to apply pressure to. Michael and Matt and working on making the button pressure more exact by applying some type of buffer material so that finger pressure is precise. Also, the fabric is being working on by the other team, so we will need time to test the device with the android app once it has been refined for fabric testing, by meeting up some time later. Once the device and the application have been combined, we will by able to further test the combined system for a user study.
Once we near the end of the semester we will need to submit our code and documents to the GIT server as to preserve the project process for another semester. This is of minimal concern right now. Currently we are using a svn server that has been working fine for uploading and collaborating our progress with other team members.

Electronics Team #11 Week April 13-19

Matt again.

When I'm not dedicating inordinate amounts of time to electronics for high altitude photography (similar to this) I'm working to test and better our touch pad interface. With Micheal. And it is going slow because I lack life experience with sewing and seaming. What we've found is a number of issues with the touchpad.

The most basic is that it doesn't like us. Our initial concern with unintended button presses has been brought to a painful forefront. Thereabouts of the only gesture that could possibly reliably work right now is slapping your hand down on your wrist. What tends to happen is that in the course of touching the pad hard enough to make a contact, a number of other contacts are also made. This leads to massive squares and square shaped entities of contact, which are useless from a recognition point of view. We expected some unintended presses, but also expected them to be confined to a small region. The current '1 press leads to chaos' mechanism isn't going to work.

In all likelyhood, this is caused by the separator material. We used felt as a compromise between pliability (for ease of making contacts) and strength, for keeping contacts broken. This doesn't seem to have worked out too well. We're going to try adding a plastic backing to reinforce the felt, ignoring compressibility in favor of usefulness, and we'll see where that takes us.

If given more time, I think the main thing that we should look at would be using a better separator. What I suspect we need is something more akin to a soft foam- like you'd get with certain pillows or foam pads. Something that basically acts as more of a spring, but with low tension- ie, can be easily compressed, but will keep enough distance that creases in the outer fabric layer won't lead to unintentional contact.

Ultimately, I suspect we're looking at a tradeoff between "pressure required to activate" and "accuracy of input". With the added bonus of "pain to work with". Some initial experiments with foam from hobby lobby reveal that cutting it to thin enough slices is a precise art, for which I am like 3 year old in an art museum. Getting holes in the separator is also rather difficult, the only reliable method I've had so far is to take a soldering iron to it and burn holes through it, hoping it doesn't catch fire or spew forth toxic fumes.

Other things to think about might just be the removal of a gesture system all together, in favor of an actual button pad. I don't believe a soldier is going to have much luck remembering more then five or so gestures; developing an actual special purpose key pad, rather then a touch pad, would probably be more successful, as it can be done with far fewer buttons, and thus without the need for multiplexing IO that led to our mass of touch detections.

Next week Micheal is up for chatting, until then we'll continue playing around with the prototype to try and get it closer to working. Somewhere in this set of things is getting the Bluetooth code integrated into Adams gesture recognition app, we'll see how far we can get with all these things.

Android Dev Team #11 Week April 13-19

I did a revamp of the application, it was looking pretty ugly. It looks much nicer now. The user interface is cleaner. The input method has been refined. What I mean by that is that whenever the finger enters pressure on the screen the gesture is input and only after the submit or compare button the user is able to submit the gesture, and the screen gets cleared. We also have a clear button to undo anything that might be wrong.
I also forgot to mention. During our first user study the general consensus was to make the gestures more understandable. For example, a 1 should actually look like a 1 on the gesture. A "s" should have the same shape. People should have to do very little re learning when using this application.

Electronics Team #10 Week April 6-12

Check this out!

This week, we made modifications:

• We had a pocket sewn on the back of the armband into which we could insert a stiff material to provide structure and allow for greater sensitivity.
• When that didn't work, we hot glued stiff material to the separator material so the the separator material would still be compressible but the stiff material would provide structural support.
• We attached Velcro to make the armband more easily wearable.

The stiff support material was added because when we would actually wear the armband, the fabric would get pulled taut and bring the layers closer together, which was a big part of why we were getting so much accidental activation. Testing went well, and we got some better results than last week.

Tune in next week!

Android Dev Team #10 Week April 6-12

So Paul decided to help me further with my frustrations on the algorithm, it simply does not work. I attempt to debug my code with something visual I can wrap my head around. I zealously take screen shots to determine where the problem lies after each one dollar comparison step. I finally figure out that my whole process of grabbing points is wrong. I reformat my zeros and ones to point by point, being in a time sequence in which they are entered. This fixes the problem but only after I dumb down the scoring algorithm even further and add and point by point difference and then sum then up. What I mean by point by point difference is taking the euclidean distance between the two points. I pretty much remove the rotation step and the angular stuff for scoring and all is well. Paul was very helpful in helping me troubleshoot this issue. Here are some screenshots of my JFrame output of the points during each step. To simplify things I ended up creating a separate Java Application with a JFrame with some database and input dummy data for testing.

The resampling (you can see here this is bad, the old way was putting points in in top left order instead of time order)

The rotation, things get weird here

The translating the origin and bounding, this parts fine

Electronics Team #9 Week March 27-April 5

Check out our new update here!

Howdy! Michael again. Interesting results from our initial test: it doesn't work. Well, I mean, it works, but too well.

We're getting a lot of accidental activation from a single input point on the armband, e.g. one input point actually pressed yields nine input points read. Unfortunately, this means that we're going to need to attach the other layer of separator material I talked about earlier. We're hoping this doesn't cause too many issues, or any of the same issues we were trying to avoid by using individual strips rather than a single layer of separator material.

In other new, the Bluetooth is a little wonky, but I don't think that it's anything Matt or I could control, so what can you do? Don Norman would probably have something to say about it, but we just carry on the best we can :)

Tune in next week, when we do the same thing all over again!