From Box Files to the British Science Festival: Part Two

Here is Part 2 of the series on how and why the MIRAGE was developed

If you remember, I was trying to make adaptation to visual shifts (like when wearing prism goggles) more flexible. To do this, I hacked into some motion tracking software (this tracked a sensor on the hand as it moved through space) so that I could read and change the x, y and z coordinates of the cursor on a moment-by-moment basis. This created a system whereby the cursor could either track the location of a subject’s hand, or could be manipulated to move independently of it. Once I could do this, I could show a cursor (a big black dot) to the subject as a representation of their hand location. When I first tried this I glued a projection screen that I found in my loft onto the top of a box, and hung a projector from the ceiling so it shone down onto the screen. The subject’s hand was inside the box with the motion tracking sensor on so wherever the hand moved inside the box, a black dot followed it around on top of the box. This, I suppose, was Mark 1, but I didn’t like it for two reasons: firstly a cursor is not a great representation for the hand and secondly, the projected cursor was not in the right spatial location as it was always above the hand. This was a problem for some people, especially patients with brain damage, as they would always lift their hand up to try to match it to the cursor location above.

Mark 2 was made out of box files, a stolen mirror and some drafting paper (a bit like greaseproof baking paper) and was designed to solve the second of these problems. The projector shone down from the ceiling onto the paper and, because it was a little bit see through, the image of the cursor could be seen on the underside of the paper as well. A big mirror, rescued from a skip, rested precariously on box files so that it lay horizontally exactly half way between the paper and the table. Now, when the subject looked down into the mirror, they could see a reflection of the cursor (above) so that it appeared to be at the same level as their hidden hand on the table (below the mirror). So the cursor was now in the right place, but it still looked like a cursor, not a hand.

Mark 2 was used for Newport and Jackson, 2006 published in Neuropsychologia

Next time: how the ill-fated Mark 3 tried to solve problem 1.

Written by Dr. Roger Newport