Hypomnemata

One thing I learned from Perceptron was that interactive art is a lot like engineering: it’s all about coping when plans fall through.  For instance, one of my goals was to have a neat IR-based camera system in the final product.  It didn’t gel by our first installation, and has kind of fallen by the wayside for Perceptron Mk II (I’ve also mostly bowed out, in favor of another project or two).  However, I’m still rather interested in IR sensing in general.  There’s just something about having cameras that can see something humans can’t that makes me smile.

The goal wasn’t to see heat, but instead to see people.  Perceptron uses a video feed to detect people and their movement.  This is then fed into something that generates video, which we wanted to project onto a wall behind the person.  Now, most people-detection algorithms use motion detection as part of their process.  If you’re projecting a moving image up, then trying to find a person moving on top of that… it’s going to get really confused.  To get around this, I wanted to use an IR camera and flood the video display surface with IR light.

Getting the IR light is easy: remote controls use IR, so there are a lot of IR LEDs on the market.  A quick email to the noisebridge mailing list (“High-output IR LEDs?“), and I had a few suggestions for IR LEDs from Mitch Altman, the guy behind TV-B-Gone.  A quick mouser order and the IR light part was done, but where to get an IR camera?

Thankfully, most cheap webcams are actually sensitive to IR!  The technology they use to collect light for an image (charge-coupled devices, CCDs) is actually pretty sensitive to everything from IR to just above the visible spectrum.  If you don’t believe me, get out a digital camera, point a remote control at it and hit some buttons.  You should see a pink-ish flash from the LED on the front on the screen of the camera: that’s IR getting into the red-sensing bit of the camera.

So, we can use cheap webcams: great!  I have four of them just lying around, and between us, we had something like a dozen of them.  But allowing IR into a camera is problematic: it blows out the reds, so they put a filter in to stop it.  On nicer webcams, you can get in and remove that filter, but on the cheapest ones, it’s basically part of the sensor itself.  Did I mention that I had four cheap webcams?  To get four, they’re going to be real cheap.  So cheap that the filter is on the sensor itself.  They still see IR, but not very well: it gets drowned out by the visible light.  Admittedly, this is what most people want, but I found it rather frustrating.  To overcome this, you can put in a visible light filter, which blocks visible light, rendering the camera nearly blind.  You can use lots of things for this: exposed 35mm film, floppy disk material, etc, but they’re all a little hacky and fragile.  I wanted a proper filter, but real filters are damned expensive (and still a bit fragile, not something you want in a project that might wind up at Burning Man).

A week after the showing, Jean emailed me a link detailing how to use the Wii controller’s IR sensor to track IR points.  It isn’t quite what we want, but it made me realize something: the Wiimote is a great source of cheap, robust IR filters.  I have a couple Wiimotes lying around at home, so I ordered a tri-wing screwdriver from DealExtreme (85¢US, shipped, par avion, from Hong Kong) and tonight, popped open my second Wiimote.  The IR filter works stunningly, and now I have a dedicated “data acquisition device” Wiimote.  3-axis accelerometer, lots of buttons, and a light-source tracker that works in visible and IR now: this thing is quite great!

(The SICKmods Wiimote disassembly tutorial has good pictures of the disassembly process.  In particular, you’ll want to be careful with the two tabs at the top of the device.  I used my vinyl pry tools, part of the kit to change an original iPod battery… As for getting accelerometer, on linux, I’m using CWiid, but will just be using wmdemo to pipe data out for now.  There’s a gotcha with building it in its current state, documented in a bug report: hci_remote_name should be hci_read_remote_name.  The change described there worked for me.)