It’s all in (front of) your head

July 7, 2010

To all appearances I am completely crazy.

I’m standing in Arroyo Park, trying to grab an object that no one else can see. I’m wearing a hat adorned with a giant chunk of Styrofoam over my eyes, a small blue chunk of Styrofoam on my finger, and a computer mouse strapped to my other arm.  And a regular school backpack with wires trailing out of it. And grabbing that thing is really difficult–it keeps squirming out of my hand just as I’m about to get it.

Except that, unlike normal insanity, I can give it to someone else. All they have to do is put the hat on their head. That’s because the hat is actually an augmented reality system, complete with a cheap webcam mounted on the brim, and an ipod touch acting as a display. The webcam forwards its video to a computer in the backpack, which processes it and sends it to the ipod for display. Welcome to the world of augmented reality.

In a nutshell, augmented reality is a combination of computer graphics and the real world. Because the laptop in my backpack can “see” exactly what I’m seeing, it can process the data with a variety of algorithms–identifying people, for instance. Because the computer sits between the actual visual image and the person viewing it, it can also modify the video it displays, so that, to a user, it looks like someone’s twitter updates are hovering over their head, or the sky is superimposed with a weather forecast.

Although there is huge potential for this technology, development has been rather slow and limited. Most available AR systems:

  • Are limited to a specific domain(i.e.dipslaying models of the earth)
  • Only run on a specific device
  • Sometimes require specialized displays(especially for “immersive” head-mounted systems) that costs thousands of dollars

…enter the HackerHat.

The HackerHat is designed to be a general-purpose, open-source, head-mounted AR system that can be assembled from a wide variety of inexpensive  hardware. Because both the video capture system and the streaming system that it uses are open protocols, the same processing software can be used even with radically different hardware setups. Check it out, in the video below:

Want to have one? Want to help test out new versions of the software? Let me know, and I’ll help you get one set up!


3 Responses to “It’s all in (front of) your head”

  1. kidma said

    That part about the object recognition is total BS. There’s no way you could do that with such a shitty camera image

  2. nathanww said

    In the words of Arthur C. Clarke, any sufficiently advanced technology is indistinguishable from magic 😛

    The specifics are a little complicated, but basically HackerHat is able to get good results from very low-quality hardware for a couple of reasons. The first is that before identifying an object, the software combines data from about 25 frames. This gets rid of most of the “noise” from the camera. If there’s any noise that can’t be removed, chances are it was also present when the device was trained to recognize the object, so it’s effectivley irrelevant

    HackerHat also uses a very robust “texture recognition” system. Instead of trying to identify an object’s 3D shape and match it to a databse, the HackerHat examines an area of an object and produces a unique “fingerprint” of it based on qualities like hue and roughness. This allows the hat to quickly and accuratley recognize objects even with a very low resolution input.

  3. […] we al refocused on building a new robot that really sucked balls. But when I started working on the hackerhat, the problem took on a new relevance. It’s hard to have an augmented reality system work if […]

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: