What would you want in a personal AR HUD?, A discussion on near future technology and real life Options

[Raven the Tricks...]

New technology is advancing fast these days and between AR games and apps on smartphones now and Google's Project Glass on the horizon our worldview is about to change dramatically. So here's my question, what do you think you would really want in an Augmented Reality Heads Up Display?

In video games the use of a HUD is obvious, it lets you see what you have equipped, how much ammo your gun has, and where exactly you're shooting. Translated to the real world though the ideas become a little blurier. What do you think you would actually want clutting up your view of the world?

For myself there are a few key things that I would love to have constantly displayed in some sort of HUD:

- Time and date, up in the periphery of my vision, where it doesn't get in the way but I can easily refer to it as I want
- Message indicators for things like SMS, email and instant messaging, with the ability to check those with an easy interface
- Some sort of way to easily identify people who you've registered a profile on your system, possibly including both an easy way to notice them from a distance as well as a way to remember their name if you've just met them

Really, that's about it for full time options, and even that last one could be relegated to an App. Which brings me to my next topic, what would you put as icons, maybe selectable with eye tracking or voice commands. For me it would be things like twitter and facebook feeds, a camera app, instant messaging, web browsing, and of course some AR games and a GPS map.

The next thing this considers is how would you intereact with these icons. I've already mentioned laser eye tracking, which is a technology already in early use in the helmets for the latest fighter planes. Another option would be finger tracking gloves or rings, possibly with some degree of force feedback, and of course, there's always voice entry, either text to speech or direct recording.

Finally there's the look, should this all have a unified theme like you would see in some sort of anime or video game? Or would you expect to see unique looks to each individual app? Maybe some sort of middle ground is in order, a universal standard for skinning all of these AR Apps?

These are all things to start thinking about now, because the future is coming faster than you might think. A realistic look at how far we've come in the last 10 years will tell you that Augmented Reality will be a full blown cultural phenomen like the smartphones of today, likely within 5-15 years at the most.

Maybe the more artistically inclined amongst us would be interested in giving an example of what the future might hold. I for one would be eager to see it.

[Shortstraw]

Diablo I/II style map not a mini map.

[ggodo]

Ooh, Yeah, I loved the Diablo map, but I'd like to have the option for minimaps as well. My sense of direction is pretty sweet, but I wouldn't want to have to depend entirely on it, and the Diablo map could get seriously distracting. I'd also like to be able to change HUDs fordifferent activities. Maybe it could be an app, but I see swapping HUD from work to travel to home being very helpful. I know there are features desktops today I find very useful in one situation and annoying in most. Being able to hot swap HUDs would be great.

[Umidori]

In-store directions to find various foodstuffs in supermarkets.

~Umi

[Sir_Psycho]

Some sort of artificial x-ray vision. Not to actually see through walls, but let's say you're across the street from that restaurant you like, but it looks pretty full. You activate the glasses and it highlights all tables so you can see if there is one available. Using a GPS map, you can see the road you need to take highlighted all the way to your destination.

A "scrawl" system where you can write on anything with a finger, pen cap, foot etc. A friend who's subscribed to your scrawlings will see "Good Coffee" written on the ground outside a cafe and know you recommend it. Visual text-to-text recognition/translation would be great too. You could write some-one a letter, then have the body of text appear on your glasses or other device for editing and sending as email.

But ultimately, I can't commit to the idea. I already wear glasses. Prescriptions are expensive, double it for an extra pair of prescription sunglasses. Finding the right frames is difficult and expensive, call it vanity, but it's an alteration to your appearance that is almost always there, so you have to make sure it's the right look. Then these AR-frames would have to have prescription lenses, and I have no idea how tracking etc. would interact with things like astigmatism.

Also, we've seen where google/youtube has gone with interruptive advertising, and I don't much like the idea of giving them free range of my sightlines.

[ShadowDragon8685]

-GPS. At the very least coordinate display, direction of travel and speed, but also preferably a full GPS unit's functionalities.
-The ability to 'subscribe' to my car and put information from it (especially information the car's instrument cluster doesn't give me) in my view: speed (obviously), revs, gear, drive method (my truck has part time 4wd,), and for the love of god, a textual read-out of any and all problems that it throws a code to.
-Battery and other indicators for all of my personal devices, assuming they don't roll them up into the glasses.
-The ability to track my eyes and take a picture of whatever I'm looking at without me doing anything obtrusive to take the snap or making it obvious that a picture has been taken.
-LIGHTS! Okay, technically there's no reason these need to be part of Augmented Reality glasses, but if I'm gonna be wearing these things all the goddamned time, it would be nice to have some headlights to go with my heads-up display.
-Headphones while we're at it. I already wear a pair of really heavy, and really awesome, Beyerdynamics, 24/7, because they go all around the ears very comfortably, as opposed to earbuds or the ones that rest on the ear and wind up hurting after a while. They should have a way to give me audio without hurting my ears, and if I look like Daft Punk as a result, so much the better.

[Dakka Dakka]

In-store directions to find various foodstuffs in supermarkets.

Not only only foodstuffs in supermarkets. I would like that for clothing stores as well, since they tend to arrange their wares by brand and not by item type.

[Shortstraw]

Perhaps something that lets you add a shopping list to your map?

[Wakshaani]

Single/dating/married status of people I glance at, along with a follow-up to a more detailed personals area, so I can see if they're interesting or not.

[SpellBinder]

Single/dating/married status of people I glance at, along with a follow-up to a more detailed personals area, so I can see if they're interesting or not.

Not things I'd want myself, but... Tie that in with what I keep expecting would be the first thing to be included, and you'll have an easier time with this: real time FaceBook and Twitter access.

Personally I like the map options mentioned, along with GPS and other navigational aids. Maybe with a FLIR add-on, too.

[CanRay]

Time/Date, IM, and a system to let people know to LEAVE ME THE HELL ALONE WHEN I'M WALKING on their AR.

I'm usually working on ideas when I walk, and I hate being interrupted. Really should get an MP3 Player someday.

[Neko Asakami]

Oh my God, the games. Games, games, games. I want emulators, new stuff, old stuff, everything. I'd also want the ability to customize my appearance in AR, things like clothing overlays (like a t-shirt that could display any image in AR), avatar items, wings, horns, tails, ears, etc. Virtual pets too. Not things like Tamagotchis, but actual real pets and things that could interact with others too. Animated agents would be cool, as long as they weren't like Clippy.

[Wakshaani]

I'll never understand that hatred that Clippy generates.

[Shortstraw]

I have an A.I based on Clippy. "it looks like you're dying can I help?"

[_Pax._]

- Navigation system
--- GPS, of course
--- Minimap that rotates so that "up" is always "the direction I am facing"
--- in-view arrows and lines, as if painted on the ground, for when I've asked for directions to a particular place
--- all of the above available with a "3D view", especially for the interior of buildings or below-street spaces like subway systems

- Social Networking
--- notify me that someone on my friend list is in the area - and point them out, even through crowds. Maybe offer directions to them.
--- an enemies list, as above?
--- the ability to update my status on social sites (using speech-to-text), tagged with my location and maybe accompanied by a picture taken by the on-board camera
--- perhaps an "audio twitter" service? quick 15- or 20-second sound byte, attached to a still image or short video?

- PAN management
--- generate a HUD for whatever vehicle I'm operating (all the stuff normally on the dashboard, plus rear-view imaging if available). Optionally, also for when I'm a passenger?
--- visual displays for active devices like music players, streaming video from my netbook, etc.

- Real-time information lookup
--- Facial recognition, which pulls up a business card and "important notes" like impending birthdays, etc, for people I know
--- QR code and barcode scanning, linked to assorted online shopping services
--- Architectural-recognition for historical sites or other places-of-interest, to give me little tour-guide blurbs about the things I see
--- Vitals monitor - heart rate, respiration, etc - for when bicycling/hiking/etc
--- Weather conditions for where I'm standing, including forecasts for later in the day
--- Traffic conditions in surrounding area - or along navigated route, if following directions to somewhere
--- Status of my home and my vehicle(s), even when I'm away from them - including access to any security cameras that might be installed in them.
--- Chaperone functions - i.e., the location of any kids I'm responsible for, the ability to "eavesdrop" through THEIR AR glasses, directions to their location, etc.

- General functionality
--- eye-movement tracking to add CONTROL functions to all those displays
--- Thermal, lowlight, and/or ultrasound-mapping overlays to enhance vision in the dark
--- self-darkening lenses to preserve vision in glare (flare compensation, IOW)
--- integrated noise-cancelling earbuds
--- on-board HD capable camera, still and video capabilities, with and without HUD elements displayed

[Ears]

Two things:
- Open APIs that give programmers access to the entire device, not some little app toolbox that's good for weather widgets and nothing else. Preferably in Ruby, but that's just me. (IMG:style_emoticons/default/wink.gif)
- A huge warning to never ever use this device near anything you don't want the entire world to see, hear or know. 'Cause even with the most restrictive vendor lockdowns and best sandboxing, it's impossible to secure this kind of device in any kind of way. You're accepting (visual) data from the entire world, analyse it and then act on it; on a machine with 24/7 internet access. That's not a losing battle, you've already surrendered before the other guy even knew there was a fight.

[ShadowDragon8685]

- A huge warning to never ever use this device near anything you don't want the entire world to see, hear or know. 'Cause even with the most restrictive vendor lockdowns and best sandboxing, it's impossible to secure this kind of device in any kind of way. You're accepting (visual) data from the entire world, analyse it and then act on it; on a machine with 24/7 internet access. That's not a losing battle, you've already surrendered before the other guy even knew there was a fight.

Oh, man, the hacking runs just write themselves.

"Smitty Jones is a noob who keeps his AR Glasses on and fully active when he's cleaning up the offices of Secure McSecurityBudget, Inc. Hack his glasses to record everything he sees and send it back to you on a nightly basis, then send me like, a week's worth of records."

[CanRay]

You mean you're going to trust thing like VENDOR lockdowns?

Anyhow, AR is display, not recording, so you can't see what he's seeing, only see his AR if you hack the glasses. Of course, if he's the type that has to have a memo about passwords and combinations... (IMG:style_emoticons/default/ork.gif)

[Ears]

Well, both from the posts here (including the OP) and existing real-world applications, AR very much includes a camera.

One neat for instance would kind of a price checker. Everytime it identifies a product/price combination (say a price next to a barcode), it uploads that info together with the location and timestamp to a DB (anonymously, LOL). If it sees you picking up a product, it checks that DB and then shows you indicators that tell you whether you can get that same thing cheaper (within walking distance, or on your drive home (maybe with a slight detour), or online, or whether your FB/Google+/Twitter peeps love/hate it, whether it's getting rad/horrible reviews, or whether there's a comparable thing which is significantly better/cheaper/more hip. You 'click' one of those indicators, you get more info. And of course, if your supermaket keeps annoyingly rearranging their stock, as soon as one person using this software has tagged your favourite cereals, this app can guide you to them, no matter which row they're now.

But seeing how things work, I bet the first really popular app will be some kind of Youporn integration; dunno, you keep oggling a woman whose boyfriend lookes like he could beat you to a bloody pulp, it downloads porn that features women looking like her. (IMG:style_emoticons/default/biggrin.gif)

[_Pax._]

Anyhow, AR is display, not recording, so you can't see what he's seeing, only see his AR if you hack the glasses. Of course, if he's the type that has to have a memo about passwords and combinations... (IMG:style_emoticons/default/ork.gif)

AR requires a camera to operate. It looks for objects in the real world - by shape, by pattern, etc - in order to know where to place certain tags / displays / etc. You look at your TV while wearing the glasses; they know you're looking at the TV, so a window pops up with what channel it's on, maybe a reminder of when one of your favorite shows is scheduled to start, and so on.

If you want to see a good AR mock up, watch this video. (I'd figure that for a wageslave, btw). Notice how various objects are being outlined/highlit. Notice too, the "virtual wallpaper" is basically spamtastic with advertisements. That all requires the AR system to have a camera pointed at the world.

[Wakshaani]

To see something in action that's a tad less spamtastic, try StarkHUD.

http://www.youtube.com/watch?v=1_u34kV9go0

[Tymeaus Jalynsfe...]

At least physically, I envision a Conexus Like Device... I think it is cool looking.

[Raven the Tricks...]

I can't believe I forgot to include PAN vehicle integration. I thought of it when I was first thinking of what to write, but I guess over the two days that passed before I actually wrote the post I forgot about it. For cars it would be useful, but I think where it would really shine would be motorcycles, since I've always thought the instrument clusters on those are less in your field of view than car instrument clusters are. In fact, I wouldn't be surprised to see non-interactive HUDs in motorcycle helmets in the next few years, now that the projection technology is becoming available at reasonable costs.

At least physically, I envision a Conexus Like Device... I think it is cool looking.

Link please, my google foo is failing me.

[CanRay]

Actually a vehicle HUD that I could control might allow me to drive at last...

[Tymeaus Jalynsfe...]

Link please, my google foo is failing me.

Appleseed: Ex Machina.
It was cool....