NSC Map with Marker Navigation

I now have the principles learned from my Earth demo applied to the map from my earlier map prototype - and am now very close to finishing my final prototype as a result of this. Check out the video below to see the AR Map navigation system I've created in action:

 

 

As you can see, it works quite nicely, but it is still not perfect. It's still quite responsive to lighting conditions and needs to be held really steady. I have three steps now to complete the project:

• Holding a focus group (which will be done with Christian Cornwell as a joint thing to show both of our NSC prototypes) to get some feedback on the prototype.
• Add content to the information boxes.
• Make final changes and revisions based on the feedback at the focus group.

So, I guess we will look forward to finding out what people think!

Spinning Earth Demo - using Markers as Buttons

I have finally made the major breakthrough of successfully implementing a marker which acts as a button into an AR application. Check out the video below to see it in action:

After looking at many tutorials, I finally managed to get this to work using FLARManager 0.6 (after being unable to get the examples to compile with versions 0.7 or 1.0). My demo closely follows the code in the Collada Loading example included with the library, except that I use only two models, one of which (the FLAR Manager) has the earth model registered to it when it is added, and the other (the HIRO Marker) acting as a button which when removed (by covering up with the finger, or 'pressing') causes the earth model to rotate.

So, with this bit of success under my belt, next step is to start applying these ideas to something more related to my originaly NSC prototypes, and see what I can come up with!

Panoramic Navigator at the ZKM in Karlsruhe, Germany

Last month I visited a really good museum called the Zentrum für Kunst und Medientechnologie (ZKM) in Karlsruhe, Germany.

The ZKM is quality, a really interesting place to visit should you ever happen to be anywhere near it!

PANORAMIC NAVIGATOR:

 

One thing in particular though that I saw there was the 'Panoramic Navigator'. Built in the late 90's, it's an interesting installation which uses an AR based system to navigate around the main hall of the ZKM, and display relevant information on different areas to users. It's quite a simple setup and idea really:

• The user holds onto a big metal pole, which has a touch screen attached.
• By spinning the pole around, the user also spins a camera attached to the top of the pole.
• Above the pole is also a panoramic view of the ZKM's main hall, which by spinning the pole the user directs the camera to look at different areas of.
• These images are relayed from the camera to the screen, on which certain areas form the image are able to be touched to reveal information about that particular area of the museum.

The pictures below should help to explain this a little more clearly:

 

 

It's quite different from what I'm envisaging for my AR navigation system for the NSC, but one thing that's definitely worth noting is that even way back in 1997, this Panoramic Navigator was using a touch screen interface. This really sends the message home to me that I need to plough forward with my plan of using markers printed in the guidebook as buttons (which could be thought of as their own 'paper' touch screen interface), because for me, 13 years on, the idea of using a mouse and keyboard as input devices really should be obsolete. So let's hope I can pull off the 'markers as buttons' idea!

Custom Marker Experiments

Thought I'd share a few experiments I've been doing with custom marker designs for FLARToolkit. Recently I designed three marker designs, with various levels of complexity:

Rocket Marker (very simple design)

NSC Logo Marker (more complex design)

Moon Marker (less complex, but with variations in tone)

The marker designs were made in Photoshop, and turned into PAT files for actionscript linkage using the Online Marker Generator on the Taro Taro Flash Blog:

flash.tarotaro.org/blog

Check out my video below to see how I fared with the testing of my three custom markers with my earlier Planet Test prototype:

As you can see, mixed results! While the Rocket marker works well, I think with the NSC Logo marker the problem is that it is within another black square, and so the program can't figure out which way is up/down/left/right very easily - the actual design within the square probably just bleeds into a symmetrical lump of white anyway, it is too complex. As for the Moon marker, my guess is that the problem here is because of the shading of the moon craters - I suspect that the markers only work properly with solid blocks of colour.

So plenty for me to work on - I still want to create my own custom designs, and although I think that AR in Flash can probably only handle solid blocks of colour, what's to say those colours have to be black and white... watch this space (no pun intended)!

Research - Using Markers as Buttons

Following on from receiving my brief from Neil at the Space Centre, about using AR markers embedded in the NSC guidebook, I've been really keen on the idea of using multiple markers, but having them work as buttons - so that actions happen when they are covered up, rather than simply when they are seen by the camera (which is how basic AR apps normally work).

The idea is that I could have multiple markers printed on the pages of the NSC guidebook, one of these would have virtual content registered to it, and the other markers would act as buttons, controlling the content by performing various functions when the user covers them up with their finger. So I've been looking at a couple of examples I've found which use this sort of idea:

FLASH AUGMENTED REALITY DEMO WITH MULTIPLE MARKERS:

This is an example I found on YouTube, from a user called MetzSolo:

http://www.youtube.com/watch?v=HfaGih85ZJgn

It's quite a simple application in essence, but I think it works really well as a demo of this sort of control using markers as buttons, and it also proves that this kind of thing can be done by anyone using Flash, and isn't just reserved for the pros at the big companies with their custom-built software packages!

NISSAN JUKE AUGMENTED REALITY:

Created by Total Immersion - exactly the kind of big company I previously referred to! This example is being used at the moment as a marketing tool for the Nissan Juke car, and has become quite a big hit on the web from what I can gather. It's obviously much more polished, but if you think about how it works, the basic principles are exactly the same as MetzSolo's effort, and probably work in much the same way.

Still, it's fun to play around with. Although you probably can't tell from the video above how much I'm enjoying it!

Now I guess it's high-time for me to get cracking on my own interpretation!

Map Prototype and Brief from the NSC

After the showcase Neil Shorrock, Kevin Yates and Darren Clegg from the National Space Centre were invited to IOCT and Fused Media labs at DMU to see the prototypes of myself and another student collaborating with them for his Major Project, Christian Cornwell - check out the blog for his NSC Moonwalk application:

Christian Cornwell's NSC Moonwalk blog

As well as the Planets Test demo, I showed them another simple AR exhibit demo I'd made.

MAP TEST (Second Prototype):

Map Test is my second prototype, developed using the basic principles from Planets Test, but applied to an example relevant to the NSC - a 3D model I created as a replica of the printed map of the museum which is featured in their guidebook. Have a look at me using Map Test in the video below:

ORIGINAL NSC MAP IN THEIR CURRENT GUIDEBOOK

PROJECT BRIEF (provided by the NSC):

After the demos, Neil agreed to co-supervise my project with Prof. Mohammad Ibrahim from De Montfort, and sent me via email a brief for what he hopes I can provide them as part of my Major Project, which is summarised here:

To develop an interactive system that will enable visitors with guide books to access additional content and information, both video and audio, from terminals located within the exhibition.

The current guide book includes sections on the different galleries within the centre and a map for overall orientation.

The aim is to make the guide book an essential part of the visit by encouraging the visitor to use it throughout their visit, thus enhancing their experience.

It would be desirable if the interactive system could also act as a guide, so that visitors could locate their current position and receive advice on where to go next.

So, to summarise even more plainly, the NSC want me to develop an AR application which uses markers embedded in their guidebook, which can then be used around the museum to interact with virtual content using webcam stations as a means of discovering more about the various exhibits. There's a few things that Neil is keen for the application to make use of:

• A unique marker symbol, embedded within the pages of the guidebook.
• Different types of A/V content making up the virtual content of the AR application - Neil has told me I am free to push the boat out in this respect!

These can be coupled with some of the things I am keen for my application to make use of:

• Differing levels of complexity for markers, including colour incorporated into designs.
• Use of multiple markers, each with different content / functions.
• Novel display of content - something more than just 3D models or text - content which can be truly interacted with.
• Novel method of interaction - something slightly more than just making use of the mouse or keyboard.

So, with all these points to consider, I think it's pretty safe to say we're in line for (hopefully) a pretty interesting outcome!

Project Proposal and Planet Prototype

Two of the first things I did concerning my project were to complete the Major Project Proposal, and then after confirming the collaboration with the National Space Centre I started to develop an initial prototype, just to test the waters in terms of basic AR techniques and features.

MAJOR PROJECT PROPOSAL:

In my proposal, as well as defining my project as an MSc and writing about my early research, I outlined my four key aims for the project:

• To research different methods for Augmented Reality development and different display techniques.
• To find out how successful exhibits are put together and what criteria they need to meet in order to deliver to their audiences.
• To evaluate how Augmented Reality applications can be used to meet these criteria in exhibition environments.
• To develop and produce, for the NSC in Leicester, an Augmented Reality exhibit which will provide a novel and engaging interactive experience for its users.

PLANETS TEST (First Prototype):

After completing my proposal, I set about creating the first of my prototypes for the project - the Planets Test. The application, made in Flash using the FLARToolkit library, features three spheres rotating around a central sphere, all registered to one paper marker, and tests out some early ideas for interactivity such as mouse rollovers, mouse clicks, and keyboard input.

In the video here, after putting down the marker, I first click on the middle ball, the information ball, to bring up a window with instructions on how to use the application. Note there is also an information symbol when I rollover this middle ball with the mouse. I then rollover the other three balls, to display their numbers (1 to 3) and then I use the keyboard input, first holding down the 1 key and using the up or down arrows to increase and then decrease ball number 1's speed, then doing the same with ball number 2 by using the up and down keys whilew also holding down the 2 key, and finally doing the same with ball 3. After this I pick up the marker to show the registration.

IOCT SHOWCASE:

One cool thing I got to do at this point was to test my prototype out on the unsuspecting public, as part of the IOCT showcase at DMU on the 6th and 7th of May 2010. I think people were generally quite taken with the idea, and most people who spoke to me could see a clear potential for AR exhibits, which is obviously good news for my project. I did decided though from the results and comments at the showcase that maybe using keyboard input removes the ability to interact by moving the marker around, as it is difficult to do both at the same time. Anyway, here are some photos, taken by Stuart Hollis...