Augmented Reality

Development

UAF-BBC Campus Facade

This image represents my starting place in working with Aurasma.  I very much wanted the building itself to trigger an overlay.  I wasted a bit of time with that and came away disappointed. I tried as well to drop some coordinates in the middle of that concrete in front of the door, alas.

However, working with Aurasma is the assignment.  So I turned to what I could get it to do.   On the phone app, I noticed a couple featured auras, one for the back of a dollar bill and the other the back of a twenty. Opened my wallet and extracted one of each and fired up the camera in the app.  Although, I found the content to be goofy both worked. Lesson learned, the triggers needed to be quickly recognizable by the application so not overly complicated.

Casting about my office, I do a fair bit of marketing here at UAF-BBC, I landed on the tri-fold brochures.  I am not a big fan of this promotion format, and I wondered if auras might make them more fun? In part landing on the Sustainable Energy brochures was because I know we have a fair bit of content that works as overlays. Since my initial foray into using Aurasma had been frustrating and ambiguous, I hoped this might give me some success to build on.

Link to Apple Store for Aurasma app download.

Link to Google Play for Android Aurasma app download.

follow rdheath on Aurasma

Sustainable Energy Brochure Aurasma Version

Sustainable Energy 1

I used my phone camera to capture images from a printed version of the brochure, actually every image because I didn’t know where this was going. My thought was that that camera was going to have to recognize the trigger and perhaps starting with it might streamline things. I cropped and made some minor image adjustments in Adobe Photoshop.  I then dropped this picture of the photovoltaic panel installation on the trigger element of Aurasma studio. I grabbed the interview video and compressed it in Camtasia and cut that into the overlay.  I felt flummoxed that it worked.

Sustainable Energy 2

My thought with adding these elements to the print brochures was to enrich them.  In my years of making flyers and brochures, I have always encountered the problem of too much content, not enough space. That is compounded now with the lack of interactivity or media one has to read them simply, and nobody reads anymore.

I had no content for the Yup’ik values at least no media.  I hit the Alaska Native Knowledge Network website and spent a moderate amount of time spinning my wheels looking for something, anything.  In the end, I decided to experiment with a simple image.  The first one I made the font is too small, and so I’ll make another with two columns of text so that the values are legible on the phone.

Yup’ik Cultural Values

Tom shaking hands with White House Press Secrtary

For us, here at UAF-BBC and Dillingham, Alaska, Dr. Marsik’s world record and President Obama’s visit are points of pride, stories that cannot be overlooked.  The actual video of the world record blower door test is nearly 11 minutes long and simply too much for this application. So, I had to edit it.  In truth were this a real work assignment I would consider crafting something altogether new for this use.  But for this assignment, revised version will have to stand as a placeholder.  I again dumped the mp4 into Camtasia knowing that I was cropping it and compressing it.  What surprised me in testing once the overlay was added, was the strange aspect ratio change that made the house seem weirdly shaped. I will edit out that opening sequence and only start with Tom and Kristen talking for these purposes that is adequate.

And that is a fundamental element of developing auras, I think, it has to be an iterative task with lots of testing. Yes, I could have watched all the Aurasma tutorial videos to learn how to do it.  Indeed, I did look at a couple, and unfortunately, I found them too cheerful and free from the struggles I was encountering.  So, instead, I just muddled and satisficed through. Seeing the output of fellow students has inspired me to continue experimenting with the studio mostly out of nerdy curiosity not out of any sense that this app has a significant place in our marketing efforts.

I have an iPhone but were I trying to develop even this type of project I would test early and often on different devices. Too situating the trigger and sizing the overlay to fit a phone display even close to optimally takes iteration.  It seems that the Aurasma server updates on a quarter or half-hour and that means waiting impatiently after every change.  It meant as well limiting the number of changes so that I could isolate variables.

I think locking the overlay so that once it is triggered it runs no matter what a person does with the trigger is a critical step.  Handheld brochure and handheld camera made for really annoying user experience because the app would lose track of the trigger, find it, and then restart the overlay, again, and again.

I found myself switching between several different programs, Camtasia, Movie Maker, and Adobe Photoshop for example to do this work.  If I was less facile with software, I suspect this project could be challenging. Despite the clunkiness of the Aurasma studio and app, I find myself intrigued with how to make better overlays. I am wondering as well if PowerPoint might have some functions that could simply contribute to overlay development.

Reflection

With this experience, I am struck by what seems the heavy landing on the side of “push marketing” that Aurasma appears to enforce. “Pull marketing” is more about conversation and co-creation of experiences and a product or service.  Social media and our topic of digital storytelling have a play in making this definition meaningful. This ambivalence is a concern I was not expecting to surface at the outset of this assignment. The part of me that is paid to tell the UAF-BBC story is reconciled in some ways to the need to push our story.

If I were still working an academic library, I would try to develop this as a self-guided tour of both services and resources in the library.  Librarians are fond of making scavenger hunts as training devices.  I think I would avoid that conceit and instead simply have point-of-service types of improved interface. While this class is about digital storytelling, I’m afraid we can take that too far particularly when we are approaching customers.

I have been kind of aggressive in my ignoring QR codes; however, the frustrations with Aurasma have inclined me to reset.  I wonder if combining QR and Aura’s might be an interesting approach.  I showed this work in progress to Dr. Marsik, he recounted giving Dillingham High School students a tour and watching them recognize and use QR codes that he was oblivious to because he didn’t know what they were. I was questioning the payback of this kind of development for a community like Dillingham.  However, hearing Dr. Marsik’s observation leads me to wonder if young people might quickly pick up on using the Aruasma app. Nonetheless, I think I would add a statement to the brochure about downloading the app and using the triggers to learn more about our program rather than assuming the customer recognized the logo.

I think this functionality that adds information (to buildings or skylines, cars, whatever real-world objects) is where I want this technology to go. In reading for this assignment the more recent articles identify the linkages between the internet of things and augmented reality as a critical moment. I particularly resonate with this scenario:

This technology could be used by emergency responders. “Moreover, those same first responders might plug certain variables into an incident as it is unfolding to ‘see’ the prediction of what will happen. They could visualize where the crowds will go, how the flood will expand, where the fire might move and which people and/or facilities would be impacted,” DeLoach says. The technology could also enable first responders to practice how they respond to challenging situations such as interacting with hazardous materials. “This would allow them to manage their response much more effectively as a result—likely saving lives,” DeLoach says. (Buntz, 2016)

I can imagine a contact lens, for example, worn by first-responders for heads-up and hands-free application of such simulation and scenario planning. While some of the information would be trended from databases real-time information might be collected from drones. At the end that is all very game-play and moviesque and probably likely and valuable.

Since some of our aim here is to reinvent ourselves as Instructional Designers, I wondered about real uses of Aurasma style augmented reality in schools. A quick search of YouTube, of course, yielded results.

And, I am left feeling like this is a pivotal moment, and both teachers and students contributed to it.  And yet I am not fulfilled.  I worry that this video shows just more “push education.” And that seems to be the limit of this technology at the moment.  I can push a story, you can push a story, but it is a struggle to pull stories, to co-create them, to have them organically branch and build from data, our interactions, and the narrative.

Above, I mention the role of the internet of things for informing augmented reality. The story the first-responders receive is a push story.  We see the purposiveness of storytelling in the initial inquiry, “Where will the flood spread? How best to respond?” It is probably a splendid and useful story, rich with information and prioritized and organized by artificial intelligence. And yet I am uncertain where the human agent comes into play as participant and co-creator of the story rather than a cyborg embodiment of the artificial intelligence. I do not feel paranoia or cynicism but rather a disappointment in this conclusion. Alas, at the point of execution the purposiveness of the story seems to have drained away.  If our robotics were sufficient, we could do away with the human first responder altogether.

But, I still want to hold my phone up, here in Dillingham, Alaska, (or insert a contact lens) and scan the panorama for hotspots. I want native place names, historical highlights, information about plants and animals as I look at a moose, or spruce. I want it to be Wikipedia-like so that I can participate in content creation as well. The language, other mnemonic devices, and our imaginations have been our augmented reality for eons. Pictured below are carved maps of the Greenland coastline serving as triggers for our Inuit hunters cultural overlay.
carved maps

 

Perhaps this technology-heavy augmented reality can mature into a real thing.  Certainly, the quality by which it seems to externalize and give body to imagination is fascinating. And yet, what we are exploring now seems thin and raw and underdeveloped.

References

Buntz, B. (2016, July 1) 10 Killer Applications of the IoT and Augmented Reality. Retrieved March 25, 2017 from http://www.ioti.com/iot-trends-and-analysis/10-killer-applications-iot-and-augmented-reality

Charles Cooper [Charles Cooper] (2016, Nov 7). Teaching with Aurasma. [Video File]. Retrieved from https://youtu.be/wolsdGbNEC

Inuit Cartography. (2016). [Graph illustration]. The Decolonial Atlas. Retrieved from https://decolonialatlas.wordpress.com/2016/04/12/inuit-cartography/

9 thoughts on “Augmented Reality”

    1. Skip, I can get the Auras to launch from the image trigger here on the blog site. I can likewise get them to launch from a paper version of the brochure. Within the application, on my phone, I have never been able to select an Aura and have it launch, yours, mine, any of them. It seemed intuitive that if I selected a something on the app that I would be able to view it but no. Indeed, I have been pretty frustrated with Aurasma and the phone app. Not sure if that helps.

  1. Robert,
    I think using Aurasma to spice up a brochure was a really smart idea. I agree with you that many brochures have a lot of content. I think too much writing pushes people away and nobody reads all that material. I could see this being used more when there are job fairs. I think this would be a neat way to incorporate more information rather than bulking a brochure up with words. Side note: I couldn’t get two of your auras to work. The Yupik one just had a small white rectangle but didn’t do anything and the BBC Building one didn’t have anything at all.

  2. You’ve covered so many aspects of AR as a technology and as a teaching tool that I fear that my response will seem disjointed. I’ll start with the technology aspect.

    I was able to acquire all of your trigger images except the first one. I suspect this speaks to the inherent difficulties in accessing AR content from a monitor. Glare, color balance, moire patterns, and other elements can vary from viewer to viewer and what might be accessible for one may not be accessible for another. AR was never designed to be delivered this way, or course. If you’re already in a digital environment, you would probably use endemic methods—e.g., hyperlinks—to deliver additional content. (“Click here…”) It’s what we have available for us for this assignment, but it’s not the ideal medium.

    It also occurred to me that the aspect ratio of the image may present a problem. Because it is so long, it is surrounded by a lot of undifferentiated black screen space, and AR images do not work well with undifferentiated space. They require complexity.

    ”I think locking the overlay so that once it is triggered it runs no matter what a person does with the trigger is a critical step.”

    I was pleased to see that you picked up on this element of aura implementation. It’s not always practical or easy to maintain camera focus on an object, and it can be frustrating to have to acquire the image again if it drops. This is particularly true with videos, and your superinsulated building video is a perfect example. In the case that you might be holding a printed brochure in one hand and a smartphone in the other, it’s not likely that you’d be able to maintain focus for this (very interesting!) video. Locking it makes viewing the video simple.

    ”And that is a fundamental element of developing auras, I think, it has to be an iterative task with lots of testing.”

    Truer words were never spoken. While I think Aurasma offers the most accessible method for creating AR content (at least as of this writing) it’s still an environment that not only encourages but really requires experimentation. People who are familiar with modular coding have an advantage when actions start triggering other actions and areas get more complex, but you’re entirely correct that testing and retesting is necessary.

    Development

    ”My thought with adding these elements to the print brochures was to enrich them.  In my years of making flyers and brochures, I have always encountered the problem of too much content, not enough space. That is compounded now with the lack of interactivity or media one has to read them simply, and nobody reads anymore.”

    I think you hit the proverbial nail here in terms of the essence of AR content. How do we deliver rich content in a compact package? My first thoughts about AR (back in the day…) had to do with business cards. In the traditional sense, they can either be works of art with little meaningful content or they can be so crammed with verbiage an imagery that they are frustrating. I had a business card printed with a QR code that let people automatically create a contact book entry, but even that seemed limited to me. When AR came into being, I immediately applied it to my (now defunct) business card, which you’ve seen on the assignment page. That was several years ago, and I suspect we’re not much further along with AR accessibility than we were then. You’d still need the Aurasma app to view it. But the idea seems immensely appealing to me. If we had standards for creating, deploying, and viewing AR content, then smart business cards and smart brochures could be rich sources of information. What intrigues me most about AR is that the content behind a trigger image can be changed at any time. One image can serve many purposes.

    Reflection

    ”I wonder if combining QR and Aura’s might be an interesting approach.”

    Conceptually they are quite similar and it’s easy to image an educational or instructional environment rich in both of them. In fact, I worked with a 1:1 fifth grade class in which students used both for a variety of purposes. In the case of both AR and QR, they were consumers as well as creators of the “hidden” content, and I think the fact that they could create content for their own purposes enhanced their understanding and use of both mediums. A technology is truly integrated when it becomes invisible, and that was the case in this particular classroom. Because they all had access to iPads whenever they needed it, the use of AR and QR was transparent to them.

    And that speaks to your concerns about “push education.” Using any technology to simply replicate what you are already doing is not a particularly effective use of that technology. A worksheet filled out with a stylus on an iPad is no different than doing the same worksheet with a pencil, except that it’s probably easier on the teacher to collect and evaluate the work. The students probably see little difference in the two. If all we’re doing in education is pushing content, then we may as well let machines do it.

    ”I want native place names, historical highlights, information about plants and animals as I look at a moose, or spruce. I want it to be Wikipedia-like so that I can participate in content creation as well. The language, other mnemonic devices, and our imaginations have been our augmented reality for eons.”

    We’re on the same page, and to some extent we’re getting there. I had an app on my iPhone that identified trees by using the camera to examine the leaves, and there are apps out there that used location-based data to deliver information about objects in the environment. None of these is perfect—some are far from it—but they point in a direction that is potentially powerful and game-changing. I have an app called Peak Scanner that uses AR to identify mountains viewed through my iPhone’s camera, and it works fairly reliably, giving me not only the name of the peak but also its height and its distance from me. I’m hoping for a convergence of standards and hardware that will do just what you say—give us more information about the world around us.

    ”And yet, what we are exploring now seems thin and raw and underdeveloped.”

    Yep.

  3. I was able to get all of your trigger images to work except for the first and last one, so that’s actually probably pretty good 🙂 I liked that you supplied the links to the apps- that makes it one step easier for your reader, which is important when the technology is a little more complicated as it is with auras. I definitely agree with you that testing is an important part of the process of developing your auras, and it becomes a reiterative process. When I was developing mine, I kept having to borrow my husband’s Android phone (I have an iPhone) since that helped me diagnose if my problems were coming from my phone or from my Aura. I also felt like I did a lot of muddling through and I agree that the tutorial videos didn’t address the real issues that popped up as I worked on mine. Obviously I came to a lot of the same conclusions about the underwhelming nature of the Aurasma app as it is now, but the potential (perhaps) for the future. Thanks for an interesting post!

  4. Hi Bob,

    First off, nice job on your project. I enjoyed reading it. I thought it was funny you mentioned how much librarians love scavenger hunts. Once I learned how to use Aurasma and put it into practice, I couldn’t wait to show my school librarian and brainstorm ways in which she could use AR in the library. We thought about using Aurasma to do book reviews. Example: Hover a device over a book and watch someone give their review.

    Also, I’m curious to know how you were able to upload such a long video (the one about sustainable energy). I couldn’t figure out how to upload anything larger than a few megabites. Am I missing something?

    You wrote, “However, hearing Dr. Marsik’s observation leads me to wonder if young people might quickly pick up on using the Aruasma app.”

    I honestly don’t think young people will pick up on using Aurasma until the app developer makes it easier to use. Example: There should be an option in settings that allows access to my trigger without a person following me. All you would need is the app. One less step might lead more people to use this platform. Why do you think Aurasma requires people to follow others? Marketing? Security?

    1. “There should be an option in settings that allows access to my trigger without a person following me. All you would need is the app. One less step might lead more people to use this platform. Why do you think Aurasma requires people to follow others? Marketing? Security?”

      Marketing and money.

      This is one of the major impediments to a more widespread use of AR, in my opinion. It underscores the lack of standards for development and consumption of AR content. You need a specific app, and a developer’s content will not work with anything other than that specific app. Aurasa does actually make it possible to create a “superaura” that works with the app without the need to follow a specific creator, but it’s a very expensive option that only organizations with large advertising budgets can afford to implement. (The HP logo on the examples page for this assignment is an example of this kind of superaura–no need to follow anyone, just run Aurasma and acquire the image.

      I’m with you. Until there is a standard or set of content compatibility rules that allow easier access to AR content, it will be a tool of organizations with funds to implement AR.

      I am hopeful, though, that Apple’s intent to include AR in future hardware and OSs will drive standardization. And, it’s still pretty cool to have a high school yearbook (for example) complete with AR content even if there is only one way to access that content and it requires an app and a subscription. Years after you graduate, wouldn’t it be nice to be able to point to an image of your high school band and see (and hear) a video?

  5. Bob, you hit on so many frustrations I have experienced with Aurasma myself! In addition to the points you made, the biggest problem I see with Aurasma though is that they limit the possibilities by concentrating only on triggers within an image. Essentially it only leaves one way of thinking of augmented reality and that is similar to cross-linking of the content on the web. It is not really supplementing the “reality” but rather adding more information than space allows. One of the things that got me excited about AR was reading Pattern Recognition by William Gibson years ago where one of the characters is an artist using the AR to create digital sculptures that reveal themselves to people in certain spaces. I thought that was amazing. Imagine the possibilities! And I know it is silly to talk about this kind of experience in regards to Aurasma, which is just a rudimentary app in many ways. And there are many other apps out there that are concentrating on more integrated with “reality” approach (Skip mentioned some of them). But that is kind of my frustration with Aurasma — I think this application is actually doing more harm for the further development of AR than good precisely because it mimics the internet… essentially it only allows you to still think of AR in sort of “linear” and very controlled way: point, click, get something else. It does not allow you to create “an experience” but rather a curated agglomeration of resources (digital on top of digital). I would love to see GPS-based triggers or a color hex trigger. That would add infinitely more possibilities to integrate it with the physical world.

  6. As an educator, I struggle with Aurasma as a viable teaching tool. While I could see it potentially working for content delivery in a flipped classroom wherein a student viewed aura triggers to complete their before-class learning, frankly for standards-based content delivery it feels a bit gimmicky and like its technology for the sake of technology. I think you hit the nail on the head with your library example that it would, however, work for exploratory or informational purposes. I struggle with its use in a content-based course because, when teaching history in Texas, my students were held accountable to the highest levels of Bloom’s Taxonomy via the standards and I have yet to see a good example of higher level thinking enacted in the Aura itself. Could a good teacher engage the students in higher level thinking after they view the Aurasma? Absolutely…but the affordable of the technology require that additional interaction. Which, I would argue, somewhat defeats the purpose…

    Although I am not actually in a marketing position, so take my comments and observations with a grain of salt, I wonder if you could engage viewers based on the content provided? For instance, if your Aura contained a video but then linked to a discussion board? Or invited the user to add to the story or experience in some other way? I chose to do travel blogging for this project, wherein I tell a personal story about my experience at a restaurant and use the aura to supplement with links to Yelp, Facebook, Google Maps, and a video of a local (me) providing tips for visiting. I hoped, with the inclusion of the story, to encourage viewers to create their own as they visit some of my favorite places. While it might just make them hungry to look at the pictures, I hope it invites them to consider food and the experience of dining in new ways…

Leave a Reply

Your email address will not be published. Required fields are marked *