Here’s a Look at the First Wave of Augmented Reality ARKit Apps Hitting the iOS App Store Today

With the launch of iOS 11 today, Apple has turned hundreds of millions of iPhones into augmented reality-capable devices thanks to the support of a new developer framework called ARKit. With this technology, iOS developers can more easily craft AR experiences for users on compatible iPhones and iPads, using each device’s built-in cameras, processors, and motion sensors.

As of now, the first wave of these apps are available for you to download and test on the iOS 11 App Store. The first apps range from game updates to practical everyday tools and even apps that encourage a healthier lifestyle, with more refined experiences likely coming in the future once developers get a grasp on what users enjoy with the first wave of apps.

Note that to use ARKit-enabled apps on iOS 11 you must have an iOS device with an A9, A10, or A11 processor. This means ARKit apps can be launched on iPhone 6s, iPhone 6s Plus, iPhone 7, iPhone 7 Plus, iPhone SE, iPhone 8, iPhone 8 Plus, and the upcoming iPhone X. For iPads, you can use the 9.7-inch iPad or the 10.5-inch and 12.9-inch iPad Pro. The older 9.7-inch model of the iPad Pro is compatible as well.

One you have iOS 11 installed on one of these devices, head over to the new App Store and check out some of the ARKit apps listed below to see how Apple’s new augmented reality technology works in your own home.

Games

Splitter Critters (left) and Egg, Inc. (right)

Splitter Critters ($2.99)

What’s it about? Use swipes of your finger to split a colorful landscape and guide alien critters back to their spaceship, avoiding enemies and solving puzzles in the process.

How’s AR used? Scan a flat surface and then place a fully playable version of the main game into the real world, housed within a small white box.

Warhammer 40,000: Freeblade (Free)

What’s it about? Control an Imperial Knight war machine in the Warhammer 40,000 universe and fight the evil forces of Chaos through 170 single player missions using cannons, missiles, and thermal blasts to defeat your enemies.

How’s AR used? Drop your Imperial Knight from the main game into an AR “Photo Mode” to take snap shots of the war machine in the real world.

Egg, Inc. (Free)

What’s it about? A farming simulation game focused on hatching eggs, building hen houses, hiring drivers, and researching advanced technologies to upgrade your egg farm.

How’s AR used? Take a glimpse at your farm in AR with a “Farm To Table” picture-taking mode.

Thomas & Friends Minis (Free)

What’s it about? Build, decorate, paint, and create full train sets and then control characters from Thomas & Friends as you drive through your customized train set.

How’s AR used? Bring all of your creations into the real world with the app’s AR mode, which places your train set on a flat surface so you can zoom in and around while still being able to interact with various tools and control characters.

Continue reading “Here’s a Look at the First Wave of Augmented Reality ARKit Apps Hitting the iOS App Store Today”

New iPhones may spur a surge in augmented reality

The runaway success of “Pokemon Go” last year taught the world at least two things. One: Lots of people love Pokemon. And two: Creating good augmented reality – the kind that superimposes 3-D objects into the real world and convinces people they’re actually chasing a Pikachu – is really, really hard.


“It’s not so hard that it’s impossible,” said Jeff Kelley, an iOS developer at app design and development firm Detroit Labs. “But it’s hard enough that you’re probably not going to get a return on your investment.”

Previously, if developers wanted to add augmented reality to an app, first they’d have to spend months building their own tools and performing a bunch of math to calculate how a 3-D object should look when light hits it from different angles, and how it interacts with real-world objects, Kelley said.

That high barrier to entry will all but disappear when iOS 11 launches Sept. 19 with AR Kit, a set of developer tools that takes out the hardest part of developing augmented reality experiences for the iPhone.

“As a developer, you don’t have to do all the hard math stuff to get it to work,” Kelley said. “The minimum time investment now goes way down.”

That means there soon could be a surge in the number of apps that feature augmented reality experiences, exposing more people to a technology that was once considered the purview of hardcore geeks.

Despite the enormous popularity of “Pokemon Go” last year, only 31 percent of Americans know what augmented reality is, according to a survey conducted in July by Skrite, a startup that makes a social augmented reality app.

As with its more immersive cousin, virtual reality, tech companies have for years tried to bring augmented reality to the mainstream, with little success. Google’s infamous Google Glass – a head-mounted display – was a flop that drew criticism over its conspicuous design and potential for privacy violations (the device could be used to record people).

Startups at the annual Consumer Electronics Show in Las Vegas year after year have hawked augmented reality screens that act as virtual dressing rooms, none of which have gained mass market adoption. And furniture stores and interior design firms have long offered tools to let people see what a sofa or coffee table might look like in their home, but these apps have been clunky or difficult to use.

Until “Pokemon Go,” most augmented reality experiences just weren’t very good, Kelley said. Even the most basic of experiences left much to be desired. Kelley recalls working on an app five years ago for wall decoration company Fathead, in which users could point a smartphone at a wall in their house and see how a Fathead wall sticker might look in their home. In order for it to work, though, users first had to print a PDF and stick it to their wall as a physical marker so that the app knew where to superimpose the virtual sticker.

With Apple’s AR Kit leveling the playing field, developers can spend less time worrying about the tech that powers augmented reality, and spend more time focusing on the experiences they want to create, Kelley said, which could ultimately lead to more experimentation and better products.

“The main push is that it’s priming the consumer field and the developer field,” said Gregory Curtin, whose Los Angeles firm CivicConnect works with city and transit agencies to integrate city data with augmented reality.

Curtin’s firm spent three years developing its own augmented reality platform, which can integrate transit schedules and commuter data so when a person opens a transit app and points a phone at a bus stop, the bus schedule appears on the screen.

Although a lower barrier to entry could mean CivicConnect will soon see more competition, Curtin welcomes it, because greater awareness of what augmented reality can do will mean more opportunities for developers in new markets.

Some challenges still will lie ahead, though.

AR Kit can solve the tech component, but many augmented reality experiences require 3-D art. Even Snapchat’s dancing hot dog, silly as it may be, had to be drawn and rendered by someone.

“For a lot of developers, that’s another difficult piece, because developers aren’t always good 3-D artists,” Kelley said.

The other challenge is that while the new iPhone 8 and iPhone X are optimized for augmented reality viewing, many phones – particularly cheaper options with lower-end cameras – aren’t. At a starting price of $699 for the iPhone 8 and $999 for the iPhone X, experiences made for those phones may exclude many potential users.

But it’ll be just a matter of time before the technology is readily available to everyone, developers said. Facebook already offers its own platform, AR Studio, for developers wanting to create augmented reality experiences for the social network, and dozens of third-party platforms such as Vuforia and EasyAR allow developers to create AR experiences across multiple platforms, including iOS and Android.

That Apple is throwing its weight behind AR Kit, with augmented reality-ready phones, is a big deal, developers said.

“I do think this will be a milestone in terms of changing the game,” Curtin said.


Explore further:
With iPhone X, Apple is hoping to augment reality for the everyman

New iPhones could make augmented reality a mainstream reality

The runaway success of “Pokémon Go” last year taught the world at least two things. One: Lots of people love Pokémon. And two: Creating good augmented reality — the kind that superimposes 3-D objects into the real world and convinces people they’re actually chasing a Pikachu — is really, really hard.

“It’s not so hard that it’s impossible,” said Jeff Kelley, an iOS developer at app design and development firm Detroit Labs. “But it’s hard enough that you’re probably not going to get a return on your investment.”

Previously, if a developer wanted to add augmented reality to an app, first they’d have to spend months building their own tools and performing a bunch of math to calculate how a 3-D object should look when light hits it from different angles, and how it interacts with real-world objects, Kelley said.

That high barrier to entry will all but disappear when iOS 11 launches Sept. 19 with AR Kit, a set of developer tools that takes out the hardest part of developing augmented reality experiences for the iPhone.

“As a developer, you don’t have to do all the hard math stuff to get it to work,” Kelley said. “The minimum time investment now goes way down.”

That means there soon could be a surge in the number of apps that feature augmented reality experiences, exposing more people to a technology that was once considered the purview of hardcore geeks.

Despite the enormous popularity of “Pokémon Go” last year, only 31% of Americans know what augmented reality is, according to a survey conducted in July by Skrite, a start-up that makes a social augmented reality app.

Like its more immersive cousin, virtual reality, tech companies have for years tried to bring augmented reality to the mainstream, with little success. Google’s infamous Google Glass — a head-mounted display — was a flop that drew criticism over its conspicuous design and potential for privacy violations (the device could be used to record people). Start-ups at the annual Consumer Electronics Show in Las Vegas year after year have hawked augmented reality screens that act as virtual dressing rooms, none of which have gained mass market adoption. And furniture stores and interior design firms have long offered tools to let people see what a sofa or coffee table might look like in their home, but these apps have been clunky or difficult to use.

Until “Pokémon Go,” most augmented reality experiences just weren’t very good, Kelley said. Even the most basic of experiences left much to be desired. Kelley recalls working on an app five years ago for wall decoration company Fathead, in which users could point a smartphone at a wall in their house and see how a Fathead wall sticker might look in their home. In order for it to work, though, users first had to print a PDF and stick it to their wall as a physical marker so that the app knew where to superimpose the virtual sticker.

With Apple’s AR Kit leveling the playing field, developers can spend less time worrying about the tech that powers augmented reality, and spend more time focusing on the experiences they want to create, Kelley said, which could ultimately lead to more experimentation and better products.

“The main push is that it’s priming the consumer field and the developer field,” said Gregory Curtin, whose Los Angeles firm CivicConnect works with city and transit agencies to integrate city data with augmented reality.

Curtin’s firm, which counts cities such as Palm Springs, San Diego and Mission Viejo among its clients, spent three years developing its own augmented reality platform, which can integrate transit schedules and commuter data so when a person opens a transit app and points their phone at a bus stop, the bus schedule appears on their screen.

Although a lower barrier to entry could mean CivicConnect will soon see more competition, Curtin welcomes it, because greater awareness of what augmented reality can do will mean more opportunities for developers in new markets.

Some challenges still will lie ahead, though.

AR Kit can solve the tech component, but many augmented reality experiences require 3-D art. Even Snapchat’s dancing hot dog, silly as it may be, had to be drawn and rendered by someone.

“For a lot of developers, that’s another difficult piece, because developers aren’t always good 3-D artists,” Kelley said.

The other challenge is that while the new iPhone 8 and iPhone X are optimized for augmented reality viewing, many phones — particularly cheaper options with lower-end cameras — aren’t. At a starting price of $699 for the iPhone 8 and $999 for the iPhone X, experiences made for those phones may exclude many potential users.

But it’ll be just a matter of time before the technology is readily available to everyone, developers said. Facebook already offers its own platform, AR Studio, for developers wanting to create augmented reality experiences for the social network, and dozens of third-party platforms such as Vuforia and EasyAR allow developers to create AR experiences across multiple platforms, including iOS and Android.

That Apple is throwing its weight behind AR Kit, with augmented reality-ready phones, is a big deal, developers said.

“I do think this will be a milestone in terms of changing the game,” Curtin said.

tracey.lien@latimes.com

Twitter: @traceylien

Augmented reality: Is Pokemon Go-style technology the future of football?

See how Frickley Athletic show their match highlights… straight from the pages of the programme

Matchday programmes used to be the place to find out the two teams, discover what your left-back’s favourite film was, or see which local companies were offering a 10% discount when you visited with your copy.

Nowadays you can use your paper programme to watch videos… straight from the page.

It might sound like magic, but a forward thinking semi-professional club from West Yorkshire are making it a reality.

Supporters of Frickley Athletic, who play in the eighth tier of English football, can hover their smartphone over a page in the programme to watch match highlights which are often inaccessible for fans of non-league football.

The club’s programme sales have increased since it was introduced at the start of this season, leading to interest from Football League outfits as they explore ways of sustaining a tradition of the game which is coming under increasing pressure from new forms of media.

“I’m a football traditionalist – programmes need to be in football,” says Chris Medwell, co-editor of Frickley’s programme.

“This is a good way of keeping them alive. If league clubs with more money want to use this technology then the options are endless.”

So how is Frickley making this ‘magic’ happen? Can it keep the matchday programme alive? And what else could fans see pop up on the page?

From Pokemon Go to non-league football

Augmented reality. Never heard of it? The chances are you have used it, or know someone who has.

That’s because Pokemon Go – the mobile game that blends the real world with computer graphics played by millions across the world – is the most high-profile example.

Augmented reality (AR) is all about enhancing the view of the real world with computer graphics, allowing users to experience it on a smartphone through an app that uses the camera and sensor to overlay information on to the real-world view.

“Augmented reality is a developing tech which has been around for 10-15 years in different forms, but is now taking more of a consumer approach,” says Liam Foy, head of social at Bolton-based digital marketing company Bring Digital.

“The big Silicon Valley companies, including Facebook, Snapchat and Instagram, all use it to an extent.”

Pokemon Go became a global hit after it was launched in July 2016

Medwell, a special educational needs (SEN) lecturer at Doncaster College, started using the technology to engage his teenage students and improve their learning experience.

“It helped them take ownership of their learning because it is more interactive,” he says. “Because they thought it looked like ‘magic’, they found it really engaging.

“When I was asked to co-edit the Frickley programme, I thought it would lend itself perfectly to football.”

Using free web-based software, Medwell creates a ‘trigger image’ which is printed on to a page in the programme and recognised by a smartphone app to play the video – filmed by his SEN students – on screen.

Frickley supporters simply have to download the free app, pay £2 for their programme and then watch.

How can non-league clubs benefit?

Clubs down the English league pyramid, particularly the semi-professional ones outside of the Football League, face a constant battle to keep the books balanced as they operate on relatively small incomes.

Frickley, like all Evo-Stik League clubs, are required by the league to produce a matchday programme.

“A programme is a labour of love and often a drain on resources,” says Medwell. “They aren’t cheap to put together and take a lot of time for a volunteer.”

Frickley, based in the former mining village of South Elmsall near Wakefield and with an average attendance of about 300, introduced the concept for their opening home match of the season against Cleethorpes and sold out of programmes.

They increased the print run for their next game – and sold out again.

Medwell adds: “This is a different way of attracting the attention of fans and it is really engaging them.”

WATCH: What is augmented reality?

Frickley say the cost-free feature has caught the imagination of fans who – unlike their Premier League and Football League counterparts – do not have instant access to match highlights through websites and social media.

“Everyone in the town is talking about it,” says Michael Johnson, a volunteer who works as Frickley’s media manager.

“The response has been amazing and the interest is taking the club places it has not been before – not for a lack of trying, just time and cost.”

That exposure – boosted by increased sales – has opened up commercial opportunities with local companies expressing interest in putting video adverts in the programme.

“Cost is a massive thing for a club like ours, but we have no parameters on ideas and creative thinking,” adds Johnson.

“Now we have been able to do it through technology and it is proving to be a massive tool, asset and resource for us.”

Player interviews & touch maps – other ways AR could engage fans

Other non-league teams, keen to discover new ways of boosting programme sales and engaging fans, have contacted Frickley to tap into their knowledge, while a Championship club have also been in touch.

While Frickley use AR to show match highlights, this might not be a selling point for fans higher up the pyramid who can watch more established broadcasts, such as Match of the Day, through apps on their mobile phone.

“One of the key areas to getting football fans using it is added some sort of value to the content,” says Foy.

“In the instance of match highlights, this is a great opportunity for lower-league clubs who can’t get highlights on a regular basis.

“It could also be used to show exclusive interviews with players, managers or ex-players, or to present data and stats – for example, touch maps or pass percentages.”

Foy believes more clubs, especially Premier League sides, will start to use AR as they look for new ways to engage with fans.

“The big clubs could probably build AR technology around the stadium so fans could go to certain points, be able to interact with it and find out more information,” he says.

“This could work, for example, on stadium tours and deliver history about the club and stadium through your phone so you’re not just standing there.

“Social media is changing in terms of algorithms and content is not always being seen by people, so AR could provide another opportunity to get content and information to fans.

“The key difference here is that it gets fans interacting rather than just sitting and consuming.”

Matchday programmes continue their battle to evolve

When football programmes were introduced in the late 19th Century, their main purpose was providing the two teams and a space for fans to write down the final result and goalscorers.

After the Second World War, they became a source of revenue through advertising and developed into a way of engaging with fans through manager columns, player interviews and statistics.

Anecdotal evidence seems to suggest sales of football programmes are falling, in an age where most youngsters are digital savvy.

But Chris Mortley, editor of Leicester City’s programme, says the 2015-16 Premier League champions have experienced record sales in recent seasons as fans seek souvenirs of days they never thought they would see.

The Foxes’ pre-season game against a Real Madrid side containing Cristiano Ronaldo and managed by Jose Mourinho in 2011 produced, at the time, the biggest selling programme in the club’s history.

Since then, two further editions have broken this record – the first home fixture of the 2014-15 Premier League season and on the day City lifted the Premier League title in May 2016.

“Naturally, the print run does increase for big games, whether that be against a big-name Premier League club, key matches or memorable occasions,” says Mortley, communications manager at Soar Media.

“For example, some of the highest print runs of last season were for the Champions League games at King Power Stadium.

“The club really do value the matchday programme and look to ensure that it reflects big occasions for the club.”

The programme industry, like all forms of media, is constantly evolving and looking for ways to attract new audiences.

“There was talk a few years ago to have programmes not in paper form but on a cassette in portable audio players. This never materialised,” says Steve Earl, a football programme expert.

“Some league clubs have tried to introduce bar codes you can scan in the programme for further information, but at the time it didn’t prove popular.

“However, with mobile phones now being able to do everything it will be interesting to see how Frickley’s venture progresses.”

Lenovo and Disney’s Star Wars Augmented Reality Experience

“The chosen one you are, with great promise I see.” Now that Disney owns the Star Wars franchise, the expansion of the universe is seemingly never ending. More films, more toys, and now more technology. We’re still a few years away from getting our own lightsabers [citation needed], but until then Disney has partnered with Lenovo to design a Star Wars experience using smartphones and augmented reality.

Lenovo is creating the hardware: a light beacon, a tracking sensor, a lightsaber controller, and the augmented reality headset designed for smartphones. The approach for Lenovo’s AR is different to how Samsung and others are approaching smartphone VR, or how Microsoft is implementing Hololens: by implementing a pre-approved smartphone into the headset, the hardware uses a four-inch diagonal portion of the screen to project an image that rebounds onto prisms and into the user’s eyes. The effect is that the user can still see ahead of them, but also images and details on the screens – limited mostly by the pixel density of the smartphone display.

Lenovo already has the hardware up for pre-order in the US ($199) and the EU (249-299 EUR), and is running a curated system of Android and iOS smartphones. This means that the smartphones have to be on Lenovo’s pre-approved list, which I suspect means that the limitation will be enforced at the Play Store level (I didn’t ask about side loading). But the headset is designed for variable sized devices.

In the two minute demo I participated in, I put on the headset and was given a lightsaber into a 10ft diameter circle, and fought Kylo Ren with my blue beam of painful light. Despite attempting harakiri in the first five seconds (to no effect), it was surprising how clear the image was without any IPD adjustment. The field of view with the headset is only 60 degrees horizontal and 30 degrees vertical, which is bigger than the Hololens and other AR headsets I have tried, but it still remains one of the biggest downsides to AR. In the demo, I had to move around and wait to counter-attack: after deflecting a blow or six from Kylo, I was given a time-slow opportunity to strike back. When waiting for him to attack, if I rushed to attack nothing seemed to happen. In typical boss-fight fashion, three successful block/hit combinations rendered me the victor – I didn’t see a health bar but this was a demo designed to encourage the user to have a positive experience.

One thing I did notice is that most of what I saw was not particularly elaborate graphically: 2D menus and a reasonable polygon model. Without the need to render the background, relying on what the user is in front of to do this job (Lenovo had it in a specific dark corner for ease of use) this is probably a walk in the park for the hardware in the headset. The lightsaber connects directly to the phone via Bluetooth, which I thought might be a little slow, but I didn’t feel any lag. The lightsaber was calibrated a bit incorrectly, but only by a few degrees. I asked about different lightsabers, such as Darth Maul’s variant, and was told that it there are possibilities in the future for different hardware, although based on what I saw it was unclear if they would implement a Wii-mote type of system with a single controller with a different skin attached. The limit at the time was that the physical lightsaber only emits a blue light for the sensor for now; it does go red, but only when there’s a low battery. Think about that next time you watch Star Wars: red saber means low batteries.

The possibilities for the AR headset could feasibly be endless. The agreement at this time is between Lenovo and Disney Interactive, so there is plenty of Disney IP that could feature in the future. Disney also likes to keep experiences on its platform locked down, so I wonder what the possibilities are for Lenovo to work with other developers and IP down the road. I was told by my Lenovo guide that it is all still very much a development in progress, with the hardware basically done and the software part going to ramp up. The current headset is given the name ‘Mirage’, and most smartphones should offer 3-4 hours of gameplay per charge.

Lenovo Mirage
Headset Mass 470g
Headset Dimensions 209 x 83 x 155 mm
Headset Cameras Dual Motion Tracking Cameras
Headset Buttons Select, Cancel, Menu
Supported Smartphones
as of (9/4)
iPhone 7 Plus
iPhone 7
iPhone 6s Plus
iPhone 6s

Samsung Galaxy S8
Samsung Galaxy S3 (?)
Google Pixel XL
Google Pixel
Moto Z

Lightsaber Mass 275g
Lightsaber Dimensions 316 x 47 mm
Package Contents Lenovo Mirage AR Headset
‘Light Sword’ Controller
Direction Finder
Smartphone Holder
Lightning-to-USB Cable
USB-C to USB Cable
2x AA Batteries
5V / 1A Charger and Power Supply

Pre-orders are being taken now, shipments expecting to start in mid-November. US price is listed as $199.99 (without tax) and EU pricing at 299.99 EUR (with tax).

Related Reading

Google releases SDK for augmented reality apps on Android

<!–Google releases SDK for augmented reality apps on Android | InfoWorld




Google releases SDK for augmented reality apps on Android

Google

“);});try{$(“div.lazyload_blox_ad”).lazyLoadAd({threshold:0,forceLoad:false,onLoad:false,onComplete:false,timeout:1500,debug:false,xray:false});}catch(exception){console.log(“error loading lazyload_ad “+exception);}});

Looking to mix physical and digital spaces, Google has released a beta SDK for augmented reality, dubbed ARCore, that is focused on bringing augmented reality (AR) to Android smartphones.

ARCore is built on Google’s Tango AR technology. Developers can build new AR apps or enhance existing ones with AR capabilities. (Apple’s forthcoming iOS 11 has augmented reality APIs as well, called ARKit.)

ARCore offers native APIs for motion tracking, environmental understanding, and light estimation. These capabilities let apps use the phone camera to observe points in a room and motion-sensor data, as well as detect horizontal surfaces and light virtual objects in ways to match their surroundings to make their appearance more realistic.

ARCore works with Java and OpenGL as well as with the Unreal and Unity AR technologies. 












Apple Experimenting With Several Augmented Reality Glasses Prototypes

Apple is working on “several different kinds” of wearable augmented reality prototypes as it tries to figure out the “most compelling application” for an AR headset, reports Financial Times.

Citing sources with knowledge of Apple’s plans, Financial Times says at least one group within Apple is pushing for a pair of glasses that feature a 3D camera but no screens, making the iPhone the main display, similar to Snap’s Spectacles, but no final design decisions have been made.

Snap’s camera-equipped screen-free Spectacles


Rumors of Apple’s work on AR smart glasses first surfaced in 2016, and previous rumors have suggested the glasses will connect wirelessly to the iPhone, much like the Apple Watch, and will display “images and other information to the wearer.”

While Robert Scoble suggested Apple could launch the smart glasses this year through a partnership with Carl Zeiss, most rumors (like today’s) suggest Apple is still in a prototyping phase and that a launch is still a ways off.

Earlier this year, Financial Times said Apple was “stepping up” development on an augmented reality wearable, but a potential launch is at least a year away or longer. Bloomberg has predicted a similar timeline, suggesting Apple is perhaps aiming to launch a product in 2018.

As Apple works on AR smart glasses, the company is preparing to make its first major move into augmented reality with the launch of iOS 11 and ARKit, a set of APIs designed to allow developers to build powerful augmented reality experiences into apps and games. When ARKit launches, the iPhone and the iPad will become the largest augmented reality platform in the world given the large number of devices already out in the wild.

Subscribe to the MacRumors YouTube channel for more videos.


Over the course of the past several months, Apple CEO Tim Cook has talked about Apple’s work on augmented reality several times. Just this past week, he called it “big and profound” and said he “could not be more excited” about AR and what developers are creating with ARKit.

“This is one of those huge things that we’ll look back and marvel at the start of it,” he said.

Merge thinks augmented reality is more fun when you can touch it

You may recognize Merge’s distinct, purple mobile VR headset, but its new device is where it really hopes to stand out. The company just announced the availability of the Merge Cube, an AR ‘toy’ that puts a new spin on augmented reality.

Working with a headset (or just a smartphone if you’d prefer) the Cube transforms before the eyes of the user, from what looks like an Indiana Jones prop into a bustling cityscape, a beating human heart, a tiny playable knock-off Minecraft – anything a developer can think to make, really.

Read next: The future of AR is…

We first tried Merge Cube back at CES, and while it could have felt like yet another AR gimmick, it quickly charmed us with the quality of its AR tracking. A combination of computer vision and some very intricate markings on the cube make for holograms that’s aren’t just dense but impressively robust.

“We offer a depth of detail that allows for close-up visuals, so when the computer vision is looking at just a portion of the cube there’s enough detail to keep track of it,” Merge co-founder Andrew Trickett told Wareable. The cube is designed to be rotated freely on one axis without distorting the AR effect. In one demo, the cube exploded into an entire replica solar system that I could rotate and even interact with using the headset’s button.

Merge thinks augmented reality is more fun when you can touch it

In another, all six faces became a grass canvas I could build and remove blocks on. It was essentially Minecraft by another name, but since when did kids care about IP? Perspective is a problem with some AR right now, so the the ability to hold this tiny ecosystem right up to my eyes and appreciate the details was, for a kid’s toy, rather exciting. Trickett said one company is taking all of NASA’s global data to make real-time AR weather patterns, while the Natural History Museum is trying it out to make AR versions of fossils that kids can now hold in their hands.

“Merge’s core audience is younger users, we just really believe they’re the ones that are going to adopt virtual reality and augmented reality in mass,” said Trickett, who uses the portmanteau “edu-tainment” to describe the category Merge is putting itself in. And indeed it seems likely the Merge Cube will have more appeal to a younger demographic.

“We want this to feel like technology from the future, an ancient alien artifact that doesn’t have any specific IP or design or anything so it could still be something generic or interesting you’ll put on your desk.”

Merge is also launching its Merge Miniverse, a portal designed to spotlight the best (and age-appropriate) AR games and experiences. “Most VR headsets, aside from Samsung’s and Google’s closed ecosystems, aren’t providing the user with quality content,” Trickett added. “They’re basically left to search the app store. We think that’s a terrible way to onboard society onto VR and AR.”

Merge thinks augmented reality is more fun when you can touch it

The Cube doesn’t require a Merge headset to use – any headset with support for an AR passthrough will work – and Merge has opened up the SDK to developers. It’s also working on a mobile VR controller, described by Trickett as “almost identical” to the Gear VR’s, which Merge will open source to devs. Both Trickett and fellow co-founder Franklin Lyons say they want to create the standards for the mass mobile AR/VR market. “For the first time ever there’s going to be a universal marker that developers can build more and more experiences on,” said Trickett about the Cube.

In the US Merge is now selling the Cube exclusively through Wal-Mart for $14.99, and online through its own site. But the mobile VR market has become incredibly saturated; earlier this year IDC predicted that the VR/AR headset market will continue to grow at an annual rate of 58%, with mobile headsets currently leading the charge.

“It’s taken some time to educate retailers in the consumer market on quality levels,” said Trickett. “We’re starting to see that dust settle. So now our headset’s going to be put alongside a much smaller array of products. I think that’s where the industry is going: more refinement, more consumer education, more retailer education.”

The Future of Augmented Reality Ain’t Pokemon Go – NewCo Shift

“After On:” Audio Episode 1 of 8

Meta’s Meron Gribetz on the present & future of AR

Five years back, Google Glass’s famous launch video trained us to think of augmented reality as a flat translucence. It would be a bunch of wee announcements slapped on our field of view like Post-Its on ski goggles. The world beheld this daring vision and hit the snooze bar. AR’s next major milestone, Pokémon Go, is also all about simple superimposition (for now, anyway). So I was surprised to find the faithful at last month’s AR in Action conference almost wholly focused on holograms and photorealism. It’s a big step forward — and it’s actually starting to work.

I attended the New York City event to meet up with Meta CEO Meron Gribetz. Meta is racing Microsoft for the early lead in commercial AR. Florida-based Magic Leap is also allegedly in the hunt, having raised over a billion dollars. But having yet to ship a product, they came in for some sharp criticism back in December, followed by bemused head-scratching, which continues to this day.

Subsequent to the conference, I sat down with Meron in Meta’s Silicon Valley HQ to record a long interview — which now is part of an eight-episode audio series I’m producing to accompany my new novel, After On. I set the novel nine seconds into the future, as this let me feature all kinds of present-tense science and technology. I figured this would also let me stuff my book full of 20-page digressions on how cooooool AR, synthetic biology, quantum computing, and other fields are (or rather, will be. You know — nine seconds from now).

As it turns out, 20-page digressions constitute lousy storytelling. Like, who knew (answer: My editor)? My consolation prize to myself is these interviews, in which world-class experts take us on deep dives into their fields. My co-host in all this is the inimitable Tom Merritt, who makes up for my complete inexperience in audio production and podcasting.

Meron Gribetz practices his “air grab” move on something that he swears is RIGHT THERE!

Meeting with Meta didn’t just add holograms to my conception of AR. I’d mainly thought of the field in consumer and entertainment terms (I blame Pokémon Go and AR’s flashy cousin, VR). But Meta is betting wholly on productivity and the enterprise. Meron says this stems from calling over 400 enterprise buyers of his first AR headset, which he launched via a 2013 Kickstarter. He found they were almost unanimously focused on productivity, and bet his company on this insight.

As a direct result, he now tethers his headsets to computers — which is fine for office-bound use cases. This lets Meta enlist beefy off-headset processors, avoid battery problems, and dodge issues with heat accumulation. A more powerful headset with a far wider field of view than Microsoft’s is the result. To me, this makes for a much more compelling experience. Microsoft’s HoloLens is like floating a mid-sized HDTV in front of your eyeballs, while Meta’s experience is much more enveloping. Of course, if AR turns out to be all about live-action role playing in the park with holographic scabbards, tethered headsets will be a lousy bet.

But for now, Meron is happy to all but inhabit his vision. He sits not at a desk, but a plank — a ten-by-one-foot slab of redwood whose thinness is allowed by the lack of a monitor. He says he wears his Meta headset throughout the working day, and that dozens of his hundred-plus employees do the same. In our interview, he cites a barrage of precedent going clear back to the Apple 2 and VisiCalc to argue that computing platform transitions are funded by enterprise use-cases — not entertainment.

As for AR’s first killer app in the enterprise, he offers collaborative 3D. Which struck me as rather niche. But then I went back to that VisiCalc analogy. This was the first commercial spreadsheet — the forerunner to Lotus, which itself was the forerunner to Excel. Meron likes to point out that Steve Jobs credited VisiCalc with much of the Apple 2’s success. And think of how nichey spreadsheets must have seemed in 1978. The analog equivalent was the accountant’s ledger — which would have been as familiar to non-accountants as cockpit controls to non-pilots. There were just a few hundred thousand American CPA’s. Yet VisiCalc went on to sell over a million copies. And Excel is now installed on over a billion computers. So, much as my Carter-era forerunners might have snorted at digital ledgers, I may not be visionary enough to imagine the countless uses collaborative 3D will have, once it democratizes.

When I ask if Meta’s more of a hardware, application, or OS company, Meron goes with door #3. And this resonates with me. He’s most excited when discussing things that unify the Meta experience across use cases. For instance, “air grab” — which he positions as AR’s equivalent of the iPhone’s pinch-to-zoom feature. It’s just what it sounds like: to move a hologram, you reach out, grab it, and yank it to a new point in the illusory matrix. An intuitive move, it contrasts with the taxonomy of “gestures” that HoloLens and others require users to learn to navigate AR space.

Meron most surprised me when I brought up the technology that appears in this section of my novel (conveniently excerpted right here on Medium). In this scene, a sketchy guy stalks a woman using an AR headset that’s indistinguishable from glasses. It identifies her using facial recognition. It then gives him scads of background data, allowing him to feign a jolly friend-of-friend connection, which takes her guard down.

Meron predicts this grade of hardware will ship within five years. Surprising— as this is one of just three points in the novel where I violate my “nine seconds in the future” rule to commit some science fiction, and I thought I’d been more aggressive than that! As for how we’ll avoid the related creepiness I depict in the novel, Meron presents a concept he calls “public by default” around 0:26:40 in our interview. Give it a listen, and post a comment if you have a strong reaction. If you’re a Medium member, you can listen right now by hitting “Play” at the top of this page. If you’re not a member, it’s worth trying out if it’s within your budget — and if it’s not, I’ll be posting all these episodes as podcasts on a weekly basis starting August 1st (not coincidentally, the release date of the full After On novel).

During the last part of the episode, Tom Merritt and I relate the interview to the first three excerpts of After On that appear on Medium. You can find those excerpts here, here, and here, and I’ll be honored if you make time to read them. This final section is the only part of the audio episode that presumes familiarity with the novel, and you can skip it if you haven’t yet read the relevant parts.

I hope you find time for the episode, either now or when it’s released as a podcast. Future topics and guests, plus release dates both on Medium and as podcasts, follow:

  • Neuroscience and consciousness, with Adam Gazzaley of UCSF. Adam is harnessing the power of video games to fight ADHD, autism and dementia. Yes, really! And this work has appeared on the cover of Nature, which is a huuuuuge deal in science. On Medium this Friday, July 21, podcast Tuesday, August 8.
  • Digital privacy and government intrusion, with Cindy Cohn, who runs the Electronic Frontier Foundation (“the leading nonprofit defending digital privacy, free speech, and innovation”). On Medium Friday, July 28, podcast Tuesday, August 15.
  • Synthetic biology’s promise & peril, with Autodesk Distinguished Researcher Andy Hessel. A major synbio thought leader, Andy is a catalyst behind GP-Write, described by some as the heir to the Human Genome Project. On Medium Monday July 31, podcast Tuesday, August 22.
  • Quantum computing, with venture capitalist Steve Jurvetson. Early backer of Elon Musk ventures including Tesla and SpaceX, Steve has spent fifteen years on the board of DWave; the world’s largest quantum computing company. On Medium Monday August 7, podcast Tuesday, August 29.
  • Nihilistic terrorism, with five-time New York Times bestselling author Sam Harris. Sam is one of the most outspoken and controversial people in America on this subject. On Medium Monday, August 14, Podcast Tuesday, September 5.
  • Superintelligence risk. Surprise guest. Just you wait! On Medium Monday, August 21, podcast Tuesday, September 12.
  • Fermi’s paradox, with British astronomer Stephen Webb, author of multiple books on this fascinating topic. On Medium Monday, August 28, podcast Tuesday, September 19.

Read more:

Acura Debuts Augmented Reality Race On Facebook Live 07/10/2017


Acura will debut an augmented reality
race today on Facebook live as the next iteration of the Acura TLX “What A Ride” campaign.

The event will happen at 8
p.m. Eastern and aims to showcase the performance attributes of the Acura TLX A-Spec sedan.

The automaker is calling it a first-of-its-kind AR driving event, plus a world’s first AR
helmet technology. The drivers are technology influencers including Zachary Levi, Sam Gorski, Dom Esposito and Maude Garrett along with hosts NSX GT3 racecar driver Ryan Eversley and online
personality Bradley Hasemeyer.

From behind the wheel of the new 2018 TLX A-Spec with SH-AWD, the four influencers will compete in the “What a Race” AR experience,
challenging one another for the fastest overall time during 3-lap individual runs. Each lap will trigger a new AR course visible to the driver and the Facebook Live audience, with a unique set of
visuals and obstacles that will task the Acura TLX A-Spec’s precise handling to overcome. Cameras mounted to the drivers’ helmets will bring viewers directly in on the action, and several
additional cameras positioned throughout the racecourse will provide excitement from multiple perspectives of the race. 

advertisement

advertisement

Drivers will each sport custom-built race helmets with augmented
reality technology embedded in the extra-wide visor, allowing for a full-color HD 80-degree viewing experience.The foundation is a racing helmet used for general driver protection and safety, while an
HD screen with mirrored lens technology has been added for an expanded field of view, along with a gyroscope to monitor head location within the car. The helmets are connected to a computer in the
rear seats that provide high-performance rendering capacity.

A custom molded plastic console that houses the remaining technology used to create the AR experience. Inside the console
there is a 5″ full-color HD screen that is connected to a high-powered gaming system behind the driver seat.

The HD Screen projects an image down into a mirrored lens technology. This is
essentially a two-way mirror system that allows for the reflection of the HD screen to appear properly and not distorted in the users field of view. An image at the top (the
screen) is projected through ocular lenses that adjust the image to display stereoscopically on a one-way mirror. The driver looks through the mirror, seeing the AR world and the
real world at the same time.

Each lap will trigger a new AR course visible to the driver and the Facebook Live audience, with a unique set of visuals and obstacles that will task the
Acura TLX A-Spec’s precise handling to overcome. Cameras mounted to the drivers’ helmets will bring viewers directly in on the action, and several additional cameras positioned throughout
the racecourse will provide excitement from multiple perspectives of the race. 

Acura teamed up with AR production experts at Current Studios to concept and execute the “What A
Race” experience. The team patched into the TLX’s ABS system to track the vehicle’s wheel speed and direction. To further mitigate any potential technical snafus during the race, the
team custom fabricated a device attached to the under carriage of the vehicle, allowing the location of the car in the AR world to be automatically reset remotely and quickly, as needed.
  

The race will not be a passive viewing experience, according to the automaker. Viewers will play an active role in the race and serve as a virtual pit crew. Prompts will
encourage fans to use reaction emojis throughout the race, revealing shortcuts and clearing obstacles to influence the course’s landscape and help the drivers succeed, providing an exhilarating
experience for everyone involved.