Apple Experimenting With Several Augmented Reality Glasses Prototypes

Apple is working on "several different kinds" of wearable augmented reality prototypes as it tries to figure out the "most compelling application" for an AR headset, reports Financial Times.

Citing sources with knowledge of Apple's plans, Financial Times says at least one group within Apple is pushing for a pair of glasses that feature 3D camera but no screens, making the iPhone the main display, similar to Snap's Spectacles, but no final design decisions have been made.

Snap's camera-equipped screen-free Spectacles

Rumors of Apple's work on AR smart glasses first surfaced in 2016, and previous rumors have suggested the glasses will connect wirelessly to the iPhone, much like the Apple Watch, and will display "images and other information to the wearer."

While Robert Scoble suggested Apple could launch the smart glasses this year through a partnership with Carl Zeiss, most rumors (like today's) suggest Apple is still in a prototyping phase and that a launch is still a ways off.

Earlier this year, Financial Times said Apple was "stepping up" development on an augmented reality wearable, but a potential launch is at least a year away or longer. Bloomberg has predicted a similar timeline, suggesting Apple is perhaps aiming to launch a product in 2018.

As Apple works on AR smart glasses, the company is preparing to make its first major move into augmented reality with the launch of iOS 11 and ARKit, a set of APIs designed to allow developers to build powerful augmented reality experiences into apps and games. When ARKit launches, the iPhone and the iPad will become the largest augmented reality platform in the world given the large number of devices already out in the wild.

Subscribe to the MacRumors YouTube channel for more videos.

Over the course of the past several months, Apple CEO Tim Cook has talked about Apple's work on augmented reality several times. Just this past week, he called it "big and profound" and said he "could not be more excited" about AR and what developers are creating with ARKit.

"This is one of those huge things that we'll look back and marvel at the start of it," he said.

Related Roundup: Apple VR Project
Tag: ft.com

Discuss this article in our forums

Apple CEO Tim Cook: ‘I Think Augmented Reality is Big and Profound’

During today's earnings call covering the third fiscal quarter of 2017, Apple CEO Tim Cook was asked about ARKit and how he foresees developers using the service when it first launches. In response, Cook once again reiterated his excitement about augmented reality, and in short, said he expects to see a wide variety of applications shortly after iOS 11 becomes available this fall.

Cook pointed towards many of the demos that we're already seeing, which have ranged from games and entertainment to more practical applications like furniture placement and measurement.

Subscribe to the MacRumors YouTube channel for more videos.

"I could not be more excited about AR and what we're seeing with ARKit," said Cook. "What categories? What's on the web in terms of what people are doing? It's all over the place, from entertainment to gaming." He went on to say that he expects to see small business solutions, consumer solutions, and enterprise solutions, though he mentioned enterprise "takes a little longer to get going sometimes."

"I think AR is big and profound," he went on to say. "This is one of those huge things that we'll look back at and marvel at the start of it. I think customers are going to see it in a variety of ways and it feels great to get [AR] going at a level that can get all of the developers behind it."

A recreation of A-ha's "Take On Me" made using ARKit

In an interview with CNBC after the earnings call, Cook said he believes the iPhone will become even more essential than it already is once augmented reality features are widely available.
"The smart phone is becoming even more important to people because it's going across so much of your life and you can tell by some of the things we did at WWDC that that will only continue," Cook told CNBC's Josh Lipton. "And with things like AR… I think it becomes even more essential than it currently is. I know it's hard to believe, but I think that's the case."
When iOS 11 launches in the fall, it's set to become the largest AR platform in the world thanks to the myriad iPhones and iPads that are in the hands of customers. Developers are already building a huge range of ARKit-based apps and games, which will also start launching this fall.

Related Roundup: Apple VR Project

Discuss this article in our forums

AR Startup Blippar Showcases ‘Halos’ Facial Recognition Feature in its Mobile App

Augmented reality startup Blippar today announced a new social feature for its image recognition app that lets users build an AR profile using facial recognition technology.

Called "Halos", the feature offers users the ability to scan their face into the app and fill various bubbles around their head with personal details, such as their latest tweets, favorite songs on Spotify, YouTube videos, and animated emoji.


Once the facial profile is uploaded, anyone who "blips" the user – or scans their face with the app – sees the information as a halo of bubbles suspended in mid-air. The feature builds on the app's Public Figure Facial Recognition, introduced last December, which lets users scan 370,000 famous faces to learn interesting facts about them.

The Blippar app generally relies on users aiming their phone camera at everyday objects, products or images and "blipping" them to unlock helpful information, interact with brands, play videos, games, music, and more.

Blippar said in a blog post that the mobile app is mainly its way of showcasing technologies for other companies interested in adopting their visual search engine APIs – or in this case, facial recognition tech, which the company claims has more than 99.6 percent accuracy.
"Our faces are our most unique and expressive form of communication. Through AR Face Profiles we are making the face accessible in digital format for the first time, providing an innovative and dynamic way of expressing ourselves and discovering more about those around you. The technology lends itself to many other forms of implementation, and we are delighted to be able to share our APIs and technology with other companies, who like us, are committed to spurring on innovation in their own industries."
Apple has previously snapped up smaller companies specializing in facial recognition and augmented reality technology – two tentpole features expected in the so-called "iPhone 8" due to launch later this year.

In February the tech giant bought Israeli firm RealFace, whose proprietary IP could be used to power facial authentication in the upcoming phone, with Apple's ARKit developer platform likely to be used to showcase the handset's next-generation augmented reality capabilities.

Blippar is a free download for iPhone available on the App Store. [Direct Link]

Related Roundup: Apple VR Project
Tags: augmented reality, Blippar

Discuss this article in our forums

Apple Users’ Mixed Reality Future Teased in Latest ARKit Demo

Developers have wasted no time testing the limits of Apple's new ARKit platform, as each passing week serves up tantalizing previews of what's possible on an iPhone or iPad now that developers can quickly and easily build augmented reality experiences into their apps.

The latest demo to raise the bar comes courtesy of New York-based virtual reality developer Normal VR and offers a sneak peek at what's potentially in store for anyone with an iOS device, an iMac, and a VR headset.


The video clip, recorded on a 10-inch iPad Pro, shows the company using ARKit and the Unity game engine to capture the physical movements of a VR artist and project her virtual painting onto the real world via the mimicking actions of a digital avatar. The interaction also appears to be two-way, with scaling and repositioning functions controlled by the external viewer using iPad gestures.

It's unknown whether the HTC Vive is connected to a Mac or linked to some other supporting machine, but either way the potential uses for such a setup could be pretty far-reaching.

We already know Apple's latest top-of-the-line iMacs will include VR headset support out of the box once High Sierra gets its final release. Steam game platform creator Valve has also developed a version of its SteamVR software development kit for Mac, offering Apple users the same 360-degree, room-scale tracking as the Windows and Linux variants, with the added bonus of Oculus Rift support.

If ARKit-wielding developers can create apps that allow VR content to be viewed and interacted with outside of a headset, cross-platform Apple users could be looking at an exciting future of shared mixed reality experiences.

The first apps powered by ARKit are set to launch on the new-look App Store alongside iOS 11 this fall.

Related Roundups: Apple VR Project, iOS 11
Tag: ARKit

Discuss this article in our forums

Three Apple Developers Join WebVR Community Group

Three Apple developers yesterday joined an open community initiative that aims to make virtual reality content viewable regardless of device or browser. As reported by UploadVR, the staff added their names to the WebVR Community Group members list on Wednesday, in what looks like a continuation of the company's recent major push into VR.

Apple's senior front-end developer Brandel Zachernuk, multimedia and software standards representative David Singer, and WebGL spec editor Dean Jackson now appear alongside the names of developers from various other major internet companies like Facebook, Google, and Microsoft.


WebVR's mission is to enable web users to enjoy online VR experiences whatever hardware or software platform they use, suggesting Apple's involvement will be focused on Safari compatibility. Working on that assumption, Apple's participation means every major web browser now has representation in WebVR development, including Google Chrome, Mozilla Firefox, and Microsoft Edge.

Google's Brandon Jones, who is chair of the WebVR Community Group, was first to notice Apple's appearance in the member list, which includes 147 participants at the time of writing. However, Jones offered the caveat that "group participation does not necessarily imply a commitment to implement".

Last month at the Worldwide Developers Conference, Apple revealed its first big push into the augmented reality and virtual reality spaces, with a new ARKit developer framework and high-performance iMacs with native support for VR content creation. Native VR support in macOS High Sierra also opens up the possibility for Mac owners to hook up VR headsets to their computer for the first time.

Related Roundup: Apple VR Project

Discuss this article in our forums

Hands-On With Apple’s ‘ARKit’ Augmented Reality Demo for Developers

With iOS 11, Apple is delving into augmented reality in a big way, introducing an ARKit development platform that will allow developers to quickly and easily build augmented reality experiences into their apps and games.

ARKit is positioned to be the largest AR platform in the world when it launches this fall, using the camera, processors, and motion sensors in the iPhone and iPad to create some incredibly impressive augmented reality interactions.

While we won't see the first augmented reality apps and games built on ARKit for a couple of months, Apple has an ARKit demo app to show off what ARKit can do. We went hands-on with the demo to give MacRumors readers just a small taste of what to expect.

Subscribe to the MacRumors YouTube channel for more videos.

ARKit uses technology called Visual Inertial Odometry to track the world around an iPad or iPhone, allowing a device to sense how it moves in a room. ARKit automatically analyzes a room's layout, detecting horizontal planes like tables and floors, which then allows virtual objects to be placed upon those surfaces.

With ARKit able to place any virtual object within a physical room, developers can create all kinds of unique experiences, and developers have already released several demos showing what might be possible.

Apple already has at least one major retailer on board to use ARKit -- IKEA. IKEA is developing a new augmented reality app built on ARKit that will let customers preview IKEA products in their own homes before making a purchase. IKEA has offered augmented reality functionality for a few years now, but the company says Apple's new platform will much improve the experience. With ARKit available, IKEA says augmented reality will now "play a key role" in new product lines.

For additional details on other developer tools and features coming in iOS 11, make sure to check out our full iOS 11 roundup.

Related Roundups: Apple VR Project, iOS 11
Tag: ARKit

Discuss this article in our forums

AR/VR Headset Shipments Forecast to Hit 100 Million Units By 2021, Buoyed By Apple, Microsoft, and Others

Earlier this month at the Worldwide Developers Conference, Apple revealed its first big push into the augmented reality and virtual reality spaces, with a new ARKit developer framework and high-performance iMacs with native support for VR content creation.

On Monday, research firm IDC published new data forecasting significant growth in both markets, with dedicated AR and VR headset adoption expected to increase from just under 10 million units last year to 100 million units in 2021.


VR headsets account for much of the device volume so far, with VR headsets powered by a smartphone proving the most popular, according to IDC. The second half of 2016 also saw an increase in volume of Sony PlayStation VR, HTC Vive, and Facebook's Oculus Rift.
"The next six to 18 months will further stimulate the VR market as PC vendors, along with Microsoft, introduce tethered headsets and high-end standalone VR headsets also enter the market," said Jitesh Ubrani senior research analyst for IDC Mobile Device Trackers. "With lower hardware requirements on the PC and lower prices on headsets, VR will be more accessible than ever before. And the introduction of additional motion tracking and hand tracking will help further blur the line between digital and physical reality."
Although IDC believes VR headsets will continue to dominate the market in terms of volume for the foreseeable future, the firm believes AR will have a much bigger impact on the industry as a whole, in part thanks to Apple's recent entry onto the scene.
In terms of dedicated devices, AR continues to sit slightly in the background of VR. The reason for this is not that AR is less important, but rather it is harder to achieve. IDC believes VR headsets will continue to lead in terms of volume throughout the forecast, but maintains that AR in general will have a much bigger impact overall on the industry. Consumers are very likely to have their first AR experience via a mobile phone or tablet rather than a dedicated headset, and Apple's recent introduction of ARKit further supports this.
IDC believes AR headsets will become increasingly popular in markets such as healthcare, manufacturing, field service workers, and design, with commercial shipments to account for just over 80 percent of all AR headsets shipped in the next 5 years. "We believe that many industrial jobs will fundamentally change because of AR in the next 5-years," said Ryan Reith, IDC program vice president. "These are much more opportunistic markets for dedicated AR headsets than the consumer market."

Mobile app developers have been sharing early creations using Apple's new ARKit, suggesting huge enthusiasm for the possibilities for AR on iOS devices. As for VR, Apple's enthusiasm was clear during its WWDC keynote, with the company showing off the power of its new iMacs through a live demo of VR content creation using a HTC Vive, made possible via a new Metal 2 developer kit that has provisions for external GPUs and VR headsets.

Related Roundup: Apple VR Project
Tags: IDC, ARKit

Discuss this article in our forums

Developers Share First Augmented Reality Creations Using Apple’s ARKit

Apple only announced its augmented reality developer platform ARKit last week at the Worldwide Developers Conference, but some video clips have already been posted online by developers eager to showcase the impressive potential of the software.

ARKit enables iPhones and iPads running iOS 11 to superimpose computer-generated graphics over the real world, allowing developers to take their apps beyond the screen and into the user's environment.

Apple ARKit running Unity and Overwatch Widowmaker, by Cody Brown

By using the built-in camera, processors, and motion sensors found in iOS devices, virtual content appears on top of real-world scenes, and users don't need any special equipment to enjoy them because ARKit does the heavy lifting.

One developer who spoke to Motherboard said Apple had improved upon existing AR solutions like Hololens and Google Tango by making the ARKit framework elegant and simple to use.
"The most impressive aspect of ARKit is that it tends to just work," said Cody Brown, founder of virtual reality production studio IRL, in an online interview with Motherboard. "Other AR software often requires some sort of physical tracking mechanism (like a QR code), which inevitably becomes a major piece of friction if you are trying to get anyone to use this stuff.

"Another incredible aspect of ARKit is how it handles lighting adjustments in real time, continued Brown. "I can only imagine the math and magic underneath this tech to make it work."
During its keynote at WWDC, Apple demoed a range of effects that ARKit is capable of. One demo by Wingnut Studios wowed the audience with a tabletop sci-fi raider assault. Another showed a user placing a steaming digital coffee mug and a lamp on a table and moving the objects around to show off the tracking and shadow effects.

Several other ARKit developer demos have since appeared online, including a virtual "iPhone 8", roving StarWars character BB-8, dog-stalking zombies, office floor naval battles, dancing Candy rock stars, and more.

Rainforest garage by 8ninths

Apple has invested heavily in augmented reality, making several VR/AR company acquisitions in recent years including PrimeSense, FaceShift, and Metaio. Apple's much-anticipated "iPhone 8", which is expected to be announced in September, is rumored to have 3D sensing capabilities, with augmented reality being one possible use for the technology.

ARKit will come to compatible devices as part of iOS 11, which Apple is set to release in the fall.

Related Roundups: Apple VR Project, iOS 11
Tag: ARKit

Discuss this article in our forums

Apple Offers External GPU Enclosure to Developers As Valve Opens Door to VR Mac Gaming

Apple announced a new version of its Metal graphics technology during Monday's keynote at the Worldwide Developers Conference. As part of macOS High Sierra, Metal 2 will officially support external GPUs, allowing any Mac with a Thunderbolt 3 port to benefit from graphics hardware powerful enough to run demanding virtual reality applications and games.

The native VR support in macOS High Sierra also opens up the possibility for Mac owners to hook up VR headsets to their computer for the first time. And in a concurrent related announcement, Steam game platform creator Valve also revealed in a blog post on Monday that it is making a beta version of its SteamVR software development kit available on Mac, offering players the same 360-degree, room-scale tracking as the Windows and Linux variants.

On the development side, we have worked closely with Epic and Unity to make Mac extensions of content built on those engine technologies as simple as possible. Extension tools for those engines, and others, are available as part of this beta.

We've also worked with Mozilla to help enable WebVR support on Firefox, so macOS-based web developers can start trying out VR.
In addition to the Valve partnership, Apple announced it is also selling its own external graphics enclosure to developers who want to work on graphically intensive VR and 3D applications and games, although Apple noted that external GPU support likely won't arrive for consumers until spring 2018.
Apps that use Metal, OpenCL, and OpenGL can now take advantage of the increased performance that external graphics processors can bring. The External Graphics Development Kit includes everything you need to start optimizing advanced VR and 3D apps on external graphics processors with macOS High Sierra.
Apple's External Graphics Development Kit comes with a Sonnet external GPU chassis with Thunderbolt 3 and 350W power supply, an AMD Radeon RX 580 8GB graphics card, a Belkin USB-C to 4-port USB-A hub, and a promo code for $100 towards the purchase of a HTC Vive VR headset.

The External Graphics Development Kit costs $599 and requires a Mac with Thunderbolt 3 running the latest beta version of macOS High Sierra. The other caveat is that customers have to be a member of the Apple Developer Program to be eligible to purchase the kit.

The kit can be bought directly from Apple's website, although Apple cautions that the HTC Vive promo codes have limited availability and are distributed on a first-come, first-served basis.


Discuss this article in our forums

Apple Hires NASA Augmented Reality Expert Jeff Norris

Apple has hired Jeff Norris, an augmented reality expert who founded the Mission Operations Innovation Office at NASA's Jet Propulsion Lab, reports Bloomberg.

Norris has reportedly joined Apple as a senior manager working on the augmented reality team led by Mike Rockwell, who formerly ran Dolby Labs. The team is said to be working on the previously-rumored augmented reality smart glasses as well as AR features for future versions of the iPhone.

Prior to joining Apple, Norris worked at the NASA Jet Propulsion Laboratory, where he was employed since 1999. Along with founding the Mission Operations Innovation Office, he founded the JPL Ops Lab for developing human-system interfaces for mission operations, and he led multiple projects focused on human-system interaction with an emphasis on virtual and augmented reality.

On his website, Norris features a speech he gave on augmented reality and Nasa's JPL Ops Lab, much of which was focused on augmented reality headsets and their uses.


Under Norris' leadership, the JPL Ops Lab provided the Microsoft HoloLens to astronauts onboard the International Space Station and developed software for virtually working on Mars with the HoloLens.


For the last couple of years, Apple has taken a deep interest in augmented and virtual reality, and is said to have a large team of employees working on the technologies and exploring ways they could be used in future Apple products.

Apple has been working on both virtual reality headsets and augmented reality smart glasses, with the aim of launching the latter in 2018. We've also heard rumors suggesting augmented reality functionality could be incorporated into the iPhone, perhaps as early as the iPhone 8 set to be released this September.

Apple CEO Tim Cook has expressed his excitement about augmented reality several times in recent months. "I think AR is that big, it's huge. I get excited because of the things that could be done that could improve a lot of lives," he said in February of 2017.

Related Roundup: Apple VR Project
Tags: bloomberg.com, NASA

Discuss this article in our forums