Apple Reportedly Working on 3D Sensor System for Rear Camera in 2019 iPhones

Apple is developing 3D depth sensing technology for the rear-facing cameras in its 2019 iPhones, according to a new report by Bloomberg on Tuesday. The 3D sensor system will be different to the one found in the iPhone X's front-facing camera, and is said to be the next big step in turning the smartphone into a leading augmented reality device.

Apple is evaluating a different technology from the one it currently uses in the TrueDepth sensor system on the front of the iPhone X, the people said. The existing system relies on a structured-light technique that projects a pattern of 30,000 laser dots onto a user's face and measures the distortion to generate an accurate 3D image for authentication. The planned rear-facing sensor would instead use a time-of-flight approach that calculates the time it takes for a laser to bounce off surrounding objects to create a three-dimensional picture of the environment.
The existing TrueDepth camera would continue to be used in the front-facing camera of future iPhones in order to power Face ID, while the new system would bring the more advanced "time-of-flight" 3D sensing capability to the rear camera, according to the sources cited. Discussions with manufacturers are reportedly already underway, and include Infineon, Sony, STMicroelectronics, and Panasonic. Testing is said to be still in the early stages, and could end up not being used in the phones at all.

With the release of iOS 11, Apple introduced the ARKit software framework that allows iPhone developers to build augmented reality experiences into their apps. The addition of a rear-facing 3D sensor could theoretically increase the ability for virtual objects to interact with environments and enhance the illusion of solidity.

Apple was reportedly beset with production problems when making the sensor in the iPhone X's front-facing camera, because the components used in the sensor array have to be assembled with a very high degree of accuracy. According to Bloomberg, while the time-of-flight technology uses a more advanced image sensor than the existing one in the iPhone X, it does not require the same level of precision during assembly. That fact alone could make a rear-facing 3D sensor easier to produce at high volume.

Related Roundup: iPhone X
Buyer's Guide: iPhone X (Buy Now)

Discuss this article in our forums

Amazon Updates iOS App With ARKit Support for Augmented Reality Holiday Shopping

Amazon is preparing its customers for the busy holiday shopping season, posting a detailed plan today on how it aims to help you get the items you want through Alexa voice shopping, curated gift guides, and a new "AR View" in the official Amazon iOS app. AR View was built using Apple's augmented reality developer framework called ARKit, and the company said it's meant to help customers "make better shopping decisions."


Similar to IKEA Place, Amazon's AR View lets you place virtual versions of real-world products sold on Amazon right into your home, helping you decide whether or not you like the item in a specific living space before you buy it. AR View can be found on the camera icon in the Amazon iOS app, which now has a new "AR View" option. Then you can select from "thousands of items" sold on Amazon, which includes home furniture, toys, Echo products, kitchen electronics, decor, and more.
Amazon’s latest augmented reality offering within the Amazon App launched today for customers with iOS 11 installed on their iPhone 6S or later. Using Apple’s ARKit, AR view helps customers make better shopping decisions by allowing them to visualize the aesthetic and fit of products in their own living space. Customers simply open the Amazon App, click on the camera icon and choose AR view.

They can then select from thousands of items – from living room, bedroom, kitchen and home office products to electronics, toys and games, home décor and more. Whether customers are buying a sofa or a kitchen appliance, they can overlay it onto their existing living space, move it and rotate it to get a full 360-degree peek in a live camera view to make sure it fits their style and aesthetic.
ARKit debuted in iOS 11 in September, and you'll need an iPhone 6s or later running iOS 11 to use the new AR View feature in Amazon's iOS app. Target also launched an augmented reality shopping feature last week, but it wasn't in its iOS app and instead debuted in the Target mobile website. Additionally, Target's new "See It In Your Space" option doesn't use ARKit.


Amazon is available to download for free from the iOS App Store. [Direct Link]

Tags: Amazon, ARKit

Discuss this article in our forums

‘Stranger Things’ Game Update and Snapchat AR Experience Mark Season 2 Debut on Netflix

Stranger Things: The Game received a feature update on Friday, coinciding with the hit TV series' season two debut on Netflix. Version 1.0.252 of the free retro-style sci-fi adventure game brings a new playable character to the Arcade and a brand new Hawkins High School dungeon.


Elsewhere, there are new quests to be gained from the Florist and the Hawk Theater Attendant, while players have another opportunity to get a 100 percent score thanks to a slew of new collectibles. In the words of the game's official blurb:
Things are stranger than ever in the town of Hawkins. Mike and the gang have new neighbors. Visit the Arcade to meet the new kid. Explore the Upside Down Gates that have opened around town. Solve the all-new throwback adventure in Hawkins High School.
Continuing the Stranger Things 2 theme, Snapchat today launched a new World Lens portal that takes users into an Upside Down version of Joyce Bryers' living room through an augmented reality doorway.

The room features several nods to the first season of Stranger Things, including fairy lights, alphabet graffiti, and a gaping hole in the living room wall. Interactive easter eggs and surprises scattered around the AR environment can also be discovered by tapping on objects.


To open the Upside Down lens in Snapchat, activate rear camera mode, tap the screen, and select the first sponsored effect in the list at the bottom of the interface. The lens can also be unlocked using Shazam in the Snapchat app when the Stranger Things them tune plays.

Two additional Stranger Things lenses are set to debut later in the day. To get the full augmented reality effect of the World Lenses, Snapchat users need to be using an iPhone 6s or later with iOS 11 installed.


Discuss this article in our forums

Games Are Most Popular ARKit-Only Apps, Accounting for 62% of ARKit App Revenue

Apps that use the ARKit framework Apple introduced in iOS 11 have been installed 3 million times around the world since September 19, according to new data shared today by analytics firm Sensor Tower (via TechCrunch).

Many of the ARKit-only apps that are available are games, accounting for 35 percent of ARKit apps worldwide. ARKit-only apps include those that are built entirely around the new ARKit technology, rather than apps that have added some ARKit features.


Utilities are the second most popular ARKit category, accounting for 19 percent of apps, while 11 percent of ARKit apps are in the Entertainment category and 7 percent are in the Education category.

Many of the top ARKit-only apps are games, with four games among the top five highest-grossing ARKit apps. Titles like AR Dragon, Zombie Gunship Revenant, The Machines, and Monster Park -- Dino World have proven to be the most popular AR games with consumers.


Games have accounted for a total of 53 percent of all ARKit-only app downloads and 62 percent of ARKit app revenue since the launch of ARKit.

With games taken out of the top app listings, apps that allow users to measure 3D spaces or demo furniture are some of the top performers. Among free apps, IKEA Place, the furniture app from IKEA that lets consumers see products in their homes before purchasing, is one of the more popular apps.


Sensor Tower's data covers only apps that are primarily focused on the augmented reality experience and it did not take into account apps that offer limited ARKit content like photo modes or mini games.

When iOS 11 launched on September 19, ARKit became the largest augmented reality platform in the world thanks to the huge number of iPhones and iPads out in the wild.

Apple CEO Tim Cook just this week said augmented reality may not be huge right now, but that it will experience a "dramatic" climb to take over the world much like the App Store did when it was first launched.


Discuss this article in our forums

Tim Cook Thinks Retailers Will Find Augmented Reality as Key as Having a Website

Apple CEO Tim Cook believes there isn't any sector or industry that will be untouched by augmented reality going forward.

Tim Cook via Alastair Nicol for Vogue

In a recent interview with Vogue, Cook said he believes the technology will transform everything from fashion runway shows to shopping.
"If you think about a runway show in the fashion world, that's a great application of AR because some of these, you want to see the dress all the way around, you do not want to just see the front." That kind of experience is all the more important now that runway shows are catering to a wider consumer audience watching online, and not just those seated in the front rows, he added.
Apple's chief envisions a world where customers will essentially be able to "point and buy" products. If your friend is wearing a pair of shoes you like, for example, you could point your iPhone at them, and a shopping app could instantly bring up information about the pair with the option to purchase them online.
"We don't have a plan to collect all of these objects, but I know companies who are working on that for their products," Cook said. "If you think about companies that offer a fair number of shoes, and [if a customer] sees a shoe and goes I want that one, you just want to point and [buy]. That will be a part of the shopping experience of the future, it absolutely will."
Cook is so confident in augmented reality's future that he believes the technology will become "as key as having a website" for brands.


Some retailers have already implemented augmented reality features using Apple's new ARKit platform on iOS 11. IKEA, for example, has released an app called IKEA Place that lets you virtually place furniture in your home, with true-to-scale models of everything from sofas and armchairs to footstools and coffee tables.

Cook thinks the current selection of augmented reality apps is only the very beginning of what's to come in the years ahead.

In one early ARKit demo, for instance, a woman is able to virtually try on various shades of lipstick and quickly choose her preferred color.


Later in the video, the woman browses a virtual aisle of images of herself with various cosmetics digitally applied, making her selection a much simpler process than the traditional hassle of physical makeup application.


Cook also said that the technology needed for augmented reality glasses "doesn't exist to do that in a quality way," suggesting that widely rumored Apple Glasses won't be released "any time soon."
"There are rumours and gossip about companies working on that, and we obviously don't talk about what we work on. But today I can tell you that the technology itself doesn't exist to do that in a quality way," Cook said. "We don't give a rats about being first, we want to be best in creating people's experiences. Something that you would see out in the market any time soon would not be something that any of us would be satisfied with."
Cook's sit-down discussion appears to be the same one that The Independent covered earlier this week.

Full Interview: Apple's Tim Cook On The Future Of Fashion & Shopping


Discuss this article in our forums

Review: Pixie’s ARKit-Enabled Item Trackers Are Neat, But Still Feel Like They’re in Beta

Apple's augmented reality developer framework, known as ARKit, launched within iOS 11 on September 19. The debut turned hundreds of millions of iPhones into advanced AR-capable devices in the span of a few days, leading to the first wave of ARKit apps on the iOS App Store.


One of these apps is called "Pixie," which existed prior to ARKit with its own proprietary AR technology, but the company updated its main app and "Pixie Point" trackers with Apple's technology last month. I've been testing Pixie's new ARKit-enabled tracking devices for about a week, and so far the app's augmented reality solution to finding lost items has provided a fun and engaging experience for item rediscovery, when it works.

Unfortunately, my experience with Pixie was too often plagued by a low-quality app guidance system and inconsistent success in its main function, making the whole package feel like a beta product for a better iteration down the line.


Like other Bluetooth trackers, you attach Pixie Points onto items you might lose frequently, or simply fear losing in the future. For Pixie's tracking technology to work reliably, one of the Points has to be attached to your iPhone via an included silicone-like case.

The Pixie Point sits embedded in the top right of the case's back, and this coupled with the slippery feel of the case made me more conscious of how I was holding my iPhone and how close it was to falling from my hand at times. You could also use the Point's adhesive backing to stick it directly to your iPhone, but for obvious reasons I preferred not do that.
Continue reading Review: Pixie’s ARKit-Enabled Item Trackers Are Neat, But Still Feel Like They’re in Beta

Here’s a Look at the First Wave of Augmented Reality ARKit Apps Hitting the iOS App Store Today

With the launch of iOS 11 today, Apple has turned hundreds of millions of iPhones into augmented reality-capable devices thanks to the support of a new developer framework called ARKit. With this technology, iOS developers can more easily craft AR experiences for users on compatible iPhones and iPads, using each device's built-in cameras, processors, and motion sensors.

As of now, the first wave of these apps are available for you to download and test on the iOS 11 App Store. The first apps range from game updates to practical everyday tools and even apps that encourage a healthier lifestyle, with more refined experiences likely coming in the future once developers get a grasp on what users enjoy with the first wave of apps.

Note that to use ARKit-enabled apps on iOS 11 you must have an iOS device with an A9, A10, or A11 processor. This means ARKit apps can be launched on iPhone 6s, iPhone 6s Plus, iPhone 7, iPhone 7 Plus, iPhone SE, iPhone 8, iPhone 8 Plus, and the upcoming iPhone X. For iPads, you can use the 9.7-inch iPad or the 10.5-inch and 12.9-inch iPad Pro. The older 9.7-inch model of the iPad Pro is compatible as well.

One you have iOS 11 installed on one of these devices, head over to the new App Store and check out some of the ARKit apps listed below to see how Apple's new augmented reality technology works in your own home.

Games


Splitter Critters (left) and Egg, Inc. (right)

Splitter Critters ($2.99)


- What's it about? Use swipes of your finger to split a colorful landscape and guide alien critters back to their spaceship, avoiding enemies and solving puzzles in the process.

- How's AR used? Scan a flat surface and then place a fully playable version of the main game into the real world, housed within a small white box.

Warhammer 40,000: Freeblade (Free)


- What's it about? Control an Imperial Knight war machine in the Warhammer 40,000 universe and fight the evil forces of Chaos through 170 single player missions using cannons, missiles, and thermal blasts to defeat your enemies.

- How's AR used? Drop your Imperial Knight from the main game into an AR "Photo Mode" to take snap shots of the war machine in the real world.

Egg, Inc. (Free)


- What's it about? A farming simulation game focused on hatching eggs, building hen houses, hiring drivers, and researching advanced technologies to upgrade your egg farm.
- How's AR used? Take a glimpse at your farm in AR with a "Farm To Table" picture-taking mode.

Thomas & Friends Minis (Free)


- What's it about? Build, decorate, paint, and create full train sets and then control characters from Thomas & Friends as you drive through your customized train set.

- How's AR used? Bring all of your creations into the real world with the app's AR mode, which places your train set on a flat surface so you can zoom in and around while still being able to interact with various tools and control characters.
Continue reading Here’s a Look at the First Wave of Augmented Reality ARKit Apps Hitting the iOS App Store Today

Here’s a Look at the First Wave of Augmented Reality ARKit Apps Hitting the iOS App Store Today

With the launch of iOS 11 today, Apple has turned hundreds of millions of iPhones into augmented reality-capable devices thanks to the support of a new developer framework called ARKit. With this technology, iOS developers can more easily craft AR experiences for users on compatible iPhones and iPads, using each device's built-in cameras, processors, and motion sensors.

As of now, the first wave of these apps are available for you to download and test on the iOS 11 App Store. The first apps range from game updates to practical everyday tools and even apps that encourage a healthier lifestyle, with more refined experiences likely coming in the future once developers get a grasp on what users enjoy with the first wave of apps.

Note that to use ARKit-enabled apps on iOS 11 you must have an iOS device with an A9, A10, or A11 processor. This means ARKit apps can be launched on iPhone 6s, iPhone 6s Plus, iPhone 7, iPhone 7 Plus, iPhone SE, iPhone 8, iPhone 8 Plus, and the upcoming iPhone X. For iPads, you can use the 9.7-inch iPad or the 10.5-inch and 12.9-inch iPad Pro. The older 9.7-inch model of the iPad Pro is compatible as well.

One you have iOS 11 installed on one of these devices, head over to the new App Store and check out some of the ARKit apps listed below to see how Apple's new augmented reality technology works in your own home.

Games


Splitter Critters (left) and Egg, Inc. (right)

Splitter Critters ($2.99)


- What's it about? Use swipes of your finger to split a colorful landscape and guide alien critters back to their spaceship, avoiding enemies and solving puzzles in the process.

- How's AR used? Scan a flat surface and then place a fully playable version of the main game into the real world, housed within a small white box.

Warhammer 40,000: Freeblade (Free)


- What's it about? Control an Imperial Knight war machine in the Warhammer 40,000 universe and fight the evil forces of Chaos through 170 single player missions using cannons, missiles, and thermal blasts to defeat your enemies.

- How's AR used? Drop your Imperial Knight from the main game into an AR "Photo Mode" to take snap shots of the war machine in the real world.

Egg, Inc. (Free)


- What's it about? A farming simulation game focused on hatching eggs, building hen houses, hiring drivers, and researching advanced technologies to upgrade your egg farm.
- How's AR used? Take a glimpse at your farm in AR with a "Farm To Table" picture-taking mode.

Thomas & Friends Minis (Free)


- What's it about? Build, decorate, paint, and create full train sets and then control characters from Thomas & Friends as you drive through your customized train set.

- How's AR used? Bring all of your creations into the real world with the app's AR mode, which places your train set on a flat surface so you can zoom in and around while still being able to interact with various tools and control characters.
Continue reading Here’s a Look at the First Wave of Augmented Reality ARKit Apps Hitting the iOS App Store Today

Pokémon GO Creator’s Next Game Will Integrate Audio Cues into the AR Experience

Pokémon GO creators Niantic are looking at ways to incorporate audio into future AR game experiences, according to the company's chief technology officer.

Speaking on a panel discussing augmented reality at TechCrunch's Disrupt event, Niantic CTO Phil Keslin said that audio cues would mean players wouldn't have to awkwardly hold their phone up while interacting with games like GO.

"I can tell you from experience that people don't do this," he said, mimicking how people playing an AR game would hold their phones. "It's very unnatural. It makes them look like a total doofus if they're doing it for an extended period of time," he added.

"In Pokémon GO, the only time they really use it is to share their encounter with the Pokémon. To take that one picture, which is natural…. Everybody takes a picture, and then they're done. It's not walking around the world with the phone in front of their face," he said.
Considering alternative solutions, Keslin suggested audio could be integrated into AR experience. "Audio is different," he said. "You can hide that." Most people today walk around with their audio earbuds stuck in their ears all the time, he noted. "Nobody knows that they're being augmented then."

Keslin later explained that audio was something Niantic had toyed with when they were building Ingress, a location-based, augmented reality game considered a precursor to Pokémon GO.

Audio integration was considered in a variety of ways, according to the CTO, for example, suggesting to players which location they should visit, or having their phone call them with further clues when they reached a waypoint. Another possibility was combining audio with a phone's sensors, like an accelerometer, to know what a person was doing. "AR is not just visual," he added.

Asked if audio clues would ever come to Pokémon GO, Keslin told TechCrunch: "Maybe. Or maybe we'd use it in other games," he said, smiling. "We're not a one-game wonder."

Related Roundup: Apple VR Project
Tags: Pokémon GO, ARKit

Discuss this article in our forums

ARKit Roundup: Tesla Customization, Spatial Audio Recording, and Friend Locator

ARKit demos have been popular over the summer, showcasing the features that iOS 11 users will gain on their iPhones once the new software update rolls out worldwide later in September. Today we've rounded up some of the latest tech demos of the software development kit, which you can check out below.

In the first ARKit demo, augmented reality and user experience designer Jelmer Verhoog has created an app that lets users design and customize a Tesla Model 3 car. This extends to the wheels, paint, and interior of the vehicle, and even includes driving controls. While Verhoog's demo is focused on the Model 3, it's easy to imagine other carmakers using Apple's ARKit to help customers when they're on the hunt for a new vehicle.


The next demo, by Zach Lieberman, shows off an interesting use of augmented reality and ARKit -- it allows users to record audio through the app, which is then visually represented by waveforms in an AR space. To play back the audio, users can physically move through it, with the ability to listen to the recorded audio both forwards and backwards. Check out the video below with audio to see Lieberman's demo in action.


An app called "Neon" was showcased recently, and it allows friends to find one another in the midst of overcrowded festivals. The app's Twitter page says it'll be out "this fall," and it appears to use its own social network to add close friends, find them through their own iPhones, and present their location to users as augmented reality chevrons that are easy to track down.


Augmented reality games will likely be a big part of the ARKit app market after the launch of iOS 11, and the developer of the new game Euclidean Lands is already working on an AR version of the puzzler. Since the iOS game's puzzles already float in space within the regular app, the developer has simply translated that concept into the real world with its ARKit demo, allowing players to walk around the puzzle, move characters, and rotate blocks.


Measuring apps continue to be consistently popular in the ARKit demo space, with a new app called MeasureKit having confirmed its launch for later in September alongside iOS 11. MeasureKit includes a bunch of detailed tools, including the ability to measure angles, make sure objects are hanging at the same level, and automatically detect someone's height. Watch a video of MeasureKit on the app's Twitter page here.

For more ARKit demos and roundups, check out our previous coverage in the articles below:

- Major App Developers Show Off ARKit Apps Ahead of iOS 11 Launch
- Latest ARKit Demos Include Sculpting, Food Ordering, Virtual Pets and Zombies
- Latest ARKit Demo Showcases Virtual Cosmetics Boutique
- Latest ARKit Demo Showcases Virtual Drawing
- Apple's ARKit Used to Recreate Classic A-ha 'Take On Me' Video
- ARKit Roundup: Turn-by-Turn Directions, Precise Room Measurements, and Pac-Man

Related Roundup: iOS 11
Tag: ARKit

Discuss this article in our forums