Apple to Offer Presentation on ARKit at This Year’s Game Developers Conference

Apple will be hosting a session at this year's Game Developers Conference for the first time, offering an introduction to ARKit, its augmented reality platform for developers.

The session will be presented by Michael Kuhn, who leads Apple's ARKit engineering team.

Entitled "Introduction to Apple's ARKit: Best practices and recent updates," the talk will cover core concepts of the ARKit framework and the ARKit API. It's designed to teach game developers how to get started with ARKit, and it will cover ARKit best practices.
This session introduces core concepts of the ARKit framework, it's underlying principles, and the ARKit API. It explains how to get started with ARKit using the different tracking and scene understanding capabilities as well integration into rendering/game engines. The session also highlights best practices for AR like starting an experience, placing objects in the real world, interacting with them and implications for games. In addition it explains basic concepts and challenges of AR and Computer Vision to help avoid common pitfalls and allow the creation of great experiences.
Apple has not previously offered developer sessions at GDC, but this is the first GDC since the launch of ARKit and Apple is likely hoping to get more game developers interested in implementing augmented reality features.

ARKit was introduced as part of iOS 11 back in September of 2017, and since then, developers have incorporated augmented reality features into more than 2,000 apps. Major improvements are coming to ARKit with the launch of iOS 11.3 and ARKit 1.5, which may come out right around when GDC takes place and will likely be a topic of discussion.

ARKit 1.5 can map irregularly shaped surfaces for better detection of ambient surroundings, it can recognize and map vertical surfaces like walls and doors, and it includes an image detection feature that works on everything from movie posters to bar codes.

The 2018 Game Developers Conference will kick off on March 19 at the Moscone Center in San Francisco, and it will last until March 23.

Tag: ARKit

Discuss this article in our forums

Developers Demo Augmented Reality Improvements Coming in iOS 11.3

The iOS 11.3 update, seeded to developers and public beta testers last week, introduces ARKit 1.5, an upgraded version of the set of tools developers can use to create augmented reality apps for the iPhone and the iPad.

As it turns out, ARKit 1.5 can do a lot of neat things. It can map irregularly shaped surfaces for better detection of your surroundings, and it can also recognize and map vertical surfaces like walls and doors, so you can use AR to place and detect items on walls.

Over the course of the last week, developers have been testing out ARKit 1.5 and sharing short demo videos on Twitter, providing a look at just what will be possible with augmented reality apps when iOS 11.3 is available.

Vertical surface detection, for example, is shown off in the video below. A realistic-looking tunnel is projected on a wall, and while this doesn't have any immediate usage implications, it's a useful demo of how ARKit sees walls in iOS 11.3.

An example of how vertical plane detection can be used in augmented reality games is demonstrated in the video below, where creatures projected into open space take advantage of the area around them.

Another demo adds virtual artwork to a blank wall, a concept that could potentially be used in an art gallery or museum where art is invisible without a smartphone.

Vertical plane detection is used in the video below to show a realistic-looking virtual cockatoo coming through a window and landing on a windowsill.

In addition to mapping oddly shaped spaces and recognizing vertical surfaces, ARKit 1.5 also includes image detection features that work on everything from movie posters to barcodes, as demoed below. In the future, you might be able to scan a barcode with ARKit to get a virtual popup of nutritional information, calories, and more.

Image detection could be useful in settings like art galleries and museums, where visitors could use it to scan paintings and exhibits to receive more information, as shown off in the video below.

Though not visible in the demo videos shared by developers, ARKit 1.5 also introduces a higher camera resolution, so passthrough video is 1080p rather than 720p, and there's also support for autofocus capabilities, another feature that will improve the augmented reality experience on iOS devices.

Recent data has suggested that the ARKit framework has seen only modest adoption from developers and stagnating growth since its debut in iOS 11, but improvements like ARKit 1.5 may change that in the future. Augmented reality on iOS devices is still in its infancy and it will take time for developers and users to discover the best real-world use cases for the technology.

Apps using ARKit 1.5 will be available starting this spring when iOS 11.3 is released to the public.

Related Roundup: iOS 11
Tag: ARKit

Discuss this article in our forums

iOS 11.3 Coming This Spring With New Animoji, ARKit Improvements, Battery Health Settings, and More

Apple today previewed iOS 11.3, its next major iPhone, iPad, and iPod touch software update. The first beta will be seeded to developers later today, followed by a public beta soon, ahead of an official release this spring.

iOS 11.3 introduces new Animoji on the iPhone X, including a lion, bear, dragon, and skull. There will now be 16 characters to choose from in total, including existing ones like a pig, fox, chicken, pile of poo, and robot.

iOS 11.3 will also feature ARKit 1.5. Apple says its updated augmented reality platform will be able to recognize and place virtual objects on vertical surfaces like walls and doors, such as posters, signs, and artwork, and can more accurately map irregularly shaped surfaces like circular tables.

In addition, the view of the real world through the camera now has 50 percent greater resolution and supports auto-focus for a sharper perspective.

The software update will introduce Business Chat, a new way for users to communicate directly with businesses within the Messages app. This feature will launch in beta following the public release of iOS 11.3 this spring, with support from select businesses, including Discover, Hilton, Lowe's, and Wells Fargo.
With Business Chat, it's easy to have a conversation with a service representative, schedule an appointment or make purchases using Apple Pay in the Messages app. Business Chat doesn’t share the user’s contact information with businesses and gives users the ability to stop chatting at any time.
In the Health app on iOS 11.3, users will be able to view health records, including available medical data from multiple providers like John Hopkins and Cedars-Sinai. The data is encrypted and protected with a passcode.

In a future beta release of iOS 11.3, users with an iPhone 6 or newer will be able to view their battery health under Settings > Battery. In the same menu, it will also be possible to disable Apple's power management feature.

Apple outlined some other features coming to iOS 11.3:

- Apple Music will soon be the home for music videos. Users can stream all the music videos they want without being interrupted by ads. They can also watch the hottest new videos, the classics or ones from their favorite artists back-to-back in new music video playlists.

- Apple News now makes it easier to stay up-to-date on the most important videos of the day with a new Video group in For You, and improved Top Stories.

- HomeKit software authentication provides a great new way for developers to add HomeKit support to existing accessories while protecting privacy and security.

- Support for Advanced Mobile Location (AML) to automatically send a user’s current location when making a call to emergency services in countries where AML is supported.

More details to follow…

Related Roundup: iOS 11

Discuss this article in our forums

Developer Use of Apple’s ARKit Framework Has Slowed Since September Launch

Apple's ARKit augmented reality framework has seen only modest adoption from developers since it was announced at the Worldwide Developers Conference, while growth has seen a steady decline since its official launch, according to App Store data gathered by Apptopia.

ARKit enables iPhones and iPads running iOS 11 to superimpose computer-generated graphics over the real world, allowing developers to take their apps beyond the screen and into the user's environment.

By using the built-in camera, processors, and motion sensors found in iOS devices, virtual content appears on top of real-world scenes, and users don't need any special equipment to enjoy them because ARKit does the heavy lifting.

But despite initial excitement over the technology and a general consensus that Apple has improved upon existing AR solutions like Google Tango by making ARKit simpler to use, take-up by iOS developers saw a steady decline in the second half of 2017.

In ARKit's launch month of September, developers released approximately 300 ARKit-related apps, while October saw an additional 200 or so enter the App Store, according to third-party data provider Apptopia. However, in November the number of ARKit-based apps fell to about 156, before recovering somewhat to around 160 in December. Overall, ARKit is said to have been used in about 825 of the over 3 million apps in the App Store since its debut.

As it stands, Apptopia reckons 30 percent of ARKit-using iOS apps fall into the games category, 13.2 percent are entertainment, and 7.5 percent are photo and video apps. Meanwhile, 11.9 percent are utilities, 7.8 percent are educational, and 5.2 percent are lifestyle apps, with the remaining 24.2 percent coming under the Other category.

Apple CEO Tim Cook has expressed his belief on several occasions that a big future lies ahead for augmented reality. In fact, Cook has said that he's so excited about the possibilities for the future of AR that he just wants to "yell out and scream", while admitting that there are limitations to the technology in its current state.

At the same time, Cook believes that those limitations are the building blocks of an "incredible runway" with a bright future, and that "when people begin to see what's possible, it's going to get them very excited—like we are, like we've been".

For a look at some of the first apps that implemented ARKit, check out this round-up.

Tag: ARKit

Discuss this article in our forums

Apple Reportedly Working on 3D Sensor System for Rear Camera in 2019 iPhones

Apple is developing 3D depth sensing technology for the rear-facing cameras in its 2019 iPhones, according to a new report by Bloomberg on Tuesday. The 3D sensor system will be different to the one found in the iPhone X's front-facing camera, and is said to be the next big step in turning the smartphone into a leading augmented reality device.

Apple is evaluating a different technology from the one it currently uses in the TrueDepth sensor system on the front of the iPhone X, the people said. The existing system relies on a structured-light technique that projects a pattern of 30,000 laser dots onto a user's face and measures the distortion to generate an accurate 3D image for authentication. The planned rear-facing sensor would instead use a time-of-flight approach that calculates the time it takes for a laser to bounce off surrounding objects to create a three-dimensional picture of the environment.
The existing TrueDepth camera would continue to be used in the front-facing camera of future iPhones in order to power Face ID, while the new system would bring the more advanced "time-of-flight" 3D sensing capability to the rear camera, according to the sources cited. Discussions with manufacturers are reportedly already underway, and include Infineon, Sony, STMicroelectronics, and Panasonic. Testing is said to be still in the early stages, and could end up not being used in the phones at all.

With the release of iOS 11, Apple introduced the ARKit software framework that allows iPhone developers to build augmented reality experiences into their apps. The addition of a rear-facing 3D sensor could theoretically increase the ability for virtual objects to interact with environments and enhance the illusion of solidity.

Apple was reportedly beset with production problems when making the sensor in the iPhone X's front-facing camera, because the components used in the sensor array have to be assembled with a very high degree of accuracy. According to Bloomberg, while the time-of-flight technology uses a more advanced image sensor than the existing one in the iPhone X, it does not require the same level of precision during assembly. That fact alone could make a rear-facing 3D sensor easier to produce at high volume.

Related Roundup: iPhone X
Buyer's Guide: iPhone X (Buy Now)

Discuss this article in our forums

Amazon Updates iOS App With ARKit Support for Augmented Reality Holiday Shopping

Amazon is preparing its customers for the busy holiday shopping season, posting a detailed plan today on how it aims to help you get the items you want through Alexa voice shopping, curated gift guides, and a new "AR View" in the official Amazon iOS app. AR View was built using Apple's augmented reality developer framework called ARKit, and the company said it's meant to help customers "make better shopping decisions."

Similar to IKEA Place, Amazon's AR View lets you place virtual versions of real-world products sold on Amazon right into your home, helping you decide whether or not you like the item in a specific living space before you buy it. AR View can be found on the camera icon in the Amazon iOS app, which now has a new "AR View" option. Then you can select from "thousands of items" sold on Amazon, which includes home furniture, toys, Echo products, kitchen electronics, decor, and more.
Amazon’s latest augmented reality offering within the Amazon App launched today for customers with iOS 11 installed on their iPhone 6S or later. Using Apple’s ARKit, AR view helps customers make better shopping decisions by allowing them to visualize the aesthetic and fit of products in their own living space. Customers simply open the Amazon App, click on the camera icon and choose AR view.

They can then select from thousands of items – from living room, bedroom, kitchen and home office products to electronics, toys and games, home décor and more. Whether customers are buying a sofa or a kitchen appliance, they can overlay it onto their existing living space, move it and rotate it to get a full 360-degree peek in a live camera view to make sure it fits their style and aesthetic.
ARKit debuted in iOS 11 in September, and you'll need an iPhone 6s or later running iOS 11 to use the new AR View feature in Amazon's iOS app. Target also launched an augmented reality shopping feature last week, but it wasn't in its iOS app and instead debuted in the Target mobile website. Additionally, Target's new "See It In Your Space" option doesn't use ARKit.

Amazon is available to download for free from the iOS App Store. [Direct Link]

Tags: Amazon, ARKit

Discuss this article in our forums

‘Stranger Things’ Game Update and Snapchat AR Experience Mark Season 2 Debut on Netflix

Stranger Things: The Game received a feature update on Friday, coinciding with the hit TV series' season two debut on Netflix. Version 1.0.252 of the free retro-style sci-fi adventure game brings a new playable character to the Arcade and a brand new Hawkins High School dungeon.

Elsewhere, there are new quests to be gained from the Florist and the Hawk Theater Attendant, while players have another opportunity to get a 100 percent score thanks to a slew of new collectibles. In the words of the game's official blurb:
Things are stranger than ever in the town of Hawkins. Mike and the gang have new neighbors. Visit the Arcade to meet the new kid. Explore the Upside Down Gates that have opened around town. Solve the all-new throwback adventure in Hawkins High School.
Continuing the Stranger Things 2 theme, Snapchat today launched a new World Lens portal that takes users into an Upside Down version of Joyce Bryers' living room through an augmented reality doorway.

The room features several nods to the first season of Stranger Things, including fairy lights, alphabet graffiti, and a gaping hole in the living room wall. Interactive easter eggs and surprises scattered around the AR environment can also be discovered by tapping on objects.

To open the Upside Down lens in Snapchat, activate rear camera mode, tap the screen, and select the first sponsored effect in the list at the bottom of the interface. The lens can also be unlocked using Shazam in the Snapchat app when the Stranger Things them tune plays.

Two additional Stranger Things lenses are set to debut later in the day. To get the full augmented reality effect of the World Lenses, Snapchat users need to be using an iPhone 6s or later with iOS 11 installed.

Discuss this article in our forums

Games Are Most Popular ARKit-Only Apps, Accounting for 62% of ARKit App Revenue

Apps that use the ARKit framework Apple introduced in iOS 11 have been installed 3 million times around the world since September 19, according to new data shared today by analytics firm Sensor Tower (via TechCrunch).

Many of the ARKit-only apps that are available are games, accounting for 35 percent of ARKit apps worldwide. ARKit-only apps include those that are built entirely around the new ARKit technology, rather than apps that have added some ARKit features.

Utilities are the second most popular ARKit category, accounting for 19 percent of apps, while 11 percent of ARKit apps are in the Entertainment category and 7 percent are in the Education category.

Many of the top ARKit-only apps are games, with four games among the top five highest-grossing ARKit apps. Titles like AR Dragon, Zombie Gunship Revenant, The Machines, and Monster Park -- Dino World have proven to be the most popular AR games with consumers.

Games have accounted for a total of 53 percent of all ARKit-only app downloads and 62 percent of ARKit app revenue since the launch of ARKit.

With games taken out of the top app listings, apps that allow users to measure 3D spaces or demo furniture are some of the top performers. Among free apps, IKEA Place, the furniture app from IKEA that lets consumers see products in their homes before purchasing, is one of the more popular apps.

Sensor Tower's data covers only apps that are primarily focused on the augmented reality experience and it did not take into account apps that offer limited ARKit content like photo modes or mini games.

When iOS 11 launched on September 19, ARKit became the largest augmented reality platform in the world thanks to the huge number of iPhones and iPads out in the wild.

Apple CEO Tim Cook just this week said augmented reality may not be huge right now, but that it will experience a "dramatic" climb to take over the world much like the App Store did when it was first launched.

Discuss this article in our forums

Tim Cook Thinks Retailers Will Find Augmented Reality as Key as Having a Website

Apple CEO Tim Cook believes there isn't any sector or industry that will be untouched by augmented reality going forward.

Tim Cook via Alastair Nicol for Vogue

In a recent interview with Vogue, Cook said he believes the technology will transform everything from fashion runway shows to shopping.
"If you think about a runway show in the fashion world, that's a great application of AR because some of these, you want to see the dress all the way around, you do not want to just see the front." That kind of experience is all the more important now that runway shows are catering to a wider consumer audience watching online, and not just those seated in the front rows, he added.
Apple's chief envisions a world where customers will essentially be able to "point and buy" products. If your friend is wearing a pair of shoes you like, for example, you could point your iPhone at them, and a shopping app could instantly bring up information about the pair with the option to purchase them online.
"We don't have a plan to collect all of these objects, but I know companies who are working on that for their products," Cook said. "If you think about companies that offer a fair number of shoes, and [if a customer] sees a shoe and goes I want that one, you just want to point and [buy]. That will be a part of the shopping experience of the future, it absolutely will."
Cook is so confident in augmented reality's future that he believes the technology will become "as key as having a website" for brands.

Some retailers have already implemented augmented reality features using Apple's new ARKit platform on iOS 11. IKEA, for example, has released an app called IKEA Place that lets you virtually place furniture in your home, with true-to-scale models of everything from sofas and armchairs to footstools and coffee tables.

Cook thinks the current selection of augmented reality apps is only the very beginning of what's to come in the years ahead.

In one early ARKit demo, for instance, a woman is able to virtually try on various shades of lipstick and quickly choose her preferred color.

Later in the video, the woman browses a virtual aisle of images of herself with various cosmetics digitally applied, making her selection a much simpler process than the traditional hassle of physical makeup application.

Cook also said that the technology needed for augmented reality glasses "doesn't exist to do that in a quality way," suggesting that widely rumored Apple Glasses won't be released "any time soon."
"There are rumours and gossip about companies working on that, and we obviously don't talk about what we work on. But today I can tell you that the technology itself doesn't exist to do that in a quality way," Cook said. "We don't give a rats about being first, we want to be best in creating people's experiences. Something that you would see out in the market any time soon would not be something that any of us would be satisfied with."
Cook's sit-down discussion appears to be the same one that The Independent covered earlier this week.

Full Interview: Apple's Tim Cook On The Future Of Fashion & Shopping

Discuss this article in our forums

Review: Pixie’s ARKit-Enabled Item Trackers Are Neat, But Still Feel Like They’re in Beta

Apple's augmented reality developer framework, known as ARKit, launched within iOS 11 on September 19. The debut turned hundreds of millions of iPhones into advanced AR-capable devices in the span of a few days, leading to the first wave of ARKit apps on the iOS App Store.

One of these apps is called "Pixie," which existed prior to ARKit with its own proprietary AR technology, but the company updated its main app and "Pixie Point" trackers with Apple's technology last month. I've been testing Pixie's new ARKit-enabled tracking devices for about a week, and so far the app's augmented reality solution to finding lost items has provided a fun and engaging experience for item rediscovery, when it works.

Unfortunately, my experience with Pixie was too often plagued by a low-quality app guidance system and inconsistent success in its main function, making the whole package feel like a beta product for a better iteration down the line.

Like other Bluetooth trackers, you attach Pixie Points onto items you might lose frequently, or simply fear losing in the future. For Pixie's tracking technology to work reliably, one of the Points has to be attached to your iPhone via an included silicone-like case.

The Pixie Point sits embedded in the top right of the case's back, and this coupled with the slippery feel of the case made me more conscious of how I was holding my iPhone and how close it was to falling from my hand at times. You could also use the Point's adhesive backing to stick it directly to your iPhone, but for obvious reasons I preferred not do that.
Continue reading Review: Pixie’s ARKit-Enabled Item Trackers Are Neat, But Still Feel Like They’re in Beta