Consumer Reports Ranks iPhone X Below iPhone 8 Because of Durability and Battery Life

Consumer Reports today shared its final iPhone X testing results, and while the site has given the iPhone X a recommendation, Apple's new flagship smartphone has been ranked below the iPhone 8 and the iPhone 8 Plus in the Consumer Reports recommended list.

The iPhone X did make the Consumer Reports list of top 10 smartphones in the number 9 slot, but the site says it did not beat out the iPhone 8 or the iPhone 8 Plus because of its poor performance on a durability test. Both the iPhone 8 and the iPhone 8 Plus better survived a tumble test for emulating real-world drops and fumbles of about 2.5 feet that can result in device damage, despite the fact that all three devices have glass bodies.


After 50-100 tumbles, one iPhone X model suffered serious body damage, while two others had screen defects. The iPhone 8 and 8 Plus ended up with just a few scrapes after the test. Front displays for the iPhone X, 8, and 8 Plus all came away unscathed, and the iPhone X did well on scratch tests and water resistance tests.

"If not for the damage in that durability test, the iPhone X would have come in ahead of the iPhone 8 and 8 Plus," says Richard Fisco, head of smartphone testing at CR.
Consumer Reports also had some complaints about the iPhone X battery life, which does not last as long as the battery in Samsung phones like the Galaxy S8. The iPhone X lasted 19.5 hours in the Consumer Reports battery test, compared to 26 hours for the Samsung Galaxy S8 and 21 hours for the iPhone 8 Plus.

The iPhone X didn't fare well on durability or battery tests, but it did earn the highest camera score out of all the smartphones tested by Consumer Reports. The site also listed the OLED display and the Face ID facial recognition system as iPhone X strengths.
The rear camera on the iPhone X is among the best we've ever seen. In fact, if you combine the performance scores for stills and video, this is the highest-rated smartphone camera Consumer Reports has tested.
Overall, Consumer Reports continues to rank the Samsung Galaxy S8 and the Samsung Galaxy S8+ as its top two recommended smartphones, mainly due to superior battery life, followed by the iPhone 8 Plus and the iPhone 8 in spots number three and four. At number nine, the iPhone X is at the bottom of the list, but only a few points separate all of the devices tested.

Related Roundup: iPhone X
Buyer's Guide: iPhone X (Buy Now)

Discuss this article in our forums

Consumer Reports Pulls Purchase Recommendation for Microsoft Surface Devices

After previously giving four Microsoft Surface devices a purchase "recommendation" status, Consumer Reports today has pulled that status from the Microsoft products. The publication said that because of "poor predicted reliability" in comparison with rival brands, it can no longer recommend any Surface laptops or tablets to consumers.

The decision specifically targets four Microsoft Surface devices, including the Surface Book (128GB and 512GB versions) and the Surface Laptop (128GB and 256GB versions). Although only four devices are losing their previously designated "recommended" status, Consumer Reports pointed out that its inability to recommend Microsoft Surface products extends across the company's laptop and tablet devices, including the Surface Pro.


As usual, Consumer Reports based its decision on a survey conducted by its subscribers and the devices they own and use each day, this time focusing on over 90,000 Microsoft products purchased between 2014 and early 2017. The study found that an estimated 25 percent of Microsoft laptops and tablets will "present their owners with problems" as soon as the end of the second year of ownership.

In its findings, the publication said the differences between the breakage rates of Microsoft devices and other brands were "statistically significant."
Microsoft’s estimated breakage rate for its laptops and tablets was higher than most other brands’. The differences were statistically significant, which is why Microsoft doesn’t meet CR’s standards for recommended products. The surveys are conducted annually. Microsoft defended the reliability of its laptops and tablets.
Originally, multiple Microsoft Surface products performed well in Consumer Reports' lab testing, but as time has passed and as more data has been collected a reliability issue has come into question. Survey respondents mentioned device startup problems, unexpected freezes or shut downs, and unresponsive touch screens, all occurring well after the devices were purchased.

Microsoft gave the following statement in an email sent to Consumer Reports regarding the new findings:
“Microsoft’s real-world return and support rates for past models differ significantly from Consumer Reports’ breakage predictability,” Microsoft said in an emailed statement. “We don’t believe these findings accurately reflect Surface owners’ true experiences or capture the performance and reliability improvements made with every Surface generation.”
Late last year, Apple was also denied a purchase recommendation from Consumer Reports for its 2016 MacBook Pro, because battery life "varied dramatically" from one trial to another. After working together to find the source of the issue at hand -- which Apple discovered to be a bug in a hidden Safari setting -- Consumer Reports reversed its decision and gave the new MacBook Pro devices a recommended purchase status.


Discuss this article in our forums

Consumer Reports Rates Galaxy S8 Over iPhone 7 as ‘iPhone 8’ Rumored to Address Most Shortcomings

The newest smartphone ratings from Consumer Reports have been shared this week, and the Samsung Galaxy S8 and Galaxy S8+ have beaten out its smartphone competitors, including Apple's iPhone 7 and iPhone 7 Plus, in categories related to design, battery life, camera, and more. Apple's current generation of iPhone (launched September 2016) is older than the Galaxy S8 family (launched April 2017), so it makes sense that Consumer Reports' ratings would end up favoring the newer devices.

Still, this week's report has some interesting points to make about why the Galaxy S8+ in particular received top marks, and how the upcoming "iPhone 8," as well as the "iPhone 7s" and "iPhone 7s Plus," might address the iPhone 7's shortcomings. Namely, Consumer Reports points out that the Galaxy S8 devices "have no bezels on the side," and only limited bars at the top and bottom. The testers particularly liked the S8's 5.8-inch screen.

The look of the S8 and S8+ is minimalist, modern, and elegant—and the design allows for a bigger screen in the same-size device.

Those numbers may not sound terribly different, but when you hold either phone in your hand, it feels novel: easy to grasp even if you have a small grip, but with lots of screen real estate. The S8 is 5.8 inches diagonally (that's the way screens are measured), while the S8+ is 6.2 inches.
Richard Fisco, Consumer Reports' lead phone tester, said that the S8 is comfortable to hold, while pointing out that one-handed operation becomes difficult on the devices, particularly the S8+'s 6.2-inch display. Even though the fingerprint scanner on the back of the S8 worked well, the report also described it as "awkwardly placed," explaining that they had to constantly poke around to find it and normally smudged the camera lens while doing so.

Following last year's Galaxy S7 battery-related issues, Consumer Reports now described the newest Galaxy smartphones as having "some of the best smartphone battery life we've seen," throughout a range of tests related to Wi-Fi, cellular, and talk-time usage. The testers explained that the beefier battery life on the S8+ is the main reason why the larger phone ranks above the smaller S8.

In terms of the camera, Consumer Reports said that the S8's photo-taking abilities produce colors that are rich even in low-light environments, and particularly excels in reducing the over-sharpening produced by the camera. The site also called out the iPhone 7 in this section, and its perceived advantage over the S8 in this category for having a dual rear-facing camera system. Despite the S8 and S8+ lacking this kind of set-up, Consumer Reports doesn't think the Samsung phones are at a disadvantage at all.
Finally, a few top-end cameras, including the iPhone 7 Plus and the LG G6, have dual rear-facing cameras, to enhance either zoom or wide-angle photography. The Samsung phones haven't gone that route yet—and we don't think they suffer for it.
Consumer Reports pointed towards water resistance as an advantage of the S8 line -- up to at least 5 feet of water for around 30 minutes -- while one major disappointment was found in Samsung's smart assistant Bixby. The AI helper hasn't rolled out wide yet, so it makes sense that the early version tested might have been unreliable, with the testers saying that Bixby "just isn't there yet."

Although the Galaxy S8 and S8+ might have won over iPhone 7 and iPhone 7 Plus in these ratings, essentially all of the categories are related to improvements that Apple is rumored to be adding into the 2017 line of iPhones. For Apple's tablet devices, a reduction of bezels and bigger display area have already been seen in the newest iPad Pros that just launched last week during WWDC.

The iPhone 8 specifically is predicted to include greatly reduced bezels, likely with a bar across the top to hold the front-facing camera and other optical sensors, a 5.8-inch OLED screen in a form factor close to the size of the current iPhone 7, Touch ID under the display, a stacked logic board design that will support longer battery life, improved vertically-aligned dual cameras with advanced AR abilities, IP68 waterproofing to rival the Galaxy S8, and an "enhanced Siri."

One of the cons related to the Galaxy S8 and S8+ is the price of Samsung's latest smartphones, starting at around $750 and $850 for 64GB carrier unlocked models, respectively. The iPhone 8 isn't expected to beat Samsung in this category, with rumors pointing towards a premium-priced iPhone that could cost more than $1,000. Some analysts disagree with that price and suggest an iPhone 8 that will have competitive pricing with the Galaxy S8+, potentially selling a 64GB iPhone 8 for $850 to $900 and a higher-end 256GB model for $950 to $1,000.

Related Roundup: iPhone 7
Tags: Samsung, Consumer Reports

Discuss this article in our forums

Consumer Reports Reverses Course, Recommends New MacBook Pro After Repeat Testing

Consumer Reports is out with an updated report on the 2016 MacBook Pro, and following retesting, the magazine is now recommending Apple's latest notebooks.

In the new test, conducted running a beta version of macOS that fixes the Safari-related bug that caused erratic battery life in the original test, all three MacBook Pro models "performed well."


The 13-inch model without a Touch Bar had an average battery life of 18.75 hours, the 13-inch model with a Touch Bar lasted for 15.25 hours on average, and the 15-inch MacBook Pro with Touch Bar had an average battery life of 17.25 hours.
Now that we've factored in the new battery-life measurements, the laptops' overall scores have risen, and all three machines now fall well within the recommended range in Consumer Reports ratings.
Consumer Reports originally denied the 2016 MacBook Pro a purchase recommendation in late December due to extreme battery life variance that didn't match up with Apple's 10 hour battery life claim.

Apple worked with Consumer Reports to figure out why the magazine encountered battery life issues, which led to the discovery of an obscure Safari caching bug. Consumer Reports used a developer setting to turn off Safari caching, triggering an "obscure and intermittent bug reloading icons" that drained excessive battery.

The bug, fixed by Apple in macOS Sierra 10.12.3 beta 3, is not one the average user will encounter as most people don't turn off the Safari caching option, but it's something done in all Consumer Reports tests to ensure uniform testing conditions. A fix for the issue will be available to the general public when macOS Sierra 10.12.3 is released, but users can get it now by signing up for Apple's beta testing program.

Each of the three 2016 MacBook Pro models, including the 13-inch MacBook Pro without Touch Bar, and the 13 and 15-inch MacBook Pro models with Touch Bars, are advertised as achieving 10 hours of battery life on a single charge when watching iTunes movies or browsing the web.

Real life Battery usage can vary significantly, however, based on factors like screen brightness and the applications being used.

Related Roundup: MacBook Pro
Tag: Consumer Reports
Buyer's Guide: MacBook Pro (Buy Now)

Discuss this article in our forums

Consumer Reports Retesting MacBook Pro Battery Life After Apple Says Safari Bug to Blame

Last month, the new MacBook Pro did not receive a purchase recommendation from Consumer Reports due to battery life issues that it encountered during testing. Apple subsequently said it was working with Consumer Reports to understand the results, which it noted do not match its "extensive lab tests or field data."

2016_macbook_pro_lineup
Apple has since concluded its work and learned that Consumer Reports was using a "hidden Safari setting" which trigged an "obscure and intermittent bug" that led to inconsistent battery life results. With "normal user settings" enabled, Consumer Reports said it "consistently" achieved expected battery life.

Apple's full statement was shared with iMore and other publications:
"We appreciate the opportunity to work with Consumer Reports over the holidays to understand their battery test results. We learned that when testing battery life on Mac notebooks, Consumer Reports uses a hidden Safari setting for developing web sites which turns off the browser cache. This is not a setting used by customers and does not reflect real-world usage. Their use of this developer setting also triggered an obscure and intermittent bug reloading icons which created inconsistent results in their lab. After we asked Consumer Reports to run the same test using normal user settings, they told us their MacBook Pro systems consistently delivered the expected battery life. We have also fixed the bug uncovered in this test. This is the best pro notebook we've ever made, we respect Consumer Reports and we're glad they decided to revisit their findings on the MacBook Pro."
Apple said it has fixed the Safari bug in the latest macOS Sierra beta seeded to developers and public testers this week.


Consumer Reports has issued its own statement on the matter to explain why it turns off Safari caching during its testing and other details:
We also turn off the local caching of web pages. In our tests, we want the computer to load each web page as if it were new content from the internet, rather than resurrecting the data from its local drive. This allows us to collect consistent results across the testing of many laptops, and it also puts batteries through a tougher workout.

According to Apple, this last part of our testing is what triggered a bug in the company’s Safari browser. Indeed, when we turned the caching function back on as part of the research we did after publishing our initial findings, the three MacBooks we’d originally tested had consistently high battery life results.
The non-profit organization also acknowledged user reports of poor battery life that have surfaced over the past three months.

Consumer Reports said it will complete its retesting of MacBook Pro battery life and report back with its update and findings when finished.

Apple advertises that the latest MacBook Pro models get up to 10 hours of battery life on a single charge when watching iTunes movies or browsing the web. This estimate can be affected by several factors, such as screen brightness, which applications are running, and other system processes.

Related Roundup: MacBook Pro
Tags: Safari, Consumer Reports
Buyer's Guide: MacBook Pro (Buy Now)

Discuss this article in our forums