Apple Finds Bug after Consumer Reports MacBook Pro Battery Test

Consumer Reports said in a blog post Tuesday that Apple has a fix for a bug uncovered in CR testing. According to statements from both CR and Apple, there was a bug in Safari that caused battery life issues on Apple’s new MacBook Pro with Touch Bar. That bug played a role in inconsistent battery performance in CR testing, results that had previously caused CR not to recommend the new MacBook Pros.

Apple has released a fix for that bug to its beta program, and said it will release it to the public in the coming weeks.

Apple’s MacBook Pro with Touch Bar

MacBook Pro with Touch Bar Battery Life

CR raised a ruckus by not recommending the new MacBook Pro models in part because of poor and inconsistent battery life. “In our tests of three different MacBook Pro models,” the magazine wrote, “we saw battery life results as long as 19.5 hours and as short as 3.75 hours.”

Many Apple fans prefer to dismiss Consumer Reports when the magazine bags on Apple, but CR does carry weight with many consumers. Apple apparently knows that, because the company reached out to CR in the wake of those tests.

The magazine said that Apple requested “multiple rounds of diagnostic data,” and analyzing that data revealed a bug.

Hidden Settings

In addition to the bug, CR uses a developer setting that disables caching when testing battery life. The magazine said it does this on all laptops from all manufacturers when testing batter life to remove variability in battery testing results.

It may produce more consistent results, but…well…that’s not how modern laptops work. macOS, Windows, and ChromeOS all make extensive use of caching to improve overall performance, including battery life. Turning off that caching is just not how consumers experience modern computing.

It’s hard to imagine why CR thinks this is a better way to test. To wit, “Many of these settings are set by default to extend battery life. That’s generally a good thing. But because these settings are so variable and situation-dependent, we turn several of them off during testing.”

Again. That’s just kind of weird to me, but you can read more about it in the blog post on this topic. I should also note that said testing decision has been applied to all manufacturers, and that Apple isn’t being singled out.

Apple Statement

For its part, Apple said in a statement:

We appreciate the opportunity to work with Consumer Reports over the holidays to understand their battery test results. We learned that when testing battery life on Mac notebooks, Consumer Reports uses a hidden Safari setting for developing web sites which turns off the browser cache…. We have also fixed the bug uncovered in this test.

Note the subtle mention of the bug slipped in at the end.

Apple’s statement also said:

This is not a setting used by customers and does not reflect real-world usage. [Consumer Reports‘] use of this developer setting also triggered an obscure and intermittent bug reloading icons which created inconsistent results in their lab. After we asked Consumer Reports to run the same test using normal user settings, they told us their MacBook Pro systems consistently delivered the expected battery life…. This is the best pro notebook we’ve ever made, we respect Consumer Reports and we’re glad they decided to revisit their findings on the MacBook Pro.

That Fix

The reality, though, is that battery performance has been an issue on the new MacBook Pro models. Apple has pushed this fix out to its beta program, but it remains to be seen if it will improve performance for a large number of users.

Share this post