A few weeks ago I looked at the speed of Core ML models running on various Apple devices. Apple’s new A12 Bionic chip paired with Core ML 2 made the latest generation of iPhones and iPads more than 10x faster than previous generations.
While faster is usually better, large differences in performance across devices make it hard for developers to keep user experience consistent. Apple’s (relatively) small product family and aggressive upgrade strategy mitigates this problem to some degree.
It’s not prohibitive to have a few devices around the office for testing. The same can’t be said for Android. Fragmentation in both hardware and software makes it infeasible to test apps on every combination of chipset and Android version.
I pulled data for TensorFlow Mobile models running on 70 different Android devices and compared their performance relative to the Google Pixel 2. Keep in mind this data is taken from devices being used in the wild. These aren’t laboratory benchmarks, so there might be some unintuitive results.
Some observations:
- Despite none of these devices having access to an ML accelerator like the Neural Engine in the A12 Bionic (TensorFlow Mobile is CPU-only and TensorFlow Lite is still too limited for most models), there is still a massive 10–20x speed difference between high and low end devices.
- The best performing device here, the Samsung Galaxy S9, is still 10X slower than the iPhone X and 100X slower than the new iPhone XS when it comes to on-device machine learning.
- Despite the fragmentation of the Android ecosystem, no manufacturer’s chips have pulled ahead. The flagship devices from every manufacturer perform at roughly the same level. I don’t think we’ll see big differentiation (>2X) until handset makers implement the Android Neural Networks API (NNAPI) for hardware acceleration.
Comments 0 Responses