MLPerf Mobile

MLCommons

About this app

MLPerf Mobile is a free, open-source benchmarking tool designed to measure the performance of mobile devices like smartphones and tablets in various artificial intelligence (AI) and machine learning (ML) tasks. Workloads tested include image classification, language understanding, super resolution upscaling, and image generation based on text prompts. This benchmark makes use of hardware AI acceleration on many of the latest mobile devices to ensure the best performance where possible.

MLPerf Mobile is built and maintained by the MLPerf Mobile working group at MLCommons®, a non-profit AI/ML engineering consortium made up of 125+ members including industry firms and academics from a host of different institutions around the globe. MLCommons produces world-class benchmarks for AI training and inference across many system scales, from large data center installations to tiny embedded devices.

Features of MLPerf Mobile include:

- Benchmark tests across various domains based on state-of-the-art AI models, including:

- Image classification
- Object detection
- Image segmentation
- Language understanding
- Super resolution
- Image generation from text prompts

- Custom-tuned AI acceleration on the latest mobile devices and SoCs.

- Broad support for Android devices via TensorFlow Lite delegate fallback acceleration.

- Tailored test modes for everyone from casual users wanting a quick performance assessment to MLCommons members intending to submit official results for publication.

- Customizable cool-down delays between tests to avoid thermal throttling and ensure accurate results.

- Optional cloud-based results storage so you can save and access your past results from multiple devices in one place. (This feature is free of charge but requires account registration.)

MLPerf Mobile is typically updated multiple times each year with new tests and acceleration support as AI models and mobile hardware capabilities evolve. Please note that some benchmark tests may not be supported, and therefore may not show up as available for testing, on older devices.

Source code and documentation for the MLPerf Mobile app are available on the MLCommons Github repo. For user support or questions, please feel free to open issues in the app’s Github repo:

github.com/mlcommons/mobile_app_open

If you or your organization is interested in becoming an MLCommons member, please contact [e-mail] for more information.

What's new

-Adds support for Mediatek Dimensity 9400 SoCs.
-The 60-second cooldown time in quick mode is consistently initialized.
-Updated about, licensing, & privacy info.
-Various UI fixes & back-end improvements.
-This release should be broadly compatible. In testing, we found issues with the following devices:

Samsung:
Galaxy Tab A9 Plus
Galaxy Tab S8
Galaxy A52
Galaxy Tab S7
Galaxy S24 Ultra

Google:
Pixel 5

For support, please open an issue in the MLPerf Mobile GitHub repo.