Scan Barcodes on Android Using the Power of ML Kit and Fotoapparat

The uses of machine learning and computer vision features in mobile applications are increasing quickly; so much so that users—with some kinds of applications—expect mobile experiences to have embedded artificial intelligence.

Training and deploying ML models to the phone can take a considerable amount of time; you need to have in-depth knowledge of neural networks or model optimization to create a performant model—or hire ML engineers to do these task.

Fortunately, there are some libraries and packages such as Fritz AI and ML Kit that can save you a lot of time, and let you focus on your core app features.

In this article, we’ll focus on ML Kit, which is a mobile SDK that lends Google’s machine learning expertise to mobile, apps with pre-trained models for a variety of uses cases. One of them being barcode scanning.

We’ll also use a library called Fotoapparat to take images from the camera for processing.

ML Kit is built primarily for mobile devs, but if you are an experienced ML engineer, you can use your own TensorFlow Lite models in your mobile apps with the ML Kit APIs.

Firebase and ML Kit Changes 👨‍🔧

As of June 3, 2020, Google made some changes to ML Kit for Firebase to distinguish between the on-device models and cloud-based models.

So the previous ML Kit product is split into two separate products:

ML Kit, which contain all the on-device APIs
Firebase ML, which focuses on cloud-based and custom model deployment.

Project Setup 📐

To start, open up Android Studio and create a new project or open an existing one.

As mentioned, we need two libraries—ML Kit for scanning barcodes, and a Fotoapparat for fetching images from the camera.

Add the following dependencies, as shown in the code snippet below. Note that, at the time of writing, the last stable version of Fotoapparat is 2.7.0 and ML Kit barcode scanning is 16.0.1 :

dependencies {
    /* other dependencies
    .....*/
  
    //ml kit barcode
    implementation 'com.google.mlkit:barcode-scanning:16.0.1'

    // fotoapparat library
    implementation 'io.fotoapparat:fotoapparat:2.7.0'
}

Fotoapparat Setup 📷

We’ll use the Fotoapparat library to take images from the camera—you can also configure it to accept real-time frames to do other tasks if you want to.

Camera Permission

First, we need to get permission to use the camera—in AndroidManifest.xml add the following line:

Then in the desired activity, add the following two methods to verify and request permission:

private val CAMEREA_REQUEST_ID = 1144

private fun isCameraPermissionGranted(): Boolean {
    return (ContextCompat.checkSelfPermission(this, Manifest.permission.CAMERA)
            == PackageManager.PERMISSION_GRANTED)
}

private fun requestCameraPermission() {
    ActivityCompat.requestPermissions(
        this,
        arrayOf(Manifest.permission.CAMERA),
        CAMEREA_REQUEST_ID
    )
}

Initialize Fotoapparat

In the activity layout, add a CameraView view component to preview the camera in real-time:

<io.fotoapparat.view.CameraView
    android:id="@+id/cameraView"
    android:layout_width="match_parent"
    android:layout_height="400dp"
    app:layout_constraintEnd_toEndOf="parent"
    app:layout_constraintStart_toStartOf="parent"
    app:layout_constraintTop_toTopOf="parent" />

Then in the activity class, declare a variable to hold the Fotoapparat object instance. In the onCreate(), get a reference to the CameraView and use it to create the Fotoapparat object, like in the following bit of code:

private lateinit var fotoapparat: Fotoapparat

override fun onCreate(savedInstanceState: Bundle?) {
    super.onCreate(savedInstanceState)
    setContentView(R.layout.activity_main)
  
    fotoapparat = Fotoapparat.with(this)
            .into(cameraView)
            .previewScaleType(ScaleType.CenterCrop)
            .build()
}

Handling the Lifecycle

We want our camera to stop when the activity stops and to resume capturing when the activity is displayed. To achieve that, override the onStart() and onStop() methods and make Fotoapparat aware of the lifecycle changes:

override fun onStart() {
    super.onStart()
    fotoapparat.start()
}

override fun onStop() {
    super.onStop()
    fotoapparat.stop()
}

Take Images from the Camera

As the last step, we’re ready to take images from the camera. Fotoapparat makes it easy to get a bitmap ready on-demand with the takePicture() method. When this picture is available, we’ll perform scanning:

private fun takeImage() {
    fotoapparat.takePicture()
               .toBitmap()
               .whenAvailable { picture ->
                    scanImageForBarcode(picture!!)
                }
}

private fun scanImageForBarcode(picture: Bitmap){
    // handle and scan the picture for barcode
}

We’ll implement the scanImageForBarcode method shortly. Up next we will define the other side—the scanner.

Scanner Configuration 🔭

After we get the image, the next step is to define the barcode scanner, and which types of barcode the scanner will scan.

Here are the different supported barcode types in ML Kit:

Code 128 (FORMAT_CODE_128)
Code 39 (FORMAT_CODE_39)
Code 93 (FORMAT_CODE_93)
Codabar (FORMAT_CODABAR)
EAN-13 (FORMAT_EAN_13)
EAN-8 (FORMAT_EAN_8)
ITF (FORMAT_ITF)
UPC-A (FORMAT_UPC_A)
UPC-E (FORMAT_UPC_E)
QR Code (FORMAT_QR_CODE)
PDF417 (FORMAT_PDF417)
Aztec (FORMAT_AZTEC)
Data Matrix (FORMAT_DATA_MATRIX)

First, start by creating an instance of BarcodeScannerOption—to set up the different formats our scanner will accept, we create the BarcodeScanner using this options object:

private lateinit var barcodeScanner: BarcodeScanner
        
override fun onCreate(savedInstanceState: Bundle?) {
    super.onCreate(savedInstanceState)
    setContentView(R.layout.activity_main)

    val options = BarcodeScannerOptions.Builder()
                    .setBarcodeFormats(
                         Barcode.FORMAT_EAN_13,
                         Barcode.FORMAT_QR_CODE
                     ).build()

     barcodeScanner = BarcodeScanning.getClient(options)
}

Connect Everything ⚙️

Lastly, let’s implement the scanImageForBarcode method—here are the steps for this:

Create an InputImage from the resulting bitmap and the camera rotation—this object is required by the scanner.
Send the InputImage to the process() method offered by the BarcodeScanner to scan the image for any barcode format defined in the BarcodeScannerOptions .
Add success and failure listeners to monitor the result of the process task.
Handle the list of barcodes if everything goes well.

Here’s a code snippet:

private fun scanImageForBarcode(it: BitmapPhoto) {
      val inputImage = InputImage.fromBitmap(it.bitmap, it.rotationDegrees)
      val task = barcodeScanner.process(inputImage)
      task.addOnSuccessListener { barCodesList ->
          for (barcodeObject in barCodesList) {
              val barcodeValue = barcodeObject.rawValue
              Log.d("Barcode", "The code %s".format(barcodeValue))
          }
      }
      task.addOnFailureListener {
            Log.d("ERROR", "An Exception occurred", it)
      }
}

Here is the complete Gist of the implemented activity.

Advanced Fotoapparat Features

You can do many other things with Fotoapparat library:

You can select the exposure compensation value with exposureCompensation .
Turn the flashlight on and off.
Process real-time image frames with the frameProcessor callback.
Adjust the lens position and image quality.
Switch between cameras and set the zoom level.

You can learn more about these features in the official documentation linked below in the resources.

Resources 📚

Conclusion 🏁

Implementing barcode scanning with ML Kit and Fotoapparat is pretty straightforward using these libraries and APIs—you can save a lot of development time, and focus on your main application’s features.

I hope you enjoyed this Android article—if you have any questions, don’t hesitate to post them below or contact me on Twitter, LinkedIn, GitHub, and Facebook.

Don’t forget to clap 👏 and follow 📌if you like what you read. Here are a few of my other articles you can read, as well!