Text Recognition in Flutter Using Firebase’s ML Kit

It’s becoming increasingly easy to incorporate machine learning into your mobile app development toolkit, courtesy of platforms and dev tools such as Firebase’s ML Kit.

These tools conceal technical complexities by packaging pre-trained machine learning models for use. They also allow you to train and import your own models.

In this fifth installment of our Flutter series, we’ll look at how you can implement text recognition in Flutter using ML Kit. The application will be able to recognize text in images as well as handwritten text.

You can check out the earlier posts in this series below:

Application and Use Cases

Firebase’s ML Kit enables you to you can recognize text in any Latin-based language. It can also detect multiple languages in a single image. Implementing text recognition into your application can automate tedious data entry tasks for receipts, credit cards, business cards — just to mention a few.

Add Firebase to Flutter

The first step involves adding Firebase to your Flutter project. This is done by creating a Firebase project and registering your app. Follow the comprehensive guide below to set that up:

Install Required Packages

Now that your app’s communication with Firebase is set up, you have to install two packages. The first is image_picker and the second is firebase_ml_vision. We’ll use the image_picker to pick the images from the phone’s gallery or take a new picture.

firebase_ml_vision will provide the functionality for recognizing the text.

Select Image

At this point, we can now select the image that contains the text. We can choose an image either from the gallery or camera. We check if the image has been loaded using imageLoaded. If the image has been loaded, we display it and read the contents of the image. We also initialize an empty text string that will contain the contents of the image:

File pickedImage;
File pickedImage;
var text = '';
bool imageLoaded = false;

var awaitImage = await ImagePicker.pickImage(source: ImageSource.gallery);

    setState(() {
      pickedImage = awaitImage;
      imageLoaded = true;
    });

If you’d rather use the camera, change the source to ImageSource.camera.

Create a FirebaseVisionImage

In this step, we create a FirebaseVisionImage object from the selected image.

Create an Instance of a Detector

The next step is to create an instance of the detector that we’d like to use. The available detectors are BarcodeDetector, ImageLabeler, FaceDetector, ImageLabeler, and TextRecognizer. In this case, we’re interested in the TextRecognizer.

Process the Image

We’re now ready to process the image using the detector. This will recognize the text in the image.

Extract the Text

With the image processed by the text recognizer, we can now extract the text. We obtain the full text by looping through the blocks and appending the words to the text variable.

The text recognizer is able to identify text from multiple languages. It works best if the characters are 16×16 pixels. Chinese, Japanese, and Korean text recognition is only supported when using the cloud-based API. For these languages, the characters should be 24×24 pixels. In order to obtain useful results, ensure that the image is clear and not blurred.

Finally, release the resources when done.

for (TextBlock block in visionText.blocks) {

      for (TextLine line in block.lines) {
        for (TextElement word in line.elements) {
          setState(() {
            text = text + word.text + ' ';
          });
        }
        text = text + 'n';
      }
    }
    textRecognizer.close();
  }

The source can be found below.

Final Thoughts

You can clearly see how fast and easy it is to incorporate machine learning in your mobile apps using Firebase’s ML Kit. Before launching this application into production ensure that you check with Firebase’s checklist to make sure you have the right configurations.

Should you choose to use the cloud-based API, ensure that you have the right API access rights. This will ensure that your application doesn’t fail at production.

Using the repo below you can try out different detectors.