Optical Mark Recognition (OMR) with FOMO

Story

Are you fed up with slow and expensive OMR readers? We've got your back. FOMR is a fast, reliable, and cheap OMR reader that can read your OMR sheet faster than the blink of your eye.

FOMO (Faster Objects, More Objects) is a novel machine learning algorithm that brings object detection to highly constrained devices. So let's use FOMO to build a cheap, faster, and flexible OMR reader.

Why FOMO?

FOMO is apt in our scenario for these reasons,

1. FOMO works better if the objects have a similar size, and in our case, all the marked dots have a very similar size
2. FOMO is designed to be small and amazingly fast, so we can use a microcontroller with limited processing capacity, which makes the device cheap and faster

How Does It Work?

To make the FOMR flexible, we use a marked answer key to identify the answers. Those answers are compared with other answer sheets to find the marks in other answer sheets.

To read the marks,

1. Place your answer key and read the answers.
2. Identify the position of answers in terms of co-ordinates using FOMO
3. Now place the answer sheet and read the marked options
4. Identify the position of markings in terms of co-ordinates
5. Compare the position of each marking to the postion of answers in the answer key
6. For each correct detection, add the marks
7. Print the final marks

Model Generation

Every machine learning model needs data, for acquiring data we stretched to the data acquisition tab. For data sampling , we used a Raspberry pi with the camera module. Actually here we are going to make a object detection model that can recognise black dot in the sheets.For getting more accuracy ,we also collected the individual dots(zoomed image) in addition to the groups and also we tried different orinetations.So we sampled 70 images(consist of 375 items) and is split in between test and training set.

After the data acquisition, we started labelling the collected data.The fact is that here we have only one class in this model.The data acquisition is over, it's time for feature extraction.To generate features, we first need to create an impulse. For this project we have been using 96x96 images and resized by means of "Fit to shortest axis". Then we added a processing block and learning block to complete the impulse.

We used gray scale image as the color depth for configuring the processing block and we think the FOMO won't work on RGB data.

Then we generated the features for the data. The feature explorer window is shown below.

With all data processed it's time to start training our FOMO model . The model will take an image as input and output objects detected using centroids. For our case, it will show centroids of dots detected on the mark sheets.These are our neural network settings.

We have trained the FOMO object detection model with a accuracy of 87%. The below confusion matrix shows our model is Good Fit.

With the model trained let's try it out on some test data. When collecting the data we split the data up between a training and a testing dataset. The model was trained only on the training data, and thus we can use the data in the testing dataset to validate how well the model will work in the real world. This will help us ensure the model has not learned to overfit the training data, which is a common occurrence. To validate our model, we will go to Model testing and select Classify all.
The results are surpising !! We hit 100% accuracy on real world data which shows that it is a functional model.

Hardware

The Raspberry Pi 3B is the powerful development of the extremely successful credit card-sized computer system. The brain of the system is Raspberry Pi. All major processes are carried out by this device.

To set this device up in Edge Impulse, run the following commands:
sudo apt update sudo apt upgrade curl -sL https://deb.nodesource.com/setup_12.x | sudo bash - sudo apt install -y gcc g++ make build-essential nodejs sox gstreamer1.0-tools gstreamer1.0-plugins-good gstreamer1.0-plugins-base gstreamer1.0-plugins-base-apps npm config set user root && sudo npm install edge-impulse-linux -g --unsafe-perm

Also we enabled the camera by running sudo raspi-config which is actually a prime thing to do.
Here I am using the REES52 5 Megapixel 160° degrees Wide Angle Fish-Eye Camera for the object detection. Due to its high viewing angle, it can cover more area than the normal camera module.

We can connect the raspberry pi to the edge impulse by running this command edge-impulse-linux. In this way we collected the data for the model generation.

Then we used Linux sdk for running the model in the raspberry pi. For running the linux sdk put these commands in the terminal.
sudo apt-get install libatlas-base-dev libportaudio0 libportaudio2 libportaudiocpp0 portaudio19-dev pip3 install edge_impulse_linux -i https://pypi.python.org/simple
For powering the Raspberry Pi we used 5V 2A adapter.

Device Setup

For setting up the OMR reader, we repurposed the old case used in AutoBill project. That's actually suits very well for this project. For proper reading of the mark sheets we fixed the camera module on the top side. We also marked the space for the mark sheets underneath the camera module. All the sheets would be placed in the markings, other wise the logic won't work as expected. In this project we made a custom size OMR sheets which makes the object detection easier.

Working Demo

Code

All the source files are availabe in the github repository. Grab it from here