techie / OMR Public

techie / OMR

This is your Edge Impulse project. From here you acquire new training data, design impulses and train models.

About this project

Optical Mark Recognition (OMR) with FOMO

OMR_DEVICE.jpg
Sequence 01.00_02_06_26.Still002.jpg

Story

Are you fed up with slow and expensive OMR readers? We've got your back. FOMR is a fast, reliable, and cheap OMR reader that can read your OMR sheet faster than the blink of your eye.

FOMO (Faster Objects, More Objects) is a novel machine learning algorithm that brings object detection to highly constrained devices. So let's use FOMO to build a cheap, faster, and flexible OMR reader.

Why FOMO?

FOMO is apt in our scenario for these reasons,

  1. FOMO works better if the objects have a similar size, and in our case, all the marked dots have a very similar size
  2. FOMO is designed to be small and amazingly fast, so we can use a microcontroller with limited processing capacity, which makes the device cheap and faster

How Does It Work?

To make the FOMR flexible, we use a marked answer key to identify the answers. Those answers are compared with other answer sheets to find the marks in other answer sheets.

To read the marks,

  1. Place your answer key and read the answers.
  2. Identify the position of answers in terms of co-ordinates using FOMO
  3. Now place the answer sheet and read the marked options
  4. Identify the position of markings in terms of co-ordinates
  5. Compare the position of each marking to the postion of answers in the answer key
  6. For each correct detection, add the marks
  7. Print the final marks

Answer_Key.jpg

Answer_Key_Coordinates.jpg

Answer_Sheet.jpg

Answer_Sheet_Coordinates.jpg

Final_Marks.jpg

Model Generation

Every machine learning model needs data, for acquiring data we stretched to the data acquisition tab. For data sampling , we used a Raspberry pi with the camera module. Actually here we are going to make a object detection model that can recognise black dot in the sheets.For getting more accuracy ,we also collected the individual dots(zoomed image) in addition to the groups and also we tried different orinetations.So we sampled 70 images(consist of 375 items) and is split in between test and training set.

Data.png

After the data acquisition, we started labelling the collected data.The fact is that here we have only one class in this model.The data acquisition is over, it's time for feature extraction.To generate features, we first need to create an impulse. For this project we have been using 96x96 images and resized by means of "Fit to shortest axis". Then we added a processing block and learning block to complete the impulse.

impulse.png

We used gray scale image as the color depth for configuring the processing block and we think the FOMO won't work on RGB data.
grayscale.png
Then we generated the features for the data. The feature explorer window is shown below.

feature_explorer.png

With all data processed it's time to start training our FOMO model . The model will take an image as input and output objects detected using centroids. For our case, it will show centroids of dots detected on the mark sheets.These are our neural network settings.

neural network settings_.png

We have trained the FOMO object detection model with a accuracy of 87%. The below confusion matrix shows our model is Good Fit.

confusion matrix.png

With the model trained let's try it out on some test data. When collecting the data we split the data up between a training and a testing dataset. The model was trained only on the training data, and thus we can use the data in the testing dataset to validate how well the model will work in the real world. This will help us ensure the model has not learned to overfit the training data, which is a common occurrence. To validate our model, we will go to Model testing and select Classify all.
The results are surpising !! We hit 100% accuracy on real world data which shows that it is a functional model.

model_testing.png

Hardware

The Raspberry Pi 3B is the powerful development of the extremely successful credit card-sized computer system. The brain of the system is Raspberry Pi. All major processes are carried out by this device.

rpi3_3_cnkiu6fD0V.jpeg

To set this device up in Edge Impulse, run the following commands:
sudo apt update sudo apt upgrade curl -sL https://deb.nodesource.com/setup_12.x | sudo bash - sudo apt install -y gcc g++ make build-essential nodejs sox gstreamer1.0-tools gstreamer1.0-plugins-good gstreamer1.0-plugins-base gstreamer1.0-plugins-base-apps npm config set user root && sudo npm install edge-impulse-linux -g --unsafe-perm

Also we enabled the camera by running sudo raspi-config which is actually a prime thing to do.
Here I am using the REES52 5 Megapixel 160° degrees Wide Angle Fish-Eye Camera for the object detection. Due to its high viewing angle, it can cover more area than the normal camera module.

camera_module_VK6TOIfB2N.jpg

We can connect the raspberry pi to the edge impulse by running this command edge-impulse-linux. In this way we collected the data for the model generation.

Then we used Linux sdk for running the model in the raspberry pi. For running the linux sdk put these commands in the terminal.
sudo apt-get install libatlas-base-dev libportaudio0 libportaudio2 libportaudiocpp0 portaudio19-dev pip3 install edge_impulse_linux -i https://pypi.python.org/simple
For powering the Raspberry Pi we used 5V 2A adapter.

Device Setup

OMR2.jpg

For setting up the OMR reader, we repurposed the old case used in AutoBill project. That's actually suits very well for this project. For proper reading of the mark sheets we fixed the camera module on the top side. We also marked the space for the mark sheets underneath the camera module. All the sheets would be placed in the markings, other wise the logic won't work as expected. In this project we made a custom size OMR sheets which makes the object detection easier.

device3.jpg
device4.jpg

Working Demo

Click to Play

Code

All the source files are availabe in the github repository. Grab it from here

Creating your first impulse (100% complete)

Acquire data

Every Machine Learning project starts with data. You can capture data from a development board or your phone, or import data you already collected.

Design an impulse

Teach the model to interpret previously unseen data, based on historical data. Use this to categorize new data, or to find anomalies in sensor readings.

Deploy

Package the complete impulse up, from signal processing code to trained model, and deploy it on your device. This ensures that the impulse runs with low latency and without requiring a network connection.

Download block output

Title Type Size
Image training data NPY file 50 windows
Image training labels JSON file 50 windows
Image testing data NPY file 10 windows
Image testing labels JSON file 10 windows
Object detection model TensorFlow Lite (float32) 48 KB
Object detection model TensorFlow Lite (int8 quantized) 40 KB
Object detection model TensorFlow SavedModel 149 KB
Object detection model Keras h5 model 52 KB

Clone project

You are viewing a public Edge Impulse project. Clone this project to add data or make changes.

Summary

Data collected
60 items

Project info

Project ID 116893
Project version 1