Nekhil R / Overload Alert in Elevators with FOMO Public

Nekhil R / Overload Alert in Elevators with FOMO

This is your Edge Impulse project. From here you acquire new training data, design impulses and train models.

Object detection

About this project



As elevators become more and more indispensable in people's lives, safety has become more of a concern. Overloading is an important cause of elevator accidents. The existing elevators use weighing sensors to find the load in elevators, they may sometimes fail. And their maintenance is too expensive. To avoid such accidents, we are going to design a device which can count passengers in real-time at high speed and can give an alert if the no of passengers is above the threshold.
This device can be attached anywhere in the elevators. In comparison with existing technology, its implementation cost is too low, and maintenance is easy.

How does it work?

In this prototype, we only consider two floors, the ground floor and 1st floor. After entering all the passengers in the elevators, anyone needs to press the close button. Then we count the number of passengers, if it is above the threshold the device will make an alarm. Otherwise, the elevator will move on. The threshold passenger limit can be set by the user. Check out the Demo Video

In addition to the overload alert, we also provide elevator statistics. This means that the device can upload the count in elevators with the specific time stamp to the excel software. The count will be always updated after pressing the close button.
One of the interesting aspects of this data is that it can be easily visualised utilizing any graphs or charts. In this way, it can be useful for any person who analyses elevator usage. This is the count coming from the device and it is logged in CSV format(opened in Microsoft excel).
excel_data2.jpg Below are the various graphs generated for the same data.

Clustered Column

clustered column.jpg

Line chart


Pie chart

Consider the case of elevators in a shopping mall, so by using these statistics, the mall owner can easily add up elevators, if the usage of the elevator is too high and can also remove the elevator if usage is too low.

Nicla Vision

In this project we are using Nicla Vision a tiny AI board from arduino. It features a 2MP color camera and has the intelligence to process and extract useful information from anything it sees.

Data collection and Labelling

For the purpose of the data collection, we mounted the board on a tripod and connected it to a laptop using a lengthy USB cable. The below image shows the data acquistion setup.
IMG_2163.jpg The whole setup was on one end of the room and we actullay stood on the opposite end. So our micro controller unit can easily pickup the passengers. You can follow this tutorial to connect Nicla vision to the Edge Impulse.
We captured 73 images and split it between testing and training. The images will contain only one person or two person at a time. Then we labelled each images one by one and here we have only one class named "people". The orientation of the nicla vision provides inverted images and that's not a problem at all.

Impulse Design

This is the machine learning pipeline for this project
Impulse.jpg We choose the image width and height as 96x96 and the resize mode to "Fit the shortest axis". After saving the impulse we moved onto the image tab and chosen "Grayscale" as the colour depth and saved the parameters and generated features for our images. The image below shows the generated features.

Model Training

This is our Neural network training settings and architecture for generating the model.
NN and architecture.jpg We only changed the training cycle from 60 to 70. Further increasing of training cycle or learning rate can overfit the data, so we sticked on to this.
Add image of training settings. As the Neural network architecture we used FOMO (MobileNet V2 0.35). The results are actually surprising. We got around 96% accuracy for the model(used quantized int version). Model output.jpg

Testing Model

It's time for testing the model. First we tested the test data which we seperated earlier and We got around 84 % accuracy. That seeems to be fine.
testing.jpg Now let's move onto the Live classification. So we are testing 3 sample images that captured from Nicla vision and let's see hoe our model perfomrms.

Test 1


Test 2


Test 3

test3.jpg In all our testing samples, the model performed very well.


Now we have our ML model, we need to deploy it to our Nicla vision. We just created a arduino library by pressing the building button, so a zip file will be downloaded. deploy.jpg Then we added that library to the Arduino. Then we modified the example sketch to get the project done. You can find code and assets in this github repo.

Additional Hardware and Casing

In addition to the Nicla Vision we used a buzzer and a LED to make alarm.
IMG_2150.jpg But the output current of Nicla(4.7 mA) is not enough to properly power up the LED and buzzer. So we used a 2N222A transistor to drive theee devices.
So we used an external power supply of 5V inaddition to the USB power supply for powering the Nicla vision itself. A push button is also used to check whether the door is closed or not. Finally We made a nice tiny case for this device.
IMG_2174.jpg Then we inserted each components into it...
IMG_2213.jpg Our device is ready to implement.


tera term.jpg We used this software to stream data from the Nicla vision. The streaming data can be logged anywhere in any format.Here we proceeded with CSV(comma seperated value) format, so the file can be easily opened with Micorsoft excel.
tera_term log.jpg
Make sure to tick the timestamp before logging the data. The below image shows a sample data streamed from our device which is actually opened n excel. original_excel.jpg
We can easily generate graphical reports from this data by selecting that text. The below figure represents the line chart for the above data. countwithgraph.jpg There are wide variety of options available and they are shown below. options_available.jpg


This device can be easily integrated with any elevators,so the elevator will only starts off when the passengers are in the permissible range. For reducing the cost we can use ESP32 Eye like microcontroller unit instead of the Nicla Vision.

Download block output

Title Type Size
Image training data NPY file 45 windows
Image training labels JSON file 45 windows
Image testing data NPY file 16 windows
Image testing labels JSON file 16 windows
Object detection model TensorFlow Lite (float32) 47 KB
Object detection model TensorFlow Lite (int8 quantized) 40 KB
Object detection model TensorFlow SavedModel 149 KB
Object detection model Keras h5 model 51 KB

Clone project

You are viewing a public Edge Impulse project. Clone this project to add data or make changes.


Data collected
64 items

Project info

Project ID 109997
Project version 1
License No license attached