Nekhil R / Vandalism Detection Public

Nekhil R / Vandalism Detection

This is your Edge Impulse project. From here you acquire new training data, design impulses and train models.

Audio

About this project

IMG_2684.jpg

Story

The direct annual cost of vandalism runs in billions of dollars annually in the United States only. Breaking glass and defacing properties are some of the serious forms of vandalism. Conventional security techniques such as direct lighting and intruder alarms are ineffective in many locations and cases. So by means of this project, we are able to detect the sound of glass breaking and can alert the user instantly about the event. In this project, we mainly focus on vandal activity such as glass breaking, however, this project can be applied to any vandal activity that produces sounds.

How Does It Work

The device will work as follows, suppose a vandal tried to break the glass then the tinyML algorithm running in the device can recognise the event using a microphone unit. Then the device will give mail notifications to the registered users regarding the vandalism detection. Visual Studio Code.lnk.jpg

Hardware

Arduino Nano 33 BLE Sense

Nano 33 BLE sense.jpg For this project we are using, Arduino nano 33 ble sense. It's a 3.3V AI-enabled board in the smallest available form factor. It comes with a series of embedded sensors including MP34DT05 (Digital Microphone).

ESP -01

Esp 01.jpg The ESP-01 is used for adding the wifi capability for the device because the Arduino nano 33 ble sense has only Bluetooth. Actually, the wifi is used for sending mail alerts. The serial communication is established between the Arduino and esp 01 for giving the mail alert.

Software

Data acquistion

One of the most important parts of any machine learning model is its dataset. The edge impulse offers us two options to create our dataset either direct uploading the file or recording via the device itself. Here we proceeded with recording the device itself because the first reason is that it's just a prototype so the data will be limited and the second reason is that it produces more accuracy. For connecting the BLE sense to the edge impulse please read this tutorial.
In this scenario, we have only two classes Glass break and Noise. Glass breaking sounds that we have used are from the vivid online resources and the major noise datasets are from this Microsoft Scalable Noisy Speech Dataset (MS-SNSD). We also included the natural noise in the room apart from MS-SNSD.
The sound recording was done for 20 seconds at 16KHz sampling rate. Something to keep in mind is that you must keep the sampling rate the same between your training dataset and your deployment device. When you are planning to use the 44.1Khz sound, you should need to downsample it to the 16KHz(when you are going to deploy in Arduino).
We collected around 10 minutes of data and split it between training and test set. In the training data we split the sample to 2s otherwise, the inferencing will fail because the BLE sense has limited memory to handle the data. data_acquistion.jpg

Impulse Design

This is our Impulse. Impulse is actually the machine learning pipeline termed by the Edge Impulse. Impulse Design.jpg
Here we used MFE as the processing block ,because it is very suitable for non-human voices. We have used default parameters of the MFE block.

Neural Network

These are our Neural Network settings which is most suitable for our data.If your are tinkering with your own dataset you should need to change this parameters if required. Neural Network settings.jpg We enabled the Data augmentation which helps us to randomnly transform data during training. So by means of this we are able to run more training cycles without overfitting and also it leads to the improved accuracy. This is our Neural Network architecture. NNA.jpg We have used the default 1D convolutional. Then we trained the model and we got 97% accuracy,which is very awesome. By looking the confusion matrix it is clear that there is no sign of underfitting and overfitting. Model training output.jpg

Model Testing

Before deploying the model it's a good practice to run the inference on the test dataset. In the model testing, we got around 92% accuracy. Model testing.jpg
Let's look into some of the misclassifications,
In this case, the noise very well resembled the Glass break that's why it is misclassified. misclassification1.jpg In this case too, the model performed very well in classifying the data, the data contains both the Glass_Break and the noise but the majority of the data was noise, that's why its misclassified. mis2.jpg In these two cases(below shown) also the noise was major.
mis3.jpg
mis4.jpg
The model is performing very well and can be deployed it in the real world.

Deployment

For deploying the impulse to BLE Sense,we created a arduino library with this optimizations.
deploy.jpg Then we add that library to the Arduino IDE. Then we modified the example sketch that is provided. You can find entire code and assets including circuit in this Github repo.

IFTT

IFTTT.png
For triggering the mail we used IFTT service. To setup the mail triggering upon any poitive detections please have a look at this tutorial.
This is the applet created by me. applet.jpg

Case

All the components were fitted inside this case. IMG_2680.jpg

Real world Testing

Let's test the model after deploying it on the BLE sense board. You can check it out in the below video. We have actually played the Glass breaking sound via a large speaker.

Creating your first impulse (100% complete)

Acquire data

Every Machine Learning project starts with data. You can capture data from a development board or your phone, or import data you already collected.

Design an impulse

Teach the model to interpret previously unseen data, based on historical data. Use this to categorize new data, or to find anomalies in sensor readings.

Deploy

Package the complete impulse up, from signal processing code to trained model, and deploy it on your device. This ensures that the impulse runs with low latency and without requiring a network connection.

Download block output

Title Type Size
MFE training data NPY file 240 windows
MFE training labels NPY file 240 windows
MFE testing data NPY file 138 windows
MFE testing labels NPY file 138 windows
NN Classifier model TensorFlow Lite (float32) 15 KB
NN Classifier model TensorFlow Lite (int8 quantized) 8 KB
NN Classifier model TensorFlow SavedModel 19 KB
NN Classifier model Keras h5 model 14 KB

Clone project

You are viewing a public Edge Impulse project. Clone this project to add data or make changes.

Summary

Data collected
9m 60s

Project info

Project ID 149095
Project version 1
License Apache 2.0