Nekhil R / Vandalism Detection Public

Nekhil R / Vandalism Detection

This is your Edge Impulse project. From here you acquire new training data, design impulses and train models.


About this project



The direct annual cost of vandalism runs in billions of dollars annually in the United States only. Breaking glass and defacing properties are some of the serious forms of vandalism. Conventional security techniques such as direct lighting and intruder alarms are ineffective in many locations and cases. So by means of this project, we are able to detect the sound of glass breaking and can alert the user instantly about the event. In this project, we mainly focus on vandal activity such as glass breaking, however, this project can be applied to any vandal activity that produces sounds.

How Does It Work

The device will work as follows, suppose a vandal tried to break the glass then the tinyML algorithm running in the device can recognise the event using a microphone unit. Then the device will give mail notifications to the registered users regarding the vandalism detection. Visual Studio Code.lnk.jpg


Arduino Nano 33 BLE Sense

Nano 33 BLE sense.jpg For this project we are using, Arduino nano 33 ble sense. It's a 3.3V AI-enabled board in the smallest available form factor. It comes with a series of embedded sensors including MP34DT05 (Digital Microphone).

ESP -01

Esp 01.jpg The ESP-01 is used for adding the wifi capability for the device because the Arduino nano 33 ble sense has only Bluetooth. Actually, the wifi is used for sending mail alerts. The serial communication is established between the Arduino and esp 01 for giving the mail alert.


Data acquistion

One of the most important parts of any machine learning model is its dataset. The edge impulse offers us two options to create our dataset either direct uploading the file or recording via the device itself. Here we proceeded with recording the device itself because the first reason is that it's just a prototype so the data will be limited and the second reason is that it produces more accuracy. For connecting the BLE sense to the edge impulse please read this tutorial.
In this scenario, we have only two classes Glass break and Noise. Glass breaking sounds that we have used are from the vivid online resources and the major noise datasets are from this Microsoft Scalable Noisy Speech Dataset (MS-SNSD). We also included the natural noise in the room apart from MS-SNSD.
The sound recording was done for 20 seconds at 16KHz sampling rate. Something to keep in mind is that you must keep the sampling rate the same between your training dataset and your deployment device. When you are planning to use the 44.1Khz sound, you should need to downsample it to the 16KHz(when you are going to deploy in Arduino).
We collected around 10 minutes of data and split it between training and test set. In the training data we split the sample to 2s otherwise, the inferencing will fail because the BLE sense has limited memory to handle the data. data_acquistion.jpg

Impulse Design

This is our Impulse. Impulse is actually the machine learning pipeline termed by the Edge Impulse. Impulse Design.jpg
Here we used MFE as the processing block ,because it is very suitable for non-human voices. We have used default parameters of the MFE block.

Neural Network

These are our Neural Network settings which is most suitable for our data.If your are tinkering with your own dataset you should need to change this parameters if required. Neural Network settings.jpg We enabled the Data augmentation which helps us to randomnly transform data during training. So by means of this we are able to run more training cycles without overfitting and also it leads to the improved accuracy. This is our Neural Network architecture. NNA.jpg We have used the default 1D convolutional. Then we trained the model and we got 97% accuracy,which is very awesome. By looking the confusion matrix it is clear that there is no sign of underfitting and overfitting. Model training output.jpg

Model Testing

Before deploying the model it's a good practice to run the inference on the test dataset. In the model testing, we got around 92% accuracy. Model testing.jpg
Let's look into some of the misclassifications,
In this case, the noise very well resembled the Glass break that's why it is misclassified. misclassification1.jpg In this case too, the model performed very well in classifying the data, the data contains both the Glass_Break and the noise but the majority of the data was noise, that's why its misclassified. mis2.jpg In these two cases(below shown) also the noise was major.
The model is performing very well and can be deployed it in the real world.


For deploying the impulse to BLE Sense,we created a arduino library with this optimizations.
deploy.jpg Then we add that library to the Arduino IDE. Then we modified the example sketch that is provided. You can find entire code and assets including circuit in this Github repo.


For triggering the mail we used IFTT service. To setup the mail triggering upon any poitive detections please have a look at this tutorial.
This is the applet created by me. applet.jpg


All the components were fitted inside this case. IMG_2680.jpg

Real world Testing

Let's test the model after deploying it on the BLE sense board. You can check it out in the below video. We have actually played the Glass breaking sound via a large speaker.

Download block output

Title Type Size
MFE training data NPY file 240 windows
MFE training labels NPY file 240 windows
MFE testing data NPY file 138 windows
MFE testing labels NPY file 138 windows
NN Classifier model TensorFlow Lite (float32) 15 KB
NN Classifier model TensorFlow Lite (int8 quantized) 8 KB
NN Classifier model TensorFlow SavedModel 19 KB
NN Classifier model Keras h5 model 14 KB

Clone project

You are viewing a public Edge Impulse project. Clone this project to add data or make changes.


Data collected
10m 0s

Project info

Project ID 149095
Project version 1
License Apache 2.0