Tutorial: Recognize sounds from audio - Create impulse

Edge Impulse Inc. / Tutorial: Recognize sounds from audio Public

Target: Raspberry Pi 4 Clone this project

Impulse #1

An impulse takes raw data, uses signal processing to extract features, and then uses a learning block to classify new data.

Description	Recommended
Time series data Officially Supported Operates on time series sensor data like vibration or audio data. Lets you slice up data into windows.
Images Officially Supported Processes discrete images for object detection or classification.
Pre-processed features Officially Supported Processes pre-processed features, or non time-series data.

Did you know? You can bring your own DSP code.

Description	Author	Recommended
Audio (MFCC) Officially Supported Extracts features from audio signals using Mel Frequency Cepstral Coefficients, great for human voice.	Edge Impulse
Audio (MFE) Officially Supported Extracts a spectrogram from audio signals using Mel-filterbank energy features, great for non-voice audio.	Edge Impulse
Spectrogram Officially Supported Extracts a spectrogram from audio or sensor data, great for non-voice audio or data with continuous frequencies.	Edge Impulse
Audio (Syntiant) Officially Supported Syntiant only. Compute log Mel-filterbank energy features from an audio signal.	Syntiant
Raw Data Officially Supported Use data without pre-processing. Useful if you want to use deep learning to learn features.	Edge Impulse
IMF (Iterative Filtering) Enterprise Extract IMFs from a signal using iterative filtering	Edge Impulse Inc.
ToF custom DSP Enterprise ToF to heatmap block example	Edge Impulse Inc.
PPG to HR Enterprise Turn PPG data into HR features	Edge Impulse Inc.
Edge Detection (Images) Enterprise Edge Detection for images	Edge Impulse Inc.
Pose estimation Enterprise Pose estimation for images	Edge Impulse Inc.
MFCC - Normalized Enterprise Extracts features from audio signals using Mel Frequency Cepstral Coefficients. Returns normalzed values (0..1)	Edge Impulse Inc.
HOG (Histogram of Oriented Gradients) Enterprise Generates a histogram of oriented gradients (HOG)	Edge Impulse Inc.
Audio (MFCC) Officially Supported Extracts features from audio signals using Mel Frequency Cepstral Coefficients, great for human voice.	Edge Impulse
Audio (MFE) Officially Supported Extracts a spectrogram from audio signals using Mel-filterbank energy features, great for non-voice audio.	Edge Impulse
Flatten Officially Supported Flatten an axis into a single value, useful for slow-moving averages like temperature data, in combination with other blocks.	Edge Impulse
Image Officially Supported Preprocess and normalize image data, and optionally reduce the color depth.	Edge Impulse
Spectral Analysis Officially Supported Great for analyzing repetitive motion, such as data from accelerometers. Extracts the frequency and power characteristics of a signal over time.	Edge Impulse
Spectrogram Officially Supported Extracts a spectrogram from audio or sensor data, great for non-voice audio or data with continuous frequencies.	Edge Impulse
Audio (Syntiant) Officially Supported Syntiant only. Compute log Mel-filterbank energy features from an audio signal.	Syntiant
IMU (Syntiant) Officially Supported Syntiant only. Great for analyzing repetitive motion, such as data from accelerometers. Extracts the frequency and power characteristics of a signal over time.	Syntiant
HR and HRV features Officially Supported Process PPG or ECG data into heart rate and heart rate variability features. This block is available for testing but requires an Enterprise license to deploy.	Edge Impulse
Raw Data Officially Supported Use data without pre-processing. Useful if you want to use deep learning to learn features.	Edge Impulse
IMF (Iterative Filtering) Enterprise Extract IMFs from a signal using iterative filtering	Edge Impulse Inc.
ToF custom DSP Enterprise ToF to heatmap block example	Edge Impulse Inc.
PPG to HR Enterprise Turn PPG data into HR features	Edge Impulse Inc.
Edge Detection (Images) Enterprise Edge Detection for images	Edge Impulse Inc.
Pose estimation Enterprise Pose estimation for images	Edge Impulse Inc.
MFCC - Normalized Enterprise Extracts features from audio signals using Mel Frequency Cepstral Coefficients. Returns normalzed values (0..1)	Edge Impulse Inc.
HOG (Histogram of Oriented Gradients) Enterprise Generates a histogram of oriented gradients (HOG)	Edge Impulse Inc.

Did you know? You can bring your own model in PyTorch, Keras or scikit-learn.

Description	Author	Recommended
Classification Officially Supported Learns patterns from data, and can apply these to new data. Great for categorizing movement or recognizing audio.	Edge Impulse
PyTorch MLP example (20x10 hidden layers) Enterprise Custom ML block with PyTorch	Edge Impulse Inc.
MCSA Enterprise A learning block designed specifically for the MCSA project.	Edge Impulse Inc.
Linear regression (scikit-learn) Enterprise Implementation of sklearn.linear_model.LogisticRegression	Edge Impulse Inc.
LGBM Random Forest Enterprise LGBM Random Forest	Edge Impulse Inc.
LGBM Binary Classification Enterprise LGBM Binary Classification	Edge Impulse Inc.
LGBM Random Forest Classifier Professional Enterprise The LightGBM Random Forest Classifier is an efficient, gradient-boosting framework that builds multiple decision trees in parallel for robust and scalable classification tasks.	Edge Impulse Inc.
XGBoost Random Forest Classifier Professional Enterprise The XGBoost Random Forest Classifier leverages the strengths of both gradient boosting and random forest methodologies to provide a powerful, efficient, and scalable solution for classification tasks.	Edge Impulse Inc.
SVM Classifier Professional Enterprise The scikit-learn SVM classifier is a powerful algorithm that constructs a hyperplane or set of hyperplanes in a high-dimensional space for classification, effectively handling both linear and non-linear data.	Edge Impulse Inc.
Random Forest Classifier Professional Enterprise The scikit-learn Random Forest Classifier is a versatile machine learning algorithm that builds multiple decision trees and merges their predictions for robust and accurate classification.	Edge Impulse Inc.
Logistic Regression Professional Enterprise The scikit-learn Logistic Regressor is a linear model for classification that estimates probabilities using a logistic function, effectively handling binary and multiclass classification tasks.	Edge Impulse Inc.
Ridge Classifier Professional Enterprise The scikit-learn Ridge classifier employs L2 regularization to optimize a linear classification model, mitigating overfitting by penalizing large coefficients, thereby enhancing model generalizability across diverse datasets.	Edge Impulse Inc.
RidgeCV Classifier Professional Enterprise The scikit-learn RidgeCV classifier is a linear regression model with built-in cross-validation that applies ridge regularization to prevent overfitting and improve prediction accuracy.	Edge Impulse Inc.
Regression Officially Supported Learns patterns from data, and can apply these to new data. Great for predicting numeric continuous values.	Edge Impulse
Transfer Learning (Keyword Spotting) Officially Supported Fine tune a pre-trained keyword spotting model on your data. Good performance even with relatively small keyword datasets.	Edge Impulse
Anomaly Detection (GMM) Professional Enterprise Find outliers in new data. A Gaussian mixture model (GMM) models the shape of data using a probability distribution. New data that is unlikely according to this model can be considered anomalous.	Edge Impulse
Anomaly Detection (K-means) Officially Supported Find outliers in new data. Good for recognizing unknown states, and to complement classifiers. Works best with low dimensionality features like the output of the spectral features block.	Edge Impulse
Classification - BrainChip Akida™ Officially Supported Learns patterns from data, and can apply these to new data. Great for categorizing movement or recognizing audio. Only works with BrainChip AKD1000 MINI PCIe board.	BrainChip
Visual Anomaly Detection - FOMO-AD Enterprise Detect visual anomalies. Extracts visual features using a pre-trained backbone, and applies a scoring function to evaluate how anomalous a sample is by comparing the extracted features to the learned model. Does not require anomalous data.	Edge Impulse
Custom regression MLP Enterprise Example of a regression block	Edge Impulse Inc.
NDP120 Dense Enterprise Dense Neural Network for Syntiant NDP120	Edge Impulse Inc.
EfficientNetRegression Enterprise EfficientNetRegression	Edge Impulse Inc.
Mathijs - Akida Enterprise Mathijs - Akida	Edge Impulse Inc.
LGBM Random Forest Regressor Professional Enterprise The LightGBM Random Forest Regressor applies an ensemble of decision trees with gradient boosting to provide fast, efficient, and accurate regression over large datasets.	Edge Impulse Inc.
XGBoost Random Forest Regressor Professional Enterprise The XGBoost Random Forest Regressor combines the speed and performance of gradient boosting with the robustness of a random forest to achieve high-accuracy regression on complex datasets.	Edge Impulse Inc.
Random Forest Regressor Professional Enterprise The scikit-learn Random Forest Regressor is an ensemble learning method that uses multiple decision trees to predict continuous values, offering high accuracy and handling complex datasets effectively.	Edge Impulse Inc.
Linear Regression Professional Enterprise The scikit-learn Linear Regressor is a straightforward approach that models the relationship between a dependent variable and one or more independent variables using a linear equation to predict continuous outcomes.	Edge Impulse Inc.
Ridge Regression Professional Enterprise The scikit-learn Ridge regressor applies L2 regularization to linear regression, balancing the fit and complexity of the model to improve prediction accuracy on new data by preventing overfitting.	Edge Impulse Inc.
RidgeCV Regression Professional Enterprise The scikit-learn RidgeCV regressor combines ridge regression with built-in cross-validation to automatically tune the regularization strength, ensuring optimal balance between model complexity and performance on unseen data.	Edge Impulse Inc.
Classification Officially Supported Learns patterns from data, and can apply these to new data. Great for categorizing movement or recognizing audio.	Edge Impulse
PyTorch MLP example (20x10 hidden layers) Enterprise Custom ML block with PyTorch	Edge Impulse Inc.
MCSA Enterprise A learning block designed specifically for the MCSA project.	Edge Impulse Inc.
Linear regression (scikit-learn) Enterprise Implementation of sklearn.linear_model.LogisticRegression	Edge Impulse Inc.
LGBM Random Forest Enterprise LGBM Random Forest	Edge Impulse Inc.
LGBM Binary Classification Enterprise LGBM Binary Classification	Edge Impulse Inc.
LGBM Random Forest Classifier Professional Enterprise The LightGBM Random Forest Classifier is an efficient, gradient-boosting framework that builds multiple decision trees in parallel for robust and scalable classification tasks.	Edge Impulse Inc.
XGBoost Random Forest Classifier Professional Enterprise The XGBoost Random Forest Classifier leverages the strengths of both gradient boosting and random forest methodologies to provide a powerful, efficient, and scalable solution for classification tasks.	Edge Impulse Inc.
SVM Classifier Professional Enterprise The scikit-learn SVM classifier is a powerful algorithm that constructs a hyperplane or set of hyperplanes in a high-dimensional space for classification, effectively handling both linear and non-linear data.	Edge Impulse Inc.
Random Forest Classifier Professional Enterprise The scikit-learn Random Forest Classifier is a versatile machine learning algorithm that builds multiple decision trees and merges their predictions for robust and accurate classification.	Edge Impulse Inc.
Logistic Regression Professional Enterprise The scikit-learn Logistic Regressor is a linear model for classification that estimates probabilities using a logistic function, effectively handling binary and multiclass classification tasks.	Edge Impulse Inc.
Ridge Classifier Professional Enterprise The scikit-learn Ridge classifier employs L2 regularization to optimize a linear classification model, mitigating overfitting by penalizing large coefficients, thereby enhancing model generalizability across diverse datasets.	Edge Impulse Inc.
RidgeCV Classifier Professional Enterprise The scikit-learn RidgeCV classifier is a linear regression model with built-in cross-validation that applies ridge regularization to prevent overfitting and improve prediction accuracy.	Edge Impulse Inc.
Transfer Learning (Images) Officially Supported Fine tune a pre-trained image classification model on your data. Good performance even with relatively small image datasets.	Edge Impulse
Object Detection (Images) Officially Supported Fine tune a pre-trained object detection model on your data. Good performance even with relatively small image datasets.	Edge Impulse
Regression Officially Supported Learns patterns from data, and can apply these to new data. Great for predicting numeric continuous values.	Edge Impulse
Transfer Learning (Keyword Spotting) Officially Supported Fine tune a pre-trained keyword spotting model on your data. Good performance even with relatively small keyword datasets.	Edge Impulse
Anomaly Detection (GMM) Professional Enterprise Find outliers in new data. A Gaussian mixture model (GMM) models the shape of data using a probability distribution. New data that is unlikely according to this model can be considered anomalous.	Edge Impulse
Anomaly Detection (K-means) Officially Supported Find outliers in new data. Good for recognizing unknown states, and to complement classifiers. Works best with low dimensionality features like the output of the spectral features block.	Edge Impulse
Classification - BrainChip Akida™ Officially Supported Learns patterns from data, and can apply these to new data. Great for categorizing movement or recognizing audio. Only works with BrainChip AKD1000 MINI PCIe board.	BrainChip
Transfer Learning (Images) - BrainChip Akida™ Officially Supported Fine tune a pre-trained image classification model on your data. Good performance even with relatively small image datasets. Only works with BrainChip AKD1000 MINI PCIe board.	BrainChip
Object Detection (Images) - BrainChip Akida™ Officially Supported Fine tune a pre-trained object detection model on your data. Good performance even with relatively small image datasets. Only works with BrainChip AKD1000 MINI PCIe board.	BrainChip
Visual Anomaly Detection - FOMO-AD Enterprise Detect visual anomalies. Extracts visual features using a pre-trained backbone, and applies a scoring function to evaluate how anomalous a sample is by comparing the extracted features to the learned model. Does not require anomalous data.	Edge Impulse
Custom regression MLP Enterprise Example of a regression block	Edge Impulse Inc.
efficientdet-lite Enterprise efficientdet-lite	Edge Impulse Inc.
NDP120 Dense Enterprise Dense Neural Network for Syntiant NDP120	Edge Impulse Inc.
YOLOv5 (yolov5n.pt) Enterprise YOLOv5 model with yolov5n.pt pretrained weights	Edge Impulse Inc.
efficientnet-mathijs Enterprise efficientnet	Edge Impulse Inc.
YOLOv3 Enterprise Fine-tune a pretrained YOLOv3 model based on yolov3-tiny.pt	Edge Impulse Inc.
EfficientNetB0 Enterprise EfficientNetB0	Edge Impulse Inc.
EfficientNetB1 Enterprise EfficientNetB1	Edge Impulse Inc.
EfficientNetB2 Enterprise EfficientNetB2	Edge Impulse Inc.
EfficientNetRegression Enterprise EfficientNetRegression	Edge Impulse Inc.
TI YOLOX Enterprise Transfer learning model based on yolox_nano_ti_lite_26p1_41p8_checkpoint.pth	Edge Impulse Inc.
mat_akida_nan_testing Enterprise testing of nans coming from akida training	Edge Impulse Inc.
coco-cats Enterprise coco-cats	Edge Impulse Inc.
raul-edgeai-regnetx-800mf-fpn-bgr-lite Enterprise "TI's EDGEAI optimization of the mmdetection regnetx-800mf-fpn-bgr-lite architecture. https://github.com/TexasInstruments/edgeai-mmdetection"	Edge Impulse Inc.
Nvidia TAO: fan_tiny_8_p4_hybrid Enterprise Test in using Nvidia TAO to train models	Edge Impulse Inc.
Mathijs - Akida Enterprise Mathijs - Akida	Edge Impulse Inc.
LGBM Random Forest Regressor Professional Enterprise The LightGBM Random Forest Regressor applies an ensemble of decision trees with gradient boosting to provide fast, efficient, and accurate regression over large datasets.	Edge Impulse Inc.
XGBoost Random Forest Regressor Professional Enterprise The XGBoost Random Forest Regressor combines the speed and performance of gradient boosting with the robustness of a random forest to achieve high-accuracy regression on complex datasets.	Edge Impulse Inc.
NVIDIA TAO RetinaNet Professional Enterprise Object detection model with superior performance on smaller objects. Configurable backbones optimized for targets from MCU to GPU. Supports rectangular input. Image width and height must be multiples of 32. Training requires GPU.	Edge Impulse Inc.
NVIDIA TAO YOLOV3 Professional Enterprise Object detection model that is fast and accurate. Configurable backbones optimized for targets from MCU to GPU. Supports rectangular input. Image width and height must be multiples of 32. Training requires GPU.	Edge Impulse Inc.
NVIDIA TAO YOLOV4 Professional Enterprise Object detection model that is fast and accurate. Configurable backbones optimized for targets from MCU to GPU. Supports rectangular input. Image width and height must be multiples of 32. Training requires GPU.	Edge Impulse Inc.
NVIDIA TAO SSD Professional Enterprise Object detection model for general purpose use. Configurable backbones optimized for targets from MCU to GPU. Supports rectangular input. Image width and height must be multiples of 32. Training requires GPU.	Edge Impulse Inc.
Random Forest Regressor Professional Enterprise The scikit-learn Random Forest Regressor is an ensemble learning method that uses multiple decision trees to predict continuous values, offering high accuracy and handling complex datasets effectively.	Edge Impulse Inc.
Linear Regression Professional Enterprise The scikit-learn Linear Regressor is a straightforward approach that models the relationship between a dependent variable and one or more independent variables using a linear equation to predict continuous outcomes.	Edge Impulse Inc.
Ridge Regression Professional Enterprise The scikit-learn Ridge regressor applies L2 regularization to linear regression, balancing the fit and complexity of the model to improve prediction accuracy on new data by preventing overfitting.	Edge Impulse Inc.
RidgeCV Regression Professional Enterprise The scikit-learn RidgeCV regressor combines ridge regression with built-in cross-validation to automatically tune the regularization strength, ensuring optimal balance between model complexity and performance on unseen data.	Edge Impulse Inc.
NVIDIA TAO Image Classification Professional Enterprise Image classification model for general purpose use. Configurable backbones optimized for targets from MCU to GPU. Only supports RGB images. Pre-trained weights only support 224x224 resolution. Image width and height must be greater than 32. Training requires GPU.	Edge Impulse Inc.

Time series data

Input axes

audio

Window size

Window increase (stride)

Frequency (Hz)

Invalid frequency

Zero-pad data

Spectrogram

Name

Input axes (1)

audio

Classification (Keras)

Name

Input features

Spectrogram

Output features

2 (faucet, noise)

Output features

2 (faucet, noise)

Time series data

Spectrogram

Classification (Keras)

Output features

No data collected yet