Tutorial: Adding sight to your sensors - Create impulse

Edge Impulse Inc. / Tutorial: Adding sight to your sensors Public

Clone this project

An impulse takes raw data, uses signal processing to extract features, and then uses a learning block to classify new data.

Description	Recommended
Images Processes discrete images for object detection or classification.
Time series data Operates on time series sensor data like vibration or audio data. Lets you slice up data into windows.

Did you know? You can bring your own DSP code.

Description	Author	Recommended
Image Preprocess and normalize image data, and optionally reduce the color depth.	Edge Impulse
Raw Data Use data without pre-processing. Useful if you want to use deep learning to learn features.	Edge Impulse
IMF (Iterative Filtering) Extract IMFs from a signal using iterative filtering	Edge Impulse Inc.
ToF custom DSP ToF to heatmap block example	Edge Impulse Inc.
PPG to HR Turn PPG data into HR features	Edge Impulse Inc.
Edge Detection (Images) Edge Detection for images	Edge Impulse Inc.
Pose estimation Pose estimation for images	Edge Impulse Inc.
MFCC - Normalized Extracts features from audio signals using Mel Frequency Cepstral Coefficients. Returns normalzed values (0..1)	Edge Impulse Inc.
Image Preprocess and normalize image data, and optionally reduce the color depth.	Edge Impulse
Flatten Flatten an axis into a single value, useful for slow-moving averages like temperature data, in combination with other blocks.	Edge Impulse
Audio (MFCC) Extracts features from audio signals using Mel Frequency Cepstral Coefficients, great for human voice.	Edge Impulse
Audio (MFE) Extracts a spectrogram from audio signals using Mel-filterbank energy features, great for non-voice audio.	Edge Impulse
Spectral Analysis Great for analyzing repetitive motion, such as data from accelerometers. Extracts the frequency and power characteristics of a signal over time.	Edge Impulse
Spectrogram Extracts a spectrogram from audio or sensor data, great for non-voice audio or data with continuous frequencies.	Edge Impulse
Audio (Syntiant) Syntiant only. Compute log Mel-filterbank energy features from an audio signal.	Syntiant
IMU (Syntiant) Syntiant only. Great for analyzing repetitive motion, such as data from accelerometers. Extracts the frequency and power characteristics of a signal over time.	Syntiant
Raw Data Use data without pre-processing. Useful if you want to use deep learning to learn features.	Edge Impulse
IMF (Iterative Filtering) Extract IMFs from a signal using iterative filtering	Edge Impulse Inc.
ToF custom DSP ToF to heatmap block example	Edge Impulse Inc.
PPG to HR Turn PPG data into HR features	Edge Impulse Inc.
Edge Detection (Images) Edge Detection for images	Edge Impulse Inc.
Pose estimation Pose estimation for images	Edge Impulse Inc.
MFCC - Normalized Extracts features from audio signals using Mel Frequency Cepstral Coefficients. Returns normalzed values (0..1)	Edge Impulse Inc.

Did you know? You can bring your own model in PyTorch, Keras or scikit-learn.

Description	Author	Recommended
Transfer Learning (Images) Fine tune a pre-trained image classification model on your data. Good performance even with relatively small image datasets.	Edge Impulse
efficientnet-mathijs efficientnet	Edge Impulse Inc.
EfficientNetB0 EfficientNetB0	Edge Impulse Inc.
EfficientNetB1 EfficientNetB1	Edge Impulse Inc.
EfficientNetB2 EfficientNetB2	Edge Impulse Inc.
Nvidia TAO: fan_tiny_8_p4_hybrid Test in using Nvidia TAO to train models	Edge Impulse Inc.
EfficientNet B0 Transfer learning model based on efficientnetb0_notop.h5 weights. This is a much larger model than MobileNet for Linux devices and accelerators.	Community blocks
Classification Learns patterns from data, and can apply these to new data. Great for categorizing movement or recognizing audio.	Edge Impulse
Regression Learns patterns from data, and can apply these to new data. Great for predicting numeric continuous values.	Edge Impulse
Classification - BrainChip Akida™ Learns patterns from data, and can apply these to new data. Great for categorizing movement or recognizing audio. Only works with BrainChip AKD1000 MINI PCIe board.	BrainChip
Transfer Learning (Images) - BrainChip Akida™ Fine tune a pre-trained image classification model on your data. Good performance even with relatively small image datasets. Only works with BrainChip AKD1000 MINI PCIe board.	BrainChip
Custom regression MLP Example of a regression block	Edge Impulse Inc.
PyTorch MLP example (20x10 hidden layers) Custom ML block with PyTorch	Edge Impulse Inc.
MCSA A learning block designed specifically for the MCSA project.	Edge Impulse Inc.
Linear regression (scikit-learn) Implementation of sklearn.linear_model.LogisticRegression	Edge Impulse Inc.
EfficientNetRegression EfficientNetRegression	Edge Impulse Inc.
LGBM Random Forest LGBM Random Forest	Edge Impulse Inc.
LGBM Binary Classification LGBM Binary Classification	Edge Impulse Inc.
[unpublished] LGBM Random Forest Classifier [unpublished] LGBM Random Forest Classifier	Edge Impulse Inc.
[unpublished] XGBoost Random Forest Classifier [unpublished] XGBoost Random Forest Classifier	Edge Impulse Inc.
[unpublished] LGBM Random Forest Regressor [unpublished] LGBM Random Forest Regressor	Edge Impulse Inc.
[unpublished] XGBoost Random Forest Regressor [unpublished] XGBoost Random Forest Regressor	Edge Impulse Inc.
[unpublished] SVM Classifier [unpublished] SVM Classifier	Edge Impulse Inc.
Transfer Learning (Images) Fine tune a pre-trained image classification model on your data. Good performance even with relatively small image datasets.	Edge Impulse
efficientnet-mathijs efficientnet	Edge Impulse Inc.
EfficientNetB0 EfficientNetB0	Edge Impulse Inc.
EfficientNetB1 EfficientNetB1	Edge Impulse Inc.
EfficientNetB2 EfficientNetB2	Edge Impulse Inc.
Nvidia TAO: fan_tiny_8_p4_hybrid Test in using Nvidia TAO to train models	Edge Impulse Inc.
EfficientNet B0 Transfer learning model based on efficientnetb0_notop.h5 weights. This is a much larger model than MobileNet for Linux devices and accelerators.	Community blocks
Classification Learns patterns from data, and can apply these to new data. Great for categorizing movement or recognizing audio.	Edge Impulse
Object Detection (Images) Fine tune a pre-trained object detection model on your data. Good performance even with relatively small image datasets.	Edge Impulse
Regression Learns patterns from data, and can apply these to new data. Great for predicting numeric continuous values.	Edge Impulse
Transfer Learning (Keyword Spotting) Fine tune a pre-trained keyword spotting model on your data. Good performance even with relatively small keyword datasets.	Edge Impulse
Anomaly Detection (GMM) Enterprise Find outliers in new data. A Gaussian mixture model (GMM) models the shape of data using a probability distribution. New data that is unlikely according to this model can be considered anomalous.	Edge Impulse
Anomaly Detection (K-means) Find outliers in new data. Good for recognizing unknown states, and to complement classifiers. Works best with low dimensionality features like the output of the spectral features block.	Edge Impulse
Classification - BrainChip Akida™ Learns patterns from data, and can apply these to new data. Great for categorizing movement or recognizing audio. Only works with BrainChip AKD1000 MINI PCIe board.	BrainChip
Transfer Learning (Images) - BrainChip Akida™ Fine tune a pre-trained image classification model on your data. Good performance even with relatively small image datasets. Only works with BrainChip AKD1000 MINI PCIe board.	BrainChip
Object Detection (Images) - BrainChip Akida™ Fine tune a pre-trained object detection model on your data. Good performance even with relatively small image datasets. Only works with BrainChip AKD1000 MINI PCIe board.	BrainChip
Custom regression MLP Example of a regression block	Edge Impulse Inc.
PyTorch MLP example (20x10 hidden layers) Custom ML block with PyTorch	Edge Impulse Inc.
MCSA A learning block designed specifically for the MCSA project.	Edge Impulse Inc.
efficientdet-lite efficientdet-lite	Edge Impulse Inc.
NDP120 Dense Dense Neural Network for Syntiant NDP120	Edge Impulse Inc.
Linear regression (scikit-learn) Implementation of sklearn.linear_model.LogisticRegression	Edge Impulse Inc.
YOLOv5 (yolov5n.pt) YOLOv5 model with yolov5n.pt pretrained weights	Edge Impulse Inc.
YOLOv3 Fine-tune a pretrained YOLOv3 model based on yolov3-tiny.pt	Edge Impulse Inc.
EfficientNetRegression EfficientNetRegression	Edge Impulse Inc.
TI YOLOX Transfer learning model based on yolox_nano_ti_lite_26p1_41p8_checkpoint.pth	Edge Impulse Inc.
mat_akida_nan_testing testing of nans coming from akida training	Edge Impulse Inc.
coco-cats coco-cats	Edge Impulse Inc.
raul-edgeai-regnetx-800mf-fpn-bgr-lite "TI's EDGEAI optimization of the mmdetection regnetx-800mf-fpn-bgr-lite architecture. https://github.com/TexasInstruments/edgeai-mmdetection"	Edge Impulse Inc.
LGBM Random Forest LGBM Random Forest	Edge Impulse Inc.
Mathijs - Akida Mathijs - Akida	Edge Impulse Inc.
LGBM Binary Classification LGBM Binary Classification	Edge Impulse Inc.
tao-tf1-retinanet-object-detection tao-tf1-retinanet-object-detection	Edge Impulse Inc.
tao-tf1-yolo_v3-object-detection tao-tf1-yolo_v3-object-detection"	Edge Impulse Inc.
tao-tf1-yolo_v4-object-detection tao-tf1-yolo_v4-object-detection	Edge Impulse Inc.
tao-tf1-ssd-object-detection tao-tf1-ssd-object-detection	Edge Impulse Inc.
[unpublished] LGBM Random Forest Classifier [unpublished] LGBM Random Forest Classifier	Edge Impulse Inc.
[unpublished] XGBoost Random Forest Classifier [unpublished] XGBoost Random Forest Classifier	Edge Impulse Inc.
[unpublished] LGBM Random Forest Regressor [unpublished] LGBM Random Forest Regressor	Edge Impulse Inc.
[unpublished] XGBoost Random Forest Regressor [unpublished] XGBoost Random Forest Regressor	Edge Impulse Inc.
[unpublished] SVM Classifier [unpublished] SVM Classifier	Edge Impulse Inc.
YOLOv5 for Renesas DRP-AI Transfer learning model using YOLOv5 v5 branch with yolov5s.pt weights. This block is only compatible with Renesas DRP-AI.	Renesas
YOLOv5 Transfer learning model based on Ultralytics YOLOv5 using yolov5n.pt weights, supports RGB input at any resolution (square images only).	Community blocks
YOLOX for TI TDA4VM TI's EDGEAI YOLOX. https://github.com/TexasInstruments/edgeai-yolox. Outputs ONNX v7 model format both with and without final detect layers using PyTorch 1.7.1. See the implementation https://github.com/edgeimpulse/example-custom-ml-block-ti-yolox/tree/onnx-v7	Texas Instruments

Title

Data type

Input axes

This block does not have any other settings

Image width

Image height

Resize mode

Resize method

For optimal accuracy with transfer learning blocks, use a 96x96 or 160x160 image size.

Window size

Window increase

Frequency (Hz)

Invalid frequency

Zero-pad data

Keras

Name

Input features

Output features

This block only works with an Audio (MFE) block with default parameters and a window size of 1000ms at 16000Hz, or with an Audio (Syntiant) block with default parameters and a window size of 968ms at 16000Hz.

Title

Custom Block

Keras

Output features

No data collected yet

Description