eye_for_blind - Deployment

Md. Khairul Alam / eye_for_blind Public

Target: Espressif ESP-EYE (ESP32 240MHz) Clone this project

Configure your deployment

You can deploy your impulse to any device. This makes the model run without an internet connection, minimizes latency, and runs with minimal power consumption. Read more.

Run your model as a Docker container

To run your model as a container with an HTTP interface on NVIDIA Jetson's GPUs (JetPack 4.6.x), use:

Container:

public.ecr.aws/g7a8t7v6/inference-container-jetson:e8051cd2501555fd945a3a2583fcff264ce229d0

Arguments:

--api-key ei_81d187033a8cfc81b05e3968894ce50c36a688b0a41300774069db062e0513c8 --run-http-server 1337

Ports to expose:

1337

For example, in a one-liner locally:

docker run --rm -it --runtime=nvidia --gpus all \
    -p 1337:1337 \
    public.ecr.aws/g7a8t7v6/inference-container-jetson:e8051cd2501555fd945a3a2583fcff264ce229d0 \
        --api-key ei_81d187033a8cfc81b05e3968894ce50c36a688b0a41300774069db062e0513c8 \
        --run-http-server 1337

This automatically builds and downloads the latest model (incl. hardware optimizations), and runs an HTTP endpoint at http://localhost:1337 with instructions.

Read the docs for information, like bundling in your model inside the container and selecting extra hardware optimizations.

Run your model as a Docker container

To run your model as a container with an HTTP interface on NVIDIA Jetson Orin's GPUs (JetPack 5.1.x), use:

Container:

public.ecr.aws/g7a8t7v6/inference-container-jetson-orin:16e177251d0e27a7955671a2081fd1e8e91afb0c

Arguments:

--api-key ei_81d187033a8cfc81b05e3968894ce50c36a688b0a41300774069db062e0513c8 --run-http-server 1337

Ports to expose:

1337

For example, in a one-liner locally:

docker run --rm -it --runtime=nvidia --gpus all \
    -p 1337:1337 \
    public.ecr.aws/g7a8t7v6/inference-container-jetson-orin:16e177251d0e27a7955671a2081fd1e8e91afb0c \
        --api-key ei_81d187033a8cfc81b05e3968894ce50c36a688b0a41300774069db062e0513c8 \
        --run-http-server 1337

This automatically builds and downloads the latest model (incl. hardware optimizations), and runs an HTTP endpoint at http://localhost:1337 with instructions.

Read the docs for information, like bundling in your model inside the container and selecting extra hardware optimizations.

Run your model as a Docker container

To run your model as a container with an HTTP interface on NVIDIA Jetson Orin's GPUs (JetPack 6.0), use:

Container:

public.ecr.aws/g7a8t7v6/inference-container-jetson-orin-6-0:1744167193cd72b937cc33932a4c941995a79d32

Arguments:

--api-key ei_81d187033a8cfc81b05e3968894ce50c36a688b0a41300774069db062e0513c8 --run-http-server 1337

Ports to expose:

1337

For example, in a one-liner locally:

docker run --rm -it --runtime=nvidia --gpus all \
    -p 1337:1337 \
    public.ecr.aws/g7a8t7v6/inference-container-jetson-orin-6-0:1744167193cd72b937cc33932a4c941995a79d32 \
        --api-key ei_81d187033a8cfc81b05e3968894ce50c36a688b0a41300774069db062e0513c8 \
        --run-http-server 1337

This automatically builds and downloads the latest model (incl. hardware optimizations), and runs an HTTP endpoint at http://localhost:1337 with instructions.

Read the docs for information, like bundling in your model inside the container and selecting extra hardware optimizations.

Model Optimizations

Model optimizations can increase on-device performance but may reduce accuracy.

Quantized (int8)


Latency
Ram
Flash
Accuracy

Image	Object detection	Total
15 ms.	1,486 ms.	1,501 ms.
4.0K	239.5K	239.5K
-	79.2K	-
		-

Unoptimized (float32)


Latency
Ram
Flash
Accuracy

Image	Object detection	Total
15 ms.	2,879 ms.	2,894 ms.
4.0K	887.1K	887.1K
-	103.9K	-
		-

To compare model accuracy, clone this project and run model testing.

Estimate for Espressif ESP-EYE (ESP32 240MHz).

Clone this project to deploy this impulse.

Configure your deployment

Deploy to any Linux-based development board

Run your model as a Docker container

Run your model as a Docker container

Run your model as a Docker container

Run your model as a Docker container

Model Optimizations

Quantized (int8)

Unoptimized (float32)

Latest build

Run this model

Build output

Model testing output

Configure your deployment

Default Deployment

C++ library

Deploy to any Linux-based development board

Run your model as a Docker container

Run your model as a Docker container

Run your model as a Docker container

Run your model as a Docker container

Model Optimizations

EON™ Compiler

EON™ Compiler (RAM optimized) Enterprise

EON™ Compiler

TensorFlow Lite

Quantized (int8)

Unoptimized (float32)

Latest build

Run this model

Build output

Model testing output

EON™ Compiler (RAM optimized)
Enterprise