guides/coral-edge-tpu-on-raspberry-pi/

[giscus[bot]]

guides/coral-edge-tpu-on-raspberry-pi/

Guide on how to use Ultralytics with a Coral Edge TPU on a Raspberry Pi for increased inference performance.

https://docs.ultralytics.com/guides/coral-edge-tpu-on-raspberry-pi/

[DAVIDNYARKO123]

Exactly the file I was looking for to reignite the Coral TPU dream.

[glenn-jocher]

Glad to hear the guide is what you were looking for! 🌟 If you're diving into the Coral TPU with a Raspberry Pi, just follow the steps outlined in the guide to get everything up and running. If you hit any snags or have questions, feel free to drop them here. Happy inferencing!

[ukicomputers]

If I understood right, this code only converts YOLO model for edgetpu platform:

from ultralytics import YOLO

# Load a model
model = YOLO('path/to/model.pt')  # Load a official model or custom model

# Export the model
model.export(format='edgetpu')

But after, it says to run the model with YOLO, with following code:

from ultralytics import YOLO

# Load a model
model = YOLO('path/to/edgetpu_model.tflite')  # Load a official model or custom model

# Run Prediction
model.predict("path/to/source.png")

So then we are not running the model on Coral, but on our local device, right? If that is true, please say if we need to use that exported model with something like pycoral or libcoral. Thanks in advance.

[aeltersCDS]

@glenn-jocher is there any way to run the Object Counting solution in the edge tpu?

[Aishneet]

Hey @glenn-jocher,
I am confused regarding the inference behavior of models, particularly those utilizing the EdgeTPU, within the Ultralytics library. Just above you mentioned that when using model.predict() with an EdgeTPU model, the inference runs locally rather than on the TPU. However, upon inspecting the code in the autobackend module on GitHub, it seems that when the model is an EdgeTPU model, delegates are loaded, tensors are allocated, and inference is executed on the TPU.

This has left me confused about the actual behavior. Could you kindly provide clarification on whether EdgeTPU models run inference locally or on the TPU?

[glenn-jocher]

Hey there! 😊 It looks like there might be a bit of confusion regarding the inference behavior with Edge TPU models within our library. Let's clear that up!

When you run model.predict() with an Edge TPU model in our Ultralytics library, the inference indeed aims to utilize the Edge TPU for acceleration. So, if the setup is correct (Coral device connected and necessary libraries installed on your system), the inference should run on the Edge TPU, not just locally on the CPU.

The details you've pointed out concerning delegates and tensor allocations in our autobackend module are crucial. They're part of the mechanism that ensures the model utilizes the TPU when available. It selects the Edge TPU delegate for TensorFlow Lite models, which directs the inference operations to run on the Coral Edge TPU.

However, to ensure everything runs smoothly, it's vital to have the correct version of the Coral Edge TPU runtime installed and to follow the setup guide carefully. Misconfigurations or mismatched library versions could potentially lead to unexpected behaviors or errors.

If you're encountering issues, double-checking the Edge TPU runtime installation and ensuring your Raspberry Pi setup matches the requirements might help. Also, using pycoral or libcoral for inference is an excellent approach to leverage the TPU further effectively.

Your observation and research definitely push us towards providing clearer documentation and support for users keen on leveraging Edge TPU acceleration. Thanks a ton for bringing this to our attention! Let me know if there's anything else I can help with. 🚀

[Aishneet]

Thank you so much for your detailed explanation regarding the inference behavior. I appreciate your emphasis on the importance of having the correct versions. 👍

[glenn-jocher]

You're welcome!

[opc241]

Hello,

i would also add a question here, since i am running a yolov8 model on a Raspberry Pi 4, baught a Coral TPU Accelerator, installed everything like described, but the following code runs into the following issue. Since there is no real pycoral support anymore, i try my luck here, maybe someone has an idea how to fix that.

from pycoral.adapters import detect
from pycoral.adapters import common
from pycoral.utils.edgetpu import make_interpreter
from PIL import Image

interpreter = make_interpreter('/home/pi/homecontrol/software/yolo-Weights/yolov8n_full_integer_quant.tflite')
interpreter.allocate_tensors()

image = Image.open('/home/pi/homecontrol/software/yolo-Weights/yolov8n_saved_model/capture.png')
_, scale = common.set_resized_input(interpreter, image.size, lambda size: image.resize(size, Image.LANCZOS))

# Run inference
interpreter.invoke()

# Process detection results
detections = detect.get_objects(interpreter, scale)
for detection in detections:
    print(detection)

And the output is:

Traceback (most recent call last):
  File "/home/pi/homecontrol/software/__test.py", line 16, in <module>
    detections = detect.get_objects(interpreter, scale)
  File "/usr/lib/python3/dist-packages/pycoral/adapters/detect.py", line 210, in get_objects
    count = int(interpreter.tensor(signature['outputs']['output_0'])()[0])
KeyError: 'output_0'

Greetings Markus

[glenn-jocher]

@opc241 hi Markus,

It seems like the issue here is with accessing the output tensor using a key that might not exist in your model's output dictionary. A quick solution could be to try using an integer index to access the first output tensor, as your model might have a different output name than 'output_0'. This can vary based on how the model was converted to TFLite. Here's how you could modify the line throwing the KeyError:

# Assuming your model has a single primary output, you can try accessing it with index [0]
count = int(interpreter.get_tensor(interpreter.get_output_details()[0]['index'])[0])

This line replaces the one causing the 'output_0' KeyError, and it dynamically fetches the first output tensor regardless of its name.

Make sure your Edge TPU is using the correct runtime version and is compatible with your TensorFlow Lite version, as mentioned in the Coral Edge TPU guide on our docs for Raspberry Pi setups. These steps are crucial, especially since projects like Coral can have specific compatibility requirements.

Let me know if this helps or if you see any other issues! 😊

[Skillnoob]

The model you need to select after exporting should be called yolov8n_full_integer_quant_edgetpu.tflite and has to contain _edgetpu for ultralytics to pick up that it is a edgetpu model.
Try the following code:

from ultralytics import YOLO

model = YOLO('yolov8n_full_integer_quant_edgetpu.tflite') 

model.predict("path/to/source.png")```

[glenn-jocher]

Hi Markus,

Thank you for reaching out! It looks like you're on the right track with your setup. The issue you're encountering seems to be related to the output tensor naming in your TFLite model. The KeyError: 'output_0' suggests that the model's output tensor might not be named 'output_0' as expected by the pycoral library.

To address this, you can try accessing the output tensor using its index rather than its name. Here's an updated version of your code:

from pycoral.adapters import detect
from pycoral.adapters import common
from pycoral.utils.edgetpu import make_interpreter
from PIL import Image

# Load the Edge TPU model
interpreter = make_interpreter('/home/pi/homecontrol/software/yolo-Weights/yolov8n_full_integer_quant_edgetpu.tflite')
interpreter.allocate_tensors()

# Load and preprocess the image
image = Image.open('/home/pi/homecontrol/software/yolo-Weights/yolov8n_saved_model/capture.png')
_, scale = common.set_resized_input(interpreter, image.size, lambda size: image.resize(size, Image.LANCZOS))

# Run inference
interpreter.invoke()

# Access the output tensor using its index
output_details = interpreter.get_output_details()
output_data = interpreter.get_tensor(output_details[0]['index'])

# Process detection results
detections = detect.get_objects(interpreter, scale)
for detection in detections:
    print(detection)

Additionally, ensure that you have the latest versions of torch and ultralytics installed. You can update them using the following commands:

pip install --upgrade torch ultralytics

If the issue persists, please provide a minimum reproducible example so we can investigate further. You can find more details on creating a minimum reproducible example here.

For more detailed instructions on setting up and using the Coral Edge TPU with a Raspberry Pi, you can refer to our Coral Edge TPU on Raspberry Pi guide.

Feel free to reach out if you have any more questions or run into further issues. We're here to help! 😊

[opc241]

Hi Glenn,

and thanks a lot for your reply. I implemented your suggested fix and this leads to the following:

Traceback (most recent call last):
  File "/home/pi/homecontrol/software/__test.py", line 16, in <module>
    detections = detect.get_objects(interpreter, scale)
  File "/usr/lib/python3/dist-packages/pycoral/adapters/detect.py", line 211, in get_objects
    count = int(interpreter.get_tensor(interpreter.get_output_details()[0]['index'])[0])
TypeError: only size-1 arrays can be converted to Python scalars

Good hint with the versions. I often read about that and the currently installed versions are:
Name: tflite-runtime
Version: 2.13.0

Name: pycoral
Version: 2.0.0

Python 3.9.2

Raspberry Pi 4: Debian GNU/Linux 11 (bullseye)

And the way i exported the yolov8 to .tflite, was with this code snippet:
(after export i chose the yolov8n_full_integer_quant.tflite model)

model = YOLO("yolo-Weights/yolov8n.pt")
model.export(format="edgetpu", imgsz=640)

Is this the proper way of exporting a yolov8 --> .tflite?

I am thankful for every hint, which i will follow.

[glenn-jocher]

Hi there!

Glad to hear you've implemented the fix, though it looks like you've hit another snag. The error you're encountering seems to point to an issue with how the model's output is being interpreted. It might be due to expecting a scalar output where there's actually an array.

Regarding your model export process to .tflite for the Coral Edge TPU, your steps look correct. However, for utilizing the Edge TPU, it's essential to ensure the model is accurately quantized and compatible. The method you've used should give you a quantized .tflite model suitable for the Edge TPU as long as the model architecture supports it. For best results, consider ensuring your model's architecture is fully compatible with Edge TPU requirements.

Here's a handy checklist:

• Verify model requirements for Edge TPU compatibility. Complex operations may not be supported.
• Make sure you're using the correct version of the Edge TPU compiler and runtime that match your hardware and software versions.
• Run the model conversion and compilation steps on a non-ARM platform like Google Colab or an x86_64 machine as recommended in our docs.

For example, to convert and check compatibility, you can do:

from ultralytics import YOLO

model = YOLO('yolo-Weights/yolov8n.pt')
model.export(format='edgetpu', imgsz=640)

And then use the appropriate Edge TPU compiler to ensure compatibility.

I hope these suggestions help. Let me know if you have any other questions or run into more issues! 🚀

[opc241]

Hi Glenn,

Of course i tried your suggestions, so i used Google Colab to compile the .tflite model to a "edgetpu.tflite" model, but now the following error occurs by running the (not changed) code:

Traceback (most recent call last):
  File "/home/pi/homecontrol/software/__test.py", line 17, in <module>
    detections = detect.get_objects(interpreter, scale)
  File "/usr/lib/python3/dist-packages/pycoral/adapters/detect.py", line 214, in get_objects
    elif common.output_tensor(interpreter, 3).size == 1:
  File "/usr/lib/python3/dist-packages/pycoral/adapters/common.py", line 29, in output_tensor
    return interpreter.tensor(interpreter.get_output_details()[i]['index'])()
IndexError: list index out of range

If you have another hint for that i will try, but to be honest i don't feel fun with Google's Coral TPU Accelerator.
This topic is not intuitive/compatible (enough) for me - with focus on all of the dependencies and the need of "older" versions of python [...]

I had a simple code, using YOLOv8 and the predict() method, but this unfortunately uses too much CPU on the Raspberry Pi. That is why i thought Google Coral could be it, but hm ... maybe not.

Thanks for your help and if you have a clue why this error now occurs, feel free to tell me, otherwise i think i will send the Coral back to Amazon :-(

[pderrenger]

Hey there,

I'm truly sorry to hear about the stumbling blocks you've encountered with the Coral TPU setup. It's indeed quite a leap from running straightforward inference with YOLOv8 on CPU, and I understand how the complexity with dependencies can be quite daunting.

The error you're facing suggests an issue with accessing the output tensor indices. This could be stemming from a mismatch between the expected output format by the Edgetpu model and the actual format being generated. An initial step could be to double-check if the model converted to edgetpu.tflite is fully compatible and correctly quantized.

Here's a small something you might want to try - ensuring the TFLite runtime and Edge TPU software are correctly set up and aligned with your model's requirements:

# First, ensure the tflite-runtime is using the correct version.
pip install --upgrade tflite-runtime

# If using the CLI export command, also ensure the model format matches the expected Edge TPU architecture.
yolo export model=path/to/model.pt format=edgetpu

If the quirk persists, it might be rooted in the conversion process itself or how the inference script is interpreting the output tensors. If you're willing to share a bit more about your model conversion code or the script leading to the error, I'd be keen to dive deeper.

Usage of Coral with Raspberry Pi indeed brings a notable inference speed boost, but setup woes can sometimes overshadow the benefits. Don't hesitate to reach back if there's more to dissect or if a fresh set of logs might offer new clues. Meanwhile, holding off on returning the Coral might still be a good call - let's troubleshoot a bit more! 🛠️

[StefansAI]

I followed the instructions above but still have problems getting it to work. After installing Ultralytics in an environment I run following:

wget https://github.com/feranick/libedgetpu/releases/download/16.0TF2.16.1-1/libedgetpu1-std_16.0tf2.16.1-1.bookworm_arm64.deb
sudo dpkg -i libedgetpu1-std_16.0tf2.16.1-1.bookworm_arm64.deb

wget https://github.com/feranick/libedgetpu/releases/download/16.0TF2.16.1-1/libedgetpu-dev_16.0tf2.16.1-1.bookworm_arm64.deb
sudo dpkg -i libedgetpu-dev_16.0tf2.16.1-1.bookworm_arm64.deb

#https://github.com/JungLearnBot/RPi5_yolov8/blob/main/Readme.RPi5.coral_tpu.picam.qt.md
wget https://github.com/oberluz/pycoral/releases/download/2.13.0/pycoral-2.13.0-cp311-cp311-linux_aarch64.whl
pip install pycoral-2.13.0-cp311-cp311-linux_aarch64.whl --no-deps
pip install tflite-runtime==2.14.0

export always fails:

Ultralytics YOLOv8.1.43 🚀 Python-3.11.2 torch-2.2.2 CPU (Cortex-A76)
YOLOv8n summary (fused): 168 layers, 3151904 parameters, 0 gradients, 8.7 GFLOPs

�[34m�[1mPyTorch:�[0m starting from 'yolov8n.pt' with input shape (1, 3, 640, 640) BCHW and output shape(s) (1, 84, 8400) (6.2 MB)

�[34m�[1mTensorFlow SavedModel:�[0m starting export with tensorflow 2.16.1...
WARNING ⚠️ tensorflow<=2.13.1 is required, but tensorflow==2.16.1 is currently installed https://github.com/ultralytics/ultralytics/issues/5161

�[34m�[1mONNX:�[0m starting export with onnx 1.16.0 opset 17...
�[34m�[1mONNX:�[0m simplifying with onnxsim 0.4.36...
�[34m�[1mONNX:�[0m export success ✅ 2.6s, saved as 'yolov8n.onnx' (12.3 MB)
�[34m�[1mTensorFlow SavedModel:�[0m starting TFLite export with onnx2tf 1.17.5...

�[07mAutomatic generation of each OP name started�[0m ========================================
�[32mAutomatic generation of each OP name complete!�[0m

�[07mModel loaded�[0m ========================================================================

�[07mModel conversion started�[0m ============================================================
�[33mWARNING:�[0m The optimization process for shape estimation is skipped because it contains OPs that cannot be inferred by the standard onnxruntime.
�[33mWARNING:�[0m module 'onnx' has no attribute '_serialize'
�[31mERROR:�[0m The trace log is below.
Traceback (most recent call last):
  File "/home/pi/yolo_env/lib/python3.11/site-packages/onnx2tf/utils/common_functions.py", line 288, in print_wrapper_func
    result = func(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^
  File "/home/pi/yolo_env/lib/python3.11/site-packages/onnx2tf/utils/common_functions.py", line 361, in inverted_operation_enable_disable_wrapper_func
    result = func(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^
  File "/home/pi/yolo_env/lib/python3.11/site-packages/onnx2tf/ops/Conv.py", line 246, in make_node
    input_tensor = get_padding_as_op(
                   ^^^^^^^^^^^^^^^^^^
  File "/home/pi/yolo_env/lib/python3.11/site-packages/onnx2tf/utils/common_functions.py", line 2009, in get_padding_as_op
    return tf.pad(x, padding)
           ^^^^^^^^^^^^^^^^^^
  File "/home/pi/yolo_env/lib/python3.11/site-packages/tensorflow/python/util/traceback_utils.py", line 153, in error_handler
    raise e.with_traceback(filtered_tb) from None
  File "/home/pi/yolo_env/lib/python3.11/site-packages/keras/src/backend/common/keras_tensor.py", line 91, in __tf_tensor__
    raise ValueError(
ValueError: A KerasTensor cannot be used as input to a TensorFlow function. A KerasTensor is a symbolic placeholder for a shape and dtype, used when constructing Keras Functional models or Keras Functions. You can only use it as input to a Keras layer or a Keras operation (from the namespaces `keras.layers` and `keras.operations`). You are likely doing something like:

x = Input(...)
...
tf_fn(x) # Invalid.

What you should do instead is wrap `tf_fn` in a layer:

class MyLayer(Layer):
def call(self, x):
return tf_fn(x)

x = MyLayer()(x)

�[31mERROR:�[0m input_onnx_file_path: yolov8n.onnx
�[31mERROR:�[0m onnx_op_name: wa/model.0/conv/Conv
�[31mERROR:�[0m Read this and deal with it. https://github.com/PINTO0309/onnx2tf#parameter-replacement
�[31mERROR:�[0m Alternatively, if the input OP has a dynamic dimension, use the -b or -ois option to rewrite it to a static shape and try again.
�[31mERROR:�[0m If the input OP of ONNX before conversion is NHWC or an irregular channel arrangement other than NCHW, use the -kt or -kat option.
�[31mERROR:�[0m Also, for models that include NonMaxSuppression in the post-processing, try the -onwdt option.

I ran the export in colab and the export worked, even though it errors out there too. So I used the colab exports and copied them to the RPi5, created the python file with make_interpreter, but get this error as described here too:

Traceback (most recent call last):
  File "/home/pi/tpu_detect.py", line 13, in <module>
    detections = detect.get_objects(interpreter, scale)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/pi/yolo_env/lib/python3.11/site-packages/pycoral/adapters/detect.py", line 210, in get_objects
    count = int(interpreter.tensor(signature['outputs']['output_0'])()[0])
                                   ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^
KeyError: 'output_0'

What am I missing? Has anyone applied this to an RPi5 successfully?

[pderrenger]

Hey there! 🌟 It seems like you're encountering some issues with exporting your model and running it successfully on the Raspberry Pi. Let's see if we can address this.

Firstly, regarding the TensorFlow version warning during export, the recommended TensorFlow version for optimal compatibility is <=2.13.1. However, since you managed to export successfully in Colab, we'll focus on the next steps.

Concerning the error you're facing when trying to run the model with the Edge TPU on your RPi5, it looks like there might be an issue with how the model outputs or Tensor names are expected by the pycoral library. The KeyError: 'output_0' suggests that the expected output tensor name doesn't match.

One potential solution is to explicitly define the output tensor names when exporting your model. However, given the complexity and potential changes in both the exporting process and how pycoral accesses model outputs, this might not always be straightforward.

Since the Coral project has had limited updates, and depending on the specific versions of TensorFlow Lite and pycoral you're working with, compatibility issues might arise. If you haven't already, make sure you're using the versions mentioned in the updated guide for Raspberry Pi with Coral Edge TPU. This might help mitigate some of the version mismatch issues.

As a temporary workaround, double-check the versions of all libraries and runtimes involved. It may also help to explore any available forums or GitHub issues related to pycoral and TensorFlow Lite compatibility for further insights.

Sorry to hear about the trouble this is causing you! If these steps don't resolve your issue, please consider providing the model or more details about the error, and I'd be happy to take a closer look. Hope this helps a bit! 🚀

[StefansAI]

@pderrenger, thanks a lot for the quick reply.
I had TensorFlow 2.16.1 installed, it may have been installed by the export function as dependency. After downgrading to 2.13.1 I could run the export successfully on the Rpi5. It still had an error at the end, not being able to install the edgetpu-compiler, which is not available anymore.

Ultralytics YOLOv8.1.43 🚀 Python-3.11.2 torch-2.2.2 CPU (Cortex-A76)
YOLOv8n summary (fused): 168 layers, 3151904 parameters, 0 gradients, 8.7 GFLOPs

�[34m�[1mPyTorch:�[0m starting from 'yolov8n.pt' with input shape (1, 3, 640, 640) BCHW and output shape(s) (1, 84, 8400) (6.2 MB)

�[34m�[1mTensorFlow SavedModel:�[0m starting export with tensorflow 2.13.1...

�[34m�[1mONNX:�[0m starting export with onnx 1.16.0 opset 17...
�[34m�[1mONNX:�[0m simplifying with onnxsim 0.4.36...
�[34m�[1mONNX:�[0m export success ✅ 2.7s, saved as 'yolov8n.onnx' (12.3 MB)
�[34m�[1mTensorFlow SavedModel:�[0m starting TFLite export with onnx2tf 1.17.5...

�[07mAutomatic generation of each OP name started�[0m ========================================
�[32mAutomatic generation of each OP name complete!�[0m

�[07mModel loaded�[0m ========================================================================

�[07mModel conversion started�[0m ============================================================
�[33mWARNING:�[0m The optimization process for shape estimation is skipped because it contains OPs that cannot be inferred by the standard onnxruntime.
�[33mWARNING:�[0m module 'onnx' has no attribute '_serialize'
�[07msaved_model output started�[0m ==========================================================
�[32msaved_model output complete!�[0m
�[32mFloat32 tflite output complete!�[0m
�[32mFloat16 tflite output complete!�[0m
�[34mInput signature information for quantization�[0m
�[34msignature_name�[0m: serving_default
�[34minput_name.0�[0m: images �[34mshape�[0m: (1, 640, 640, 3) �[34mdtype�[0m: <dtype: 'float32'>
�[32mDynamic Range Quantization tflite output complete!�[0m
�[32mINT8 Quantization tflite output complete!�[0m
�[32mFull INT8 Quantization tflite output complete!�[0m
�[32mINT8 Quantization with int16 activations tflite output complete!�[0m
�[32mFull INT8 Quantization with int16 activations tflite output complete!�[0m
�[34m�[1mTensorFlow SavedModel:�[0m export success ✅ 291.0s, saved as 'yolov8n_saved_model' (41.0 MB)
�[34m�[1mEdge TPU:�[0m WARNING ⚠️ Edge TPU known bug https://github.com/ultralytics/ultralytics/issues/1185

�[34m�[1mEdge TPU:�[0m export requires Edge TPU compiler. Attempting install from https://coral.ai/docs/edgetpu/compiler/
OK
deb https://packages.cloud.google.com/apt coral-edgetpu-stable main
Hit:1 http://deb.debian.org/debian bookworm InRelease
Hit:2 http://deb.debian.org/debian-security bookworm-security InRelease
Hit:3 http://deb.debian.org/debian bookworm-updates InRelease
Hit:4 https://packages.cloud.google.com/apt coral-edgetpu-stable InRelease
Hit:5 http://archive.raspberrypi.com/debian bookworm InRelease
Reading package lists...
Reading package lists...
Building dependency tree...
Reading state information...
�[34m�[1mEdge TPU:�[0m export failure ❌ 3.4s: Command 'sudo apt-get install edgetpu-compiler' returned non-zero exit status 100.

I downloaded the bus image and ran the suggested python code:

from pycoral.adapters import detect
from pycoral.adapters import common
from pycoral.utils.edgetpu import make_interpreter
from PIL import Image

interpreter = make_interpreter('yolov8n_saved_model/yolov8n_full_integer_quant.tflite')
interpreter.allocate_tensors()

image = Image.open('bus.jpg')
_, scale = common.set_resized_input(interpreter, image.size, lambda size: image.resize(size, Image.LANCZOS))
interpreter.invoke()

detections = detect.get_objects(interpreter, scale)

The output error has changed now to following:

python tpu_detect.py 
Traceback (most recent call last):
  File "/home/pi/tpu_detect.py", line 13, in <module>
    detections = detect.get_objects(interpreter, scale)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/pi/yolo_env/lib/python3.11/site-packages/pycoral/adapters/detect.py", line 214, in get_objects
    elif common.output_tensor(interpreter, 3).size == 1:
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/pi/yolo_env/lib/python3.11/site-packages/pycoral/adapters/common.py", line 29, in output_tensor
    return interpreter.tensor(interpreter.get_output_details()[i]['index'])()
                              ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^
IndexError: list index out of range

To round it up, I ran a pip freeze and reduced the output to what I thought was relevant:

flatbuffers==24.3.25
h5py==3.10.0
keras==2.13.1
matplotlib==3.8.4
numpy==1.24.3
onnx==1.16.0
onnx-graphsurgeon==0.3.27
onnx2tf==1.17.5
onnxruntime==1.17.1
onnxsim==0.4.36
opencv-python==4.9.0.80
pandas==2.2.1
pillow==10.3.0
protobuf==4.25.3
pycoral @ file:///home/pi/pycoral-2.13.0-cp311-cp311-linux_aarch64.whl
python-dateutil==2.9.0.post0
PyYAML==6.0.1
tensorboard==2.13.0
tensorboard-data-server==0.7.2
tensorflow-aarch64==2.13.1
tensorflow-estimator==2.13.0
tensorflow-io-gcs-filesystem==0.36.0
tflite-runtime==2.14.0
tflite-support==0.1.0a1
torch==2.2.2
torchvision==0.17.2
tqdm==4.66.2
ultralytics==8.1.43

Not sure, which versions are really required here. Is there a list to check?

[pderrenger]

@StefansAI hey there! 🌞 Great to hear you've made progress with TensorFlow 2.13.1 on your Rpi5. It looks like you're encountering some final hurdles related to the Edge TPU compiler, which indeed can be a bit tricky since the compiler for ARM architecture isn’t readily available. Your detailed documentation and adaptation process are commendable! 🚀

Regarding the KeyError: 'output_0' and your following IndexError, these suggest there might be a disconnect between the expected model output tensor names and what's being provided. Since pycoral and the Edge TPU compiler expect specific outputs, ensuring the exported .tflite model aligns with these expectations is critical.

For the Edge TPU, TensorFlow Lite models need to be fully quantized, and it seems like you're on the right track with quantization. If you're still facing issues, one workaround could involve double-checking the model output names before and after quantization to ensure they match pycoral's expectations.

Since the Edge TPU compiler installation directly on the Raspberry Pi is a challenge, your approach of exporting the model elsewhere and transferring it is wise. If anyone has found an innovative solution or workaround for the Edge TPU compiler on ARM architecture, this is a valuable opportunity for them to share insights.

For the Pip freeze and library version concerns, the relevant ones mostly align with TensorFlow Lite runtime and pycoral versions. Given your setup, you might not need TensorFlow if you're running inferences directly with TensorFlow Lite runtime (tflite-runtime). This could simplify your environment a tad more.

In essence, focusing on the compatibility of exported model tensor names with the pycoral adapter and ensuring all model operations are Edge TPU compatible through careful quantization should be the priority. If the community has additional insights or solutions, chiming in would greatly benefit everyone working on similar projects. Best of luck, and keep pushing the boundaries! 🌟

[DAVIDNYARKO123]

I encountered similar challenges and ultimately developed a "solution". I contribute to a PyPI package designed to support the Ultralytics community, specifically for Raspberry Pi users employing the EDGE TPU USB Accelerator. This package simplifies the setup process by ensuring the correct versions of necessary runtimes are installed with just a single pip and setup command. It also provides several default YOLO models for immediate download and experimentation.

I hope this information is useful to you.

GitHub:

https://github.com/DAVIDNYARKO123/edge-tpu-silva

PyPi:

https://pypi.org/project/edge-tpu-silva/

YouTube:

https://www.youtube.com/watch?v=sOxQTRRh9tw

I just posted the video above after seeing this post.

[DAVIDNYARKO123]

Hey there! 🌟 Thanks for sharing your work on the edge-tpu-silva package - it sounds like a fantastic resource for the community, especially for those working with Raspberry Pi and the Coral Edge TPU. Your effort to simplify the setup process and provide readily available YOLO models is much appreciated. 🙏

For anyone interested in leveraging the power of Coral's Edge TPU on a Raspberry Pi, this package could be a great starting point. I'll definitely check out your YouTube video for a deeper dive into how it all works. 📺💡

Please keep up the great work, and feel free to reach out if you think there are ways we could collaborate or further support your efforts. The link to our guide on using Ultralytics YOLOv8 with a Coral Edge TPU on a Raspberry Pi might also complement your solution: Coral Edge TPU on Raspberry Pi with Ultralytics YOLOv8.

Best of luck with your project, and looking forward to seeing more from you! 🚀

Hello @pderrenger
Thank you for the encouraging words! ✨I’m glad you found the edge-tpu-silva package interesting and I’m eager to explore the guide you provided. It was a great complement and inspiration to this work.

I’m open to collaborating and would love to discuss how we can support this project. Looking forward to connecting soon! 🚀

[StefansAI]

Hi David,

I tried your approach. I always start from a new SD card image of the 64bit Bookworm in the RPi5 and then run the suggested commands. In your case it is:

python3 -m venv .venv
source .venv/bin/activate
pip install edge-tpu-silva
silvatpu-linux-setup

Then I downloaded the first of your exported models and run this line:

silvatpu -p det -m Downloads/240_yolov8n_full_integer_quant_edgetpu.tflite -i /dev/video0 -z 240 -t 0.5 -v True

Then I get this error:

(.venv) pi@pi5:~ $ silvatpu -p det -m Downloads/240_yolov8n_full_integer_quant_edgetpu.tflite -i /dev/video0 -z 240 -t 0.5 -v True
Traceback (most recent call last):
  File "/home/pi/.venv/lib/python3.11/site-packages/ultralytics/nn/autobackend.py", line 321, in __init__
    from tflite_runtime.interpreter import Interpreter, load_delegate
ModuleNotFoundError: No module named 'tflite_runtime'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/pi/.venv/bin/silvatpu", line 8, in <module>
    sys.exit(silvatpu())
             ^^^^^^^^^^
  File "/home/pi/.venv/lib/python3.11/site-packages/edge_tpu_silva/__main__.py", line 97, in silvatpu
    for _, _ in outs:
  File "/home/pi/.venv/lib/python3.11/site-packages/edge_tpu_silva/silva/silva_detect.py", line 38, in process_detection
    outs = model.predict(
           ^^^^^^^^^^^^^^
  File "/home/pi/.venv/lib/python3.11/site-packages/ultralytics/engine/model.py", line 445, in predict
    self.predictor.setup_model(model=self.model, verbose=is_cli)
  File "/home/pi/.venv/lib/python3.11/site-packages/ultralytics/engine/predictor.py", line 297, in setup_model
    self.model = AutoBackend(
                 ^^^^^^^^^^^^
  File "/home/pi/.venv/lib/python3.11/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/pi/.venv/lib/python3.11/site-packages/ultralytics/nn/autobackend.py", line 323, in __init__
    import tensorflow as tf
ModuleNotFoundError: No module named 'tensorflow'

I then tried to install tensorflow-aarch64==2.13.1, but ran into problems with dependencies of h5py. Any thoughts?

[DAVIDNYARKO123]

Hi @StefansAI
I will be glad to assist. Please what was the terminal output after running the setup command:

silvatpu-linux-setup

Did you see a message indicating successful installation, including the runtime version, at the end of the setup process?

Please list your Python version.

Be sure to use the recommended Python versions. As well as your apt-get is updated and upraded

sudo apt-get update
sudo apt-get upgrade

[Decoo13]

Hey David, thanks again for the information!

everything went smoothly for me I followed the README instructiions and everything worked well on my end.
I would like to note that I did place everything in the home directory to make things simpler in terms of placing the correct path for each of the files.

[pderrenger]

Hey David,

That's fantastic to hear! 🚀 Glad the setup went smoothly for you. Placing everything in the home directory can indeed simplify managing paths, especially for newcomers. It's a practical tip that could benefit many users, so thanks for highlighting it!

For anyone else following this thread, remember to check the Raspberry Pi model and OS compatibility, especially for tasks involving hardware accelerators like the Coral Edge TPU. Our documentation provides a detailed guide for setups like these, ensuring you get the best performance out of your Edge TPU and Raspberry Pi combo.

Keep exploring and sharing your insights, David! If you've got more questions or run into hiccups, feel free to drop them here. 🌟

[torayeff]

Is this also supported for M.2 Accelerator A+E key instead of USB? I am currently connecting directly to that interface on my RPi 5.

[pderrenger]

Hey there! 👋 The direct support for the M.2 Accelerator with the A+E key on Raspberry Pi isn’t explicitly detailed in our documentation. However, as long as your Raspberry Pi 5 can recognize the accelerator as a compatible device and you have the proper drivers/runtime installed, our guide for using the Coral Edge TPU with a Raspberry Pi should still be applicable. The main steps involve setting up the Edge TPU runtime correctly and exporting your model to a format compatible with the Coral Edge TPU (i.e., TensorFlow Lite with full integer quantization).

Here’s a quick snippet on how to export your model for the Edge TPU:

from ultralytics import YOLO

# Load your model
model = YOLO('path/to/model.pt')  # Replace with your model path

# Export the model for Edge TPU
model.export(format='edgetpu')

Make sure the Edge TPU runtime is installed properly on your RPi, and you should be good to go. If you encounter specific issues with the M.2 interface, it might be worth checking on compatibility forums or the official Coral documentation. 😊

[StefansAI]

Hi @pderrenger:
I gave it one more shot to install and run your libraries for Python3.11 as described here together with Pycoral for 3.11 from Oberluz. But in the end, there is still that mismatch between interpreter and detect.get_objects().
interpreter.get_output_details() returns a list with only one element, but the element of the index = 3 is requested. I tried to debug with VSS code, but there was nothing I else I could see. I have a screenshot, but I don't see how to attach it here.

Hi @DAVIDNYARKO123:
As said, I tried the bookworm out of the box with Python 3.11 and was really hoping I get it with 3.11 to work. I saw now, you required <3.10. So, I installed 3.9 in an venv and followed your instructions. It works and I get text outputs, no video or images. I can dig into that to see something. Thanks, I will check it out more.

[DAVIDNYARKO123]

@StefansAI
Great, I am glad 🎉!
To get the frame outputted, you can set the —-show/-s to True.

You can refer to the git link for more on that.

Future version of edge-tpu-silva which i am currently working on will include 3.10 and 3.11.

[StefansAI]

@DAVIDNYARKO123, thanks, I found it, started to play around with the code and added displaying the inference times.
With 640 yolov8n export and camera input the times are around 240ms average. My own test video with only 4 classes is around 210ms. For comparison, my Jetson Nano with only 4 classes does the same around 70ms.
With your 240 exports, the inference time on life stream video drops to ~30ms, which sounds great.

[StefansAI]

@DAVIDNYARKO123:
I had to realize, that it doesn't use the TPU at all and the acceleration of less than factor 2 compared to yolo without TPU must have come from the integer export. While streaming from the camera I unplugged the TPU and it kept running with the same speed. Can you confirm?

[oliverlibaw]

Thanks so much @StefansAI.

[UsamaRehman001]

@StefansAI how do you give camera source to the model. or frame by frame of 640 x 640. because even when i export the model. some layers are not quantized for the edgetpu shown by log file. also i get segmentation fault for any size greater than 192. can you help me figure out

[StefansAI]

@UsamaRehman001

Look for predict examples like here for instance here:

model = YOLO("model.pt")
# accepts all formats - image/dir/Path/URL/video/PIL/ndarray. 0 for webcam
results = model.predict(source="0")

source=0 is the webcam (USB). stream=True and show=True

[UsamaRehman001]

@StefansAI my camera is CSI camera. thats why i am confused for the source. thats a separate issue but any model exported for greater than 192x192 give "Segmentation Fault"

[restless-robin]

@glenn-jocher do you have any thoughts on why @StefansAI would be getting a lack of performance improvement using the TPU even at the smaller resolutions? Has anyone else been able to show an actual performance improvement (at any resolution level) for YOLOv8 on the Coral TPU?

[FearL0rd]

Hello my friends from Earth.
This is the compiler for ARM
https://github.com/FearL0rd/edgetpu_compiler_for_ARM64

[glenn-jocher]

@FearL0rd hello! 🌍 Thanks for sharing the ARM compiler link for the Edge TPU. It's definitely useful for those looking to run inference on ARM64 devices with Edge TPUs.

For incorporating it with Ultralytics YOLOv8, you would generally export the YOLO model to a TensorFlow Lite format that's compatible with the Edge TPU. Just make sure the exported model is fully quantized to leverage the Edge TPU's capabilities. Here's how you might export the model using the Ultralytics framework:

from ultralytics import YOLO

# Load your model
model = YOLO('path/to/model.pt')

# Export the model to a TFLite format compatible with Edge TPU
model.export(format='tflite', int8=True)

Then, you can compile this .tflite model for the Edge TPU using the ARM-compatible compiler that you've mentioned. If there's anything specific you need help with regarding this process, feel free to ask! 🚀

[StefansAI]

@glenn-jocher, @FearL0rd,

I downloaded the compiler, did a new export with the code above, but get an error from the compiler when using

./edgetpu_compiler model_full_integer_quant.tflite

Internal compiler error. Aborting!

Other export outputs give "Model not quantized". Any thoughts, why the fully quantized model throws an error?

[glenn-jocher]

To fix the "Internal compiler error. Aborting!" when running edgetpu_compiler:

1. Update the Compiler: Ensure you are using the latest version of the Edge TPU compiler. You can download it from the official Coral website.

2. Check Model Compatibility: Verify that your model is fully integer-quantized and compatible with Edge TPU. Ensure all operations in the model are supported by the TPU.

3. Simplify the Model: Try reducing the model complexity by removing or simplifying unsupported operations.

4. Check for File Corruption: Ensure the .tflite model file is not corrupted.

5. Run on a Different Machine: Sometimes, trying the compilation on a different machine or environment can help identify if the issue is system-specific.

6. Consult Logs: Look at the detailed logs or error messages provided by the compiler for more specific issues.

By following these steps, you should be able to resolve or identify the cause of the internal compiler error.

[ninopleno]

Hi there everyone!

Any stable solution for using Coral Accelerator USB?

Thanks in advanced!

[pderrenger]

@ninopleno hey there!

It looks like you're encountering a common issue related to missing operation registrations in TensorFlow Lite when using the Edge TPU. This typically happens when the model uses ops that are not supported by the TensorFlow Lite runtime configured for the Edge TPU.

Here are a couple of steps to potentially resolve this:

1. Ensure Compatibility: Double-check that all operations used in your model are supported by the TensorFlow Lite Edge TPU. You might need to modify your model to replace or remove unsupported operations.

2. Recompile the Model: Use the latest version of the Edge TPU Compiler to recompile your .tflite model. Sometimes, recompiling with the latest tools can resolve these issues as support for operations can improve over time.

3. TensorFlow Lite Runtime: Make sure you're using a compatible version of the TensorFlow Lite runtime. If there's a mismatch between the runtime and the model's requirements, that could cause the errors you're seeing.

For detailed steps on model conversion and setup, you might find our guide on Coral Edge TPU usage helpful. 😊

If the issue persists, please provide more details about the operations used in your model, and we can look into this further!

[ninopleno]

Hi everyone,

is there any updates from Ultralytics? My installation is failing...

ERROR: Cannot install ultralytics==0.0.13, ultralytics==0.0.14, ultralytics==0.0.15, ultralytics==0.0.16, ultralytics==0.0.17, ultralytics==0.0.18, ultralytics==0.0.19, ultralytics==0.0.20, ultralytics==0.0.21, ultralytics==0.0.22, ultralytics==0.0.23, ultralytics==0.0.24, ultralytics==0.0.25, ultralytics==0.0.26, ultralytics==0.0.27, ultralytics==0.0.28, ultralytics==0.0.29, ultralytics==0.0.30, ultralytics==0.0.31, ultralytics==0.0.32, ultralytics==0.0.33, ultralytics==0.0.34, ultralytics==0.0.35, ultralytics==0.0.36, ultralytics==0.0.37, ultralytics==0.0.38, ultralytics==0.0.39, ultralytics==0.0.40, ultralytics==0.0.41, ultralytics==0.0.42, ultralytics==0.0.43, ultralytics==0.0.44, ultralytics==8.0.0, ultralytics==8.0.1, ultralytics==8.0.10, ultralytics==8.0.100, ultralytics==8.0.101, ultralytics==8.0.102, ultralytics==8.0.103, ultralytics==8.0.104, ultralytics==8.0.105, ultralytics==8.0.106, ultralytics==8.0.107, ultralytics==8.0.108, ultralytics==8.0.109, ultralytics==8.0.11, ultralytics==8.0.110, ultralytics==8.0.111, ultralytics==8.0.112, ultralytics==8.0.113, ultralytics==8.0.114, ultralytics==8.0.115, ultralytics==8.0.116, ultralytics==8.0.117, ultralytics==8.0.118, ultralytics==8.0.119, ultralytics==8.0.120, ultralytics==8.0.121, ultralytics==8.0.122, ultralytics==8.0.123, ultralytics==8.0.124, ultralytics==8.0.125, ultralytics==8.0.126, ultralytics==8.0.127, ultralytics==8.0.128, ultralytics==8.0.13, ultralytics==8.0.130, ultralytics==8.0.131, ultralytics==8.0.132, ultralytics==8.0.133, ultralytics==8.0.134, ultralytics==8.0.135, ultralytics==8.0.136, ultralytics==8.0.137, ultralytics==8.0.138, ultralytics==8.0.139, ultralytics==8.0.14, ultralytics==8.0.140, ultralytics==8.0.141, ultralytics==8.0.142, ultralytics==8.0.143, ultralytics==8.0.144, ultralytics==8.0.145, ultralytics==8.0.146, ultralytics==8.0.147, ultralytics==8.0.148, ultralytics==8.0.149, ultralytics==8.0.150, ultralytics==8.0.151, ultralytics==8.0.152, ultralytics==8.0.153, ultralytics==8.0.154, ultralytics==8.0.155, ultralytics==8.0.156, ultralytics==8.0.157, ultralytics==8.0.158, ultralytics==8.0.159, ultralytics==8.0.16, ultralytics==8.0.160, ultralytics==8.0.161, ultralytics==8.0.162, ultralytics==8.0.163, ultralytics==8.0.164, ultralytics==8.0.165, ultralytics==8.0.166, ultralytics==8.0.167, ultralytics==8.0.168, ultralytics==8.0.169, ultralytics==8.0.17, ultralytics==8.0.170, ultralytics==8.0.171, ultralytics==8.0.172, ultralytics==8.0.173, ultralytics==8.0.175, ultralytics==8.0.176, ultralytics==8.0.178, ultralytics==8.0.179, ultralytics==8.0.18, ultralytics==8.0.180, ultralytics==8.0.181, ultralytics==8.0.182, ultralytics==8.0.183, ultralytics==8.0.184, ultralytics==8.0.185, ultralytics==8.0.186, ultralytics==8.0.187, ultralytics==8.0.188, ultralytics==8.0.189, ultralytics==8.0.19, ultralytics==8.0.190, ultralytics==8.0.191, ultralytics==8.0.192, ultralytics==8.0.193, ultralytics==8.0.194, ultralytics==8.0.195, ultralytics==8.0.196, ultralytics==8.0.197, ultralytics==8.0.198, ultralytics==8.0.199, ultralytics==8.0.2, ultralytics==8.0.20, ultralytics==8.0.200, ultralytics==8.0.201, ultralytics==8.0.202, ultralytics==8.0.203, ultralytics==8.0.204, ultralytics==8.0.205, ultralytics==8.0.206, ultralytics==8.0.207, ultralytics==8.0.208, ultralytics==8.0.209, ultralytics==8.0.21, ultralytics==8.0.210, ultralytics==8.0.211, ultralytics==8.0.212, ultralytics==8.0.213, ultralytics==8.0.214, ultralytics==8.0.215, ultralytics==8.0.216, ultralytics==8.0.217, ultralytics==8.0.218, ultralytics==8.0.219, ultralytics==8.0.22, ultralytics==8.0.220, ultralytics==8.0.221, ultralytics==8.0.222, ultralytics==8.0.223, ultralytics==8.0.224, ultralytics==8.0.225, ultralytics==8.0.226, ultralytics==8.0.227, ultralytics==8.0.228, ultralytics==8.0.229, ultralytics==8.0.23, ultralytics==8.0.230, ultralytics==8.0.231, ultralytics==8.0.232, ultralytics==8.0.233, ultralytics==8.0.234, ultralytics==8.0.235, ultralytics==8.0.236, ultralytics==8.0.237, ultralytics==8.0.238, ultralytics==8.0.239, ultralytics==8.0.24, ultralytics==8.0.25, ultralytics==8.0.26, ultralytics==8.0.27, ultralytics==8.0.28, ultralytics==8.0.29, ultralytics==8.0.3, ultralytics==8.0.30, ultralytics==8.0.31, ultralytics==8.0.32, ultralytics==8.0.33, ultralytics==8.0.34, ultralytics==8.0.35, ultralytics==8.0.36, ultralytics==8.0.37, ultralytics==8.0.38, ultralytics==8.0.39, ultralytics==8.0.4, ultralytics==8.0.40, ultralytics==8.0.41, ultralytics==8.0.42, ultralytics==8.0.43, ultralytics==8.0.44, ultralytics==8.0.45, ultralytics==8.0.46, ultralytics==8.0.47, ultralytics==8.0.48, ultralytics==8.0.49, ultralytics==8.0.5, ultralytics==8.0.50, ultralytics==8.0.51, ultralytics==8.0.52, ultralytics==8.0.53, ultralytics==8.0.54, ultralytics==8.0.55, ultralytics==8.0.56, ultralytics==8.0.57, ultralytics==8.0.58, ultralytics==8.0.59, ultralytics==8.0.6, ultralytics==8.0.60, ultralytics==8.0.61, ultralytics==8.0.62, ultralytics==8.0.63, ultralytics==8.0.64, ultralytics==8.0.65, ultralytics==8.0.66, ultralytics==8.0.67, ultralytics==8.0.68, ultralytics==8.0.69, ultralytics==8.0.7, ultralytics==8.0.70, ultralytics==8.0.71, ultralytics==8.0.72, ultralytics==8.0.73, ultralytics==8.0.74, ultralytics==8.0.75, ultralytics==8.0.76, ultralytics==8.0.77, ultralytics==8.0.78, ultralytics==8.0.79, ultralytics==8.0.8, ultralytics==8.0.80, ultralytics==8.0.81, ultralytics==8.0.82, ultralytics==8.0.83, ultralytics==8.0.84, ultralytics==8.0.85, ultralytics==8.0.86, ultralytics==8.0.87, ultralytics==8.0.88, ultralytics==8.0.89, ultralytics==8.0.9, ultralytics==8.0.90, ultralytics==8.0.91, ultralytics==8.0.92, ultralytics==8.0.93, ultralytics==8.0.94, ultralytics==8.0.95, ultralytics==8.0.96, ultralytics==8.0.97, ultralytics==8.0.98, ultralytics==8.0.99, ultralytics==8.1.0, ultralytics==8.1.1, ultralytics==8.1.10, ultralytics==8.1.11, ultralytics==8.1.12, ultralytics==8.1.13, ultralytics==8.1.14, ultralytics==8.1.15, ultralytics==8.1.16, ultralytics==8.1.17, ultralytics==8.1.18, ultralytics==8.1.19, ultralytics==8.1.2, ultralytics==8.1.20, ultralytics==8.1.22, ultralytics==8.1.23, ultralytics==8.1.24, ultralytics==8.1.25, ultralytics==8.1.26, ultralytics==8.1.27, ultralytics==8.1.28, ultralytics==8.1.29, ultralytics==8.1.3, ultralytics==8.1.30, ultralytics==8.1.32, ultralytics==8.1.33, ultralytics==8.1.34, ultralytics==8.1.35, ultralytics==8.1.36, ultralytics==8.1.37, ultralytics==8.1.38, ultralytics==8.1.39, ultralytics==8.1.4, ultralytics==8.1.40, ultralytics==8.1.41, ultralytics==8.1.42, ultralytics==8.1.43, ultralytics==8.1.44, ultralytics==8.1.45, ultralytics==8.1.46, ultralytics==8.1.47, ultralytics==8.1.5, ultralytics==8.1.6, ultralytics==8.1.7, ultralytics==8.1.8, ultralytics==8.1.9, ultralytics==8.2.0, ultralytics==8.2.1, ultralytics==8.2.10, ultralytics==8.2.11, ultralytics==8.2.12, ultralytics==8.2.13, ultralytics==8.2.14, ultralytics==8.2.15, ultralytics==8.2.16, ultralytics==8.2.17, ultralytics==8.2.18, ultralytics==8.2.19, ultralytics==8.2.2, ultralytics==8.2.20, ultralytics==8.2.21, ultralytics==8.2.22, ultralytics==8.2.23, ultralytics==8.2.24, ultralytics==8.2.25, ultralytics==8.2.26, ultralytics==8.2.27, ultralytics==8.2.28, ultralytics==8.2.3, ultralytics==8.2.4, ultralytics==8.2.5, ultralytics==8.2.6, ultralytics==8.2.8 and ultralytics==8.2.9 because these package versions have conflicting dependencies.

The conflict is caused by:
    ultralytics 8.2.28 depends on torchvision>=0.9.0
    ultralytics 8.2.27 depends on torchvision>=0.9.0
    ultralytics 8.2.26 depends on torchvision>=0.9.0
    ultralytics 8.2.25 depends on torchvision>=0.9.0
    ultralytics 8.2.24 depends on torchvision>=0.9.0
    ultralytics 8.2.23 depends on torchvision>=0.9.0
    ultralytics 8.2.22 depends on torchvision>=0.9.0
    ultralytics 8.2.21 depends on torchvision>=0.9.0
    ultralytics 8.2.20 depends on torchvision>=0.9.0
    ultralytics 8.2.19 depends on torchvision>=0.9.0
    ultralytics 8.2.18 depends on torchvision>=0.9.0
    ultralytics 8.2.17 depends on torchvision>=0.9.0
    ultralytics 8.2.16 depends on torchvision>=0.9.0
    ultralytics 8.2.15 depends on torchvision>=0.9.0
    ultralytics 8.2.14 depends on torchvision>=0.9.0
    ultralytics 8.2.13 depends on torchvision>=0.9.0
    ultralytics 8.2.12 depends on torchvision>=0.9.0
    ultralytics 8.2.11 depends on torchvision>=0.9.0
    ultralytics 8.2.10 depends on torchvision>=0.9.0
    ultralytics 8.2.9 depends on torchvision>=0.9.0
    ultralytics 8.2.8 depends on torchvision>=0.9.0
    ultralytics 8.2.6 depends on torchvision>=0.9.0
    ultralytics 8.2.5 depends on torchvision>=0.9.0
    ultralytics 8.2.4 depends on torchvision>=0.9.0
    ultralytics 8.2.3 depends on torchvision>=0.9.0
    ultralytics 8.2.2 depends on torchvision>=0.9.0
    ultralytics 8.2.1 depends on torchvision>=0.9.0
    ultralytics 8.2.0 depends on torchvision>=0.9.0
    ultralytics 8.1.47 depends on torchvision>=0.9.0
    ultralytics 8.1.46 depends on torchvision>=0.9.0
    ultralytics 8.1.45 depends on torchvision>=0.9.0
    ultralytics 8.1.44 depends on torchvision>=0.9.0
    ultralytics 8.1.43 depends on torchvision>=0.9.0
    ultralytics 8.1.42 depends on torchvision>=0.9.0
    ultralytics 8.1.41 depends on torchvision>=0.9.0
    ultralytics 8.1.40 depends on torchvision>=0.9.0
    ultralytics 8.1.39 depends on torchvision>=0.9.0
    ultralytics 8.1.38 depends on torchvision>=0.9.0
    ultralytics 8.1.37 depends on torchvision>=0.9.0
    ultralytics 8.1.36 depends on torchvision>=0.9.0
    ultralytics 8.1.35 depends on torchvision>=0.9.0
    ultralytics 8.1.34 depends on torchvision>=0.9.0
    ultralytics 8.1.33 depends on torchvision>=0.9.0
    ultralytics 8.1.32 depends on torchvision>=0.9.0
    ultralytics 8.1.30 depends on torchvision>=0.9.0
    ultralytics 8.1.29 depends on torchvision>=0.9.0
    ultralytics 8.1.28 depends on torchvision>=0.9.0
    ultralytics 8.1.27 depends on torchvision>=0.9.0
    ultralytics 8.1.26 depends on torchvision>=0.9.0
    ultralytics 8.1.25 depends on torchvision>=0.9.0
    ultralytics 8.1.24 depends on torchvision>=0.9.0
    ultralytics 8.1.23 depends on torchvision>=0.9.0
    ultralytics 8.1.22 depends on torchvision>=0.9.0
    ultralytics 8.1.20 depends on torchvision>=0.9.0
    ultralytics 8.1.19 depends on torchvision>=0.9.0
    ultralytics 8.1.18 depends on torchvision>=0.9.0
    ultralytics 8.1.17 depends on torchvision>=0.9.0
    ultralytics 8.1.16 depends on torchvision>=0.9.0
    ultralytics 8.1.15 depends on torchvision>=0.9.0
    ultralytics 8.1.14 depends on torchvision>=0.9.0
    ultralytics 8.1.13 depends on torchvision>=0.9.0
    ultralytics 8.1.12 depends on torchvision>=0.9.0
    ultralytics 8.1.11 depends on torchvision>=0.9.0
    ultralytics 8.1.10 depends on torchvision>=0.9.0
    ultralytics 8.1.9 depends on torchvision>=0.9.0
    ultralytics 8.1.8 depends on torchvision>=0.9.0
    ultralytics 8.1.7 depends on torchvision>=0.9.0
    ultralytics 8.1.6 depends on torchvision>=0.9.0
    ultralytics 8.1.5 depends on torchvision>=0.9.0
    ultralytics 8.1.4 depends on torchvision>=0.9.0
    ultralytics 8.1.3 depends on torchvision>=0.9.0
    ultralytics 8.1.2 depends on torchvision>=0.9.0
    ultralytics 8.1.1 depends on torchvision>=0.9.0
    ultralytics 8.1.0 depends on torchvision>=0.9.0
    ultralytics 8.0.239 depends on torchvision>=0.9.0
    ultralytics 8.0.238 depends on torchvision>=0.9.0
    ultralytics 8.0.237 depends on torchvision>=0.9.0
    ultralytics 8.0.236 depends on torchvision>=0.9.0
    ultralytics 8.0.235 depends on torchvision>=0.9.0
    ultralytics 8.0.234 depends on torchvision>=0.9.0
    ultralytics 8.0.233 depends on torchvision>=0.9.0
    ultralytics 8.0.232 depends on torchvision>=0.9.0
    ultralytics 8.0.231 depends on torchvision>=0.9.0
    ultralytics 8.0.230 depends on torchvision>=0.9.0
    ultralytics 8.0.229 depends on torchvision>=0.9.0
    ultralytics 8.0.228 depends on torchvision>=0.9.0
    ultralytics 8.0.227 depends on torchvision>=0.9.0
    ultralytics 8.0.226 depends on torchvision>=0.9.0
    ultralytics 8.0.225 depends on torchvision>=0.9.0
    ultralytics 8.0.224 depends on torchvision>=0.9.0
    ultralytics 8.0.223 depends on torchvision>=0.9.0
    ultralytics 8.0.222 depends on torchvision>=0.9.0
    ultralytics 8.0.221 depends on torchvision>=0.9.0
    ultralytics 8.0.220 depends on torchvision>=0.9.0
    ultralytics 8.0.219 depends on torchvision>=0.9.0
    ultralytics 8.0.218 depends on torchvision>=0.9.0
    ultralytics 8.0.217 depends on torchvision>=0.9.0
    ultralytics 8.0.216 depends on torchvision>=0.9.0
    ultralytics 8.0.215 depends on torchvision>=0.9.0
    ultralytics 8.0.214 depends on torchvision>=0.9.0
    ultralytics 8.0.213 depends on torchvision>=0.9.0
    ultralytics 8.0.212 depends on torchvision>=0.9.0
    ultralytics 8.0.211 depends on torchvision>=0.9.0
    ultralytics 8.0.210 depends on torchvision>=0.9.0
    ultralytics 8.0.209 depends on torchvision>=0.9.0
    ultralytics 8.0.208 depends on torchvision>=0.9.0
    ultralytics 8.0.207 depends on torchvision>=0.9.0
    ultralytics 8.0.206 depends on torchvision>=0.9.0
    ultralytics 8.0.205 depends on torchvision>=0.9.0
    ultralytics 8.0.204 depends on torchvision>=0.9.0
    ultralytics 8.0.203 depends on torchvision>=0.9.0
    ultralytics 8.0.202 depends on torchvision>=0.9.0
    ultralytics 8.0.201 depends on torchvision>=0.9.0
    ultralytics 8.0.200 depends on torchvision>=0.9.0
    ultralytics 8.0.199 depends on torchvision>=0.9.0
    ultralytics 8.0.198 depends on torchvision>=0.9.0
    ultralytics 8.0.197 depends on torchvision>=0.9.0
    ultralytics 8.0.196 depends on torchvision>=0.9.0
    ultralytics 8.0.195 depends on torchvision>=0.9.0
    ultralytics 8.0.194 depends on torchvision>=0.9.0
    ultralytics 8.0.193 depends on torchvision>=0.9.0
    ultralytics 8.0.192 depends on torchvision>=0.9.0
    ultralytics 8.0.191 depends on torchvision>=0.9.0
    ultralytics 8.0.190 depends on torchvision>=0.9.0
    ultralytics 8.0.189 depends on torchvision>=0.9.0
    ultralytics 8.0.188 depends on torchvision>=0.9.0
    ultralytics 8.0.187 depends on torchvision>=0.9.0
    ultralytics 8.0.186 depends on torchvision>=0.9.0
    ultralytics 8.0.185 depends on torchvision>=0.9.0
    ultralytics 8.0.184 depends on torchvision>=0.9.0
    ultralytics 8.0.183 depends on torchvision>=0.9.0
    ultralytics 8.0.182 depends on torchvision>=0.9.0
    ultralytics 8.0.181 depends on torchvision>=0.9.0
    ultralytics 8.0.180 depends on torchvision>=0.9.0
    ultralytics 8.0.179 depends on torchvision>=0.9.0
    ultralytics 8.0.178 depends on torchvision>=0.9.0
    ultralytics 8.0.176 depends on torchvision>=0.9.0
    ultralytics 8.0.175 depends on torchvision>=0.9.0
    ultralytics 8.0.173 depends on torchvision>=0.9.0
    ultralytics 8.0.172 depends on torchvision>=0.9.0
    ultralytics 8.0.171 depends on torchvision>=0.9.0
    ultralytics 8.0.170 depends on torchvision>=0.9.0
    ultralytics 8.0.169 depends on torchvision>=0.9.0
    ultralytics 8.0.168 depends on torchvision>=0.9.0
    ultralytics 8.0.167 depends on torchvision>=0.9.0
    ultralytics 8.0.166 depends on torchvision>=0.9.0
    ultralytics 8.0.165 depends on torchvision>=0.9.0
    ultralytics 8.0.164 depends on torchvision>=0.9.0
    ultralytics 8.0.163 depends on torchvision>=0.9.0
    ultralytics 8.0.162 depends on torchvision>=0.9.0
    ultralytics 8.0.161 depends on torchvision>=0.9.0
    ultralytics 8.0.160 depends on torchvision>=0.9.0
    ultralytics 8.0.159 depends on torchvision>=0.9.0
    ultralytics 8.0.158 depends on torchvision>=0.9.0
    ultralytics 8.0.157 depends on torchvision>=0.9.0
    ultralytics 8.0.156 depends on torchvision>=0.9.0
    ultralytics 8.0.155 depends on torchvision>=0.9.0
    ultralytics 8.0.154 depends on torchvision>=0.9.0
    ultralytics 8.0.153 depends on torchvision>=0.9.0
    ultralytics 8.0.152 depends on torchvision>=0.9.0
    ultralytics 8.0.151 depends on torchvision>=0.8.1
    ultralytics 8.0.150 depends on torchvision>=0.8.1
    ultralytics 8.0.149 depends on torchvision>=0.8.1
    ultralytics 8.0.148 depends on torchvision>=0.8.1
    ultralytics 8.0.147 depends on torchvision>=0.8.1
    ultralytics 8.0.146 depends on torchvision>=0.8.1
    ultralytics 8.0.145 depends on torchvision>=0.8.1
    ultralytics 8.0.144 depends on torchvision>=0.8.1
    ultralytics 8.0.143 depends on torchvision>=0.8.1
    ultralytics 8.0.142 depends on torchvision>=0.8.1
    ultralytics 8.0.141 depends on torchvision>=0.8.1
    ultralytics 8.0.140 depends on torchvision>=0.8.1
    ultralytics 8.0.139 depends on torchvision>=0.8.1
    ultralytics 8.0.138 depends on torchvision>=0.8.1
    ultralytics 8.0.137 depends on torchvision>=0.8.1
    ultralytics 8.0.136 depends on torchvision>=0.8.1
    ultralytics 8.0.135 depends on torchvision>=0.8.1
    ultralytics 8.0.134 depends on torchvision>=0.8.1
    ultralytics 8.0.133 depends on torchvision>=0.8.1
    ultralytics 8.0.132 depends on torchvision>=0.8.1
    ultralytics 8.0.131 depends on torchvision>=0.8.1
    ultralytics 8.0.130 depends on torchvision>=0.8.1
    ultralytics 8.0.128 depends on torchvision>=0.8.1
    ultralytics 8.0.127 depends on torchvision>=0.8.1
    ultralytics 8.0.126 depends on torchvision>=0.8.1
    ultralytics 8.0.125 depends on torchvision>=0.8.1
    ultralytics 8.0.124 depends on torchvision>=0.8.1
    ultralytics 8.0.123 depends on torchvision>=0.8.1
    ultralytics 8.0.122 depends on torchvision>=0.8.1
    ultralytics 8.0.121 depends on torchvision>=0.8.1
    ultralytics 8.0.120 depends on torchvision>=0.8.1
    ultralytics 8.0.119 depends on torchvision>=0.8.1
    ultralytics 8.0.118 depends on torchvision>=0.8.1
    ultralytics 8.0.117 depends on torchvision>=0.8.1
    ultralytics 8.0.116 depends on torchvision>=0.8.1
    ultralytics 8.0.115 depends on torchvision>=0.8.1
    ultralytics 8.0.114 depends on torchvision>=0.8.1
    ultralytics 8.0.113 depends on torchvision>=0.8.1
    ultralytics 8.0.112 depends on torchvision>=0.8.1
    ultralytics 8.0.111 depends on torchvision>=0.8.1
    ultralytics 8.0.110 depends on torchvision>=0.8.1
    ultralytics 8.0.109 depends on torchvision>=0.8.1
    ultralytics 8.0.108 depends on torchvision>=0.8.1
    ultralytics 8.0.107 depends on torchvision>=0.8.1
    ultralytics 8.0.106 depends on torchvision>=0.8.1
    ultralytics 8.0.105 depends on torchvision>=0.8.1
    ultralytics 8.0.104 depends on torchvision>=0.8.1
    ultralytics 8.0.103 depends on torchvision>=0.8.1
    ultralytics 8.0.102 depends on torchvision>=0.8.1
    ultralytics 8.0.101 depends on torchvision>=0.8.1
    ultralytics 8.0.100 depends on torchvision>=0.8.1
    ultralytics 8.0.99 depends on torchvision>=0.8.1
    ultralytics 8.0.98 depends on torchvision>=0.8.1
    ultralytics 8.0.97 depends on torchvision>=0.8.1
    ultralytics 8.0.96 depends on torchvision>=0.8.1
    ultralytics 8.0.95 depends on torchvision>=0.8.1
    ultralytics 8.0.94 depends on torchvision>=0.8.1
    ultralytics 8.0.93 depends on torchvision>=0.8.1
    ultralytics 8.0.92 depends on torchvision>=0.8.1
    ultralytics 8.0.91 depends on torchvision>=0.8.1
    ultralytics 8.0.90 depends on torchvision>=0.8.1
    ultralytics 8.0.89 depends on torchvision>=0.8.1
    ultralytics 8.0.88 depends on torchvision>=0.8.1
    ultralytics 8.0.87 depends on torchvision>=0.8.1
    ultralytics 8.0.86 depends on torchvision>=0.8.1
    ultralytics 8.0.85 depends on torchvision>=0.8.1
    ultralytics 8.0.84 depends on torchvision>=0.8.1
    ultralytics 8.0.83 depends on torchvision>=0.8.1
    ultralytics 8.0.82 depends on torchvision>=0.8.1
    ultralytics 8.0.81 depends on torchvision>=0.8.1
    ultralytics 8.0.80 depends on torchvision>=0.8.1
    ultralytics 8.0.79 depends on torchvision>=0.8.1
    ultralytics 8.0.78 depends on torchvision>=0.8.1
    ultralytics 8.0.77 depends on torchvision>=0.8.1
    ultralytics 8.0.76 depends on torchvision>=0.8.1
    ultralytics 8.0.75 depends on torchvision>=0.8.1
    ultralytics 8.0.74 depends on torchvision>=0.8.1
    ultralytics 8.0.73 depends on torchvision>=0.8.1
    ultralytics 8.0.72 depends on torchvision>=0.8.1
    ultralytics 8.0.71 depends on torchvision>=0.8.1
    ultralytics 8.0.70 depends on torchvision>=0.8.1
    ultralytics 8.0.69 depends on torchvision>=0.8.1
    ultralytics 8.0.68 depends on torchvision>=0.8.1
    ultralytics 8.0.67 depends on torchvision>=0.8.1
    ultralytics 8.0.66 depends on torchvision>=0.8.1
    ultralytics 8.0.65 depends on torchvision>=0.8.1
    ultralytics 8.0.64 depends on torchvision>=0.8.1
    ultralytics 8.0.63 depends on torchvision>=0.8.1
    ultralytics 8.0.62 depends on torchvision>=0.8.1
    ultralytics 8.0.61 depends on torchvision>=0.8.1
    ultralytics 8.0.60 depends on torchvision>=0.8.1
    ultralytics 8.0.59 depends on torchvision>=0.8.1
    ultralytics 8.0.58 depends on torchvision>=0.8.1
    ultralytics 8.0.57 depends on torchvision>=0.8.1
    ultralytics 8.0.56 depends on torchvision>=0.8.1
    ultralytics 8.0.55 depends on torchvision>=0.8.1
    ultralytics 8.0.54 depends on torchvision>=0.8.1
    ultralytics 8.0.53 depends on torchvision>=0.8.1
    ultralytics 8.0.52 depends on torchvision>=0.8.1
    ultralytics 8.0.51 depends on torchvision>=0.8.1
    ultralytics 8.0.50 depends on torchvision>=0.8.1
    ultralytics 8.0.49 depends on torchvision>=0.8.1
    ultralytics 8.0.48 depends on torchvision>=0.8.1
    ultralytics 8.0.47 depends on torchvision>=0.8.1
    ultralytics 8.0.46 depends on torchvision>=0.8.1
    ultralytics 8.0.45 depends on torchvision>=0.8.1
    ultralytics 8.0.44 depends on torchvision>=0.8.1
    ultralytics 8.0.43 depends on torchvision>=0.8.1
    ultralytics 8.0.42 depends on torchvision>=0.8.1
    ultralytics 8.0.41 depends on torchvision>=0.8.1
    ultralytics 8.0.40 depends on torchvision>=0.8.1
    ultralytics 8.0.39 depends on torchvision>=0.8.1
    ultralytics 8.0.38 depends on torchvision>=0.8.1
    ultralytics 8.0.37 depends on torchvision>=0.8.1
    ultralytics 8.0.36 depends on torchvision>=0.8.1
    ultralytics 8.0.35 depends on torchvision>=0.8.1
    ultralytics 8.0.34 depends on torchvision>=0.8.1
    ultralytics 8.0.33 depends on torchvision>=0.8.1
    ultralytics 8.0.32 depends on torchvision>=0.8.1
    ultralytics 8.0.31 depends on torchvision>=0.8.1
    ultralytics 8.0.30 depends on torchvision>=0.8.1
    ultralytics 8.0.29 depends on torchvision>=0.8.1
    ultralytics 8.0.28 depends on torchvision>=0.8.1
    ultralytics 8.0.27 depends on torchvision>=0.8.1
    ultralytics 8.0.26 depends on torchvision>=0.8.1
    ultralytics 8.0.25 depends on torchvision>=0.8.1
    ultralytics 8.0.24 depends on torchvision>=0.8.1
    ultralytics 8.0.23 depends on torchvision>=0.8.1
    ultralytics 8.0.22 depends on torchvision>=0.8.1
    ultralytics 8.0.21 depends on torchvision>=0.8.1
    ultralytics 8.0.20 depends on torchvision>=0.8.1
    ultralytics 8.0.19 depends on torchvision>=0.8.1
    ultralytics 8.0.18 depends on torchvision>=0.8.1
    ultralytics 8.0.17 depends on torchvision>=0.8.1
    ultralytics 8.0.16 depends on torchvision>=0.8.1
    ultralytics 8.0.14 depends on torchvision>=0.8.1
    ultralytics 8.0.13 depends on torchvision>=0.8.1
    ultralytics 8.0.11 depends on torch>=1.7.0
    ultralytics 8.0.10 depends on torch>=1.7.0
    ultralytics 8.0.9 depends on torch>=1.7.0
    ultralytics 8.0.8 depends on torch>=1.7.0
    ultralytics 8.0.7 depends on torchvision>=0.8.1
    ultralytics 8.0.6 depends on torchvision>=0.8.1
    ultralytics 8.0.5 depends on torchvision>=0.8.1
    ultralytics 8.0.4 depends on torchvision>=0.8.1
    ultralytics 8.0.3 depends on torchvision>=0.8.1
    ultralytics 8.0.2 depends on torchvision>=0.8.1
    ultralytics 8.0.1 depends on torchvision>=0.8.1
    ultralytics 8.0.0 depends on torchvision>=0.8.1
    ultralytics 0.0.44 depends on torchvision>=0.8.1
    ultralytics 0.0.43 depends on torchvision>=0.8.1
    ultralytics 0.0.42 depends on torchvision>=0.8.1
    ultralytics 0.0.41 depends on torchvision>=0.8.1
    ultralytics 0.0.40 depends on torchvision>=0.8.1
    ultralytics 0.0.39 depends on torchvision>=0.8.1
    ultralytics 0.0.38 depends on torchvision>=0.8.1
    ultralytics 0.0.37 depends on torchvision>=0.8.1
    ultralytics 0.0.36 depends on torchvision>=0.8.1
    ultralytics 0.0.35 depends on torchvision>=0.8.1
    ultralytics 0.0.34 depends on torchvision>=0.8.1
    ultralytics 0.0.33 depends on torchvision>=0.8.1
    ultralytics 0.0.32 depends on torchvision>=0.8.1
    ultralytics 0.0.31 depends on torchvision>=0.8.1
    ultralytics 0.0.30 depends on torchvision>=0.8.1
    ultralytics 0.0.29 depends on torchvision>=0.8.1
    ultralytics 0.0.28 depends on torchvision>=0.8.1
    ultralytics 0.0.27 depends on torchvision>=0.8.1
    ultralytics 0.0.26 depends on torchvision>=0.8.1
    ultralytics 0.0.25 depends on torchvision>=0.8.1
    ultralytics 0.0.24 depends on torchvision>=0.8.1
    ultralytics 0.0.23 depends on torchvision>=0.8.1
    ultralytics 0.0.22 depends on torchvision>=0.8.1
    ultralytics 0.0.21 depends on torchvision>=0.8.1
    ultralytics 0.0.20 depends on torchvision>=0.8.1
    ultralytics 0.0.19 depends on torchvision>=0.8.1
    ultralytics 0.0.18 depends on torchvision>=0.8.1
    ultralytics 0.0.17 depends on torchvision>=0.8.1
    ultralytics 0.0.16 depends on torchvision>=0.8.1
    ultralytics 0.0.15 depends on torchvision>=0.8.1
    ultralytics 0.0.14 depends on torchvision>=0.8.1
    ultralytics 0.0.13 depends on torchvision>=0.8.1

To fix this you could try to:
1. loosen the range of package versions you've specified
2. remove package versions to allow pip attempt to solve the dependency conflict

ERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/user_guide/#fixing-conflicting-dependencies

This is in BullsEye 64bits RPI 4

[pderrenger]

Hello!

It looks like you're encountering a dependency conflict related to torchvision. To resolve this, you can try the following steps:

1. Ensure Compatibility: Make sure that your Python environment is compatible with the versions of torchvision and torch required by ultralytics. You might need to create a new virtual environment.

2. Install Specific Versions: Sometimes, specifying the exact versions of torch and torchvision before installing ultralytics can help. For example:

   pip install torch==1.10.0 torchvision==0.11.1
   pip install ultralytics

3. Use a Fresh Environment: If the above doesn't work, setting up a fresh virtual environment and installing the dependencies from scratch can often resolve these conflicts:

   python -m venv env
   source env/bin/activate
   pip install torch torchvision
   pip install ultralytics

If these steps do not resolve the issue, could you please provide more details about your environment (Python version, OS, etc.)? This will help us diagnose the problem more effectively.

Thank you for reaching out, and we hope to get this resolved for you soon! 😊

[ninopleno]

Thanks @pderrenger

I will try it today. But the specs are as follow:

• Python 3.9.2
• RPi OS Bullseye 64 Bits

The weird thing is since my last build it worked perfectly. I'm wondering if there is new stable version in Ultralytics? Hence some updated dependencies e.g. torch and torchvision?

[pderrenger]

Hello @ninopleno,

Thanks for sharing your environment details! It's indeed possible that dependency updates could be causing the issue if there have been recent updates to torch and torchvision. We continuously work to improve and update Ultralytics YOLOv8, which might include updates to dependencies for better performance and compatibility.

I recommend checking the releases page for any recent stable versions and their corresponding dependency requirements. Sometimes, aligning with the specific versions mentioned in the release notes can resolve such conflicts.

If the problem persists after aligning with the recommended versions, please let us know. We're here to help!

[zoldaten]

to get it working (USB Asselerator) on raspberry pi 4b (Bullseye or Bookworm):

1. do not use https://github.com/feranick/libedgetpu, do not follow instruction from ultralytics on HOW TO INSTALL coral usb asselerator from this page - it leads to Segmentation fault (i checked it out on my pi).

follow official https://coral.ai/docs/accelerator/get-started/#requirements to install on Linux.

After that:

Download pycoral-2.0.0-cp39-cp39-linux_aarch64.whl and tflite_runtime-2.5.0.post1-cp39-cp39-linux_aarch64.whl from https://github.com/google-coral/pycoral/releases/tag/v2.0.0
pip install ./pycoral-2.0.0-cp39-cp39-linux_aarch64.whl
pip install ./tflite_runtime-2.5.0.post1-cp39-cp39-linux_aarch64.whl

2. i have updated example from @DAVIDNYARKO123 to use yolov8 with image not video file:

from edge_tpu_silva import process_detection
import cv2,time
import numpy as np
from ultralytics import YOLO

# Example Usage with Required Parameters
#model_path = '240_yolov8n_full_integer_quant_edgetpu.tflite'
#input_path = 'path/to/your/input/video.mp4'
input_path = 'obama.jpg'
imgsz = 240

model = YOLO(model='240_yolov8n_full_integer_quant_edgetpu.tflite', task="detect", verbose=False)

input_path=cv2.imread(input_path)
print(input_path.shape)
input_path = input_path[:,:,:3]
#image = cv2.resize(np_array, (240, 240))

#input_path = np.expand_dims(input_path, axis=0)

# Run the object detection process
start_time = time.time()
outs = model.predict(
    source=input_path,
    conf=0.5,
    imgsz=imgsz,
    verbose=False,
    stream=False,
    show=True,    )
#outs = process_detection(model_path, input_path, imgsz, show=True)


#frame_count = 0
start_time = time.time()
for out in outs:
    
    objs_lst = []
    for box in out.boxes:
        obj_cls, conf, bb = (
            box.cls.numpy()[0],
            box.conf.numpy()[0],
            box.xyxy.numpy()[0],
        )
        label = out.names[int(obj_cls)]
        ol = {
            "id": obj_cls,
            "label": label,
            "conf": conf,
            "bbox": bb,
        }
        objs_lst.append(ol)

        
        print(label)
        #print("  id:    ", obj_cls)
        #print("  score: ", conf)
        #print("  bbox:  ", bb)

    #frame_count += 1
    #elapsed_time = time.time() - start_time
    #fps = frame_count / elapsed_time

    #print (objs_lst, fps)
    print (objs_lst)
    print("--- %s seconds ---" % (time.time() - start_time))

    # Break the loop if 'esc' key is pressed for video or camera
    if cv2.waitKey(1) == 27:
        break
#for _, _ in outs:
  #pass

you can also see example with picamera2 on Bullseye - https://youtu.be/37PwrRVP9j4

3. there is a problem with Coral USB on Raspbian Bookworm as it uses python3.11 as system default. It could be solved with pyenv or venv but there will be a problem with picamera2 as it uses libcamera. So only i got reached there - gstreamer using instead picamera2.

Have a nice day.

[zoldaten]

i also tried to convert by hands yolov8m to edgetpu on another linux PC:

yolo export model=yolov8m.pt imgsz=640

and got this:

Operation log: yolov8m_saved_model/yolov8m_full_integer_quant_edgetpu.log

Model successfully compiled but not all operations are supported by the Edge TPU. A percentage of the                                                       model will instead run on the CPU, which is slower. If possible, consider updating your model to use o                                                      nly operations supported by the Edge TPU. For details, visit g.co/coral/model-reqs.
Number of operations that will run on Edge TPU: 297
Number of operations that will run on CPU: 26

Operator                       Count      Status

PAD                            7          Mapped to Edge TPU
TRANSPOSE                      1          Mapped to Edge TPU
TRANSPOSE                      4          Operation is otherwise supported, but not mapped due to some                                                       unspecified limitation
STRIDED_SLICE                  2          More than one subgraph is not supported
STRIDED_SLICE                  16         Mapped to Edge TPU
STRIDED_SLICE                  2          Operation is otherwise supported, but not mapped due to some                                                       unspecified limitation
RESHAPE                        2          More than one subgraph is not supported
RESHAPE                        1          Mapped to Edge TPU
RESHAPE                        3          Operation is otherwise supported, but not mapped due to some                                                       unspecified limitation
MUL                            77         Mapped to Edge TPU
MUL                            2          More than one subgraph is not supported
SUB                            2          More than one subgraph is not supported
CONCATENATION                  16         Mapped to Edge TPU
CONCATENATION                  1          More than one subgraph is not supported
CONCATENATION                  2          Operation is otherwise supported, but not mapped due to some                                                       unspecified limitation
SOFTMAX                        1          More than one subgraph is not supported
RESIZE_NEAREST_NEIGHBOR        2          Mapped to Edge TPU
CONV_2D                        1          More than one subgraph is not supported
CONV_2D                        83         Mapped to Edge TPU
QUANTIZE                       1          Operation is otherwise supported, but not mapped due to some                                                       unspecified limitation
QUANTIZE                       2          Mapped to Edge TPU
MAX_POOL_2D                    3          Mapped to Edge TPU
ADD                            2          More than one subgraph is not supported
ADD                            12         Mapped to Edge TPU
LOGISTIC                       1          Operation is otherwise supported, but not mapped due to some                                                       unspecified limitation
LOGISTIC                       77         Mapped to Edge TPU
Compilation child process completed within timeout period.
Compilation succeeded!

after test on raspberry in the same code (as above):

Loading yolov8m_full_integer_quant_edgetpu.tflite for TensorFlow Lite Edge TPU inference...
F driver/usb/usb_driver.cc:857] transfer on tag 1 failed. Abort. Deadline exceeded: USB transfer error 2 [LibUsbDataOutCallback]

Process ended with exit code -6.

and leads to hang on Coral USB ...

[zoldaten]

ok. fixed issue with conversion.
should be:
yolo export model=yolov8m.pt imgsz=224 not
yolo export model=yolov8m.pt imgsz=640

image size matter. hmm.

[glenn-jocher]

@zoldaten great catch on the image size adjustment for the model export! Indeed, the input size can significantly impact the compatibility and performance of the model on the Edge TPU. Using an image size of 224 for yolov8m.pt seems to be more effective for your setup.

Thanks for sharing this update; it's a valuable tip for others working in similar configurations. If you encounter any more insights or need further assistance, feel free to reach out. Happy coding! 🚀

[Skillnoob]

Hey @zoldaten
Im the one who wrote this guide and you getting a segmentation fault with this guide is unusual, since it is made to prevent the segmentation fault many people got because of the libedgetpu version that is out of date by 3 years. The version by feranic was specifically made to fix this issue and the changes are also now merged into the official libedgetpu repository. I am using feranic's version and it works fine for me on both my RPI 5 and 4B.
The coral tpu doesn't work well with a imgsz of 640 and maps most of the OP's to the cpu. Additionally I had issues running models larger than s because they would exceed the 8MB cache limit of the tpu.
Here's some benchmark results I posted: #8035
I always had to use a imgsz of 512 or lower for it to not run at CPU speeds.

[faheem-khaskheli]

can anyone share metrics of FPS for Ultralytics models on Coral.AI USD TPU?

[glenn-jocher]

For FPS metrics of Ultralytics models on the Coral Edge TPU, performance can vary based on the specific model and setup. Generally, you can expect significant improvements in inference speed compared to running on a Raspberry Pi alone. For detailed benchmarks and user experiences, you might want to check the Ultralytics community forums or relevant GitHub issues. For setup and installation, refer to our guide on using Coral Edge TPU with Raspberry Pi: https://docs.ultralytics.com/guides/coral-edge-tpu-on-raspberry-pi/.

[Skillnoob]

@faheem-khaskheli I've posted some benchmarks results here: #8035

[Ildaron]

Hello, I moved ahead with your manual, thank you so much for that, but finally spent already few days, and still haven't receive any positive result
RAspberryPi5, RAM-8

my pip list 

+(test1) ildaron@raspberrypi:~ $ pip list

+Package            Version
+------------------ -----------
+certifi            2024.7.4
+charset-normalizer 3.3.2
+contourpy          1.2.1
+cycler             0.12.1
+filelock           3.15.4
+fonttools          4.53.1
+fsspec             2024.6.1
+idna               3.7
+Jinja2             3.1.4
+kiwisolver         1.4.5
+MarkupSafe         2.1.5
+matplotlib         3.9.2
+mpmath             1.3.0
+networkx           3.3
+numpy              1.26.4
+opencv-python      4.10.0.84
+packaging          24.1
+pip                23.0.1
+psutil             6.0.0
+py-cpuinfo         9.0.0
+pyparsing          3.1.2
+python-dateutil    2.9.0.post0
+pytz               2024.1
+PyYAML             6.0.2
+requests           2.32.3
+scipy              1.14.1
+setuptools         66.1.1
+six                1.16.0
+sympy              1.13.2
+torch              2.4.0
+torchvision        0.19.0
+tqdm               4.66.5
+typing_extensions  4.12.2
+tzdata             2024.1
+ultralytics        8.2.79
+ultralytics-thop   2.0.5
+urllib3            2.2.2


2. Information about RAaspberry  

-PRETTY_NAME="Debian GNU/Linux 12 (bookworm)"
-NAME="Debian GNU/Linux"
-VERSION_ID="12"
-VERSION="12 (bookworm)"
-VERSION_CODENAME=bookworm
-ID=debian
-HOME_URL="https://www.debian.org/"
-SUPPORT_URL="https://www.debian.org/support"
-BUG_REPORT_URL="https://bugs.debian.org/"
ildaron@raspberrypi:~ $ 

ildaron@raspberrypi:~ $ uname -a
-Linux raspberrypi 6.6.31+rpt-rpi-2712 #1 SMP PREEMPT Debian 1:6.6.31-1+rpt1 (2024-05-29) aarch64 GNU/Linux



3. 
When I want to install the library,  I receive the next error, I tried all versions that I found in GitHub in different environments but the result always was negative, can you give me some advice on what I can do?

(test1) ildaron@raspberrypi:~ $ sudo dpkg -i ~/Downloads/libedgetpu-dev_16.0tf2.17.0-1.bookworm_arm64.deb
dpkg: warning: downgrading libedgetpu-dev from 16.0tf2.17.0-1.bullseye to 16.0tf2.17.0-1.bookworm
(Reading database ... 268821 files and directories currently installed.)
Preparing to unpack .../libedgetpu-dev_16.0tf2.17.0-1.bookworm_arm64.deb ...
Unpacking libedgetpu-dev (16.0tf2.17.0-1.bookworm) over (16.0tf2.17.0-1.bullseye) ...
dpkg: dependency problems prevent configuration of libedgetpu-dev:
 libedgetpu-dev depends on libedgetpu1-std (= 16.0tf2.17.0-1.bookworm) | libedgetpu1 (= 16.0tf2.17.0-1.bookworm); however:
  Package libedgetpu1 is not installed.

dpkg: error processing package libedgetpu-dev (--install):
 dependency problems - leaving unconfigured
Errors were encountered while processing:
 libedgetpu-dev

[Ildaron]

tensorflow==2.15.0
Hi @Skillnoob thank you, unfortunately I cant install this version
"TensorFlow 2.15.0 is not directly compatible with most Raspberry Pi models. Raspberry Pi typically uses ARM-based processors, while TensorFlow 2.15.0 is primarily designed for x86 architectures"

[Skillnoob]

@Ildaron where did you get this from? I can install that version just fine on my Raspberry Pi. Also, as the guide states, you're supposed to export on a x86 linux machine, something like google colab which is free.

[Ildaron]

@Ildaron where did you get this from? I can install that version just fine on my Raspberry Pi. Also, as the guide states, you're supposed to export on a x86 linux machine, something like google colab which is free.
Hi @Skillnoob thank you for quick response
I spent few hours to try install myself, without any result, just example of one of my negative examples from my attempts

(myenv) ildaron@raspberrypi:~/Desktop/beehaviour/final $ pip3 install tensorflow==2.15.0
Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='www.piwheels.org', port=443): Read timed out. (read timeout=15)")': /simple/tensorflow/
Collecting tensorflow==2.15.0
WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x7ffef33a1750>: Failed to establish a new connection: [Errno 101] Network is unreachable')': /packages/cf/ac/6d884eba6d30196baf8f8284448f4d5388681f386f1150ad2d54398bc33a/tensorflow-2.15.0-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
WARNING: Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x7ffef33a1d90>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution')': /packages/cf/ac/6d884eba6d30196baf8f8284448f4d5388681f386f1150ad2d54398bc33a/tensorflow-2.15.0-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Downloading tensorflow-2.15.0-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (2.2 kB)
ERROR: Could not find a version that satisfies the requirement tensorflow-cpu-aws==2.15.0; platform_system == "Linux" and (platform_machine == "arm64" or platform_machine == "aarch64") (from tensorflow) (from versions: none)
ERROR: No matching distribution found for tensorflow-cpu-aws==2.15.0; platform_system == "Linux" and (platform_machine == "arm64" or platform_machine == "aarch64")

[Ildaron]

@Skillnoob and also small question, I need operate in real-time to connect my camera to RaspberryPi for object detection in real-time, if I correctly understood I cant operate with Google colab and Pi in real-time for machine vision task, or I am wrong ? if not, why I need Coral since Google Colab it is a quite powerful system that allow me works with high FPS and without Coral

and I following steps that described here https://docs.ultralytics.com/guides/coral-edge-tpu-on-raspberry-pi/
I didint see here any mentioned about "o export on a x86 linux machine" or I missed that

[Skillnoob]

@Ildaron Under the Export your model to a Edge TPU compatible model section:

To use the Edge TPU, you need to convert your model into a compatible format. It is recommended that you run export on Google Colab, x86_64 Linux machine, using the official Ultralytics Docker container, or using Ultralytics HUB, since the Edge TPU compiler is not available on ARM. See the Export Mode for the available arguments.

Then in the warning box of Running the model is a link to a tflite-runtime that is compatible for the pi.

Again, export using google colab and transfer the model to your pi.

[Ildaron]

Hi @zoldaten in the link that you provided I tuck in the next step, do you know how I can fix it

(myenv) ildaron@raspberrypi:/ $ sudo apt-get install libedgetpu1-std
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
libedgetpu1-std is already the newest version (16.0tf2.17.0-1.bookworm).
You might want to run 'apt --fix-broken install' to correct these.
The following packages have unmet dependencies:
 libedgetpu-dev:armhf : Depends: libedgetpu1-std:armhf (= 16.0tf2.17.0-1.ubuntu24.04) but 16.0tf2.17.0-1.bookworm is to be installed or
                                 libedgetpu1:armhf (= 16.0tf2.17.0-1.ubuntu24.04)
E: Unmet dependencies. Try 'apt --fix-broken install' with no packages (or specify a solution).

[Skillnoob]

@Ildaron lower your imgsz to 512 by doing imgsz=512 when exporting and running inference. For some reason a lot of operations get mapped to the CPU when using the default image size.

I would recommend exporting to tensorrt with a jetson nano. Find the guide here.

[Ildaron]

thank you @Skillnoob
do you mean make the next command

model.export(format="edgetpu", imgsz=512) ?

[Skillnoob]

@Ildaron yes.
You have to add imgsz=512 to both the export and prediction.
For example:
model.predict(..., imgsz=512) or model(..., imgsz=512)

[Ildaron]

@Ildaron yes. You have to add imgsz=512 to both the export and prediction. For example: model.predict(..., imgsz=512) or model(..., imgsz=512)

Hi @Skillnoob thank you, it gave me 10 FPS, thas is better.
I wanted to ask, in Jetson Nano how do you think it will be better ? because, generally Jetson Nano is better since use GPU, but in this case need to use Coral accelerator and CPU for Jetson Nano and Pi5 similar in power

[Skillnoob]

@Ildaron
You will probably have much more fps if you follow the jetson integration and export with half=True at also better accuracy

[wb666greene]

I'm not using RaspberryPi, but I appear to have succesfully converted s couple of yolov8 models and seem to be running them on the TPU. I get ~45mS inference time with the nano model and ~335ms with the small model, presumably becasuse it is too large for the TPU.

I have multiple TPUs, can I specify which to use when i do:

model = YOLO('yolo8/yolov8n_saved_model/yolov8n_full_integer_quant_edgetpu.tflite',task='detect')
results = model.predict(image, conf=0.70, verbose=False)

My goal is to use one TPU for MobilenetSSD_v2 and the other for yolo8 , so I can run on lessor hardware, (Celeron or Atom class) right now it needs either an 8th generation iGPU or a system with NVidia GPU.

My project is here:
https://github.com/wb666greene/AI-Person-Detector-with-YOLO-verification-Version-2/tree/main

[glenn-jocher]

@wb666greene currently, specifying a particular TPU for inference isn't supported directly in the Ultralytics framework. You might need to manage TPU allocation at the system level or through custom scripts. For further details on using Coral Edge TPU, please refer to the Ultralytics documentation.

[wb666greene]

Thanks for the answers. I'd ran across the 512 size limitation while going through the other comments here, I just haven't tried it yet. Reading the Ultralytics docs about thread safe inference , I instance the MobilenetSSD_v2 on the TPU in its own thread, and instance the yolo8 model in a different thread. I can specify the USB TPU in the MobilenetSSD thread since it uses pycoral:

model = make_interpreter(modelPath, "usb")
0r do this:
from pycoral.utils.edgetpu import list_edge_tpus

# list installed tpus, and figure out if any are M.2 (pci)
pci_tpu = list()
usb_tpu = list()
tpus = list_edge_tpus()

for i in range(len(tpus)):
    if tpus[i]['type'] == 'pci': pci_tpu.append(i)
    if tpus[i]['type'] == 'usb': usb_tpu.append(i)

model = make_interpreter(modelPath, device=':'+str(usb_tpu[1]))

And then hope the M.2 TPU gets used by the Ultralytics thread. Guess I'll just have to play with it and see what happens. The pycoral docs I've seen says if you have both M.2 and USB TPUs you can't predict which will be used if you don't specify.

I guess "Plan B" could be to use pycoral to load the yolov8n_full_integer_quant_edgetpu.tflite model and do the inference in pycoral as well. But the Ultralytics model loading and inference drops into my yolo8 CUDA thread with only minor changes and can be selected via a thread parameter. One M.2 and one USB TPU would be my usual use case, but I'd try to also make it work with two USB TPUs.

[Skillnoob]

@wb666greene Since PyCoral also only uses the tflite intepreter in the backend, you could try looking at their code on how they select the TPU and modify Ultralytics locally to specifically use one TPU.

[wb666greene]

I should follow-up just for closure, was a total non-issue. When I use two TPUs I use the code snippet I posted to use the M.2 TPU in the initial MoblenetSSD_v2 detection and whatever is left is used by the Yolo thread to verify the detection after zooming in on the initial detection. Works great, the key was the imgsz=512 parameter for the "small" model. The "nano model" just wasn't accurate enough to use. I also have it automatically downloading and converting the yolov8s.pt model. I ran into an issue converting with a Gemini-Lake iGPU where the conversion died with "illegal instruction" core dump. But copying over the edgetpu.tflite file converted on a different system worked fine. The USB TPU gets quite hot if used for the initial MobilenetSSD detection, so I decided to use the M.2 version first since it does the most inferences.

[Skillnoob]

@wb666greene A friend of mine designed a cooler for the USB TPU if you want to be able to run the max lib with the tpu for long periods.
https://makerworld.com/en/models/233023?from=search#profileId-249779

[Ildaron]

Hello, @Skillnoob really, thank you so much for your support, I hope in this time you also will help me,

I was sure that all my problems were solved but now I have an unusual situation.
My model worked well, but when I increased the amount of my dataset from 100 images to 700 images my model stopped working from this moment, it was a quite tricky situation because I use the same scripts.

Shortly just repeat my message from 2 weeks ago about what I did

"start repeat the last message"
launch the next script to convert .pt model to .tflite

from ultralytics import YOLO
model = YOLO("/content/drive/MyDrive/Bee/yolov8n.pt") 
 model.export(format="edgetpu")

In RaspberryPi
download for Pi5 - yolov8n_full_integer_quant_edgetpu.tflite from https://github.com/feranick/libedgetpu/releases
in env

sudo dpkg -i /home/ildaron/Downloads/libedgetpu1-std_16.0tf2.17.0-1.bookworm_arm64.deb
pip innstall tensorflow

launch python.py with yolov8n_full_integer_quant.tflite model and you have 7 FPS
"stop repeat the last message"

Why is strange, because just Yolo model works, directly in RaspberryPi in format - ".pt" I can detect objects in low FPS, so I can confirm that ".pt" is correct

But after I converted the model to Coral format (I used really all available models for Coral )
with the next scrips (which works for 100 samples)

model = YOLO("/content/drive/MyDrive/Bee/yolov8n.pt") 
 model.export(format="edgetpu")`

I receive the next result.

It is totally wrong, detected 300 objects, except my object.

So again everything works for 100 images but didn't work for 600 images

Probably need a clear memory or something? Because it's very strange, it all worked for a small dataset, also the YOLO model definitely works, but the conversion is incorrect now.

[Ildaron]

unsure why this happens. Try updating torch and torchvision.

I use Google Colab for training and convertation, do you mean update libraries in Google colab?

[Skillnoob]

@Ildaron both the pi and colab

[Ildaron]

@Ildaron both the Pi and Colab
thank you, for your advice @Skillnoob

I did this, and the result was the same, and again the first model (on a hundred images) works correctly, the second model (600 images) does not work, as you can see in the image below

The only thing I changed for the second model (600 images) it is the new labeling program, for the second model (600 images) I used YoloLabel_v1.2.1 software

But to check whether the second model was trained correctly, I launched the model in the ".pt" format in python IDLE, and everything is fine but slow. Does this mean that the model is trained correctly and can be transferred to coral if it works in ".pt" format?

I updated the libraries as you recommended

Only want to say, in fact in Google Collab originally I only installed
pip install ultralytics
and did the command
model.export(format="edgetpu", imgsz=512)

after that on the RaspberryPi I installed only ultralytics
pip install ultralytics
and
pip install tensorflow
and launched the module in different coral formats

Again I very surprised that my first model is works always but the second model with the same scripts doesn't work

[Skillnoob]

@Ildaron sadly I don't know how to fix this.
I've had the pretty much same issue with my TPU and the cause seems to be something in the conversion.

@Y-T-G can you maybe take a look?

[Y-T-G]

@Ildaron the old model still work? Try converting the old model again and see if the conversion still works for it.

[Ildaron]

Hi @Skillnoob finally it works. I used exactly the same scripts in Google Colab (for training and convert model), and removed ultralytics after that I installed !pip install ultralytics==8.0.134 which didn't help
I again installed ultralytics and suddenly my model started working,

It definitely doesn't make sense to me because for a few days I used these scripts without any results and I believe it is not necessary to reinstall the library for Google Colab because after disconnecting from Google Drive, let say, RAM is erasing for all installed libraries, so don't what happened

[Skillnoob]

@Ildaron python libaries are very weird, can happen from time to time that something just breaks for no reason

[szym1998]

It would be fantastic to have something as straightforward for Hailo. While Hailo is theoretically superior to Coral in nearly every aspect, working with it has proven to be quite a challenge for me.

[Skillnoob]

Ultralytics is trying to integrate the hailo at some point, but their compiler is very buggy and in general their software and documentation is very lackluster

[szym1998]

Absolutely, I know exactly what you mean. I've been trying to work with Hailo myself, and it really feels like dealing with a closed-source ecosystem. Despite its potential, the software is challenging to navigate, and the documentation leaves much to be desired.

[Skillnoob]

Check this repo out, it should hopefully work. It is possible to run but I haven't been able to find a way to properly install their python library or documentation about it.
https://github.com/cyclux/HailoConverter

[szym1998]

It worked have you tried yolo11?

[Skillnoob]

I have not. I'm currently not actively using my PI's, so I haven't tried it yet.

[y71eyg7hi8]

(detect) jinxin@raspberrypi:/YOLO/jiemian $ python 111.py
[{'name': 'serving_default_images:0', 'index': 0, 'shape': array([ 1, 640, 640, 3], dtype=int32), 'shape_signature': array([ 1, 640, 640, 3], dtype=int32), 'dtype': <class 'numpy.int8'>, 'quantization': (0.01865844801068306, -14), 'quantization_parameters': {'scales': array([0.01865845], dtype=float32), 'zero_points': array([-14], dtype=int32), 'quantized_dimension': 0}, 'sparsity_parameters': {}}]
[{'name': 'PartitionedCall:0', 'index': 46, 'shape': array([ 1, 84, 8400], dtype=int32), 'shape_signature': array([ 1, 84, 8400], dtype=int32), 'dtype': <class 'numpy.int8'>, 'quantization': (2.551055431365967, -128), 'quantization_parameters': {'scales': array([2.5510554], dtype=float32), 'zero_points': array([-128], dtype=int32), 'quantized_dimension': 0}, 'sparsity_parameters': {}}]
(detect) jinxin@raspberrypi:/YOLO/jiemian $ python 111.py
WARNING ⚠️ Unable to automatically guess model task, assuming 'task=detect'. Explicitly define task for your model, i.e. 'task=detect', 'segment', 'classify','pose' or 'obb'.
Loading /home/jinxin/YOLO/yolov8n_saved_model/yolov8n_full_integer_quant_edgetpu.tflite for TensorFlow Lite Edge TPU inference...

F driver/usb/usb_driver.cc:857] transfer on tag 1 failed. Abort. Deadline exceeded: USB transfer error 2 [LibUsbDataOutCallback]

[glenn-jocher]

To resolve issues with YOLOv8 on Edge TPU, ensure you've installed the correct Edge TPU runtime for your Raspberry Pi OS Bookworm 64-bit. Reinstalling the runtime or tflite-runtime might help. For detailed guidance, refer to the Coral Edge TPU on Raspberry Pi guide.

[y71eyg7hi8]

Project Overview

Results with Edge TPU on Raspberry Pi

I was pleasantly surprised by the performance achieved using the Edge TPU on the Raspberry Pi. The acceleration results were impressive!

Overall, the Edge TPU significantly improved the processing speed for my project, making it a valuable addition to the Raspberry Pi ecosystem.

[y71eyg7hi8]

[y71eyg7hi8]

[glenn-jocher]

Thank you for sharing your experience with the Edge TPU on Raspberry Pi. For any issues with YOLOv8 on Edge TPU, ensure the correct runtime is installed and consider reinstalling the libedgetpu runtime. For detailed setup instructions, please refer to the Coral Edge TPU guide.

[snavalm]

In ultralytics==8.3.23 I'm finding an issue when trying to run model.predict. The issue appears during preprocessing, the preprocessor is trying to put the image in the device ":0" and that obviously fails.
I worked around this issue modifying autobackend.py, setting device='cpu' after the interpreter has been initialised

        if edgetpu:  # TF Edge TPU https://coral.ai/software/#edgetpu-runtime
            device = device[3:] if str(device).startswith("tpu") else ":0"
            LOGGER.info(f"Loading {w} on device {device[1:]} for TensorFlow Lite Edge TPU inference...")
            delegate = {"Linux": "libedgetpu.so.1", "Darwin": "libedgetpu.1.dylib", "Windows": "edgetpu.dll"}[
                platform.system()
            ]
            interpreter = Interpreter(
                model_path=w,
                experimental_delegates=[load_delegate(delegate, options={"device": device})],
            )
            ##### CHANGE: CHANGE TO CPU SO PREPROCESSING DOESN'T FAIL ######
            device='cpu'

[glenn-jocher]

@snavalm thank you for sharing your workaround. It seems like adjusting the device setting to 'cpu' resolved the issue for you. For further guidance on using Coral Edge TPU with Ultralytics, please refer to our documentation at https://docs.ultralytics.com/guides/coral-edge-tpu-on-raspberry-pi/. If you have more questions, feel free to ask here.

[Skillnoob]

Oh yeah, thanks for catching that. Confirmed the bug on my tpu. Created a PR to fix this: #17199

[Skillnoob]

@snavalm should be fixed now and you can update with pip install ultralytics -U

[snavalm]

Thank you!

[glenn-jocher]

Thank you for your contribution and for creating the PR to address the issue. Please ensure to update your package with pip install ultralytics -U to get the latest fixes. For further assistance with Coral Edge TPU, you can refer to our guide at https://docs.ultralytics.com/guides/coral-edge-tpu-on-raspberry-pi/.

[marcivanmanalac]

Hello,

I noticed the compatible hardware is only limited to pi 4B or 5 recommended. Would this run on a pi zero 2W? The original coral docs shows it is compatible but I was curious if so with YOLO11.

[Skillnoob]

You would need to test if it works, I don't have a Zero 2W so I didn't include it while writing the guide. Also you may get significantly worse performance because the Zero's CPU is very slow and pre/post processing may take a lot longer.

[marcivanmanalac]

I am testing on the pi zero 2W because of physical size constraints for the system I want to put it on.

I have it running Raspberry Pi OS Lite 64-bit Deb 12 Bookworm

I downloaded the bookworm arm64 release

I am having difficulty depackaging. This is the output:

~ $ sudo dpkg -i libedgetpu-dev_16.0tf2.17.1-1.bookworm_arm64.deb
Selecting previously unselected package libedgetpu-dev.
(Reading database ... 57935 files and directories currently installed.)
Preparing to unpack libedgetpu-dev_16.0tf2.17.1-1.bookworm_arm64.deb ...
Unpacking libedgetpu-dev (16.0tf2.17.1-1.bookworm) ...
dpkg: dependency problems prevent configuration of libedgetpu-dev:
libedgetpu-dev depends on libedgetpu1-std (= 16.0tf2.17.1-1.bookworm) | libedgetpu1 (= 16.0tf2.17.1-1.bookworm); however:
Package libedgetpu1-std is not installed.
Package libedgetpu1 is not installed.

dpkg: error processing package libedgetpu-dev (--install):
dependency problems - leaving unconfigured
Errors were encountered while processing:
libedgetpu-dev

The guide says to uninstall libedgetpu1-std so I didn't and if I can only do so following the On Linux portion of the coral install guide as mentioned in the comments.

[Skillnoob]

You're not supposed to install the -dev version, the version table never mentions it. You're supposed to either install the std or max variant, where you can find the correct one you want from the table. Also, don't follow the official coral guide, its outdated.

[marcivanmanalac]

Ahh yes, thank you.

If I wanted to do C++ inferencing I would still need to build the LITE RT aka TF-LITE instead of pip installing to go with the TPU edge runtime library?

[glenn-jocher]

For C++ inferencing with the Coral Edge TPU, you'll need to build the TensorFlow Lite runtime from source, as the pip-installed version is typically for Python use. This ensures compatibility with the TPU runtime library.

[sansoy]

first of all thank you for the amazing instructions for both creating the .tflite using colab and running inference on the raspberry pi.

I'm waiting on delivery of my 8Gb rPI5 but in the interim I ran the following tests using a 2Gb rPI4 and found that Yolov8n outperformed Yolov11n.

YOLOV8
Loading yolov8_full_integer_quant_edgetpu.tflite on device 0 for TensorFlow Lite Edge TPU inference...

0: 640x640 1 person, 145.8ms
Speed: 133.6ms preprocess, 145.8ms inference, 139.2ms postprocess per image at shape (1, 3, 640, 640)

0: 640x640 1 person, 142.8ms
Speed: 23.7ms preprocess, 142.8ms inference, 6.7ms postprocess per image at shape (1, 3, 640, 640)

0: 640x640 1 person, 115.8ms
Speed: 25.2ms preprocess, 115.8ms inference, 2.9ms postprocess per image at shape (1, 3, 640, 640)

YOLOV11
Loading yolov11_full_integer_quant_edgetpu.tflite on device 0 for TensorFlow Lite Edge TPU inference...

0: 640x640 1 person, 422.6ms
Speed: 29.0ms preprocess, 422.6ms inference, 4.7ms postprocess per image at shape (1, 3, 640, 640)

0: 640x640 1 person, 430.1ms
Speed: 22.0ms preprocess, 430.1ms inference, 2.7ms postprocess per image at shape (1, 3, 640, 640)

0: 640x640 1 person, 382.6ms
Speed: 17.9ms preprocess, 382.6ms inference, 2.8ms postprocess per image at shape (1, 3, 640, 640)

Any insights into why the older version is faster?
Sabri

[sansoy]

you hit the nail on the head. YoloV11
Number of operations that will run on Edge TPU: 275
Number of operations that will run on CPU: 222

[Skillnoob]

Thats the issue then. I can investigate this later on my tpu.

[sansoy]

awesome! thank you for all your help

[sansoy]

heres the output from colab for yolov11
!yolo mode=export model=runs/detect/train2/weights/best.pt format=edgetpu imgsz=512

Creating new Ultralytics Settings v0.0.6 file ✅
View Ultralytics Settings with 'yolo settings' or at '/root/.config/Ultralytics/settings.json'
Update Settings with 'yolo settings key=value', i.e. 'yolo settings runs_dir=path/to/dir'. For help see https://docs.ultralytics.com/quickstart/#ultralytics-settings.
Ultralytics 8.3.40 🚀 Python-3.10.12 torch-2.5.1+cu121 CPU (Intel Xeon 2.20GHz)
YOLO11n summary (fused): 238 layers, 2,582,737 parameters, 0 gradients, 6.3 GFLOPs

PyTorch: starting from 'runs/detect/train2/weights/best.pt' with input shape (1, 3, 512, 512) BCHW and output shape(s) (1, 7, 5376) (5.2 MB)
requirements: Ultralytics requirements ['sng4onnx>=1.0.1', 'onnx_graphsurgeon>=0.3.26', 'onnx>=1.12.0', 'onnx2tf>1.17.5,<=1.22.3', 'onnxslim>=0.1.31', 'tflite_support', 'onnxruntime'] not found, attempting AutoUpdate...
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
grpcio-status 1.62.3 requires protobuf>=4.21.6, but you have protobuf 3.20.3 which is incompatible.
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com/
Collecting sng4onnx>=1.0.1
Downloading sng4onnx-1.0.4-py3-none-any.whl.metadata (4.6 kB)
Collecting onnx_graphsurgeon>=0.3.26
Downloading onnx_graphsurgeon-0.5.2-py2.py3-none-any.whl.metadata (8.1 kB)
Collecting onnx>=1.12.0
Downloading onnx-1.17.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (16 kB)
Collecting onnx2tf<=1.22.3,>1.17.5
Downloading onnx2tf-1.22.3-py3-none-any.whl.metadata (136 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 136.6/136.6 kB 36.7 MB/s eta 0:00:00
Collecting onnxslim>=0.1.31
Downloading onnxslim-0.1.43-py3-none-any.whl.metadata (4.2 kB)
Collecting tflite_support
Downloading tflite_support-0.4.4-cp310-cp310-manylinux2014_x86_64.whl.metadata (2.4 kB)
Collecting onnxruntime
Downloading onnxruntime-1.20.1-cp310-cp310-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl.metadata (4.5 kB)
Requirement already satisfied: numpy in /usr/local/lib/python3.10/dist-packages (from onnx_graphsurgeon>=0.3.26) (1.26.4)
Requirement already satisfied: protobuf>=3.20.2 in /usr/local/lib/python3.10/dist-packages (from onnx>=1.12.0) (4.25.5)
Requirement already satisfied: sympy in /usr/local/lib/python3.10/dist-packages (from onnxslim>=0.1.31) (1.13.1)
Requirement already satisfied: packaging in /usr/local/lib/python3.10/dist-packages (from onnxslim>=0.1.31) (24.2)
Requirement already satisfied: absl-py>=0.7.0 in /usr/local/lib/python3.10/dist-packages (from tflite_support) (1.4.0)
Requirement already satisfied: flatbuffers>=2.0 in /usr/local/lib/python3.10/dist-packages (from tflite_support) (24.3.25)
Collecting protobuf>=3.20.2 (from onnx>=1.12.0)
Downloading protobuf-3.20.3-cp310-cp310-manylinux_2_12_x86_64.manylinux2010_x86_64.whl.metadata (679 bytes)
Collecting sounddevice>=0.4.4 (from tflite_support)
Downloading sounddevice-0.5.1-py3-none-any.whl.metadata (1.4 kB)
Collecting pybind11>=2.6.0 (from tflite_support)
Downloading pybind11-2.13.6-py3-none-any.whl.metadata (9.5 kB)
Collecting coloredlogs (from onnxruntime)
Downloading coloredlogs-15.0.1-py2.py3-none-any.whl.metadata (12 kB)
Requirement already satisfied: CFFI>=1.0 in /usr/local/lib/python3.10/dist-packages (from sounddevice>=0.4.4->tflite_support) (1.17.1)
Collecting humanfriendly>=9.1 (from coloredlogs->onnxruntime)
Downloading humanfriendly-10.0-py2.py3-none-any.whl.metadata (9.2 kB)
Requirement already satisfied: mpmath<1.4,>=1.1.0 in /usr/local/lib/python3.10/dist-packages (from sympy->onnxslim>=0.1.31) (1.3.0)
Requirement already satisfied: pycparser in /usr/local/lib/python3.10/dist-packages (from CFFI>=1.0->sounddevice>=0.4.4->tflite_support) (2.22)
Downloading sng4onnx-1.0.4-py3-none-any.whl (5.9 kB)
Downloading onnx_graphsurgeon-0.5.2-py2.py3-none-any.whl (56 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 56.4/56.4 kB 280.9 MB/s eta 0:00:00
Downloading onnx-1.17.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (16.0 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 16.0/16.0 MB 303.4 MB/s eta 0:00:00
Downloading onnx2tf-1.22.3-py3-none-any.whl (435 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 435.0/435.0 kB 355.3 MB/s eta 0:00:00
Downloading onnxslim-0.1.43-py3-none-any.whl (142 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 142.9/142.9 kB 322.3 MB/s eta 0:00:00
Downloading tflite_support-0.4.4-cp310-cp310-manylinux2014_x86_64.whl (60.8 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 60.8/60.8 MB 164.9 MB/s eta 0:00:00
Downloading onnxruntime-1.20.1-cp310-cp310-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl (13.3 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 13.3/13.3 MB 64.1 MB/s eta 0:00:00
Downloading protobuf-3.20.3-cp310-cp310-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (1.1 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.1/1.1 MB 207.7 MB/s eta 0:00:00
Downloading pybind11-2.13.6-py3-none-any.whl (243 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 243.3/243.3 kB 370.7 MB/s eta 0:00:00
Downloading sounddevice-0.5.1-py3-none-any.whl (32 kB)
Downloading coloredlogs-15.0.1-py2.py3-none-any.whl (46 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 46.0/46.0 kB 274.4 MB/s eta 0:00:00
Downloading humanfriendly-10.0-py2.py3-none-any.whl (86 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 86.8/86.8 kB 284.7 MB/s eta 0:00:00
Installing collected packages: sng4onnx, pybind11, protobuf, onnx2tf, humanfriendly, sounddevice, onnx, coloredlogs, tflite_support, onnxslim, onnxruntime, onnx_graphsurgeon
Attempting uninstall: protobuf
Found existing installation: protobuf 4.25.5
Uninstalling protobuf-4.25.5:
Successfully uninstalled protobuf-4.25.5
Successfully installed coloredlogs-15.0.1 humanfriendly-10.0 onnx-1.17.0 onnx2tf-1.22.3 onnx_graphsurgeon-0.5.2 onnxruntime-1.20.1 onnxslim-0.1.43 protobuf-3.20.3 pybind11-2.13.6 sng4onnx-1.0.4 sounddevice-0.5.1 tflite_support-0.4.4

requirements: AutoUpdate success ✅ 18.5s, installed 7 packages: ['sng4onnx>=1.0.1', 'onnx_graphsurgeon>=0.3.26', 'onnx>=1.12.0', 'onnx2tf>1.17.5,<=1.22.3', 'onnxslim>=0.1.31', 'tflite_support', 'onnxruntime']
requirements: ⚠️ Restart runtime or rerun command for updates to take effect

TensorFlow SavedModel: starting export with tensorflow 2.17.1...

ONNX: starting export with onnx 1.17.0 opset 19...
ONNX: slimming with onnxslim 0.1.43...
ONNX: export success ✅ 3.0s, saved as 'runs/detect/train2/weights/best.onnx' (10.1 MB)
TensorFlow SavedModel: starting TFLite export with onnx2tf 1.22.3...
WARNING: All log messages before absl::InitializeLog() is called are written to STDERR
I0000 00:00:1733410559.653469 2686 devices.cc:67] Number of eligible GPUs (core count >= 8, compute capability >= 0.0): 0
WARNING: All log messages before absl::InitializeLog() is called are written to STDERR
W0000 00:00:1733410560.694263 2686 tf_tfl_flatbuffer_helpers.cc:392] Ignored output_format.
W0000 00:00:1733410560.694317 2686 tf_tfl_flatbuffer_helpers.cc:395] Ignored drop_control_dependency.
I0000 00:00:1733410561.853448 2686 devices.cc:67] Number of eligible GPUs (core count >= 8, compute capability >= 0.0): 0
W0000 00:00:1733410562.770901 2686 tf_tfl_flatbuffer_helpers.cc:392] Ignored output_format.
W0000 00:00:1733410562.770935 2686 tf_tfl_flatbuffer_helpers.cc:395] Ignored drop_control_dependency.
I0000 00:00:1733410563.900146 2686 devices.cc:67] Number of eligible GPUs (core count >= 8, compute capability >= 0.0): 0
W0000 00:00:1733410564.813312 2686 tf_tfl_flatbuffer_helpers.cc:392] Ignored output_format.
W0000 00:00:1733410564.813348 2686 tf_tfl_flatbuffer_helpers.cc:395] Ignored drop_control_dependency.
W0000 00:00:1733410569.064287 2686 tf_tfl_flatbuffer_helpers.cc:392] Ignored output_format.
W0000 00:00:1733410569.064316 2686 tf_tfl_flatbuffer_helpers.cc:395] Ignored drop_control_dependency.
fully_quantize: 0, inference_type: 6, input_inference_type: FLOAT32, output_inference_type: FLOAT32
W0000 00:00:1733410583.668178 2686 tf_tfl_flatbuffer_helpers.cc:392] Ignored output_format.
W0000 00:00:1733410583.668217 2686 tf_tfl_flatbuffer_helpers.cc:395] Ignored drop_control_dependency.
fully_quantize: 0, inference_type: 6, input_inference_type: INT8, output_inference_type: INT8
W0000 00:00:1733410598.161528 2686 tf_tfl_flatbuffer_helpers.cc:392] Ignored output_format.
W0000 00:00:1733410598.161560 2686 tf_tfl_flatbuffer_helpers.cc:395] Ignored drop_control_dependency.
W0000 00:00:1733410623.275389 2686 tf_tfl_flatbuffer_helpers.cc:392] Ignored output_format.
W0000 00:00:1733410623.275422 2686 tf_tfl_flatbuffer_helpers.cc:395] Ignored drop_control_dependency.
TensorFlow SavedModel: export success ✅ 151.9s, saved as 'runs/detect/train2/weights/best_saved_model' (35.4 MB)
Edge TPU: WARNING ⚠️ Edge TPU known bug #1185

Edge TPU: export requires Edge TPU compiler. Attempting install from https://coral.ai/docs/edgetpu/compiler/
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 1022 100 1022 0 0 12841 0 --:--:-- --:--:-- --:--:-- 12936
Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)).
OK
deb https://packages.cloud.google.com/apt coral-edgetpu-stable main
Get:1 https://cloud.r-project.org/bin/linux/ubuntu jammy-cran40/ InRelease [3,626 B]
Get:2 https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64 InRelease [1,581 B]
Get:3 https://packages.cloud.google.com/apt coral-edgetpu-stable InRelease [1,423 B]
Get:4 https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64 Packages [1,190 kB]
Get:5 http://security.ubuntu.com/ubuntu jammy-security InRelease [129 kB]
Hit:6 http://archive.ubuntu.com/ubuntu jammy InRelease
Get:7 https://packages.cloud.google.com/apt coral-edgetpu-stable/main amd64 Packages [6,888 B]
Get:8 https://packages.cloud.google.com/apt coral-edgetpu-stable/main all Packages [1,865 B]
Get:9 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [128 kB]
Get:10 https://r2u.stat.illinois.edu/ubuntu jammy InRelease [6,555 B]
Get:11 https://ppa.launchpadcontent.net/deadsnakes/ppa/ubuntu jammy InRelease [18.1 kB]
Get:12 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [2,454 kB]
Get:13 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [127 kB]
Hit:14 https://ppa.launchpadcontent.net/graphics-drivers/ppa/ubuntu jammy InRelease
Get:15 https://r2u.stat.illinois.edu/ubuntu jammy/main all Packages [8,525 kB]
Hit:16 https://ppa.launchpadcontent.net/ubuntugis/ppa/ubuntu jammy InRelease
Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [3,446 kB]
Get:18 https://ppa.launchpadcontent.net/deadsnakes/ppa/ubuntu jammy/main amd64 Packages [35.0 kB]
Get:19 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [1,225 kB]
Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [2,738 kB]
Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,514 kB]
Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [53.3 kB]
Get:23 https://r2u.stat.illinois.edu/ubuntu jammy/main amd64 Packages [2,624 kB]
Fetched 24.2 MB in 4s (6,308 kB/s)
Reading package lists... Done
W: https://packages.cloud.google.com/apt/dists/coral-edgetpu-stable/InRelease: Key is stored in legacy trusted.gpg keyring (/etc/apt/trusted.gpg), see the DEPRECATION section in apt-key(8) for details.
W: Skipping acquire of configured file 'main/source/Sources' as repository 'https://r2u.stat.illinois.edu/ubuntu jammy InRelease' does not seem to provide it (sources.list entry misspelt?)
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following NEW packages will be installed:
edgetpu-compiler
0 upgraded, 1 newly installed, 0 to remove and 51 not upgraded.
Need to get 7,913 kB of archives.
After this operation, 31.2 MB of additional disk space will be used.
Get:1 https://packages.cloud.google.com/apt coral-edgetpu-stable/main amd64 edgetpu-compiler amd64 16.0 [7,913 kB]
Fetched 7,913 kB in 0s (22.7 MB/s)
debconf: unable to initialize frontend: Dialog
debconf: (No usable dialog-like program is installed, so the dialog based frontend cannot be used. at /usr/share/perl5/Debconf/FrontEnd/Dialog.pm line 78, <> line 1.)
debconf: falling back to frontend: Readline
debconf: unable to initialize frontend: Readline
debconf: (This frontend requires a controlling tty.)
debconf: falling back to frontend: Teletype
dpkg-preconfigure: unable to re-open stdin:
Selecting previously unselected package edgetpu-compiler.
(Reading database ... 123632 files and directories currently installed.)
Preparing to unpack .../edgetpu-compiler_16.0_amd64.deb ...
Unpacking edgetpu-compiler (16.0) ...
Setting up edgetpu-compiler (16.0) ...
Processing triggers for libc-bin (2.35-0ubuntu3.4) ...
/sbin/ldconfig.real: /usr/local/lib/libtcm.so.1 is not a symbolic link

/sbin/ldconfig.real: /usr/local/lib/libtbbmalloc_proxy.so.2 is not a symbolic link

/sbin/ldconfig.real: /usr/local/lib/libhwloc.so.15 is not a symbolic link

/sbin/ldconfig.real: /usr/local/lib/libur_loader.so.0 is not a symbolic link

/sbin/ldconfig.real: /usr/local/lib/libtbbmalloc.so.2 is not a symbolic link

/sbin/ldconfig.real: /usr/local/lib/libtcm_debug.so.1 is not a symbolic link

/sbin/ldconfig.real: /usr/local/lib/libur_adapter_level_zero.so.0 is not a symbolic link

/sbin/ldconfig.real: /usr/local/lib/libur_adapter_opencl.so.0 is not a symbolic link

/sbin/ldconfig.real: /usr/local/lib/libumf.so.0 is not a symbolic link

/sbin/ldconfig.real: /usr/local/lib/libtbbbind.so.3 is not a symbolic link

/sbin/ldconfig.real: /usr/local/lib/libtbbbind_2_0.so.3 is not a symbolic link

/sbin/ldconfig.real: /usr/local/lib/libtbbbind_2_5.so.3 is not a symbolic link

/sbin/ldconfig.real: /usr/local/lib/libtbb.so.12 is not a symbolic link

Edge TPU: starting export with Edge TPU compiler 16.0.384591198...
Edge TPU: running 'edgetpu_compiler --out_dir "runs/detect/train2/weights/best_saved_model" --show_operations --search_delegate --delegate_search_step 30 --timeout_sec 180 "runs/detect/train2/weights/best_saved_model/best_full_integer_quant.tflite"'
Edge TPU Compiler version 16.0.384591198
Searching for valid delegate with step 30
Try to compile segment with 497 ops
Started a compilation timeout timer of 180 seconds.

Model compiled successfully in 3526 ms.

Input model: runs/detect/train2/weights/best_saved_model/best_full_integer_quant.tflite
Input size: 3.12MiB
Output model: runs/detect/train2/weights/best_saved_model/best_full_integer_quant_edgetpu.tflite
Output size: 3.09MiB
On-chip memory used for caching model parameters: 1.36MiB
On-chip memory remaining for caching model parameters: 5.84MiB
Off-chip memory used for streaming uncached model parameters: 8.00KiB
Number of Edge TPU subgraphs: 1
Total number of operations: 497
Operation log: runs/detect/train2/weights/best_saved_model/best_full_integer_quant_edgetpu.log

Model successfully compiled but not all operations are supported by the Edge TPU. A percentage of the model will instead run on the CPU, which is slower. If possible, consider updating your model to use only operations supported by the Edge TPU. For details, visit g.co/coral/model-reqs.
Number of operations that will run on Edge TPU: 275
Number of operations that will run on CPU: 222

Operator Count Status

TRANSPOSE 3 Operation is otherwise supported, but not mapped due to some unspecified limitation
TRANSPOSE 7 More than one subgraph is not supported
TRANSPOSE 3 Mapped to Edge TPU
PAD 2 More than one subgraph is not supported
PAD 5 Mapped to Edge TPU
STRIDED_SLICE 2 Operation is otherwise supported, but not mapped due to some unspecified limitation
STRIDED_SLICE 11 Mapped to Edge TPU
STRIDED_SLICE 10 More than one subgraph is not supported
PACK 2 More than one subgraph is not supported
SOFTMAX 2 More than one subgraph is not supported
DEPTHWISE_CONV_2D 6 More than one subgraph is not supported
RESIZE_NEAREST_NEIGHBOR 2 More than one subgraph is not supported
MUL 34 Mapped to Edge TPU
MUL 46 More than one subgraph is not supported
CONV_2D 163 Mapped to Edge TPU
CONV_2D 46 More than one subgraph is not supported
LOGISTIC 44 More than one subgraph is not supported
LOGISTIC 34 Mapped to Edge TPU
FULLY_CONNECTED 4 More than one subgraph is not supported
RESHAPE 10 More than one subgraph is not supported
RESHAPE 2 Operation is otherwise supported, but not mapped due to some unspecified limitation
RESHAPE 5 Mapped to Edge TPU
MAX_POOL_2D 3 Mapped to Edge TPU
CONCATENATION 1 Operation is otherwise supported, but not mapped due to some unspecified limitation
CONCATENATION 8 Mapped to Edge TPU
CONCATENATION 14 More than one subgraph is not supported
QUANTIZE 3 More than one subgraph is not supported
SUB 2 More than one subgraph is not supported
ADD 7 Mapped to Edge TPU
ADD 10 More than one subgraph is not supported
SPLIT 4 For example, a fully-connected or softmax layer with 2D output
SPLIT 2 Mapped to Edge TPU
Compilation child process completed within timeout period.
Compilation succeeded!
Edge TPU: export success ✅ 17.6s, saved as 'runs/detect/train2/weights/best_saved_model/best_full_integer_quant_edgetpu.tflite' (3.1 MB)

Export complete (171.5s)
Results saved to /content/drive/MyDrive/VOXL2_YOLOV8/runs/detect/train2/weights
Predict: yolo predict task=detect model=runs/detect/train2/weights/best_saved_model/best_full_integer_quant_edgetpu.tflite imgsz=512 int8
Validate: yolo val task=detect model=runs/detect/train2/weights/best_saved_model/best_full_integer_quant_edgetpu.tflite imgsz=512 data=dataset.yaml int8
Visualize: https://netron.app/
💡 Learn more at https://docs.ultralytics.com/modes/export

[glenn-jocher]

The key issue causing YOLOv11's slower inference on the Coral Edge TPU is the higher number of operations being forced to run on the CPU (222 compared to YOLOv8's much smaller count). This creates a performance bottleneck since CPU fallback is significantly slower. For better performance, consider simplifying the YOLOv11 model or switching to a YOLOv8 model, as it has better compatebility with the TPU.