AttributeError: module 'tensorflow._api.v2.lite' has no attribute 'load_delegate'

@zldrobit

Search before asking

I have searched the YOLOv5 issues and found no similar bug report.

YOLOv5 Component

Export

Bug

@zldrobit I think recent changes to EdgeTPU inference created a bug where load_delegate import was attempted from tensorflow.lite.load_delegate instead of tensorflow.lite.experimental.load_delegate causing this error message:

AttributeError: module 'tensorflow._api.v2.lite' has no attribute 'load_delegate'

If I attempt a fix in #6536 I still see a new error message on EdgeTPU inference:

AttributeError: 'Delegate' object has no attribute '_library'

Environment

YOLOv5 🚀 v6.0-231-g9bc72a3 torch 1.10.0+cu111 CPU

Minimal Reproducible Example

In Colab Notebook
https://colab.research.google.com/github/ultralytics/yolov5/blob/master/tutorial.ipynb

!python export.py --include edgetpu
!python detect.py --weights yolov5s-int8_edgetpu.tflite

Additional

Will submit PR with fix for original error.

Are you willing to submit a PR?

Yes I'd like to help by submitting a PR!

Edge TPU `tf.lite.experimental.load_delegate` fix

Edge TPU `tf.lite.experimental.load_delegate` fix (#6536)

@zldrobit

@zldrobit ok I've merged #6536 which I think fixes original issue.

I think the second error is just because I didn't install Edge TPU runtime in Colab. Is it possible to run Edge TPU models without Edge TPU accelerator plugged into the machine? I'm asking because I have a separate branch that I'm working on that is creating a new benchmarks.py to benchmark all of the export formats (currently CPU-only), i.e.:
https://github.com/ultralytics/yolov5/blob/updates/benchmarks/utils/benchmarks.py

benchmarks: weights=/usr/src/yolov5/yolov5s.pt, imgsz=640, batch_size=1, data=/usr/src/yolov5/data/coco128.yaml
...

Benchmarks complete (782.19s)
                   Weights  mAP@0.5:0.95  Inference time (ms)
0               yolov5s.pt      0.407554           102.777191
1      yolov5s.torchscript      0.402908           132.848348
2             yolov5s.onnx      0.402908            89.061137
3   yolov5s_openvino_model      0.402908            67.093970
4                   engine           NaN                  NaN
5                   coreml           NaN                  NaN
6      yolov5s_saved_model      0.402908           133.983964
7               yolov5s.pb      0.402908           101.405423
8      yolov5s-fp16.tflite      0.402851           502.289245
9                  edgetpu           NaN                  NaN
10                    tfjs           NaN                  NaN

@glenn-jocher

Hello @glenn-jocher I am facing similar issue ModuleNotFoundError: No module named 'tensorflow.lite.experimental.load_delegate'. when trying to do this !python /content/yolov5/detect.py --weights /content/yolov5s-fp16.tflite.. Any fix?

Tensorflow Version -: 2.7.0
Environment -: colab

@Rishav-hub

@Rishav-hub yes I think we need to follow the official example more closely, which does work in my tests:
https://www.tensorflow.org/api_docs/python/tf/lite/experimental/load_delegate

@glenn-jocher

@glenn-jocher but what is causing this issue do I need to make changes in the common.py in yolov5. Is there any permanent fix you suggest or do I need to make changes in the source code?

@Rishav-hub

@Rishav-hub we need a PR to update the repo with a fix. It's difficult for me to debug since I don't have any Edge TPU devices to test on. The requirements are:

Prefer Edge TPU Runtime to TF if installed
Fix load_delegate() issues
Keep code as concise as possible

The area of the code that needs updating is:

yolov5/models/common.py

Lines 376 to 390 in cba4303

    
           elif tflite:  # https://www.tensorflow.org/lite/guide/python#install_tensorflow_lite_for_python 
        
               try:  # prefer tflite_runtime if installed 
        
                   from tflite_runtime.interpreter import Interpreter, load_delegate 
        
               except ImportError: 
        
                   import tensorflow.lite.experimental.load_delegate as load_delegate 
        
                   import tensorflow.lite.Interpreter as Interpreter 
        
               if 'edgetpu' in w.lower():  # Edge TPU https://coral.ai/software/#edgetpu-runtime 
        
                   LOGGER.info(f'Loading {w} for TensorFlow Lite Edge TPU inference...') 
        
                   delegate = {'Linux': 'libedgetpu.so.1', 
        
                               'Darwin': 'libedgetpu.1.dylib', 
        
                               'Windows': 'edgetpu.dll'}[platform.system()] 
        
                   interpreter = Interpreter(model_path=w, experimental_delegates=[load_delegate(delegate)]) 
        
               else:  # Lite 
        
                   LOGGER.info(f'Loading {w} for TensorFlow Lite inference...') 
        
                   interpreter = Interpreter(model_path=w)  # load TFLite model

Edge TPU TF imports fix

Edge TPU TF imports fix (#6542)

@Rishav-hub

@Rishav-hub good news 😃! Your original issue may now be fixed ✅ in PR #6542. This PR fixes TF module loading only for Edge TPU TFLite exports, i.e. python detect.py --weights yolov5s-int8_edgetpu.tflite

To receive this update:

Git – git pull from within your yolov5/ directory or git clone https://github.com/ultralytics/yolov5 again
PyTorch Hub – Force-reload model = torch.hub.load('ultralytics/yolov5', 'yolov5s', force_reload=True)
Notebooks – View updated notebooks
Docker – sudo docker pull ultralytics/yolov5:latest to update your image

Thank you for spotting this issue and informing us of the problem. Please let us know if this update resolves the issue for you, and feel free to inform us of any other issues you discover or feature requests that come to mind. Happy trainings with YOLOv5 🚀!

@glenn-jocher

@glenn-jocher Edge TPU models cannot be run on CPU only, because of the Edge TPU operator.

EDIT: If there is no delegate given when initializing TFLite interpreter, the output message running an Edge TPU model without an accelerator is as follows,

@zldrobit

@zldrobit ok got it!

Update Neuralmagic-YoloV5 integration to v6.1 (#29)

Edge TPU `tf.lite.experimental.load_delegate` fix (ultralytics#6536)

Edge TPU TF imports fix (ultralytics#6542)

Edge TPU TF imports fix (#6542)

Navigation Menu

Repository navigation

AttributeError: module 'tensorflow._api.v2.lite' has no attribute 'load_delegate' #6535

Search before asking

YOLOv5 Component

Bug

Environment

Minimal Reproducible Example

Additional

Are you willing to submit a PR?

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

	elif tflite: # https://www.tensorflow.org/lite/guide/python#install_tensorflow_lite_for_python
	try: # prefer tflite_runtime if installed
	from tflite_runtime.interpreter import Interpreter, load_delegate
	except ImportError:
	import tensorflow.lite.experimental.load_delegate as load_delegate
	import tensorflow.lite.Interpreter as Interpreter
	if 'edgetpu' in w.lower(): # Edge TPU https://coral.ai/software/#edgetpu-runtime
	LOGGER.info(f'Loading {w} for TensorFlow Lite Edge TPU inference...')
	delegate = {'Linux': 'libedgetpu.so.1',
	'Darwin': 'libedgetpu.1.dylib',
	'Windows': 'edgetpu.dll'}[platform.system()]
	interpreter = Interpreter(model_path=w, experimental_delegates=[load_delegate(delegate)])
	else: # Lite
	LOGGER.info(f'Loading {w} for TensorFlow Lite inference...')
	interpreter = Interpreter(model_path=w) # load TFLite model

AttributeError: module 'tensorflow._api.v2.lite' has no attribute 'load_delegate' #6535

Description

Search before asking

YOLOv5 Component

Bug

Environment

Minimal Reproducible Example

Additional

Are you willing to submit a PR?

Activity

glenn-jocher commented on Feb 4, 2022

Rishav-hub commented on Feb 5, 2022

glenn-jocher commented on Feb 5, 2022

Rishav-hub commented on Feb 5, 2022

glenn-jocher commented on Feb 5, 2022

glenn-jocher commented on Feb 5, 2022

zldrobit commented on Feb 8, 2022

glenn-jocher commented on Feb 8, 2022

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions