TFLite export issue from yolov3.pt weights (works for yolov3-spp.pt)

Search before asking

I have searched the YOLOv3 issues and found no similar bug report.

YOLOv3 Component

Export

Bug

python export.py --weights yolov3.pt --include saved_model pb tflite --int8 --img 416
export: data=data/coco128.yaml, weights=yolov3.pt, imgsz=[416], batch_size=1, device=cpu, half=False, inplace=False, train=False, optimize=False, int8=True, dynamic=False, simplify=False, opset=13, topk_per_class=100, topk_all=100, iou_thres=0.45, conf_thres=0.25, include=['saved_model', 'pb', 'tflite']
YOLOv3 🚀 v9.6.0-6-g0f80f2f torch 1.10.1+cu102 CPU

Fusing layers... 
Model Summary: 261 layers, 61922845 parameters, 0 gradients, 156.1 GFLOPs

PyTorch: starting from yolov3.pt (124.3 MB)
2022-01-17 16:05:01.871223: W tensorflow/stream_executor/platform/default/dso_loader.cc:60] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /home/corehalt/src/yolov3_test/venv/lib/python3.8/site-packages/cv2/../../lib64:
2022-01-17 16:05:01.871262: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.

TensorFlow saved_model: starting export with tensorflow 2.4.1...

                 from  n    params  module                                  arguments                     
  0                -1  1       928  models.common.Conv                      [3, 32, 3, 1]                 
2022-01-17 16:05:03.584306: I tensorflow/compiler/jit/xla_cpu_device.cc:41] Not creating XLA devices, tf_xla_enable_xla_devices not set
2022-01-17 16:05:03.584784: W tensorflow/stream_executor/platform/default/dso_loader.cc:60] Could not load dynamic library 'libcuda.so.1'; dlerror: libcuda.so.1: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /home/corehalt/src/yolov3_test/venv/lib/python3.8/site-packages/cv2/../../lib64:
2022-01-17 16:05:03.584829: W tensorflow/stream_executor/cuda/cuda_driver.cc:326] failed call to cuInit: UNKNOWN ERROR (303)
2022-01-17 16:05:03.584867: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:156] kernel driver does not appear to be running on this host (pop-os): /proc/driver/nvidia/version does not exist
2022-01-17 16:05:03.585266: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2022-01-17 16:05:03.585612: I tensorflow/compiler/jit/xla_gpu_device.cc:99] Not creating XLA devices, tf_xla_enable_xla_devices not set
  1                -1  1     18560  models.common.Conv                      [32, 64, 3, 2]                
  2                -1  1     20672  models.common.Bottleneck                [64, 64]                      
  3                -1  1     73984  models.common.Conv                      [64, 128, 3, 2]               
  4                -1  2    164608  models.common.Bottleneck                [128, 128]                    
  5                -1  1    295424  models.common.Conv                      [128, 256, 3, 2]              
  6                -1  8   2627584  models.common.Bottleneck                [256, 256]                    
  7                -1  1   1180672  models.common.Conv                      [256, 512, 3, 2]              
  8                -1  8  10498048  models.common.Bottleneck                [512, 512]                    
  9                -1  1   4720640  models.common.Conv                      [512, 1024, 3, 2]             
 10                -1  4  20983808  models.common.Bottleneck                [1024, 1024]                  
 11                -1  1   5245952  models.common.Bottleneck                [1024, 1024, False]           

TensorFlow saved_model: export failure: Convolution with multiple kernels are not allowed.

TensorFlow GraphDef: starting export with tensorflow 2.4.1...

TensorFlow GraphDef: export failure: 'NoneType' object has no attribute 'inputs'

TensorFlow Lite: starting export with tensorflow 2.4.1...

TensorFlow Lite: export failure: 'NoneType' object has no attribute 'call'

Export complete (5.98s)
Results saved to /home/corehalt/src/yolov3_test/yolov3
Visualize with https://netron.app

Environment

YOLO: YOLOv3 🚀 v9.6.0-6-g0f80f2f torch 1.10.1+cu102 CPU
OS: Ubuntu 20.04 (Pop OS)
Python: Python 3.8.1

Minimal Reproducible Example

python export.py --weights yolov3.pt --include saved_model pb tflite --int8 --img 416

Additional

I'm trying to export the yolov3 model to TFLite.
I installed the requirements in a clean virtualenv with Python 3.8.10 (uncommenting the tflite export section in the file) and run the export command:

# works with `yolov3-spp.pt` (automatically downloaded previously by detect.py)
python export.py --weights yolov3-spp.pt --include saved_model pb tflite --int8 --img 416

but, on the other hand, it fails to export the standard yolov3.pt model. It fails with the specified error:

# Fails with `yolov3.pt` (automatically downloaded previously by detect.py)
python export.py --weights yolov3.pt --include saved_model pb tflite --int8 --img 416

I also tried to to use different versions of TensorFlow (2.7.0, 2.6.1, 2.4.1) but doesn't work as well.

Thank you.

Are you willing to submit a PR?

Yes I'd like to help by submitting a PR!

@corehalt

👋 Hello @corehalt, thank you for your interest in YOLOv3 🚀! Please visit our ⭐️ Tutorials to get started, where you can find quickstart guides for simple tasks like Custom Data Training all the way to advanced concepts like Hyperparameter Evolution.

If this is a 🐛 Bug Report, please provide screenshots and minimum viable code to reproduce your issue, otherwise we can not help you.

If this is a custom training ❓ Question, please provide as much information as possible, including dataset images, training logs, screenshots, and a public link to online W&B logging if available.

For business inquiries or professional support requests please visit https://ultralytics.com or email Glenn Jocher at glenn.jocher@ultralytics.com.

Requirements

Python>=3.6.0 with all requirements.txt installed including PyTorch>=1.7. To get started:

$ git clone https://github.com/ultralytics/yolov3
$ cd yolov3
$ pip install -r requirements.txt

Environments

YOLOv3 may be run in any of the following up-to-date verified environments (with all dependencies including CUDA/CUDNN, Python and PyTorch preinstalled):

Google Colab and Kaggle notebooks with free GPU:
Google Cloud Deep Learning VM. See GCP Quickstart Guide
Amazon Deep Learning AMI. See AWS Quickstart Guide
Docker Image. See Docker Quickstart Guide

Status

If this badge is green, all YOLOv3 GitHub Actions Continuous Integration (CI) tests are currently passing. CI tests verify correct operation of YOLOv3 training (train.py), validation (val.py), inference (detect.py) and export (export.py) on MacOS, Windows, and Ubuntu every 24 hours and on every commit.

@corehalt

@corehalt yes I'm able to reproduce the same. Not sure what the issue might be. You could try exporting via the YOLOv5 repo, which is more up to date, though the repo doesn't seem to be the problem since YOLOv3-SPP exports correctly.

@corehalt

@corehalt unfortunately our support levels for the YOLOv3 repo are minimal as we support and grow YOLOv5, so I'll put a TODO on this, but I can't give you an ETA for when it might be solved.

@glenn-jocher

@glenn-jocher just for the record, same issue also happens for yolov3-tiny.pt.

@corehalt

@corehalt hmm interesting. We probably just need to retrain all the YOLOv3 models with the latest YOLOv5 master, i.e. just use the yolov3 yamls with the YOLOv5 repo and retrain, and then these should be exportable.

When we released these models TFLite export was not fully supported, and it's still not part of our CI checks (though it should be).

@corehalt

@corehalt all https://github.com/ultralytics/yolov3 models are now retrained but original issue is not resolved with these new models.

New models are attached as updated release assets at https://github.com/ultralytics/yolov3/releases/tag/v9.6.0

@glenn-jocher

@glenn-jocher Thank you for retraining the models.
An update on this is that even using the new yolov3 retrained model under yolov5 repository I'm still getting the same issue:

TensorFlow Lite: export failure: 'NoneType' object has no attribute 'call'

@corehalt

@corehalt yeah sorry. Can you try debugging by removing the TFLite export try: except and tracing the source of the error?

@glenn-jocher

@glenn-jocher I think I found the issue, now I can export correctly using yolov3.yaml
Seems that here https://github.com/ultralytics/yolov3/blob/master/models/yolov3.yaml#L31 there is a typo:

# YOLOv3 head
head:
  [[-1, 1, Bottleneck, [1024, False]],
   [-1, 1, Conv, [512, [1, 1]]],   <-------
   [-1, 1, Conv, [1024, 3, 1]],

to be able to export later in TF it should be changed to:

# YOLOv3 head
head:
  [[-1, 1, Bottleneck, [1024, False]],
   [-1, 1, Conv, [512, 1, 1]],   <------ no brackets
   [-1, 1, Conv, [1024, 3, 1]],

The problem is that any training that starts from the pre-trained yolov3.pt model will have this issue.
Only a train from the scratch will allow to export later to TF.

Is it possible for you to retrain the model with the updated yaml?

Thank you

Ah, the typo also appears in the yaml file configuration of the YoloV5 repo:
https://github.com/ultralytics/yolov5/blob/master/models/hub/yolov3.yaml#L31

@corehalt

@corehalt oh awesome, nice work! Yes I can update and retrain.

Fix yolov3.yaml remove extra bracket

Fix yolov3.yaml remove list

@glenn-jocher

@glenn-jocher just for the record, same issue also happens for yolov3-tiny.pt.

I can confirm that yolov3 is fixed but not the yolov3-tiny

@daniele-salerno

@daniele-salerno sure. YOLOv3-tiny has special maxpool layers that have not been translated into corresponding TF layers in tf.py. These are required for TF exports.

Please see our ✅ Contributing Guide to get started.

Update Neuralmagic-YoloV5 integration to v6.1 (#29)

For those having trouble converting Yolov3-tiny to tflite see this.

@glenn-jocher

@glenn-jocher
Hello, I succeeded in changing best.pt to best-fp16.tflite looking at the above issues.
However, when I tried to detect a jpg image of 416x416 size with detect.py, the following error occurred.

WARNING: --img-size [416, 416] must be multiple of max stride 64, updating to [448, 448]
ValueError: Cannot set tensor: Dimension mismatch. Got 448 but expected 416 for dimension 1 of input 0.

In this case, when training yolov3, should I have trained only with 448x448 images?
Can't I use yolov3 trained with a 416x416 image?

This is the code that executed detect.py.
!python detect.py --weights best-fp16.tflite --img 416 --conf 0.4 --iou-thres 0.8 --source /content/datasets/custom/images/med_36.jpg

@Euijune

@Euijune I would export and detect with the YOLOv5 repo (even if you trained with YOLOv3 it will still work) for best results.

Yes you can train at any size and detect at any other size (pytorch), though many export formats are fixed and not dynamic, so whatever you export at will be the size you must run inference at.

@glenn-jocher

@glenn-jocher
Thank you.
I converted from yolo_v5 repo, I was able to complete detect successfully.
Thank you very much for your quick response.

Fix yolov3.yaml remove list (ultralytics#6655)

@glenn-jocher

@glenn-jocher I am trying to convert the best.pt weights but keep encountering this error.

export: data=yolov5/data/coco128.yaml, weights=['/home/ec2-user/SageMaker/yolov5/runs/train/results_128/weights/best.pt'], imgsz=[256], batch_size=1, device=cpu, half=False, inplace=False, keras=False, optimize=False, int8=False, dynamic=False, simplify=False, opset=17, verbose=False, workspace=4, nms=False, agnostic_nms=False, topk_per_class=100, topk_all=100, iou_thres=0.45, conf_thres=0.25, include=['saved_model', 'pb', 'tflite']
YOLOv5 🚀 v7.0-66-g9650f16 Python-3.9.13 torch-1.12.1 CPU

Fusing layers...
Model summary: 267 layers, 46113663 parameters, 0 gradients

PyTorch: starting from /home/ec2-user/SageMaker/yolov5/runs/train/results_128/weights/best.pt with output shape (1, 4032, 7) (88.4 MB)
2023-02-03 16:54:24.112721: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 AVX512F FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.

TensorFlow SavedModel: starting export with tensorflow 2.11.0...

             from  n    params  module                                  arguments

2023-02-03 16:54:25.573277: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 AVX512F FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
0 -1 1 7040 models.common.Conv [3, 64, 6, 2, 2]
1 -1 1 73984 models.common.Conv [64, 128, 3, 2]
2 -1 1 156928 models.common.C3 [128, 128, 3]
3 -1 1 295424 models.common.Conv [128, 256, 3, 2]
4 -1 1 1118208 models.common.C3 [256, 256, 6]
5 -1 1 1180672 models.common.Conv [256, 512, 3, 2]
6 -1 1 6433792 models.common.C3 [512, 512, 9]
7 -1 1 4720640 models.common.Conv [512, 1024, 3, 2]
8 -1 1 9971712 models.common.C3 [1024, 1024, 3]
9 -1 1 2624512 models.common.SPPF [1024, 1024, 5]
10 -1 1 525312 models.common.Conv [1024, 512, 1, 1]
11 -1 1 0 torch.nn.modules.upsampling.Upsample [None, 2, 'nearest']
12 [-1, 6] 1 0 models.common.Concat [1]
13 -1 1 2757632 models.common.C3 [1024, 512, 3, False]
14 -1 1 131584 models.common.Conv [512, 256, 1, 1]
15 -1 1 0 torch.nn.modules.upsampling.Upsample [None, 2, 'nearest']
16 [-1, 4] 1 0 models.common.Concat [1]
17 -1 1 690688 models.common.C3 [512, 256, 3, False]
18 -1 1 590336 models.common.Conv [256, 256, 3, 2]
19 [-1, 14] 1 0 models.common.Concat [1]
20 -1 1 2495488 models.common.C3 [512, 512, 3, False]
21 -1 1 2360320 models.common.Conv [512, 512, 3, 2]
22 [-1, 10] 1 0 models.common.Concat [1]
23 -1 1 9971712 models.common.C3 [1024, 1024, 3, False]
24 [17, 20, 23] 1 37695 models.yolo.Detect [2, [[10, 13, 16, 30, 33, 23], [30, 61, 62, 45, 59, 119], [116, 90, 156, 198, 373, 326]], [256, 512, 1024], [256, 256]]
TensorFlow SavedModel: export failure ❌ 2.1s: Exception encountered when calling layer 'sequential' (type Sequential).

One of the dimensions in the output is <= 0 due to downsampling in conv2d. Consider increasing the input size. Received input shape [1, 260, 260, 3] which would produce output shape with a zero or negative value in a dimension.

Call arguments received by layer 'sequential' (type Sequential):
• inputs=tf.Tensor(shape=(1, 256, 256, 3), dtype=float32)
• training=None
• mask=None

TensorFlow GraphDef: starting export with tensorflow 2.11.0...
TensorFlow GraphDef: export failure ❌ 0.0s: 'NoneType' object has no attribute 'inputs'

TensorFlow Lite: starting export with tensorflow 2.11.0...
TensorFlow Lite: export failure ❌ 0.0s: 'NoneType' object has no attribute 'call'
Traceback (most recent call last):
File "/home/ec2-user/SageMaker/yolov5/export.py", line 653, in
main(opt)
File "/home/ec2-user/SageMaker/yolov5/export.py", line 648, in main
run(**vars(opt))
File "/home/ec2-user/anaconda3/envs/pytorch_p39/lib/python3.9/site-packages/torch/autograd/grad_mode.py", line 27, in decorate_context
return func(*args, **kwargs)
File "/home/ec2-user/SageMaker/yolov5/export.py", line 589, in run
add_tflite_metadata(f[8] or f[7], metadata, num_outputs=len(s_model.outputs))
AttributeError: 'NoneType' object has no attribute 'outputs'

Fix yolov3.yaml remove list (#6655)

@paramkaur10

@paramkaur10 It seems the export script is encountering issues with the tensorflow export.

You may need to adjust the input size to meet the model's requirements. You can try increasing the input size to see if that resolves the issue, as indicated by the error message "Consider increasing the input size". Additionally, you may want to review the code related to the downsampling in conv2d to ensure it aligns with the desired input size and output shape.

Finally, for TensorFlow export, ensure that the input and output tensors are correctly defined and that they align with the model's architecture.

If these steps do not resolve the issue, further investigation into the model architecture and the export script may be necessary to identify the root cause of the error.

I hope this helps!

Fix yolov3.yaml remove list (#6655)

Navigation Menu

Repository navigation

TFLite export issue from yolov3.pt weights (works for yolov3-spp.pt) #1887

Search before asking

YOLOv3 Component

Bug

Environment

Minimal Reproducible Example

Additional

Are you willing to submit a PR?

Requirements

Environments

Status

12 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

TFLite export issue from yolov3.pt weights (works for yolov3-spp.pt) #1887

Description

Search before asking

YOLOv3 Component

Bug

Environment

Minimal Reproducible Example

Additional

Are you willing to submit a PR?

Activity

github-actions commented on Jan 17, 2022

Requirements

Environments

Status

glenn-jocher commented on Jan 17, 2022

glenn-jocher commented on Jan 17, 2022

corehalt commented on Jan 20, 2022

glenn-jocher commented on Jan 20, 2022

glenn-jocher commented on Jan 24, 2022

corehalt commented on Feb 14, 2022

glenn-jocher commented on Feb 14, 2022

corehalt commented on Feb 16, 2022

corehalt commented on Feb 16, 2022

glenn-jocher commented on Feb 16, 2022

12 remaining items

daniele-salerno commented on Mar 22, 2022

glenn-jocher commented on Mar 22, 2022

wakasturner commented on Jun 13, 2022

Euijune commented on Aug 13, 2022

Euijune commented on Aug 13, 2022

glenn-jocher commented on Aug 14, 2022

Euijune commented on Aug 17, 2022

paramkaur10 commented on Feb 3, 2023

glenn-jocher commented on Nov 14, 2023

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions