2020年9月23日 星期三

Jetson Nano DeepStream YoloV3 tiny custom model

紀錄關於在Jetson Nano 使用YoloV3 tiny 自製的預訓練檔案

1.預訓練檔案來源

https://github.com/amineHY/YOLOv3-for-custum-objects


2.將以下檔案放置到Jetson Nano上 /opt/nvidia/deepstream/deepstream-5.0//sources/objectDetector_Yolo目錄下

fire/model/yolov3-tiny-obj_final.weights

cfg/yolov3-tiny-obj.cfg


3.切換至 /opt/nvidia/deepstream/deepstream-5.0//sources/objectDetector_Yolo目錄

4.新建檔案 lables-obj.txt

fire


5.編輯檔案 config_infer_primary_yoloV3_tiny.txt

#custom-network-config=yolov3-tiny.cfg

custom-network-config=yolov3-tiny-obj.cfg

#model-file=yolov3-tiny.weights

model-file=yolov3-tiny-obj_final.weights

#labelfile-path=labels.txt

labelfile-path=labels-obj.txt


#network-mode=1

network-mode=2

#num-detected-classes=80

num-detected-classes=1


6.編輯檔案 nvdsinfer_custom_impl_Yolo/nvdsparsebbox_Yolo.cpp

-static const int NUM_CLASSES_YOLO = 80;

+static const int NUM_CLASSES_YOLO = 1;

7.Compile 檔案

export CUDA_VER=10.2
make -C nvdsinfer_custom_impl_Yolo

8.清除暫存檔 rm -rf ~/.cache/gstreamer-1.0/registry.aarch64.bin

9.執行 deepstream-app -c deepstream_app_config_yoloV3_tiny.txt

(deepstream-app:18137): GLib-GObject-WARNING **: 01:42:51.150: g_object_set_is_valid_property: object class 'GstNvArgusCameraSrc' has no property named 'maxperf'

(deepstream-app:18137): GStreamer-CRITICAL **: 01:42:51.152: passed '0' as denominator for `GstFraction'
Unknown or legacy key specified 'is-classifier' for group [property]
Warn: 'threshold' parameter has been deprecated. Use 'pre-cluster-threshold' instead.

Using winsys: x11 
gstnvtracker: Loading low-level lib at /opt/nvidia/deepstream/deepstream-5.0/lib/libnvds_mot_klt.so
gstnvtracker: Optional NvMOT_RemoveStreams not implemented
gstnvtracker: Batch processing is OFF
gstnvtracker: Past frame output is OFF
0:00:00.258438073 18137      0x983e930 INFO                 nvinfer gstnvinfer.cpp:619:gst_nvinfer_logger:<primary_gie> NvDsInferContext[UID 1]: Info from NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:1715> [UID = 1]: Trying to create engine from model files
Loading pre-trained weights...
Loading weights of yolov3-tiny complete!
Total Number of weights read : 8676244
Loading pre-trained weights...
Loading weights of yolov3-tiny complete!
Total Number of weights read : 8676244
Building Yolo network...
      layer               inp_size            out_size       weightPtr
(0)   conv-bn-leaky     3 x 608 x 608      16 x 608 x 608    496   
(1)   maxpool          16 x 608 x 608      16 x 304 x 304    496   
(2)   conv-bn-leaky    16 x 304 x 304      32 x 304 x 304    5232  
(3)   maxpool          32 x 304 x 304      32 x 152 x 152    5232  
(4)   conv-bn-leaky    32 x 152 x 152      64 x 152 x 152    23920 
(5)   maxpool          64 x 152 x 152      64 x  76 x  76    23920 
(6)   conv-bn-leaky    64 x  76 x  76     128 x  76 x  76    98160 
(7)   maxpool         128 x  76 x  76     128 x  38 x  38    98160 
(8)   conv-bn-leaky   128 x  38 x  38     256 x  38 x  38    394096
(9)   maxpool         256 x  38 x  38     256 x  19 x  19    394096
(10)  conv-bn-leaky   256 x  19 x  19     512 x  19 x  19    1575792
(11)  maxpool         512 x  19 x  19     512 x  19 x  19    1575792
(12)  conv-bn-leaky   512 x  19 x  19    1024 x  19 x  19    6298480
(13)  conv-bn-leaky  1024 x  19 x  19     256 x  19 x  19    6561648
(14)  conv-bn-leaky   256 x  19 x  19     512 x  19 x  19    7743344
(15)  conv-linear     512 x  19 x  19      18 x  19 x  19    7752578
(16)  yolo             18 x  19 x  19      18 x  19 x  19    7752578
(17)  route                  -            256 x  19 x  19    7752578
(18)  conv-bn-leaky   256 x  19 x  19     128 x  19 x  19    7785858
INFO: [TRT]: mm1_19: broadcasting input0 to make tensors conform, dims(input0)=[1,38,19][NONE] dims(input1)=[128,19,19][NONE].
INFO: [TRT]: mm2_19: broadcasting input1 to make tensors conform, dims(input0)=[128,38,19][NONE] dims(input1)=[1,19,38][NONE].
(19)  upsample        128 x  19 x  19     128 x  38 x  38        - 
(20)  route                  -            384 x  38 x  38    7785858
(21)  conv-bn-leaky   384 x  38 x  38     256 x  38 x  38    8671618
(22)  conv-linear     256 x  38 x  38      18 x  38 x  38    8676244
(23)  yolo             18 x  38 x  38      18 x  38 x  38    8676244
Output yolo blob names :
yolo_17
yolo_24
Total number of yolo layers: 49
Building yolo network complete!
Building the TensorRT Engine...
INFO: [TRT]: mm1_19: broadcasting input0 to make tensors conform, dims(input0)=[1,38,19][NONE] dims(input1)=[128,19,19][NONE].
INFO: [TRT]: mm2_19: broadcasting input1 to make tensors conform, dims(input0)=[128,38,19][NONE] dims(input1)=[1,19,38][NONE].
INFO: [TRT]: Some tactics do not have sufficient workspace memory to run. Increasing workspace size may increase performance, please check verbose output.
INFO: [TRT]: Detected 1 inputs and 2 output network tensors.
Building complete!
0:02:35.752148292 18137      0x983e930 INFO                 nvinfer gstnvinfer.cpp:619:gst_nvinfer_logger:<primary_gie> NvDsInferContext[UID 1]: Info from NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:1748> [UID = 1]: serialize cuda engine to file: /opt/nvidia/deepstream/deepstream-5.0/sources/objectDetector_Yolo/model_b1_gpu0_fp16.engine successfully
INFO: [Implicit Engine Info]: layers num: 3
0   INPUT  kFLOAT data            3x608x608       
1   OUTPUT kFLOAT yolo_17         18x19x19        
2   OUTPUT kFLOAT yolo_24         18x38x38        

0:02:35.795951757 18137      0x983e930 INFO                 nvinfer gstnvinfer_impl.cpp:313:notifyLoadModelStatus:<primary_gie> [UID 1]: Load new model:/opt/nvidia/deepstream/deepstream-5.0/sources/objectDetector_Yolo/config_infer_primary_yoloV3_tiny.txt sucessfully

Runtime commands:
h: Print this help
q: Quit

p: Pause
r: Resume

NOTE: To expand a source in the 2D tiled display and view object details, left-click on the source.
      To go back to the tiled display, right-click anywhere on the window.


**PERF:  FPS 0 (Avg)
**PERF:  0.00 (0.00)
** INFO: <bus_callback:181>: Pipeline ready

** INFO: <bus_callback:167>: Pipeline running

GST_ARGUS: Creating output stream
CONSUMER: Waiting until producer is connected...
GST_ARGUS: Available Sensor modes :
GST_ARGUS: 3264 x 2464 FR = 21.000000 fps Duration = 47619048 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 3264 x 1848 FR = 28.000001 fps Duration = 35714284 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1920 x 1080 FR = 29.999999 fps Duration = 33333334 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1280 x 720 FR = 59.999999 fps Duration = 16666667 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1280 x 720 FR = 120.000005 fps Duration = 8333333 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: Running with following settings:
   Camera index = 0 
   Camera mode  = 4 
   Output Stream W = 1280 H = 720 
   seconds to Run    = 0 
   Frame Rate = 120.000005 
GST_ARGUS: Setup Complete, Starting captures for 0 seconds
GST_ARGUS: Starting repeat capture requests.
CONSUMER: Producer has connected; continuing.
KLT Tracker Init
**PERF:  29.67 (29.46)
**PERF:  29.04 (29.10)
**PERF:  29.10 (29.20)
**PERF:  29.04 (29.10)
**PERF:  29.10 (29.16)
**PERF:  29.04 (29.10)
**PERF:  28.87 (29.11)
**PERF:  29.24 (29.10)

10.使用Overlay sink直接輸出結果至HDMI Screen. 

編輯檔案 deepstream_app_config_yoloV3_tiny.txt

[sink0]
enable=1
#Type - 1=FakeSink 2=EglSink 3=File 5=overlaysink
type=5
width=1280
height=720

編輯 /etc/ssh/sshd_config
+   #X11Forwarding no
-    X11Forwarding yes

設定環境參數

unset DISPLAY

11.使用 CSI camera作為輸入來源

編輯檔案 deepstream_app_config_yoloV3_tiny.txt

[source0]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI 5=CsiCamera
type=5
camera-width=1280
camera-height=720

12. Reference