netcat을 사용하여 v4l2loopback에서 v4l2src(edgepuvision)로 비디오 전달

2024-10-28 • tag-icon

netcat을 사용하여 v4l2loopback에서 v4l2src(edgepuvision)로 비디오 전달

태블릿의 웹캠을 Coral Dev Board의 TensorFlow 모델용 스트리밍 소스로 사용하려고 합니다. 내 생각은 netcat을 사용하여 태블릿에서 스트림을 전달하는 것입니다.

ffmpeg -i /dev/video0 -filter:v fps=30 -f matroska - | nc -l 9999

그런 다음 Coral 보드에서 v4l2loopback을 사용합니다.

modprobe v4l2loopback exclusive_caps=1 card_label="Golf:Cam1"

스트림을 전달할 수 있도록 가상 비디오 장치를 만듭니다.

nc 192.168.1.50 9999 | ffmpeg -i /dev/stdin -f v4l2 /dev/video2

스트림이 올바르게 전달된 것 같습니다.

그런 다음 edgepu_Detect_server를 시작합니다. 하지만 브라우저에 결과를 표시하려고 하면 검은색 화면만 표시되고 터미널에는 다음 오류가 표시됩니다.

오류: gst-stream-error-quark: 내부 데이터 스트림 오류입니다. (1): gstbasesrc.c(3055): gst_base_src_loop (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: 스트리밍이 중지됨, 이유가 협상되지 않음 (-4)

netcat으로의 스트리밍이 아직 실행 중입니다.

다음은 좀 더 자세한 출력입니다.

태블릿에서

# ffmpeg -i /dev/video0 -filter:v fps=30 -f matroska - | nc -l 9999

ffmpeg version 6.0-6ubuntu1 Copyright (c) 2000-2023 the FFmpeg developers
  built with gcc 13 (Ubuntu 13.2.0-2ubuntu1)
  configuration: --prefix=/usr --extra-version=6ubuntu1 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libdav1d --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libglslang --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librabbitmq --enable-librist --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libsvtav1 --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzimg --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --disable-sndio --enable-libjxl --enable-pocketsphinx --enable-librsvg --enable-libvpl --disable-libmfx --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --enable-libplacebo --enable-librav1e --enable-shared
  libavutil      58.  2.100 / 58.  2.100
  libavcodec     60.  3.100 / 60.  3.100
  libavformat    60.  3.100 / 60.  3.100
  libavdevice    60.  1.100 / 60.  1.100
  libavfilter     9.  3.100 /  9.  3.100
  libswscale      7.  1.100 /  7.  1.100
  libswresample   4. 10.100 /  4. 10.100
  libpostproc    57.  1.100 / 57.  1.100
Input #0, video4linux2,v4l2, from '/dev/video0':
  Duration: N/A, start: 76152.953085, bitrate: 147456 kb/s
  Stream #0:0: Video: rawvideo (YUY2 / 0x32595559), yuyv422, 1280x720, 147456 kb/s, 10 fps, 10 tbr, 1000k tbn
Stream mapping:
  Stream #0:0 -> #0:0 (rawvideo (native) -> h264 (libx264))
Press [q] to stop, [?] for help
[libx264 @ 0x5896e86b2680] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
[libx264 @ 0x5896e86b2680] profile High 4:2:2, level 3.1, 4:2:2, 8-bit
[libx264 @ 0x5896e86b2680] 264 - core 164 r3095 baee400 - H.264/MPEG-4 AVC codec - Copyleft 2003-2022 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=6 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
Output #0, matroska, to 'pipe:':
  Metadata:
    encoder         : Lavf60.3.100
  Stream #0:0: Video: h264 (H264 / 0x34363248), yuv422p(tv, progressive), 1280x720, q=2-31, 30 fps, 1k tbn
    Metadata:
      encoder         : Lavc60.3.100 libx264
    Side data:
      cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: N/A
frame=    0 fps=0.0 q=0.0 size=       0kB time=-577014:32:22.77 bitrate=  -0.0kbits/s speed=N/A   
frame=    0 fps=0.0 q=0.0 size=       0kB time=-577014:32:22.77 bitrate=  -0.0kbits/s speed=N/A   
frame=    0 fps=0.0 q=0.0 size=       0kB time=-577014:32:22.77 bitrate=  -0.0kbits/s speed=N/A   
frame=46781 fps= 30 q=29.0 size=   51378kB time=00:25:59.30 bitrate= 269.9kbits/s speed=0.998x

Coral 개발 보드에서

# nc 192.168.1.50 9999 | ffmpeg -i /dev/stdin -f v4l2 /dev/video2

ffmpeg version 4.1.6-1~deb10u1 Copyright (c) 2000-2020 the FFmpeg developers
  built with gcc 8 (Debian 8.3.0-6)
  configuration: --prefix=/usr --extra-version='1~deb10u1' --toolchain=hardened --libdir=/usr/lib/aarch64-linux-gnu --incdir=/usr/include/aarch64-linux-gnu --arch=arm64 --enable-gpl --disable-stripping --enable-avresample --disable-filter=resample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librsvg --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared
  libavutil      56. 22.100 / 56. 22.100
  libavcodec     58. 35.100 / 58. 35.100
  libavformat    58. 20.100 / 58. 20.100
  libavdevice    58.  5.100 / 58.  5.100
  libavfilter     7. 40.101 /  7. 40.101
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  3.100 /  5.  3.100
  libswresample   3.  3.100 /  3.  3.100
  libpostproc    55.  3.100 / 55.  3.100
Input #0, matroska,webm, from '/dev/stdin':
  Metadata:
    ENCODER         : Lavf60.3.100
  Duration: N/A, start: 0.000000, bitrate: N/A
    Stream #0:0: Video: h264 (High 4:2:2), yuv422p(tv, progressive), 1280x720, 30 fps, 30 tbr, 1k tbn, 60 tbc
    Metadata:
      ENCODER         : Lavc60.3.100 libx264
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> rawvideo (native))
Output #0, video4linux2,v4l2, to '/dev/video2':
  Metadata:
    encoder         : Lavf58.20.100
    Stream #0:0: Video: rawvideo (Y42B / 0x42323459), yuv422p, 1280x720, q=2-31, 442368 kb/s, 30 fps, 30 tbn, 30 tbc
    Metadata:
      encoder         : Lavc58.35.100 rawvideo
frame=54113 fps= 30 q=-0.0 size=N/A time=00:30:03.76 bitrate=N/A speed=   1x

그런 다음 모델/서버를 시작합니다.

edgetpu_detect_server \
      --source "/dev/video2:YUY2:848x480:30/1 identity drop-allocation=true" \
      --model "/home/mendel/Edge/examples-camera/all_models/mobilenet_ssd_v2_coco_quant_postprocess_edgetpu.tflite" \
      --labels "/home/mendel/Edge/examples-camera/all_models/coco_labels.txt" \
      --filter "person,bicycle,car,motorcycle,bus,train,truck,bench,cat,dog,horse,cow,sheep,cell phone" \
      --loop \
      --threshold=0.45

INFO:edgetpuvision.streaming.server:Listening on ports tcp: 4665, web: 4664, annexb: 4666
INFO:edgetpuvision.streaming.server:New web connection from 192.168.2.1:62538
INFO:edgetpuvision.streaming.server:Number of active clients: 1
INFO:edgetpuvision.streaming.server:New web connection from 192.168.2.1:62539
INFO:edgetpuvision.streaming.server:Number of active clients: 2
INFO:edgetpuvision.streaming.server:[192.168.2.1:62538] Rx thread finished
INFO:edgetpuvision.streaming.server:[192.168.2.1:62538] Tx thread finished
INFO:edgetpuvision.streaming.server:New web connection from 192.168.2.1:62540
INFO:edgetpuvision.streaming.server:Number of active clients: 3
INFO:edgetpuvision.streaming.server:[192.168.2.1:62538] Stopping...
INFO:edgetpuvision.streaming.server:[192.168.2.1:62538] Stopped.
INFO:edgetpuvision.streaming.server:Number of active clients: 2
INFO:edgetpuvision.streaming.server:New web connection from 192.168.2.1:62541
INFO:edgetpuvision.streaming.server:[192.168.2.1:62539] Tx thread finished
INFO:edgetpuvision.streaming.server:[192.168.2.1:62539] Rx thread finished
INFO:edgetpuvision.streaming.server:[192.168.2.1:62540] Rx thread finished
INFO:edgetpuvision.streaming.server:[192.168.2.1:62540] Tx thread finished
INFO:edgetpuvision.streaming.server:[192.168.2.1:62541] Rx thread finished
INFO:edgetpuvision.streaming.server:Number of active clients: 3
INFO:edgetpuvision.streaming.server:[192.168.2.1:62539] Stopping...
INFO:edgetpuvision.streaming.server:[192.168.2.1:62539] Stopped.
INFO:edgetpuvision.streaming.server:Number of active clients: 2
INFO:edgetpuvision.streaming.server:[192.168.2.1:62540] Stopping...
INFO:edgetpuvision.streaming.server:[192.168.2.1:62540] Stopped.
INFO:edgetpuvision.streaming.server:Number of active clients: 1
INFO:edgetpuvision.streaming.server:New web connection from 192.168.2.1:62542
INFO:edgetpuvision.streaming.server:[192.168.2.1:62541] Tx thread finished
INFO:edgetpuvision.streaming.server:Number of active clients: 2
INFO:edgetpuvision.streaming.server:[192.168.2.1:62541] Stopping...
INFO:edgetpuvision.streaming.server:[192.168.2.1:62541] Stopped.
INFO:edgetpuvision.streaming.server:Number of active clients: 1
INFO:edgetpuvision.streaming.server:New web connection from 192.168.2.1:62543
INFO:edgetpuvision.streaming.server:[192.168.2.1:62542] Rx thread finished
INFO:edgetpuvision.streaming.server:[192.168.2.1:62542] Tx thread finished
INFO:edgetpuvision.streaming.server:[192.168.2.1:62543] Rx thread finished
INFO:edgetpuvision.streaming.server:Number of active clients: 2
INFO:edgetpuvision.streaming.server:[192.168.2.1:62542] Stopping...
INFO:edgetpuvision.streaming.server:[192.168.2.1:62542] Stopped.
INFO:edgetpuvision.streaming.server:Number of active clients: 1
INFO:edgetpuvision.streaming.server:New web connection from 192.168.2.1:62544
INFO:edgetpuvision.streaming.server:[192.168.2.1:62543] Tx thread finished
INFO:edgetpuvision.streaming.server:Number of active clients: 2
INFO:edgetpuvision.streaming.server:[192.168.2.1:62543] Stopping...
INFO:edgetpuvision.streaming.server:[192.168.2.1:62544] Rx thread finished
INFO:edgetpuvision.streaming.server:[192.168.2.1:62543] Stopped.
INFO:edgetpuvision.streaming.server:Number of active clients: 1
INFO:edgetpuvision.streaming.server:New web connection from 192.168.2.1:62545
INFO:edgetpuvision.streaming.server:[192.168.2.1:62544] Tx thread finished
INFO:edgetpuvision.streaming.server:Number of active clients: 2
INFO:edgetpuvision.streaming.server:[192.168.2.1:62544] Stopping...
INFO:edgetpuvision.streaming.server:[192.168.2.1:62545] Rx thread finished
INFO:edgetpuvision.streaming.server:[192.168.2.1:62545] Tx thread finished
INFO:edgetpuvision.streaming.server:[192.168.2.1:62544] Stopped.
INFO:edgetpuvision.streaming.server:Number of active clients: 1
INFO:edgetpuvision.streaming.server:[192.168.2.1:62545] Stopping...
INFO:edgetpuvision.streaming.server:[192.168.2.1:62545] Stopped.
INFO:edgetpuvision.streaming.server:Number of active clients: 0
INFO:edgetpuvision.streaming.server:New web connection from 192.168.2.1:62546
INFO:edgetpuvision.streaming.server:Number of active clients: 1
INFO:edgetpuvision.streaming.server:New web connection from 192.168.2.1:62547
INFO:edgetpuvision.streaming.server:Number of active clients: 2
INFO:edgetpuvision.streaming.server:New web connection from 192.168.2.1:62548
INFO:edgetpuvision.streaming.server:[192.168.2.1:62546] Rx thread finished
INFO:edgetpuvision.streaming.server:[192.168.2.1:62546] Tx thread finished
INFO:edgetpuvision.streaming.server:[192.168.2.1:62547] Rx thread finished
INFO:edgetpuvision.streaming.server:[192.168.2.1:62547] Tx thread finished
INFO:edgetpuvision.streaming.server:Number of active clients: 3
INFO:edgetpuvision.streaming.server:[192.168.2.1:62546] Stopping...
INFO:edgetpuvision.streaming.server:[192.168.2.1:62548] Rx thread finished
INFO:edgetpuvision.streaming.server:[192.168.2.1:62546] Stopped.
INFO:edgetpuvision.streaming.server:Number of active clients: 2
INFO:edgetpuvision.streaming.server:[192.168.2.1:62547] Stopping...
INFO:edgetpuvision.streaming.server:[192.168.2.1:62547] Stopped.
INFO:edgetpuvision.streaming.server:Number of active clients: 1
INFO:edgetpuvision.streaming.server:New web connection from 192.168.2.1:62549
INFO:edgetpuvision.streaming.server:Number of active clients: 2
INFO:edgetpuvision.streaming.server:[192.168.2.1:62549] Rx thread finished
INFO:edgetpuvision.streaming.server:[192.168.2.1:62549] Tx thread finished
INFO:edgetpuvision.streaming.server:New web connection from 192.168.2.1:62550
INFO:edgetpuvision.streaming.server:Number of active clients: 3
INFO:edgetpuvision.streaming.server:[192.168.2.1:62550] Rx thread finished
INFO:edgetpuvision.streaming.server:[192.168.2.1:62549] Stopping...
INFO:edgetpuvision.streaming.server:[192.168.2.1:62550] Tx thread finished
INFO:edgetpuvision.streaming.server:[192.168.2.1:62549] Stopped.
INFO:edgetpuvision.streaming.server:Number of active clients: 2
INFO:edgetpuvision.streaming.server:[192.168.2.1:62550] Stopping...
INFO:edgetpuvision.streaming.server:[192.168.2.1:62550] Stopped.
INFO:edgetpuvision.streaming.server:Number of active clients: 1
INFO:edgetpuvision.streaming.server:New web connection from 192.168.2.1:62551
INFO:edgetpuvision.streaming.server:Number of active clients: 2
INFO:edgetpuvision.streaming.server:New web connection from 192.168.2.1:62552
INFO:edgetpuvision.streaming.server:[192.168.2.1:62552] Rx thread finished
INFO:edgetpuvision.streaming.server:[192.168.2.1:62552] Tx thread finished
INFO:edgetpuvision.streaming.server:[192.168.2.1:62548] Tx thread finished
INFO:edgetpuvision.streaming.server:Number of active clients: 3
INFO:edgetpuvision.streaming.server:[192.168.2.1:62552] Stopping...
INFO:edgetpuvision.streaming.server:[192.168.2.1:62551] Rx thread finished
INFO:edgetpuvision.streaming.server:[192.168.2.1:62552] Stopped.
INFO:edgetpuvision.streaming.server:Number of active clients: 2
INFO:edgetpuvision.streaming.server:[192.168.2.1:62551] Tx thread finished
INFO:edgetpuvision.streaming.server:[192.168.2.1:62548] Stopping...
INFO:edgetpuvision.streaming.server:[192.168.2.1:62548] Stopped.
INFO:edgetpuvision.streaming.server:Number of active clients: 1
INFO:edgetpuvision.streaming.server:[192.168.2.1:62551] Stopping...
INFO:edgetpuvision.streaming.server:[192.168.2.1:62551] Stopped.
INFO:edgetpuvision.streaming.server:Number of active clients: 0
INFO:edgetpuvision.streaming.server:New web connection from 192.168.2.1:62553
INFO:edgetpuvision.streaming.server:Number of active clients: 1
INFO:edgetpuvision.streaming.server:[192.168.2.1:62553] Rx thread finished
INFO:edgetpuvision.streaming.server:[192.168.2.1:62553] Tx thread finished
INFO:edgetpuvision.streaming.server:New web connection from 192.168.2.1:62554
INFO:edgetpuvision.streaming.server:[192.168.2.1:62554] Upgraded to WebSocket
INFO:edgetpuvision.streaming.server:Number of active clients: 2
INFO:edgetpuvision.streaming.server:[192.168.2.1:62553] Stopping...
INFO:edgetpuvision.streaming.server:[192.168.2.1:62553] Stopped.
INFO:edgetpuvision.streaming.server:Number of active clients: 1
INFO:edgetpuvision.streaming.server:[192.168.2.1:62554] stream_control True
INFO:edgetpuvision.streaming.server:[192.168.2.1:62554] Enabling client
INFO:edgetpuvision.streaming.server:New web connection from 192.168.2.1:62555
INFO:edgetpuvision.streaming.server:Number of active clients: 2
INFO:edgetpuvision.streaming.server:Camera start recording
INFO:edgetpuvision.streaming.server:[192.168.2.1:62555] Rx thread finished
v4l2src device=/dev/video2 ! video/x-raw,format=YUY2,width=848,height=480,framerate=30/1 ! tee name=t
t. ! queue max-size-buffers=1 leaky=downstream ! videoconvert ! x264enc speed-preset=ultrafast tune=zerolatency threads=4 key-int-max=5 bitrate=1000 aud=False ! video/x-h264,profile=baseline ! h264parse ! video/x-h264,stream-format=byte-stream,alignment=nal ! appsink name=h264sink emit-signals=True max-buffers=1 drop=False sync=False
t. ! queue ! glfilterbin filter=glbox ! video/x-raw,format=RGB,width=300,height=300 ! appsink name=appsink emit-signals=True max-buffers=1 drop=True sync=False
INFO:edgetpuvision.streaming.server:[192.168.2.1:62555] Tx thread finished
INFO:edgetpuvision.streaming.server:[192.168.2.1:62555] Stopping...
INFO:edgetpuvision.streaming.server:[192.168.2.1:62555] Stopped.
INFO:edgetpuvision.streaming.server:Number of active clients: 1
INFO:OpenGL.acceleratesupport:No OpenGL_accelerate module loaded: No module named 'OpenGL_accelerate'
INFO:edgetpuvision.streaming.server:New web connection from 192.168.2.1:62560
INFO:edgetpuvision.streaming.server:[192.168.2.1:62560] Rx thread finished
INFO:edgetpuvision.streaming.server:Number of active clients: 2
INFO:edgetpuvision.streaming.server:[192.168.2.1:62560] Tx thread finished
INFO:edgetpuvision.streaming.server:[192.168.2.1:62560] Stopping...
INFO:edgetpuvision.streaming.server:[192.168.2.1:62560] Stopped.
INFO:edgetpuvision.streaming.server:Number of active clients: 1
Error: gst-stream-error-quark: Internal data stream error. (1): gstbasesrc.c(3055): gst_base_src_loop (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
streaming stopped, reason not-negotiated (-4)

누구든지 문제가 어디에 있는지 볼 수 있습니까?

감사합니다, 라데크

관련 정보