GStreamer Compositing for Streaming H.264 Video

Gateworks recently featured a blog in which 8 video cameras were connected to a Gateworks Ventana SBC and then displayed on a HDMI monitor. This is useful for localized applications. For remote applications there is another solution.

Remote applications require streaming the multiple video streams over the network (Ethernet or WiFi). For bandwidth efficiency, all camera inputs can be joined together into a single frame and then transmitted across the network.

streamingdiagram3

To join all the streams into a single frame, a software element of GStreamerĀ called a compositor is used. Older versions of the compositor relied on the CPU and caused choppy video. Gateworks recently started using gstreamer-imx which contains a hardware accelerated compositor which is far superior. With this compositor, each stream can be positioned on the frame and then linked to a RTSP stream in the H.264 format.

An example is shown with two Gateworks Ventana SBCs that are on the same network.

Start the following pipeline on the SBC with the cameras connected:

gst-variable-rtsp-server -u \
 "imxv4l2videosrc device=/dev/video2 queue-size=55 ! queue2 ! c.sink_0 \
 imxv4l2videosrc device=/dev/video3 queue-size=55 ! queue2 ! c.sink_1 \
 imxv4l2videosrc device=/dev/video4 queue-size=55 ! queue2 ! c.sink_2 \
 imxv4l2videosrc device=/dev/video5 queue-size=55 ! queue2 ! c.sink_3 \
 imxv4l2videosrc device=/dev/video6 queue-size=55 ! queue2 ! c.sink_4 \
 imxv4l2videosrc device=/dev/video7 queue-size=55 ! queue2 ! c.sink_5 \
 imxv4l2videosrc device=/dev/video8 queue-size=55 ! queue2 ! c.sink_6 \
 imxv4l2videosrc device=/dev/video9 queue-size=55 ! queue2 ! c.sink_7 \
 imxg2dcompositor name=c background-color=0xffffff \
 sink_0::xpos=0 sink_0::ypos=0 sink_0::width=320 sink_0::height=360 sink_0::fill_color=0x00000000 \
 sink_1::xpos=320 sink_1::ypos=0 sink_1::width=320 sink_1::height=360 sink_1::fill_color=0x00000000 \
 sink_2::xpos=640 sink_2::ypos=0 sink_2::width=320 sink_2::height=360 sink_2::fill_color=0x00000000 \
 sink_3::xpos=960 sink_3::ypos=0 sink_3::width=320 sink_3::height=360 sink_3::fill_color=0x00000000 \
 sink_4::xpos=0 sink_4::ypos=360 sink_4::width=320 sink_4::height=360 sink_4::fill_color=0x00000000 \
 sink_5::xpos=320 sink_5::ypos=360 sink_5::width=320 sink_5::height=360 sink_5::fill_color=0x00000000 \
 sink_6::xpos=640 sink_6::ypos=360 sink_6::width=320 sink_6::height=360 sink_6::fill_color=0x00000000 \
 sink_7::xpos=960 sink_7::ypos=360 sink_7::width=320 sink_7::height=360 sink_7::fill_color=0x00000000 \
 ! queue2 ! video/x-raw, width=1280, height=720 ! imxipuvideotransform \
 ! imxvpuenc_h264 bitrate=20000 ! rtph264pay name=pay0 pt=96"

Then, on the receiving board that is connected to an HDMI display, start the following pipeline with the actual IP address (example IP below) of the board with the cameras:

gst-launch-1.0 rtspsrc location=rtsp://172.24.10.210:9099/stream latency=100 ! \
queue2 ! decodebin ! autovideosink

For more information andĀ code examples, please visit the related Gateworks Software Wiki links below:

 

OpenWrt Laguna Ethernet Latency Bug Fix

Under certain high load conditions in OpenWrt, using dual-core Laguna products, Ethernet ping latencies could grow to an unacceptable value. After investigation, Gateworks found an SMP race condition as the cause in the CNS3xxx Ethernet driver in Gateworks’ 13-06 and trunk OpenWrt BSPs. Note that the 12-10 OpenWrt BSP is not affected. Gateworks has resolved the issue with upstream patches to OpenWrt:
https://dev.openwrt.org/changeset/39761/trunk/target/linux/cns3xxx
https://dev.openwrt.org/changeset/39762/trunk/target/linux/cns3xxx

These fixes have been committed in both the Gateworks 13-06 branch and trunk. Additionally, the prebuilt 13-06 binary firmware images have been updated and they are located here: Laguna Binary Images.

Gateworks recommends all customers using dual-core Laguna products update their firmware.