The problem#

I want to make my smartphone's camera appear as an actual camera device on my desktop so any application (primarily Discord) can use it like it were a normal USB web cam.

My previous post introduced minimal-webrtc-gstreamer, which got as far as getting the video stream from any web browser into a GStreamer pipeline, which reduces the problem to outputting a GStreamer pipeline into a virtual web cam device.

The solution#

Download minimal-webrtc-gstreamer and install v4l2loopback. Then run

sudo modprobe v4l2loopback video_nr="42"\
    'card_label=virtcam'\
    exclusive_caps=1 max_buffers=2
./minimal-webrtc-host.py\
    --url "https://apps.aweirdimagination.net/camera/"\
    --receiveVideoTo /dev/video42\
    --sendAudio false

You can test by watching the stream with

gst-launch-1.0 v4l2src device=/dev/video42 ! autovideosink

Note that some applications, including the current desktop release of Discord may not support the virtual camera, showing a solid black square or failing to connect to it at all. It should work in the latest Chromium/Chrome browser, including for the Discord web app.

When done, remove the virtual camera device:

sudo modprobe -r v4l2loopback

The details#

GStreamer tutorial#

GStreamer is a big, complicated multimedia framework with lots of features and therefore a lot of its own jargon and concepts. It does a good job of hiding away the complexity in simple cases, but that magic does sometimes get in the way of understanding more complicated workflows.

I wish I had found this tutorial much earlier. It does a great job of explaining a lot of the concepts behind GStreamer alongside Python example code.

Multimonitor screen sharing#

There was a recently fixed bug in libwebrtc that meant on Linux screen sharing from a multimonitor setup would only show an option for sharing the "entire screen", not each monitor separately. Chromium/Chrome (before version 83) and Firefox were both impacted because they both use libwebrtc. I actually encountered the bug as a bug in Discord because the Discord desktop application uses Electron which is based on Chromium (Electron 9 is based on Chromium 83 and therefore should have the fix, but Discord has not yet updated to the latest Electron.)

As this was a long-standing bug, there were multiple workarounds developed including Hangouts Linux Individual Screen Share and Mon2Cam, which use v4l2loopback and FFmpeg to create a virtual camera device and stream a section of the screen to it or show the section of the screen in a window that itself can be shared.

Therefore, through this investigation, I knew that v4l2loopback existed and could be used to create a virtual camera device that I could point Discord (or some other video chat application) at.

First attempt#

GStreamer has the v4l2sink element for outputting to Video4Linux2 devices. That documentation includes the example

gst-launch-1.0 videotestsrc ! v4l2sink device=/dev/video1

so I just replaced the existing output code

disp = Gst.ElementFactory.make('autovideosink')

with

sink = Gst.ElementFactory.make('v4l2sink')
sink.set_property('device', '/dev/video42')

(After some research to figure out the programmatic equivalent of the device= part was to call the .set_property() method.)

To create the v4l2loopback device, I ran

sudo modprobe v4l2loopback video_nr="42" 'card_label=virtcam'

Then

gst-launch-1.0 v4l2src device=/dev/video42 ! autovideosink

successfully showed my camera's video stream. But no other applications were happy. Either they didn't show the camera at all or they would show a single frame and then freeze.

v4l2loopback settings#

The README for v4l2loopback suggests setting exclusive_caps=1 when using the device with Chrome/WebRTC, which is a workaround for programs that expect camera devices to not accept video input.

This bug discussion suggests setting max_buffers=2 may fix issues with the video freezing after a single frame.

Then the whole command is

sudo modprobe v4l2loopback video_nr="42"\
    'card_label=virtcam'\
    exclusive_caps=1 max_buffers=2

Note that in my testing, I often had to reset the v4l2loopback device by running

sudo modprobe -r v4l2loopback

and then re-initializing it, so if you're experimenting and something that seems like it should work isn't working, that's worth a try.

Format conversions#

You may have noticed that despite supporting a lot of different media formats, GStreamer pipelines often get away with not mentioning any formats explicitly. This is due to the GStreamer concept of capabilities where different elements can advertise what kinds of streams they support and elements like videoconvert will just automatically apply the necessary conversions based on their input and output.

When the default capabilities are not appropriate, you'll see pipelines written like

gst-launch-1.0  videotestsrc ! \
  video/x-raw, width=320, height=240 ! \
  xvimagesink

where there's a pipeline element (sometimes shown in quotes) with a desired video/audio format and possibly other information like the size or framerate. Since there's no keyword, it took me a while to eventually find in the tutorial I linked above that omitting the element name is equivalent to using the capsfilter element with that string as its caps property. Therefore, the equivalent Python code is

caps = Gst.Caps.from_string("video/x-raw, width=320, height=240")
filter = Gst.ElementFactory.make("capsfilter", "filter")
filter.set_property("caps", caps)

Correct capabilities#

Various tips about using FFmpeg to output to a virtual camera mention explicitly setting -pix_fmt yuv420p or -pix_fmt yuyv422, so I set the capabilities filter to "video/x-raw,format=YUY2".

I also tried explicitly setting the framerate and size in case those mattered, but none of the variants got Discord to accept the video. I even tried following those tips to have FFmpeg clone the virtual camera to another virtual camera with various conversions set but nothing I tried worked.

Giving up on desktop Discord#

While at one point I did get the desktop Discord client to show a short snippet of video from my camera, I've been unable to recreate it. As it works fine in Chromium and Discord is based on Chromium (1) I assume this will be fixed in some future version of Discord whenever they next upgrade the version of Electron they use, and (2) an easy workaround is to just use Discord through the website when I want video.