Step by Step Video Recording Configuration Guide For Jitsi Meet with Jibri

Step by Step Video Recording Configuration Guide For Jitsi Meet with Jibri

To enable recording of conferences made with on-premise Jitsi meet installations, Jibri(JItsi BRoadcasting Infrastructure) is required. Jibri launches a chrome instance in a virtual framebuffer. The output of this frame buffer is feed into FFmpeg which saves the recording into a file. After the conference ends, FFmpeg shuts down and finishes writing to file. A finalization bash script runs and stores the file for permanent storage.

Because of the recording architecture, Jibri needs an ALSA loopback device on the host. This makes scaling the Jibri little challenging for Linux and impossible for Windows. Linux kernel module allows the creating of multiple such devices. Since Jibri works as a client, it can be deployed to another server.

To check if required modules are loaded into the kernel:

# lsmod | grep snd_aloop

If this command returns empty, the module needs to be loaded as root used before using Jibri:

# modprobe snd-aloop
# lsmod | grep snd_aloop
snd_aloop              28672  6
snd_pcm               114688  9 snd_hda_codec_hdmi,snd_hda_intel,snd_usb_audio,snd_hda_codec,snd_aloop,snd_hda_core
snd                    94208  48 snd_hda_codec_generic,snd_seq_device,snd_hda_codec_hdmi,snd_hwdep,snd_hda_intel,snd_usb_audio,snd_usbmidi_lib,snd_hda_codec,snd_hda_codec_realtek,snd_timer,snd_aloop,snd_pcm,snd_rawmidi

These commands are enough to create loopback sound devices. Yet the naming standards for the devices can be affected by the existing sound card devices. Jibri expects a device at pcmC0D0p by default configuration, which means pcmCard0 of Device0p. Since everything is a file in Linux systems, we can simply move the device with the mv command.

# mv pcmC0D3p pcmC0D0p
# ls /dev/snd/
by-path    controlC1  controlC3  hwC0D0    pcmC0D7p  pcmC0D9p  pcmC1D0p  pcmC1D1p  pcmC2D0p  pcmC2D1p  pcmC3D0p  pcmC3D1p  pcmC4D0p  pcmC4D1p  timer
controlC0  controlC2  controlC4  pcmC0D0p  pcmC0D8p  pcmC1D0c  pcmC1D1c  pcmC2D0c  pcmC2D1c  pcmC3D0c  pcmC3D1c  pcmC4D0c  pcmC4D1c  seq

Now the host is ready to run Jibri. To enable the same modules at the next reboot, /etc/modules can be used.

# echo "snd-aloop" >> /etc/modules

Jibri comes as a package and as a docker-compose config. Using the Jibri with docker-jitsi-meet is pretty straight forward. All that is required is to add a Jitsi container to the docker-compose stack.

docker-compose -f docker-compose.yml -f jibri.yml up -d

After running Jibri you can check the logs of the container with

# docker logs docker-jitsi-meet_jibri_1

By default, Jibri will save the recordings under /config/recordings but the best practice is to use a finalization script to push the data to a central location. NFS, S3 or FTP server can be used for this.

Compile a custom jibri for docker

If Jibri is run with docker, recompiling the package is not straightforward as a traditional install. The best approach is to create a custom container to build the Jibri. Since the Jibri container is based on a Debian container, it is best to run a Debian container.

# docker run -it debian bash
Unable to find image 'debian:latest' locally
latest: Pulling from library/debian
b9a857cbf04d: Pull complete
Digest: sha256:b16f66714660c4b3ea14d273ad8c35079b81b35d65d1e206072d226c7ff78299
Status: Downloaded newer image for debian:latest
root@39953920f6ee:/#

In the docker container, run the following commands to enable package manager and install the build tools. Linux images are optimized for docker by removing the nonessential packages. Build a development environment on this slim base requires many packages and this step can take some time.

# apt update
Get:1 http://security.debian.org/debian-security buster/updates InRelease [65.4 kB]
...
All packages are up to date.
# apt-get install libxml2-utils fakeroot build-essential dpkg-dev debhelper git devscripts maven vim -y
...
update-alternatives: using /usr/lib/jvm/java-11-openjdk-amd64/bin/rmiregistry to provide /usr/bin/rmiregistry (rmiregistry) in auto mode

Then the source code for jibri has to be downloaded from GitHub.

# git clone https://github.com/jitsi/jibri.git
Cloning into 'jibri'...
remote: Enumerating objects: 11, done.
remote: Counting objects: 100% (11/11), done.
remote: Compressing objects: 100% (8/8), done.
remote: Total 6821 (delta 1), reused 5 (delta 0), pack-reused 6810
Receiving objects: 100% (6821/6821), 1.03 MiB | 1.21 MiB/s, done.
Resolving deltas: 100% (2883/2883), done.
# cd jibri

Now we are ready to build the Jibri from the source in a docker container. We need to download the required packages with maven.

# mvn package

To create a .deb package for the custom build, release.sh script can be used.

export WORKSPACE=`pwd`
resources/jenkins/release.sh Minor
# ls ..
-rw-r--r--  1 root root     4223 Jan 16 20:10 ../jibri_8.0-64-g268e33d-1_all.buildinfo
-rw-r--r--  1 root root     1022 Jan 16 20:10 ../jibri_8.0-64-g268e33d-1_all.changes
-rw-r--r--  1 root root 44379684 Jan 16 20:10 ../jibri_8.0-64-g268e33d-1_all.deb

Now we need to patch the jibri container with the custom .deb file. Docker-compose file uses the prebuilt image from GitHub by default. To rebuild docker-compose containers.

docker-compose -f docker-compose.yml -f jibri.yml build -d

Since docker images are used in the docker-compose file images cannot be rebuilt directly. This means containers created by the developers are pulled and used. Since it is an open-source project, build files for the containers are public and they are in the same git repository. Dockerfile for Jibri is under the folder named jibri just like the other components of jitsi. We can alter docker-compose to build a custom container by replacing the image address with a Dockerfile path for the build.

version: '3'
services:
    jibri:
+        build: ./jibri
        restart: ${RESTART_POLICY}
        volumes:
            - ${CONFIG}/jibri:/config:Z
            - /dev/shm:/dev/shm

After this step, changes can be tested with docker-compose.

# docker-compose -f docker-compose.yml -f jibri.yml build -d

Since we didn't apply the new package yet, everything should be the same. Installing packages to docker containers is futile as they tend to restart and discard all the changes made to them. The best way to persistently change the package in a container is to copy and install that package in Dockerfile.

ARG JITSI_REPO=jitsi
FROM ${JITSI_REPO}/base-java

#ARG CHROME_RELEASE=latest
#ARG CHROMEDRIVER_MAJOR_RELEASE=latest
ARG CHROME_RELEASE=78.0.3904.97
ARG CHROMEDRIVER_MAJOR_RELEASE=78

SHELL ["/bin/bash", "-o", "pipefail", "-c"]
+COPY patched.deb /tmp/patched.deb
RUN \
	apt-dpkg-wrap apt-get update \
	&& apt-dpkg-wrap apt-get install -y jibri libgl1-mesa-dri procps \
+	dpkg -i /tmp/patched.deb  \
	&& apt-cleanup

Now to build a patched container with custom Jibri packages.

docker-compose -f docker-compose.yml -f jibri.yml build -d