FFmpeg on Clear Linux including H.264 and VP9 acceleration

General Discussion
1

Hello. I am filled with joy in completing testing H.264 and vp9 hardware acceleration with FFmpeg and Firefox. This includes a YouTube player script calling ffmpeg and ffplay so able to watch HDR YouTube videos in 10-bits output.

This is not an April Fools joke, but rather a completed effort for running on NVIDIA GPUs. My motivation is simply wanting hardware acceleration in Firefox. Now, the Clear Linux desktop experience is superb and fun.

Well, I made a GitHub repository ffmpeg-on-clear-linux.

8 Likes
2

Good job…

3

Good job!!!

And I need 20 characters to post.

4

It has been a while :slight_smile: I added instructions for Chromium to the README file. Both Chromium and Firefox work great including hardware acceleration. @nobodyatandnothing and @Mazotis helped with debugging. Thank you. I never imagined for this repo to reach this point.

Disclaimer: I have an AMD box. Unfortunately, I’m unable to test using Intel graphics.

4 Likes
5

For Chromium, one needs to pass --enable-features=VaapiVideoDecoder flag. I updated the README file. Below, Feature Status from the chrome://gpu page.

Graphics Feature Status

  • Canvas: Hardware accelerated
  • Canvas out-of-process rasterization: Disabled
  • Direct Rendering Display Compositor: Disabled
  • Compositing: Hardware accelerated
  • Multiple Raster Threads: Enabled
  • Out-of-process Rasterization: Hardware accelerated
  • OpenGL: Enabled
  • Rasterization: Hardware accelerated
  • Raw Draw: Disabled
  • Skia Renderer: Enabled
  • Video Decode: Hardware accelerated
  • Video Encode: Software only. Hardware acceleration disabled
  • Vulkan: Disabled
  • WebGL: Hardware accelerated
  • WebGL2: Hardware accelerated

nvidia-smi

Notice ...eatures=VaapiVideoDecoder in the nvidia-smi output.

+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|    0   N/A  N/A       896      G   /usr/bin/X                        194MiB |
|    0   N/A  N/A       982      G   /usr/bin/gnome-shell               54MiB |
|    0   N/A  N/A     17698      G   ...eatures=VaapiVideoDecoder       36MiB |
+-----------------------------------------------------------------------------+

Both Chromium and Firefox work well for me. As mentioned in a prior post, I’m not able to test against Intel graphics. I have an AMD box with NVIDIA graphics.

3 Likes
6

Update: Added results for Vivaldi and Brave.

What a journey this has been… and saw the mess the README file has become. Well, I polished the file. It has a new section “Determine the VAAPI driver to use”. The flow is nicer now.

There are three new sections for Google Chrome, Vivaldi, and Brave. For the first time, I saw the Video Engine (NVIDIA) being used (VP9 media) along with the GPU while watching videos. This is unlike Chromium and Firefox where the Video Engine is not utilized, just GPU.

In closing, I ran various benchmarks on the web. Note that results can only be compared to other browsers running on the same machine. Difference CPU or GPU may very likely produce different results.

Version

   Chrome 97.0.4692.71  Firefox 96.0         Brave 1.34.80
 Chromium 99.0.4829.0   Vivaldi 5.0.2497.38

html5test

   Chrome  528.0 (*)    Firefox  512.0       Brave  521.0
 Chromium  526.0        Vivaldi  526.0

Speedometer 2.0

   Chrome  212.0 (*)    Firefox  172.0       Brave  158.9
 Chromium  147.2        Vivaldi  153.0

Octane 2.0

   Chrome  58623 (*)    Firefox  29641       Brave  57586
 Chromium  57215        Vivaldi  56398

JetStream 2.0

   Chrome  184.5 (*)    Firefox  118.8       Brave  175.4
 Chromium  178.2        Vivaldi  172.2

MotionMark 1.2

   Chrome 1002.0 (*)    Firefox  690.7       Brave  767.2
 Chromium  627.3        Vivaldi  783.2

SunSpider 1.0.2

   Chrome  137.8        Firefox  116.5 (*)   Brave  154.7
 Chromium  147.5        Vivaldi  150.2

MazeSolver 40x40

   Chrome    3.0 (*)    Firefox    3.1       Brave    3.3
 Chromium    3.4        Vivaldi    3.4

StyleBench

   Chrome   43.4        Firefox  116.0 (*)   Brave   33.1
 Chromium   31.8        Vivaldi   34.1

WebAssembly

   Chrome   2852        Firefox   2672 (*)   Brave   2916
 Chromium   2803        Vivaldi   2894

WebXPRT 3.0

   Chrome    306        Firefox    310 (*)   Brave    272
 Chromium    280        Vivaldi    285

At last, I hope the project is finished. Can that be for real? :slight_smile: What to make of this? Depending on the site or web application, Firefox may feel faster to you. See the StyleBench results above. Google Chrome and variants are quite fast otherwise. Brave and Vivaldi spend less time loading pages due to blocking creepy ads. IMO, enjoy the browser that you like using.

5 Likes
7

Vivaldi uses an optimized Chromium engine.

Could you include it in the benchmark ?

2 Likes
8

GNOME 42 is going to speed up Firefox considerably :

1 Like
9

source code is on Source | Vivaldi Browser

1 Like
10

Current Vivaldi (v5) is around 5-10 % faster than v4 in the benchmark.

3 Likes
11

See results for Vivaldi above. Moreover, I added instructions for Vivaldi to the README file.

A delightful experience is seeing the use of the Video Engine along with the GPU when watching a video, similarly to Google Chrome. That’s the case for VP9 format and saves me 15 watts in power consumption.

2 Likes
12

I completed testing Brave on Clear Linux. See results above. The README now has a table of contents. What a treat it is to view the web without all the creepy ads.

Update: It turns out that software rendering may be helpful for Chromium-based browsers, similarly to Firefox. Especially if the GPU is lacking or you want the desktop to remain fluid while watching a video, which the GPU is handling along with desktop composition. For example, moving-resizing the browser window itself or other windows.

From testing, a balanced CPU-GPU utilization may result in a more fluid desktop experience.

  • GPU: Desktop composition, H.264/VP9 decode, WebGL
  • CPU: HTML rendering, basically not all on the GPU

Update 2: Firefox configured to render web on the CPU still decodes videos on the GPU. From further testing, that’s not the case for Chromium-based browsers – all software.

So now I’m back to Hardware acceleration enabled in Chromium-based browsers and leaving ForceCompositionPipeline to Off in /etc/X11/xorg.conf.d/nvidia-device-opt.conf.

The mysterious choppiness that I experienced when first using Clear Linux ended up being my Logitech mouse. Enabling hires-smooth-resolution made a noticeable improvement.

$ sudo swupd bundle-add Solaar-gui
$ sudo solaar show
$ sudo solaar config M705 hires-smooth-resolution True

The project is finished! :slight_smile:

2 Likes
13

Desktop integration is completed. One may launch via Application Finder (optionally add to Favorites) or use the command-line. The script for Chromium creates its desktop file the first time launched.

# Run scripts for $HOME/bin
~/git/ffmpeg-on-clear-linux/bin
run-brave-stable
run-chrome-stable
run-chromium-latest
run-vivaldi-stable

# Desktop files for $HOME/.local/share/applications
~/git/ffmpeg-on-clear-linux/desktop
brave-browser.desktop
google-chrome.desktop
vivaldi-stable.desktop

Periodic Updates

Chromium is a one step process i.e. run ~/chromium-latest-linux/update.sh. Brave, Chrome, and Vivaldi involve two steps. Basically, obtain current stable package followed by rpm -U.

2 Likes
14

I apologize for the many iterative updates. The last phase of the project “Desktop Integration” is now completed and polished. What follows are output that some might find interesting. For example, what files are being used from /usr/local/lib and /usr/lib64/dri? On my system nvidia_drv_video.so is a symbolic link to vdpau_drv_video.so.

Brave, Chromium, Google Chrome, Vivaldi

lsof |grep -e '/usr/local' -e '/usr/lib64/dri' |grep brave
lsof |grep -e '/usr/local' -e '/usr/lib64/dri' |grep chro
lsof |grep -e '/usr/local' -e '/usr/lib64/dri' |grep vival

/usr/lib64/dri/vdpau_drv_video.so
/usr/local/lib/libvdpau.so.1.0.0

Firefox

lsof |grep -e '/usr/local' -e '/usr/lib64/dri' |grep Isolat

/usr/local/lib/libaom.so.3.0.0
/usr/local/lib/libavcodec.so.58.134.100
/usr/local/lib/libavutil.so.56.70.100
/usr/local/lib/libdav1d.so.5.0.1
/usr/local/lib/libfdk-aac.so.2.0.2
/usr/local/lib/libmp3lame.so.0.0.0
/usr/local/lib/libopus.so.0.8.0
/usr/local/lib/libSvtAv1Enc.so.0.8.7
/usr/local/lib/libswresample.so.3.9.100
/usr/local/lib/libvdpau.so.1.0.0
/usr/local/lib/libvmaf.so.1.1.1
/usr/local/lib/libx264.so.164
/usr/local/lib/libx265_main12.so.199
/usr/local/lib/libx265.so.199
/usr/local/lib/libxvidcore.so.4.3
15

Similarly, a list of opened libraries from /opt/nvidia while watching a 2K or 4K video.

lsof |grep '/opt/nvidia' |grep brave |awk '{print $NF}' |sort -u
lsof |grep '/opt/nvidia' |grep chrom |awk '{print $NF}' |sort -u
lsof |grep '/opt/nvidia' |grep vival |awk '{print $NF}' |sort -u
lsof |grep '/opt/nvidia' |grep Isola |awk '{print $NF}' |sort -u

Brave, Chromium, Google Chrome, Vivaldi

The libvdpau_nvidia.so is used. :slight_smile: That explains the Video Engine being utilized for VP9 media.

/opt/nvidia/lib64/libEGL.so.1.1.0
/opt/nvidia/lib64/libGLdispatch.so.0
/opt/nvidia/lib64/libGL.so.1.7.0
/opt/nvidia/lib64/libGLX_nvidia.so.495.29.05
/opt/nvidia/lib64/libGLX.so.0
/opt/nvidia/lib64/libnvidia-glcore.so.495.29.05
/opt/nvidia/lib64/libnvidia-glsi.so.495.29.05
/opt/nvidia/lib64/libnvidia-tls.so.495.29.05
/opt/nvidia/lib64/vdpau/libvdpau_nvidia.so.495.29.05

Firefox

/opt/nvidia/lib64/libEGL.so.1.1.0
/opt/nvidia/lib64/libGLdispatch.so.0
/opt/nvidia/lib64/libGL.so.1.7.0
/opt/nvidia/lib64/libGLX.so.0
16

A screenshot. Notice GPU Utilization and Video Engine Utilization. CPU utilization (not shown here) is under 10% with occasional spikes up to 22% for the Brave process while watching a 2K video.

image
image888Ă—785 146 KB

17

The following is the Internet category in my Application Finder screenshot. The various web browsers are all there as one would expect.

image
image893Ă—697 98.2 KB

Each Chromium-based application launch their custom launch script residing in the user’s ~/bin folder.

$ grep Exec ~/.local/share/applications/brave-browser.desktop 
Exec=/bin/bash -c "\$HOME/bin/run-brave-stable %U"
Exec=/bin/bash -c "\$HOME/bin/run-brave-stable"
Exec=/bin/bash -c "\$HOME/bin/run-brave-stable --incognito"

# ditto for google-chrome.desktop and vivaldi-stable.desktop

$ grep Exec ~/.local/share/applications/chromium-latest.desktop 
Exec=/bin/bash -c "$HOME/bin/run-chromium-latest %U"

My ~/bin folder, obtained from the ffmpeg-on-clear-linux project. One is able to customize options as needed. Particularly passing the --enable-features=VaapiVideoDecoder flag.

$ ls -1 ~/bin
run-brave-stable
run-chrome-stable
run-chromium-latest
run-vivaldi-stable
1 Like
18

This is for me so when looking back and ask why did I add this option --disable-accelerated-2d-canvas. It turns out that disabling it makes 2D canvas faster for me. I’m posting this in the event someone finds this helpful. I ran several tests on the web.

http://www.smashcat.org/av/canvas_test
CanvasTest 249fps --disable-accelerated-2d-canvas (*)
CanvasTest 182fps --enable-accelerated-2d-canvas

http://www.kevs3d.co.uk/dev/canvasmark
CanvasMark 23115  --disable-accelerated-2d-canvas (*)
CanvasMark 21202  --enable-accelerated-2d-canvas

https://browserbench.org/MotionMark1.2
MotionMark 809.7  --disable-accelerated-2d-canvas (*)
MotionMark 754.6  --enable-accelerated-2d-canvas

https://pspdfkit.com/webassembly-benchmark (lower is better)
 WasmScore 2920   --disable-accelerated-2d-canvas (*)
 WasmScore 3397   --enable-accelerated-2d-canvas
5 Likes
19

Dear friends! Today I completed testing on an Intel NUC (Iris 6100 graphics) connected to a 2560x1440 2K display. Hardware acceleration is enabled out of the box from what I can tell and works quite well.

Making no changes, the GPU consumes 62% utilization while watching Big Buck Bunny (1440p60) full screen. The fans rev up, but not too much.

It is lesser GPU utilization (45%) including CPU while watching the same video (1440p60) full screen after building FFmpeg and configuring Firefox as per the README.

Unfortunately chromium-based browsers consume 75% on the Iris 6100 GPU. However 1080p plays fine, no roaring fans.

2 Likes
20

I added (later updated) a new section titled Initial preparation in the README file. It is for folks using the NVIDIA proprietary driver. Basically running swupd bundle-add devpkg-libva or devpkg-mediasdk or devpkg-mesa restores the Clear Linux OS provided libGL files which breaks the NVIDIA installation.

The new section walks the reader into adding a service which removes libGL files that shouldn’t be there i.e. libEGL.so*, libGLESv1_CM.so*, libGLESv2.so*, and libGL.so*.

What a vicious cycle (search on YouTube: Who’s On First Base).

  1. Initial swupd bundle-add adds mesa libraries
  2. NVIDIA installer removes mesa libraries
  3. swupd bundle-add devpkg also adds mesa libraries
  4. now the NVIDIA installation is broken
  5. so we need a systemd service to fix the NVIDIA installation
  6. swupd update mesa-new-version adds mesa libraries
  7. and we need the service to fix the installation again

Simulate swupd running in the background and the user launching applications that require libGL libraries. What are the chances for both NVIDIA and Mesa libGL libraries being utilized during an update window?

3 Likes