GPU Programming: Raspberry Pi (VideoCore IV, VideoCore VI, VideoCore VII)

Table of contents

VideoCore versions

There exist two versions of the VideoCore GPU whose instruction sets differs quite a lot:

• VideoCore IV: used in the Raspberry Pi 1-3 and Raspberry Pi Zero 1-2 (W)
• VideoCore VI: used in the Raspberry Pi 4

A more detailled list can be found at Raspberry Pi Documentation - Processors [visited 2022-06-18T17:43:32Z].

Disassemblers

Ressources

For building the disassembler and analyzing the instruction set, we use the following ressources:

VideoCore IV:

• VideoCore® IV 3D Architecture Reference Guide [published 2013-09-16; visited 2021-10-31T23:08:23Z] (local copy): documentation of the instruction set.

• src/gallium/drivers/vc4 · main · Mesa / mesa · GitLab [visited 2022-08-05T17:41:09Z]: source code of the vc4 driver that is built as libvc4.a. This library contains a disassembler for VideoCore IV instructions:

  • src/gallium/drivers/vc4/vc4_qpu.h · main · Mesa / mesa · GitLab [visited 2022-08-29T22:27:14Z]
  • src/gallium/drivers/vc4/vc4_qpu_defines.h · main · Mesa / mesa · GitLab [visited 2022-11-05T13:21:33Z]
  • src/gallium/drivers/vc4/vc4_qpu_disasm.c · main · Mesa / mesa · GitLab [visited 2022-08-29T22:28:13Z]

  Also of interest is:

  • Validation: src/gallium/drivers/vc4/vc4_qpu_validate.c · main · Mesa / mesa · GitLab [visited 2022-11-05T13:27:18Z]: contains the function vc4_qpu_validate to validate a GPU program. For this function not to generate into a trivial return, the DEBUG macro has to be defined when building libvc4.a.
  • Code generation:
    • src/gallium/drivers/vc4/vc4_program.c · main · Mesa / mesa · GitLab [2023-05-06T19:56:49Z]
    • src/gallium/drivers/vc4/vc4_qir.h · main · Mesa / mesa · GitLab [2023-05-06T20:00:04Z]
    • src/gallium/drivers/vc4/vc4_qir.c · main · Mesa / mesa · GitLab [2023-05-06T20:00:15Z]

VideoCore VI:

• src/broadcom/qpu · main · Mesa / mesa · GitLab [visited 2022-08-05T17:38:25Z]: source code of the libbroadcom_qpu.a library that is a disassembler for VideoCore VI instructions
• Also of potential interest: src/gallium/drivers/v3d · main · Mesa / mesa · GitLab [visited 2022-08-29T22:36:24Z]: source code of the v3d driver

Another library of Mesa that we use is libmesa_util.a. Its source code can be found at src/util · main · Mesa / mesa · GitLab [visited 2022-08-29T21:59:22Z].

Links to test cases for the instruction sets are documented further below in section .

Build disassemblers

A disassembler for machine instructions of the VideoCore IV and VideoCore VI QPUs can be found at nubok/vcqpudisasm [visited 2022-08-28T10:50:40Z]. This repository was inspired by Terminus-IMRC/vc6qpudisas: Disassembler of VideoCore VI QPU [visited 2022-08-28T12:29:55Z].

Let's go through its install instructions to set it up. The following steps were tested on Raspberry Pi OS (32 bit; based on Debian Bullseye) and Ubuntu 20.04 (WSL2; x86-64).

Build Mesa

First setup Mesa. We need to install various packages that are required for building Mesa:

sudo apt-get install meson
sudo apt-get install python3-mako
sudo apt-get install libdrm-dev
sudo apt-get install flex
sudo apt-get install bison
sudo apt-get install libx11-dev
sudo apt-get install libxext-dev
sudo apt-get install libxfixes-dev
sudo apt-get install libxcb-glx0-dev
sudo apt-get install libxcb-shm0-dev
sudo apt-get install libx11-xcb-dev
sudo apt-get install libxcb-dri2-0-dev
sudo apt-get install libxcb-dri3-dev
sudo apt-get install libxcb-present-dev
sudo apt-get install libxshmfence-dev
sudo apt-get install libxxf86vm-dev
sudo apt-get install libxrandr-dev

If you do these steps in Ubuntu 20.04 or Ubuntu 22.04 LTS (for example under WSL2), you need to add

sudo apt-get install pkg-config

Now run

git clone https://gitlab.freedesktop.org/mesa/mesa.git --depth=1
cd mesa/
git fetch --all --tags

• If you use Ubuntu 20.04 (for example under WSL2) or Raspberry Pi OS 11 (based on Debian Bullseye), type
  git checkout tags/mesa-21.3.9 -b mesa21_3_9
• If you use Ubuntu 22.04 LTS (for example under WSL2), type
  git checkout tags/mesa-23.1.3 -b mesa23_1_3

git checkout tags/mesa-21.3.9 -b mesa21_3_9

git checkout tags/mesa-23.1.3 -b mesa23_1_3

mkdir build
cd build
meson .. -Dgallium-drivers=vc4,v3d -Dvulkan-drivers=broadcom -Dplatforms=x11
ninja src/broadcom/qpu/libbroadcom_qpu.a src/util/libmesa_util.a src/gallium/drivers/vc4/libvc4.a
cd ../../

Build vcqpudisasm

If necessary, install cmake via

sudo apt install cmake

Start with

git clone https://github.com/nubok/vcqpudisasm.git
cd vcqpudisasm
mkdir build
cd build
cmake .. -DCMAKE_PREFIX_PATH="$(realpath ../../mesa);$(realpath ../../mesa/build)"
make

Run examples

Now for running some examples:

VideoCore IV: Run

./vc4qpudisasm <<< '0x10025020cc9e7081'
add rb0, r0, r2 ; v8adds r0, r0, r1

Other examples can be found at userland/host_applications/linux/apps/hello_pi/hello_fft/hex at master · raspberrypi/userland [visited 2022-08-28T12:51:09Z] (also available at firmware/opt/vc/src/hello_pi/hello_fft/hex at master · raspberrypi/firmware [visited 2022-08-28T12:52:14Z]).

VideoCore VI: There seem to exist at least four versions of V3D: 3.3, 4.0, 4.1 and 4.2 (corresponding to values 33, 40, 41 and 42 of the ver field of struct v3d_device_info). See

• src/broadcom/common/v3d_device_info.h · main · Mesa / mesa · GitLab [visited 2022-07-24T14:10:56Z] for a description of the fields of struct v3d_device_info data structure,
• src/broadcom/qpu/tests/qpu_disasm.c · main · Mesa / mesa · GitLab [visited 2022-07-24T14:17:12Z] for test cases in which the versions 3.3, 4.1 and 4.2 occur,
• src/broadcom/qpu/qpu_pack.c · main · Mesa / mesa · GitLab [visited 2022-12-30T23:29:25Z]. There in the definition of the function v3d_qpu_sig_unpack the versions 3.3, 4.0 and 4.1 are referenced.

The GPU of the respective Raspberry Pi versions seems to use version 4.2 (value 42).

With this knowledge, you can run the example:

./vc6qpudisasm 42 <<< '0x54001f4038f91fbf'
add  r0, r1, r2      ; fmul  rf61, rf62, rf63

Other examples can, as mentioned above, be found at src/broadcom/qpu/tests/qpu_disasm.c · main · Mesa / mesa · GitLab [visited 2022-07-24T14:17:12Z].

Instruction set of VideoCore IV

Registers

• ra0-ra31
• rb0-rb31

Instructions

An instruction consists of 64 bit (8 byte) in little-endian format.

The sig field determines the type of instruction (branch instruction, load immediate instruction or ALU instruction):

Branch instructions

TODO

Load immediate instructions

TODO

ALU instructions

TODO

Instruction set of VideoCore VI

Registers

TODO

Instructions

An instruction consists of 64 bit (8 byte) in little-endian format.

The op_mul field determines the type of instruction (branch instruction or ALU instruction):

Branch instructions

For the value of the cond field:

The disassembler in libbroadcom_qpu.a decodes 001 in the cond field identical to 000 (unconditional branch).

msfign field

For the msfign field:

bdi field

The bdi field encodes the type of destination:

bdu field

If ub = 1 (bit 14), the bdu field has the following encoding:

ALU instructions

op_mul field

TODO

op_add field

TODO

sig field

For the sig field (signaling bits) see the declarations

• v33_sig_map
• v40_sig_map
• v41_sig_map

in src/broadcom/qpu/qpu_pack.c · main · Mesa / mesa · GitLab [visited 2022-12-30T23:29:25Z]. The disassembler code can be found at src/broadcom/qpu/qpu_disasm.c · main · Mesa / mesa · GitLab [visited 2023-07-20T20:07:53Z] (v3d_qpu_disasm_sig, which calls v3d_qpu_disasm_sig_addr if necessary).

Version 3.3:

Version 4.0:

Version ≥4.1: From version 4.1 on, the disassembly of some these signals uses another value, the sig_addr; their cells are highlighted with a █-colored background.

Connecting to application code

VideoCore IV

Mailbox functionality

TODO

VideoCore VI

TODO

GPGPU examples

VideoCore IV

FFT example

Setup

Setup an SD card with the Raspberry Pi OS (formerly called Raspbian) based on Debian 10 (“buster”). The newer versions of Raspberry Pi OS that are based on Debian 11 (“bullseye”) don't seem to work because they don't support the fkms driver. See BULLSEYE: "dtoverlay=vc4-fkms-v3d" OR "dtoverlay=vc4-kms-v3d" That is one of many Questions - Raspberry Pi Forums [visited 2022-06-21T18:31:19Z] for details. See also "New" old functionality with Raspberry Pi OS (Legacy) - Raspberry Pi [visited 2022-06-21T18:31:31Z].

Then do the following steps:

• Check whether the file /boot/config.txt contains the line dtoverlay=vc4-fkms-v3d (not to be confused with dtoverlay=vc4-kms-v3d), for example via sudo pico /boot/config.txt.
• If this is not the case, either add this line (and ensure that nowhere else in this file the parameter dtoverlay is set to a different value) or run sudo raspi-config → 6 Advanced Options → A2 GL Driver → G2 GL (Fake KMS).

Just for the sake of completeness, here is a table with the various possible settings and whether the hello_fft.bin demo that we run in section section does run with this setting set:

Run the FFT example

Sources:

• Accelerating Fourier transforms using the GPU [published 2014-01-30; visited 2021-10-31T22:29:19Z]; former link (now dead): https://www.raspberrypi.org/accelerating-fourier-transforms-using-the-gpu/
• Corresponding Git repository: firmware/opt/vc/src/hello_pi/hello_fft at master · raspberrypi/firmware [visited 2021-11-05T22:32:41Z]

The Raspberry Pi OS version based on “Buster” contains a folder /opt/vc. If it is missing (or - more interesting - you want to use the latest available version), you can create it manually by doing the following steps:

• Install cmake: sudo apt-get install cmake
• Clone the userland repository (raspberrypi/userland: Source code for ARM side libraries for interfacing to Raspberry Pi GPU [visited 2022-06-19T22:19:26Z]): git clone https://github.com/raspberrypi/userland.git
• In the checked out userland folder, type ./buildme

Best copy /opt/vc/src/hello_pi/hello_fft to your home folder:

cp -r /opt/vc/src/hello_pi/hello_fft ~

Now run

cd ~/hello_fft
make
sudo ./hello_fft.bin 8

(the command sudo mknod char_dev c 100 0 that is mentioned in the above linked blog article is not necessary anymore).

Explanation of the FFT example

An explanation of the FFT example by the original author Andrew Holme with lots of additional information and links can be found at GPU_FFT [visited 2022-11-02T09:52:58Z].

VideoCore VI

Idein Inc. (GitHub page: Idein Inc. · GitHub [visited 2022-07-22T19:46:58Z]) did a lot of work for GPGPU on the VideoCore VI.

We consider the repository py-videocore6: GitHub - Idein/py-videocore6: Python library for GPGPU programming on Raspberry Pi 4 [visited 2022-07-22T19:52:46Z].

Various material

VideoCore general

• V3DLib: C++ library for programming the VideoCore GPU on all Raspberry Pi's. Builds upon QPULib (below); wimrijnders/V3DLib: C++ library for programming the VideoCore GPU on all Raspberry Pi's. [visited 2023-09-29T11:16:00Z]

VideoCore IV

• QPULib: Language and compiler for the Raspberry Pi GPU; mn416/QPULib: Language and compiler for the Raspberry Pi GPU [visited 2023-09-29T11:15:34Z]

QPU assembler code

• GPGPU hacking on the Pi [published 2014-06-20; visited 2021-10-31T22:26:46Z]; former link (now dead): https://www.raspberrypi.org/gpgpu-hacking-on-the-pi/

• Hacking The GPU For Fun And Profit [visited 2021-10-31T22:40:09Z] (SHA-256)

  Parts:

  • Hacking The GPU For Fun And Profit (Pt. 1) [published 2014-05-03; visited 2021-10-31T23:00:44Z]
  • Hacking The GPU For Fun And Profit (Pt. 2) [published 2014-05-06; visited 2021-10-31T23:00:53Z]
  • Hacking The GPU For Fun And Profit (Pt. 3) [published 2014-05-13; visited 2021-10-31T23:01:00Z]
  • Hacking The GPU For Fun And Profit (Pt. 4) [published 2014-05-21; visited 2021-10-31T23:01:17Z]

VC4ASM/vcio2

• VC4ASM: Macro assembler for Broadcom VideoCore IV aka Raspberry Pi GPU
  • VC4ASM Macro Assembler for VideoCore IV [visited 2022-12-28T01:06:36Z]
  • maazl/vc4asm: Macro assembler for Broadcom VideoCore IV aka Raspberry Pi GPU [visited 2022-12-28T01:00:20Z]
• vcio2: Kernel driver to access Raspberry Pi VideoCore IV GPU without root privileges
  • vcio2 - Kernel driver to access Raspberry Pi VideoCore IV GPU without root privileges [visited 2022-12-28T01:10:33Z]
  • maazl/vcio2: Kernel driver to access Rapsperry Pi's VideoCore IV GPU frum user space [visited 2022-12-28T01:12:36Z]

OpenCL

• doe300 (doe300) / Repositories [visited 2022-12-28T00:55:55Z]; see in particular
  • VC4C: doe300/VC4C: Compiler for the VC4CL OpenCL implementation [visited 2022-12-28T15:29:41Z]
  • VC4CL: doe300/VC4CL: OpenCL implementation running on the VideoCore IV GPU of the Raspberry Pi models [visited 2022-12-28T15:27:54Z]
  • VC4CLStdLib: doe300/VC4CLStdLib: GPU-side implementation of the OpenCL standard-library for VC4CL [visited 2022-12-28T15:35:27Z]
  • VC4CL-Jenkins: doe300/VC4CL-Jenkins: Jenkins scripts for running various tests against VC4C(L) [visited 2022-12-28T15:37:34Z]
• Further repositories that are interesting concerning OpenCL:
  • OpenCL-CTS: KhronosGroup/OpenCL-CTS: The OpenCL Conformance Tests [visited 2022-12-28T15:40:35Z]
  • Boost.Compute: boostorg/compute: A C++ GPU Computing Library for OpenCL [visited 2022-12-28T15:41:57Z]
  • clpeak: krrishnarraj/clpeak: A tool which profiles OpenCL devices to find their peak capacities [visited 2022-12-28T15:43:03Z]

VideoCore VI

Vulkan on the Raspberry Pi:

• v3dv: quick guide to build and run some demos – infapi00 [published 2020-06-09; last updated 2020-11-25; visited 2022-04-13T21:23:22Z]
• src/broadcom · main · Mesa / mesa · GitLab [visited 2022-04-23T20:59:20Z]: the folder in the Git repository of Mesa that is likely relevant for our purposes

VideoCore VI:

• vv3d (for Rpi4) - Mobile GPU [visited 2022-08-05T20:13:17Z]