QEMU ARM

QEMU for ARM supports a special ‘virt’ machine designed for emulation and virtualization purposes. This document describes how to run U-Boot under it. Both 32-bit ARM and AArch64 are supported.

The ‘virt’ platform provides the following as the basic functionality:

A freely configurable amount of CPU cores

U-Boot loaded and executing in the emulated flash at address 0x0

A generated device tree blob placed at the start of RAM

A freely configurable amount of RAM, described by the DTB

A PL011 serial port, discoverable via the DTB

An ARMv7/ARMv8 architected timer

PSCI for rebooting the system

A generic ECAM-based PCI host controller, discoverable via the DTB

Additionally, a number of optional peripherals can be added to the PCI bus.

See Devicetree in QEMU for information on how to see the devicetree actually generated by QEMU.

Building (secure)

U-Boot

For AArch64:
```
make qemu_arm64_defconfig
make
```

On successful build ‘u-boot.bin’ should be created. It’s necessary in the following steps (building TF-A).

OP-TEE

For AArch64:

git clone https://github.com/OP-TEE/optee_os.git
cd optee_os
git checkout 4.9.0
export CROSS_COMPILE64=aarch64-none-elf-
export CROSS_COMPILE32=arm-none-eabi-
make PLATFORM=vexpress-qemu_armv8a CFG_TRANSFER_LIST=y CFG_MAP_EXT_DT_SECURE=y

At least OP-TEE v4.9.0 for AArch64 needs both compiler (64-Bit and 32-Bit edition) for a successful build. On a successful build following files should be created under the directory ‘out/arm-plat-vexpress/core’ from OP-TEE:

optee_os/out/arm-plat-vexpress/core/tee-header_v2.bin
optee_os/out/arm-plat-vexpress/core/tee-pageable_v2.bin
optee_os/out/arm-plat-vexpress/core/tee-pager_v2.bin

TF-A

For AArch64:

git clone https://github.com/ARM-software/arm-trusted-firmware.git
cd arm-trusted-firmware
git submodule update --init
git checkout v2.14.0
export CROSS_COMPILE=aarch64-none-elf-
export BL32=path/to/tee-header_v2.bin
export BL32_EXTRA1=path/to/tee-pager_v2.bin
export BL32_EXTRA2=path/to/tee-pageable_v2.bin
export BL33=path/to/u-boot.bin
make PLAT=qemu BL32_RAM_LOCATION=tdram SPD=opteed TRANSFER_LIST=1 all fip

On successful build the following files should be created under the directory ‘build/qemu/release’ from TF-A:

arm-trusted-firmware/build/qemu/release/bl1.bin
arm-trusted-firmware/build/qemu/release/fip.bin

The following file is at least created with TF-A v2.14.0 and can be directly passed with the ‘-bios’ option to QEMU:

arm-trusted-firmware/build/qemu/release/qemu_fw.bios

If the single file (‘qemu_fw.bios’) doesn’t exist, ‘bl1.bin’ and ‘fip.bin’ can be concatenated with the command ‘dd’ alternatively:

dd if=bl1.bin of=qemu_fw.bios bs=4096 conv=notrunc
dd if=fip.bin of=qemu_fw.bios seek=64 bs=4096 conv=notrunc

Building (non-secure)

U-Boot

Set the CROSS_COMPILE environment variable as usual, and run:

For ARM:
```
make qemu_arm_defconfig
make
```
For AArch64:
```
make qemu_arm64_defconfig
make
```

Running U-Boot (secure)

For AArch64:

qemu-system-aarch64 -machine virt,secure=on,virtualization=on \
-nographic -cpu cortex-a57 -bios qemu_fw.bios

For additional QEMU command description see running U-Boot in non-secure state.

Running U-Boot (non-secure)

The minimal QEMU command line to get U-Boot up and running is:

For ARM:

qemu-system-arm -machine virt -nographic -bios u-boot.bin

For AArch64:

qemu-system-aarch64 -machine virt -nographic -cpu cortex-a57 -bios u-boot.bin

Note that for some odd reason qemu-system-aarch64 needs to be explicitly told to use a 64-bit CPU or it will boot in 32-bit mode. The -nographic argument ensures that output appears on the terminal. Use Ctrl-A X to quit.

Additional persistent U-Boot environment support can be added as follows:

Create envstore.img using qemu-img:

qemu-img create -f raw envstore.img 64M

Add a pflash drive parameter to the command line:

-drive if=pflash,format=raw,index=1,file=envstore.img

Additional peripherals that have been tested to work in both U-Boot and Linux can be enabled with the following command line parameters:

To add a video console, remove “-nographic” and add e.g.:
```
-serial stdio -device VGA
```

To add a Serial ATA disk via an Intel ICH9 AHCI controller, pass e.g.:

-drive if=none,file=disk.img,format=raw,id=mydisk \
-device ich9-ahci,id=ahci -device ide-drive,drive=mydisk,bus=ahci.0

To add an Intel E1000 network adapter, pass e.g.:
```
-netdev user,id=net0 -device e1000,netdev=net0
```
To add an EHCI-compliant USB host controller, pass e.g.:
```
-device usb-ehci,id=ehci
```
To add a USB keyboard attached to an emulated xHCI controller, pass e.g.:
```
-device qemu-xhci,id=xhci -device usb-kbd,bus=xhci.0
```

To add an NVMe disk, pass e.g.:

-drive if=none,file=disk.img,id=mydisk -device nvme,drive=mydisk,serial=foo

To add a random number generator, pass e.g.:
```
-device virtio-rng-pci
```

These have been tested in QEMU 2.9.0 but should work in at least 2.5.0 as well.

Booting distros

It is possible to install and boot a standard Linux distribution using qemu_arm64 by setting up a root disk:

qemu-img create root.img 20G

then using the installer to install. For example, with Debian 12:

qemu-system-aarch64 \
  -machine virt -cpu cortex-a53 -m 4G -smp 4 \
  -bios u-boot.bin \
  -serial stdio -device VGA \
  -nic user,model=virtio-net-pci \
  -device virtio-rng-pci \
  -device qemu-xhci,id=xhci \
  -device usb-kbd -device usb-tablet \
  -drive if=virtio,file=debian-12.0.0-arm64-netinst.iso,format=raw,readonly=on,media=cdrom \
  -drive if=virtio,file=root.img,format=raw,media=disk

The output will be something like this:

U-Boot 2023.10-rc2-00075-gbe8fbe718e35 (Aug 11 2023 - 08:38:49 +0000)

DRAM:  4 GiB
Core:  51 devices, 14 uclasses, devicetree: board
Flash: 64 MiB
Loading Environment from Flash... *** Warning - bad CRC, using default environment

In:    serial,usbkbd
Out:   serial,vidconsole
Err:   serial,vidconsole
Bus xhci_pci: Register 8001040 NbrPorts 8
Starting the controller
USB XHCI 1.00
scanning bus xhci_pci for devices... 3 USB Device(s) found
Net:   eth0: virtio-net#32
Hit any key to stop autoboot:  0
Scanning for bootflows in all bootdevs
Seq  Method       State   Uclass    Part  Name                      Filename
---  -----------  ------  --------  ----  ------------------------  ----------------
Scanning global bootmeth 'efi_mgr':
Scanning bootdev 'fw-cfg@9020000.bootdev':
fatal: no kernel available
scanning bus for devices...
Scanning bootdev 'virtio-blk#34.bootdev':
  0  efi          ready   virtio       2  virtio-blk#34.bootdev.par efi/boot/bootaa64.efi
** Booting bootflow 'virtio-blk#34.bootdev.part_2' with efi
Using prior-stage device tree
Failed to load EFI variables
Error: writing contents
** Unable to write file ubootefi.var **
Failed to persist EFI variables
Missing TPMv2 device for EFI_TCG_PROTOCOL
Booting /efi\boot\bootaa64.efi
Error: writing contents
** Unable to write file ubootefi.var **
Failed to persist EFI variables
Welcome to GRUB!

Standard boot looks through various available devices and finds the virtio disks, then boots from the first one. After a second or so the grub menu appears and you can work through the installer flow normally.

After the installation, you can boot into the installed system by running QEMU again without the drive argument corresponding to the installer CD image.

Enabling TPMv2 support

To emulate a TPM the swtpm package may be used. It can be built from the following repositories:

https://github.com/stefanberger/swtpm.git

Swtpm provides a socket for the TPM emulation which can be consumed by QEMU.

In a first console invoke swtpm with:

swtpm socket --tpmstate dir=/tmp/mytpm1   \
--ctrl type=unixio,path=/tmp/mytpm1/swtpm-sock --log level=20

In a second console invoke qemu-system-aarch64 with:

-chardev socket,id=chrtpm,path=/tmp/mytpm1/swtpm-sock \
-tpmdev emulator,id=tpm0,chardev=chrtpm \
-device tpm-tis-device,tpmdev=tpm0

Enable the TPM on U-Boot’s command line with:

tpm autostart

Debug UART

The debug UART on the ARM virt board uses these settings:

CONFIG_DEBUG_UART=y
CONFIG_DEBUG_UART_PL010=y
CONFIG_DEBUG_UART_BASE=0x9000000
CONFIG_DEBUG_UART_CLOCK=0