iTranslated by AI
Building a Statically Linked QEMU with Virtual GPU and WHPX Support Using Windows Docker
qemu is very useful, but for various reasons (mentioned later), I needed several specific features. Since I couldn't find a version that had all the features I wanted, I decided to build it myself.
- A Windows version
- Statically linked
- Supports OpenGL (WebGL) acceleration
- Works alongside Hyper-V (WSL2, Windows Containers, etc.)
Summary
- Even with native Windows Docker, the build environment can be containerized. Using
--isolation processallows it to be used with low overhead, much like a simplechroot, making it practical. - Just as Linux has KVM, Windows has WHPX (Windows Hypervisor Platform), which can accelerate QEMU.
- OpenGL can be used via a virtual GPU with VirGL even in the Windows version of QEMU.
As a result, I was able to build a statically linked QEMU on Windows with fully functional GPU virtualization.
However, it only seems to support up to OpenGL ES 2.0, so compute shaders cannot be used...
EDIT: After debugging, it properly supported ES 3.1 (mentioned later).
What are Docker's "Process Isolation" Containers?
I believe every Windows user has thought, "I wish Windows had chroot(2)" so they could use software without cluttering their environment. Docker's "process isolation" containers are almost exactly that.
(However, GUI applications cannot be used. In the past, it was possible to set up things like Remote Desktop, but that seems to have been blocked, likely because it was too convenient.)
If you have Docker installed, you can start a container in process isolation mode by passing --isolation process to docker run as follows:
Installing Docker
For home use, you should be able to use it by installing Docker Desktop normally and switching to Windows containers. However, I don't use Docker Desktop locally, so I'm not entirely sure.
If you don't need Linux container support, installing the command-line tools directly is simple. This method doesn't require a Docker Desktop license.
Note: The available container images vary depending on the version of Windows.
# For users on Windows 10 host
docker run --rm -it --isolation process mcr.microsoft.com/windows/servercore:2004 cmd
# For users on Windows 11 host
docker run --rm -it --isolation process mcr.microsoft.com/windows/servercore:ltsc2022 cmd
There are several official documents for Windows 10 that state process isolation mode is unsupported, and it doesn't seem to be seriously supported. On my local Windows 10 22H2, it works fine with the servercore:2004 image. However, support for this image itself has already expired.
On Windows 11, you can use the 2022 or 2025 images.
Regular containers that are not process-isolated become Hyper-V containers, which run a separate Windows instance on top of Hyper-V (similar to WSL2). Windows Docker defaults to Hyper-V containers.
Benefits
Compared to Windows containers on Hyper-V, they are faster and more resource-efficient. They are at a level practical enough for building software.
Additionally, being usable in environments where Hyper-V is disabled is also an advantage.
Drawbacks
It is important to note that there is no security isolation. Processes inside the container can be observed normally from the host (similar to Docker on Linux).
(↓ A ping running inside the container being observed via Process Explorer on the host)
Furthermore, container images cannot be shared between systems with different kernel versions—specifically between Windows 10 and Windows 11 (Hyper-V containers can run Windows 10 containers on Windows 11). Well, Windows 10 support is ending soon, so it's not a major issue...
A phenomenon that symbolizes the poor isolation of process-isolated containers is that a Windows 10 container can spontaneously wake the host from sleep. You should avoid using containers for sandboxing suspicious programs.
Also, compatibility is slightly suboptimal; for example, Cygwin will not function correctly unless ASLR is disabled: https://cygwin.com/pipermail/cygwin/2022-December/252720.html.
Installing MSYS2 (gcc) in the Container
Since we want to build QEMU this time, we will install gcc (specifically MSYS2, a major distribution for Windows) into the container. Following the official MSYS2 site (https://www.msys2.org/docs/ci/), we add the necessary packages:
Note: As mentioned at the beginning, you need to pre-tag a base image as qemubuild-windows-base.
# Before building this image, tag Windows image based on your kernel
# Win10: docker tag mcr.microsoft.com/windows/servercore:2004 qemubuild-windows-base
# Win11: docker tag mcr.microsoft.com/windows/servercore:ltsc2022 qemubuild-windows-base
FROM qemubuild-windows-base
SHELL ["powershell", "-Command", "$ErrorActionPreference = 'Stop'; $ProgressPreference = 'SilentlyContinue';"]
RUN [Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12; \
Invoke-WebRequest -UseBasicParsing -uri "https://github.com/msys2/msys2-installer/releases/download/nightly-x86_64/msys2-base-x86_64-latest.sfx.exe" -OutFile msys2.exe; \
.\msys2.exe -y -oC:\; \
Remove-Item msys2.exe ; \
function msys() { C:\msys64\usr\bin\bash.exe @('-lc') + @Args; } \
msys ' '; \
msys 'pacman --noconfirm -Syuu'; \
msys 'pacman --noconfirm -Syuu'; \
msys 'pacman --noconfirm -Scc'; \
msys 'pacman --noconfirm -S git python ninja mingw-w64-ucrt-x86_64-toolchain base-devel python-setuptools mingw-w64-ucrt-x86_64-meson mingw-w64-ucrt-x86_64-ca-certificates mingw-w64-ucrt-x86_64-egl-headers mingw-w64-ucrt-x86_64-gles-headers mingw-w64-ucrt-x86_64-cmake mingw-w64-ucrt-x86_64-python-yaml' ; \
msys 'pacman --noconfirm -Scc';
LABEL org.opencontainers.image.source=https://github.com/okuoku/qemubuild
In this project, we want to build for UCRT (Visual Studio's runtime library), so we select packages starting with mingw-w64-ucrt-.
The build—or rather, saving the image—takes a very long time.
In addition to the standard build environment packages like git, python, ninja, and mingw-w64-ucrt-x86_64-toolchain, I have added mingw-w64-ucrt-x86_64-ca-certificates (because some projects use curl to fetch dependencies during the build) and mingw-w64-ucrt-x86_64-python-yaml (required for building libepoxy from a Git repository), among others.
Building QEMU and its Dependencies
In this project, I decided to basically make all dependencies Git submodules and check them out together. However, GLib and QEMU are designed to check out or download additional dependencies during the build, so source code not included in this Git repository is also required.
Prerequisites
- Clone the above repository with
git clone --recursive. (Don't forget to setgit config --global core.longpaths trueif necessary.) - Tag the base Windows image as
qemubuild-windows-base(if the host is Windows 10, usedocker tag mcr.microsoft.com/windows/servercore:2004 qemubuild-windows-base). - Prepare the Docker image
qemubuild(as described in the Dockerfile above). - Prepare Docker volumes
objs,libs, anddist. They can be empty. For example,docker volume create objs.
Actual Build
After completing the above preparations, running cmake -P build.cmake will output the built QEMU to the out directory within the repository. If CMake is not installed on the host, install it and add it to your path.
In this repository, I am using CMake like a shell script for the build process. Since CMake is available almost everywhere...
Explanation of the Build Process
...If I stopped here, the article would be very short...
Using MSYS inside Docker
To use MSYS normally, commands must be launched via a login shell (bash). Since the commands are long, I decided to turn them into functions. In the repository:
execute_process(COMMAND
# Create a process isolation container by specifying --isolation process
docker run --isolation process --rm
# Volume settings: libs is the installation destination for dependencies, dist is the final QEMU
"-vtmp:c:\\objs"
"-vlibs:c:\\libs"
"-vdist:c:\\dist"
# e:/repos/qemubuild is the source code repository
"-ve:/repos/qemubuild:c:\\srcs"
"-ve:/repos/qemubuild/out:c:\\out"
# qemubuild: The tag of the container we created with MinGW
qemubuild
# Command line to execute
"c:\\msys64\\msys2_shell.cmd" -here -no-start -ucrt64 -defterm
-c "${script}"
Note: In this snippet, I've hardcoded values for clarity, but since I thought that wasn't ideal, I switched to automatic detection in the repository.
The key is to use msys2_shell.cmd like c:\msys64\msys2_shell.cmd -here -no-start -ucrt64 -defterm -c "shell script". By invoking commands through this, you can launch them exactly as you would in an MSYS terminal.
Building Dependencies with Meson
Many projects, including QEMU and those from GNOME or Freedesktop.org, generally adopt Meson as their build system. The build procedure for these projects can be standardized as follows:
- Generate project files with
meson setup. - Perform the actual build with
meson compile. - Install the results with
meson install.
Since this is consistent, I decided to script it for reuse. In the repository:
# setup: call meson setup with the PKG_CONFIG_PATH environment variable set
PKG_CONFIG_PATH=c:/libs/lib/pkgconfig /ucrt64/bin/meson setup
# Common options
--prefix=c:/libs --buildtype=release -Ddefault_library=static
# Additional options (only libepoxy needs this)
${adddef}
# Source code directory
c:/srcs/deps/${projname}
# Build directory -- Ninja project files are output here
c:/objs/${projname}
# compile: call Ninja to compile
/ucrt64/bin/meson compile -C c:/objs/${projname}
# install
/ucrt64/bin/meson install -C c:/objs/${projname}
I specify the following common options for every project:
-
--prefix: Installs to a non-standard location to make it easier to clean up after the build. -
--buildtype: Specifiesrelease(build with optimizations). -
-Ddefault_library=static: Builds static libraries only.
Additionally, since libepoxy could not be built correctly on Windows for some reason, I am passing -Degl=yes as an extra option.
QEMU is incredibly feature-rich, and building a "complete" version requires building a massive number of dependency libraries. However, only a few are actually needed just to get it running.
The following are built with Meson:
- glib2: https://gitlab.gnome.org/GNOME/glib The portability library for GNOME. Mandatory because QEMU depends on glib for features like threading.
- libslirp: https://gitlab.freedesktop.org/slirp/libslirp A user-land TCP/IP implementation. It is a historic piece of software that will celebrate its 30th anniversary next year and appeared in last year's Internet Old-timers' Association Advent Calendar. It's currently a "hot" library, having a bit of a dispute with QEMU maintainers after suddenly breaking its ABI. Necessary for the VM to communicate via TCP/IP.
- pixman: https://gitlab.freedesktop.org/pixman/pixman Used by the VNC implementation.
- libepoxy: https://github.com/anholt/libepoxy An OpenGL(ES) loader. OpenGL availability and its handling as a DLL vary by environment; this library abstracts those differences. Required by virglrenderer.
- virglrenderer: https://gitlab.freedesktop.org/virgl/virglrenderer The host-side implementation of Mesa's VirGL. It executes rendering commands created via VirGL on the host's OpenGL or OpenGL ES. Recently, it has also begun supporting Vulkan passthrough.
Projects with their own unique build systems include SDL2 (CMake) and QEMU itself (configure script).
Building SDL2
By default, SDL2 would load OpenGL itself, causing function name collisions and crashes. Therefore, I build it normally except for passing SDL_OPENGL=OFF.
/ucrt64/bin/cmake -G Ninja
-DSDL_STATIC=ON -DSDL_SHARED=OFF -DSDL_OPENGL=OFF
-DCMAKE_BUILD_TYPE=RelWithDebInfo
-DCMAKE_INSTALL_PREFIX=c:/libs
-B c:/objs/SDL2 -S c:/srcs/deps/SDL2
Building QEMU
This involved a lot of trial and error...
PKG_CONFIG_PATH=c:/libs/lib/pkgconfig c:/srcs/qemu/configure
--enable-whpx --enable-system --enable-slirp --enable-vnc
--target-list=aarch64-softmmu,arm-softmmu,avr-softmmu,riscv32-softmmu,riscv64-softmmu,x86_64-softmmu
--prefix=c:/dist
--disable-gio --disable-curl --disable-zstd --disable-bzip2 --disable-curses
--static --extra-ldflags=-liconv --disable-werror --extra-cflags=-DLIBSLIRP_STATIC
-
--enable-whpx: Enables the Windows Hypervisor Platform. Without this, QEMU cannot utilize the hypervisor and becomes a slow, standard emulator. Since the WHPX version of QEMU is also the core of the modern Android Emulator, it is expected to be reasonably stable. For environments without Hyper-V, Google maintains a Windows port of KVM. Intel used to provide HAXM for this purpose but has since discontinued it. (Even further back, there was a QEMU-specific kernel module called KQEMU.) -
--disable-gio --disable-curl --disable-zstd --disable-bzip2 --disable-curses: Features I don't use but would otherwise be enabled automatically. -
--static: Statically links the binary (the goal of this project). -
--extra-ldflags=-liconv: A workaround for an issue where iconv is not included in the pkgconfig specification despite glib2 requiring it to be explicitly linked (likely a bug). This isn't an issue with dynamic linking because iconv is linked naturally via DLL dependencies. -
--disable-werror: A workaround for the issue where functions were deprecated in libslirp. -
--extra-cflags=-DLIBSLIRP_STATIC: A define to utilize the fact that static linking was recently supported in libslirp for Windows, whereas DLLs were previously mandatory.
Extracting Build Results
Running build.cmake stores the build results in the Docker volume dist. To extract them to the host side, start the container and copy them.
In the build script, I simply use cp:
cp -rp c:/dist/* c:/out/
Trying it Out
Let's try booting the built QEMU. This also requires some preparation.
Testing the Default Settings First
If you double-click the built qemu-system-x86_64.exe, something should appear for now. If nothing appears, something likely went wrong during the build.
Enabling Windows Hypervisor Platform
...Wait, was this not enabled by default? Check "Windows Hypervisor Platform." cf. Android Emulator documentation
By the way, this is a different checkbox from "Hyper-V."
ANGLE
Since Windows does not include OpenGL ES by default, we will introduce ANGLE, Google's OpenGL ES implementation. For this project, I downloaded it from https://github.com/mmozeiko/build-angle and copied the DLLs included in the release to the same location as QEMU.
Disabling HiDPI
If you are using a display with a DPI different from the Windows standard, such as a 4K monitor, the display might look slightly blurry, as seen in the screenshots above. This happens because Windows assumes the application is not HiDPI-compatible and automatically scales it up.
Since proper HiDPI support was only implemented in SDL starting with SDL3, I'll manually spoof it as a HiDPI-compatible app for now. It doesn't have a UI anyway.
Right-click the QEMU executable → Properties → Compatibility → Change high DPI settings → Override high DPI scaling behavior → Set to Application.
Boot Options
qemu-system-x86_64.exe \
# Set appropriate CPU model
-machine q35 -smp 6 -cpu qemu64-v1 \
# Use WHPX. kernel-irqchip specification should be unnecessary on recent Windows hosts
-accel whpx,kernel-irqchip=off \
# Japanese keyboard mapping
-k ja \
# Detailed device settings
-m 8192 -netdev user,id=net1,hostfwd=tcp:127.0.0.1:5522-:22 \
-device virtio-net-pci,netdev=net1 \
-drive file=e:/vm/alpine/alpine.qcow2,id=blk1,if=none \
-device virtio-blk-pci,drive=blk1 \
-usb -usbdevice tablet -audio driver=sdl,model=virtio \
# VirGL settings
-device virtio-vga-gl -display sdl,gl=es
While other hypervisors intelligently auto-configure various settings, with QEMU, you have to search for a combination that works. In my case, -machine q35 -cpu qemu64-v1 -accel whpx,kernel-irqchip=off proved to be the most stable, so I went with that.
Checking if GPU Virtualization is Working
On most Linux distributions, you can check the API support status using eglinfo from the mesa-utils package.
Running eglinfo -p x11 outputs the status for X11 applications, and eglinfo -p wayland for Wayland applications.
Wayland platform:
EGL API version: 1.5
EGL vendor string: Mesa Project
EGL version string: 1.5
EGL client APIs: OpenGL OpenGL_ES
EGL driver name: virtio_gpu
:
:
OpenGL compatibility profile vendor: Mesa
OpenGL compatibility profile renderer: virgl (ANGLE (Intel, Intel(R) HD Graphics 530 (0x00001912) ...)
OpenGL compatibility profile version: 2.1 Mesa 24.3.2
:
:
OpenGL ES profile vendor: Mesa
OpenGL ES profile renderer: virgl (ANGLE (Intel, Intel(R) HD Graphics 530 (0x00001912) ...)
OpenGL ES profile version: OpenGL ES 2.0 Mesa 24.3.2
OpenGL ES profile shading language version: OpenGL ES GLSL ES 1.0.16
However, for some reason, it ends up as OpenGL ES 2.0... (Currently investigating). Although I'm using OpenGL ES as the VirGL backend this time, standard Desktop OpenGL is also provided to the guest as shown above.
By the way, if it's not enabled correctly, it automatically falls back to llvmpipe. This performs rendering using only the CPU, without utilizing the GPU.
Various Circumstances (Motivation)
As always, I think the natural reaction is "Why go to all this trouble?"
Dynamic linking is inconvenient
In the first place, if you just want to use qemu, there is a QEMU package in MSYS2.
However, I needed to build it from source for reasons like debugging. Considering coexistence with the original MSYS2 packages, I thought it would be easier to make my own build statically linked.
The problem where WSL2's virtual GPU doesn't work on old GPUs
GPU virtualization on Windows requires relatively recent GPU drivers. Virtual GPUs for Windows Containers require WDDM 2.5, and DirectX 12 passthrough for WSL2 requires WDDM 2.9. What about my environment?
... WDDM 2.1. When I ran dxdiag to check, I literally blurted out, "Are you serious?!"
Being able to use RenderDoc on the entire desktop environment
By using QEMU's VirGL, you can treat the entire desktop environment as a single OpenGL ES application. This means you can perform GPU tracing on the entire desktop environment.
(Tracing a WebGL demo https://webglsamples.org/field/field.html being displayed in Firefox: Since I set RENDERDOC_HOOK_EGL=0 to trace the DirectX side, it's vertically inverted.)
Why? One of the goals of WasmLinux is to run web browsers and desktop environments within a web browser, and as a preliminary step, I want to port QEMU to WebGL. To achieve this, I need to investigate what GPU features are being used through GPU tracing.
By the way, a WebAssembly port of QEMU already exists as part of container2wasm (impressive!) https://github.com/ktock/qemu-wasm. I already made WebGL C-bindings a long time ago, with enough completion to run Unity WebGL at that time:
I've also implemented an emulator that wraps those WebGL bindings into EGL + OpenGL ES to the point where DirectX 8 games can run on Wine:
After that, I should be able to do it if I just spend enough time... although finding that time is the hardest part.
Final Thoughts
Well, I've spent a lot of time talking about something that ultimately boils down to "Just upgrade your PC," but I've already migrated most of my development to Mac and Linux, so I haven't really maintained my Windows environment... and I don't have much time.
Windows 10 is getting old
Mainstream support will end in October next year (2025). https://www.microsoft.com/ja-jp/atlife/article-windows10-portal-eos :
Windows 10 Home and Pro
Support started: July 29, 2015 Support ends: October 14, 2025
Since Windows Containers (or rather, Docker on Windows) only received serious support (?) from Windows 11 onwards, you should probably adopt Windows 11 if you want to use them for this purpose. In this case, the Windows image that can properly use process isolation containers on Windows 10 is 2004 (April 2020), a version whose support has long since expired.
...Well, I only have Win11 in my test environment. I'm a bit sentimental about Windows 10 because it's the last Windows with a 32-bit kernel... (and you can put the taskbar on the side).
The issue where the guest side defaults to OpenGL ES 2.0
EDIT: In my environment, the cause was that GL_MAX_VERTEX_UNIFORM_BLOCKS was only 12, which is the minimum requirement of the specification.
For now, desktop environments like GNOME and web browsers work fine with OpenGL ES 2.0, but this means WebGL2 cannot be used, so I'd like to do something about it...
According to an old comment (https://www.collabora.com/news-and-blog/blog/2019/08/28/virglrenderer-state-of-virtualized-virtual-worlds/#qcom1089), the extension GL_NV_conditional_render is supposedly required, but I can't think of a situation where it would be necessary (in GLES).
The combination of ANGLE + virglrenderer + QEMU used here is the same combination used by UTM, which runs on macOS and iOS. Since GLES 3.0 is visible there, it shouldn't be impossible on ANGLE... hopefully. Unless ANGLE's backend requires OpenGL, that is.
Venus and gfxstream
QEMU 9.2, released this month, supports not only OpenGL via VirGL but also Vulkan passthrough via venus. https://wiki.qemu.org/ChangeLog/9.2
virtio-gpu now support venus encapsulation for vulkan (need recent virglrenderer on host and mesa on guest)
Therefore, I think using Vulkan would be better, but it feels like it might be "mission impossible" to bring it to the Web. The functional gap between Vulkan and WebGPU is quite significant.
The ChromeOS blog provides a clear explanation of the differences between Venus and VirGL:
Something even closer to a "passthrough" is gfxstream, which is used in the Android Emulator. (There are also issues raised to try and use Venus or VirGL... https://gitlab.freedesktop.org/virgl/virglrenderer/-/issues/563)
The Significance of Building QEMU Locally
In addition to the WebAssembly version of QEMU mentioned earlier (https://github.com/ktock/qemu-wasm), I find it interesting that various fascinating hacks are being explored, such as the high-speed interpreter version used in UTM. I also ported it to Cygwin myself a long time ago:
However, QEMU has become much more complex than it used to be, and the preparation can be quite tedious, so I suspect many people avoid it. That's why I wrote this article—to demonstrate that it can be done (relatively) easily.
Personally, I use QEMU built this way as a hypervisor for CI. Lately, KVM has become available on GitHub Actions workers (largely for running the Android emulator), and I feel its compatibility with build automation is superior to hypervisors implemented as services like Hyper-V, VMware, or VirtualBox.
However, using it as a regular desktop is a bit hit-or-miss, as clicking the "Close" button shuts it down without any confirmation (I've been burned by this many times).
Discussion