Exploring Zig - comptime

Intro

Welcome, everyone.
It is time for the first installment of Exploring Zig.
Your host for this series is smallkirby—an eternal NEET who's already been using Zig for six months.
In "Exploring Zig," I’ll be picking out and introducing features and characteristics of Zig that I find interesting.
...or I might not.
The theme for this first installment is comptime, a concept that is particularly vital within Zig.

What is Zig - Everything is Explicit

A Recap of Zig

Since this is the first post, let's start with a brief recap of Zig.

Zig is a compiled general-purpose programming language whose development began in 2016.
Development started the year after Rust hit its 1.0 release in 2015.
The latest release is v0.12.0, and it seems to receive minor updates roughly once a year.
Since it hasn't reached 1.0 yet, breaking changes are introduced with every minor update.
Usually, when the minor version bumps, your old code stops building. Isn't it just adorable?

Currently, there are five corporate sponsors, including Shiguredo.
Unlike certain other languages like Rust or Go, it doesn't have world-renowned tech giants backing it.
If there's anyone out there troubled by having far too much money, perhaps you should consider becoming a sponsor.

The community is decentralized.
On GitHub Issues, asking questions is explicitly forbidden.
Instead, they recommend asking in one of the many existing communities.
While seeing IRC listed first feels a bit old-school, there's also Discord, Slack, and various others.
Each community exists independently with its own moderators, with no official vs. unofficial distinction.
Zig's lead developer, Andrew Kelley, is simply a moderator on GitHub, not the moderator of the entire community.
My personal recommendation is Ziggit.

While the community is friendly and provides quick answers, Zig as a whole is still overwhelmingly lacking in documentation.
While the official documentation is comprehensive in its scope, it often fails to reach the "itchy spots," and you'll still see many sections marked as TODO.
In particular, it's no exaggeration to say that documentation for the build system is non-existent.
The official site recently released some documentation, but it still only explains a tiny fraction.
Currently, the best learning material for Zig is Zig's stdlib.
Alternatively, Bun is also a good learning resource (honestly, developing an all-in-one JS runtime in a language that hasn't even reached 1.0 is pretty insane, right?).
You can also learn a lot by looking at Andrew Kelley's personal projects on GitHub.
For now, it's enough to understand that there is no documentation.

If You See It, You Get It—That's Zig

Now, wandering slightly off-topic, I believe the hallmark of Zig is that "Everything is Explicit."
To put it another way, it's a language that is easy to read even in a simple text editor.

For example, suppose we have some C++ code like this:

auto handle = device.open(desc + 2);

At this point, a wandering C++ beginner like me would start thinking like this:

• 🤔 "Let's see... first, the type of device is class Device, so I have to jump to the header file..."
  • 🤔 "Okay, device::Device has an open() method, so it's this one..."
    • 🤔 "Wait, this function does almost nothing! What's going on?"
    • 🤔 "Ah, there's a child class called class UsbDevice. It's calling that one."
      • 🤔 "Wait, hold on. Which one was this device originally instantiated as?"
      • 🤔 "Oh man, it's a further child class called class MouseDevice!"
  • 🤔 "Wait, the variable desc isn't defined in the current scope...?"
    • 🤔 "Ah, okay. It's a member variable of this class. Got it, got it."
• 🤔 "The desc type looks like a struct Descriptor which just wraps an int."
  • 🤔 "Huh? This struct overloads the + operator. You've got to be kidding me."
• 🤔 "There are two definitions for MouseDevice::open(). This overloading is confusing."
• 🤔 "What? Neither definition takes a struct Descriptor."
  • 🤔 "Ah, right. The constructor of the class used as the argument for open() isn't marked as explicit. Implicit conversion is happening."
• 🤔 "This return class allocates memory from the heap in its constructor."
  • 🤔 "By the way, when should I free this heap memory?"
    • 🤔 "Well, I guess it'll be freed automatically in the destructor. Wait, there's also a Finalize() method?"

Okay, maybe I wrote that a bit too redundantly. It's an exaggeration, of course, but if you're not used to it, you can end up running back and forth like this.
It's one thing when you're reading in VSCode, but when you're reading on GitHub, you start to lose track of what's what.

In contrast, Zig has stripped away various features and made everything explicit:

• Function overloading: You can't do it.
• Default arguments: There aren't any.
• Implicit type conversion: Absolutely not allowed. You can't even go from i8 to i16 (though you can with comptime). You can't do 1 and true. Class constructors aren't called automatically.
• Exceptions: There aren't any. There is syntax sugar to early-return on errors, but it doesn't do that salmon-like thing of swimming back up the stack until caught.
• Inheritance: There isn't any. You can do something similar using union, but there's likely a better way.
• Operator overloading: No way.
• Constructors/Destructors: None. It's scary if functions get called without permission, right? You can use defer to describe processing when leaving a scope, similar to Go.
• Implicit memory allocation: None. All heap allocations must be done explicitly. Standard library functions that use the heap require an Allocator to be passed in.
• Macros: None. Don't you think it's scary not to have types? Ever had a nonsensical error just because you forgot a parenthesis? Instead, there's a feature called comptime that handles most of those use cases.

As a result of stripping away these features, Zig code becomes extremely readable.
In the example above, the one and only open() function of struct Device would be called, the type of its argument (the exact type of desc) would be uniquely determined, and there would be no conversions.
If device doesn't hold an allocator, then open() won't be allocating from the heap.
desc + 2 is literally nothing other than the addition of numbers, and it's obvious that desc is a numeric type (integer/float).
No destructor will be called automatically when leaving the scope.
Simple!

So, Zig has stripped away many features found in other languages, but it still possesses several powerful language features.
Today, I'll be introducing comptime, which is one of the most important yet often difficult-to-handle features when you've just started learning.

comptime Overview

Zig is extremely particular about whether a value is determined at compile-time or not. It's quite meticulous.
Expressions that are determined at compile-time are called comptime values.
The compiler decides whether a value is comptime based on certain rules.
Additionally, developers can explicitly use the comptime keyword on types to force a value to be comptime.

For formal documentation, I'll point you to the Zig Language Reference, but below, let's look at some interesting examples.

Case 1: MMIO

Since I'm a NEET, I've been spending my free time porting MikanOS to Zig, and in the process, I had to write an xHCI (USB) driver.
The xHC has various registers:

pub const CapabilityRegisters = packed struct {
    cap_length: u8,
    hci_version: u16,
    ...
}

packed struct is a struct that removes padding between members.
Also, in Zig, you can take arbitrary bit widths for integer types, so you can use u8, u16, or even u99.
That's great, but for these MMIO registers, the bit width used when accessing them is often fixed by the specifications.
For instance, some registers must be accessed as a WORD (16-bit), while others must be accessed as a DWORD (32-bit).
This becomes a hassle when you want to change only the cap_length field.
If this register mandates DWORD access, accessing only cap_length would result in an access violation.

This is where comptime comes in handy.
First, we define an AccessWidth enum to represent the bit widths we can access:

pub const AccessWidth = enum(u8) {
    QWORD = 8,
    DWORD = 4,
    WORD = 2,
    BYTE = 1,

    pub fn utype(comptime self: AccessWidth) type {
        return switch (self) {
            .QWORD => u64,
            .DWORD => u32,
            .WORD => u16,
            .BYTE => u8,
        };
    }

    pub fn size(comptime self: AccessWidth) usize {
        return @sizeOf(self.utype());
    }
};

Extremely simple.
enum(u8) explicitly states that an instance of this enum is represented in 8 bits.
It has four members representing the bit widths.
utype() is a utility function that returns the integer type corresponding to the access width.
In Zig, as long as it is comptime, you can have type as the return value of a function.
size() is another utility that returns the bit width represented by the enum instance.

With that in mind, we define a struct to represent an MMIO register:

pub fn Register(
    comptime T: type,
    comptime access_width: AccessWidth,
) type {
    return packed struct {
        const Self = @This();
        const asize = access_width.size();
        const atype = access_width.utype();
        const len = @sizeOf(T) / asize;

        /// Underlying data
        _data: T,

        /// Read the data from the underlying register with the correct access width.
        pub fn read(self: *volatile Self) T {
            var ret: T = mem.zeroes(T);
            const ret_bytes: [*]volatile atype = @ptrCast(mem.asBytes(&ret));
            const val: [*]volatile atype = @ptrCast(mem.asBytes(&self._data));
            for (0..len) |i| {
                ret_bytes[i] = val[i];
            }

            return ret;
        }
    };
}

This function takes a comptime type T and an AccessWidth, and returns a register struct holding that type.
Once again, the function is returning a type.
Moreover, the returned type contains a member _data: T using the type passed as an argument.
The _data member represents the actual contents of the register.
Using the previous example, it would look like this:

capability_regs: *volatile Register(CapabilityRegisters, .DWORD),

Now, the capability_regs variable is a pointer to a CapabilityRegisters struct, but it has a type that allows for DWORD access.

Back to the definition of Register().
The asize and atype members represent the size and type corresponding to the access width.
They are treated like static variables of the returned struct type.
read() is the core part; it performs access at the width corresponding to the access width while returning data in the original struct type.
What it's doing is:

• Treat the MMIO register as an array of the type (atype) corresponding to the access width.
• Prepare an empty variable ret of the actual register type (T).
• Copy the data as many times (len) as necessary.

That's the gist of it. The key point here isn't exactly what it's doing, but rather that by taking a type as an argument, you can create a struct with methods tailored to that type.

Case 2: Partial Type

Scripting languages like TypeScript have features to represent types that consist of only some fields of another type. While this is a rare feature in compiled languages, it can be realized in Zig using comptime.

In the previous example, we were reading from MMIO registers; here, let's suppose we want to write to only specific fields. In such a case, we can write it like this:

pub fn modify(self: *volatile Self, value: anytype) void {
    var new = self.read();
    const info = @typeInfo(@TypeOf(value));
    inline for (info.Struct.fields) |field| {
        @field(new, field.name) = @field(value, field.name);
    }

    self.write(new);
}

First, we obtain the current value via read(). Then, we retrieve the type information of the value argument into the info variable. value is of type anytype and can accept any comptime type (so there's no need to explicitly mark it as comptime). Here, info.Struct.fields is an array representing all the fields of the value struct. By iterating through this array with a for loop, we copy the fields from value into the corresponding fields of the new variable. The inline for used here is only available for comptime and is unrolled during compilation. Other similar constructs include inline switch and inline while.

Now, if you want to change only specific fields, you can write it like this:

capability_regs.modify(.{
    .cap_length = 0x10,
});

Isn't that amazing? It's incredibly convenient!
(By the way, the cap_length in the xHCI Capability Register is RO, so you shouldn't actually be writing to it.)

Case 3: Let's Return Functions Too

Let's change the mood slightly; this time, suppose we want to write interrupt handling.
On x64, when the CPU receives an interrupt, it looks up the IDT and calls the handler corresponding to the interrupt vector.
Since this handler performs tasks like saving registers, we generally want to use something common for all interrupts.
However, a problem arises here.
If we use a common handler, we lose track of which interrupt vector (interrupt number) was triggered.
This means we cannot call the vector-specific handler from the common handler.

Therefore, in Linux and similar systems, it is written in assembly like this:

SYM_CODE_START(early_idt_handler_array)
	i = 0
	.rept NUM_EXCEPTION_VECTORS
	(...)
	pushq $i		# 72(%rsp) Vector number
	jmp early_idt_handler_common
	i = i + 1
	.fill early_idt_handler_array + i*EARLY_IDT_HANDLER_SIZE - ., 1, 0xcc
	.endr
SYM_CODE_END(early_idt_handler_array)

It pushes the interrupt vector i first, then jumps to early_idt_handler_common, the common handler.
This logic generates as many handlers as necessary using macros.

This process can also be written in Zig (some parts omitted):

pub const Isr = fn () callconv(.Naked) void;

pub fn generateIsr(comptime vector: usize) Isr {
    return struct {
        fn handler() callconv(.Naked) void {
            // If the interrupt does not provide an error code, push a dummy one.
            if (vector != 8 and !(vector >= 10 and vector <= 14) and vector != 17) {
                asm volatile (
                    \\pushq $0
                );
            }
            asm volatile (
                \\pushq %[vector]
                :
                : [vector] "n" (vector),
            );
            // Jump to the common ISR.
            asm volatile (
                \\jmp isrCommon
            );
        }
    }.handler;
}

export fn isrCommon() callconv(.Naked) void {
    asm volatile (
        \\(Common processing goes here)
    );
}

In Zig, you can specify the calling convention using callconv:

• .Naked: Does not generate a function prologue or epilogue. Used when calling from assembly.
• .Win: Follows Windows conventions. Useful when calling from UEFI, etc.
• .C: Follows C conventions. Used when calling from C.

generateIsr() takes a vector number and returns a function that pushes that vector and then calls the common handler.
You can control the contents of the returned function using high-level language syntax without having to resort to assembly macros like .if.
Zig is a language that enables this kind of metaprogramming—yes, with comptime.

The returned function has a real existence as a function and can be generated as follows:

pub fn init() void {
    inline for (0..num_system_exceptions) |i| {
        idt.setGate(
            ...,
            isr.generateIsr(i),
        );
    }
    ...
}

There it is: inline for.
With this, interrupt handler functions are actually generated at compile-time for each value from 0 to num_system_exceptions - 1.
Looking at it with nm, it looks something like this:

000000000011ce40 t arch.x86.isr.generateIsr__struct_2549.handler
000000000011ce50 t arch.x86.isr.generateIsr__struct_2552.handler
000000000011ce60 t arch.x86.isr.generateIsr__struct_2555.handler
000000000011ce70 t arch.x86.isr.generateIsr__struct_2558.handler
000000000011ce80 t arch.x86.isr.generateIsr__struct_2561.handler
000000000011ce90 t arch.x86.isr.generateIsr__struct_2564.handler
000000000011cea0 t arch.x86.isr.generateIsr__struct_2567.handler
000000000011ceb0 t arch.x86.isr.generateIsr__struct_2570.handler

No more struggling with incomprehensible assembly macros! Hooray!

Outro

Catssssssssssssssss.
I want a Shiba Inuuuuuuuuuuu.
I don't want to work at alllllllllllll.
Doggo. Cattooooo.
Dog cafeeeeeee. Pomeraniannnnn.
🐕
🐕
🐕
🐕
😺
🐕
🐕
🐕

---

See you next time in "Exploring Zig - Ecosystem Edition."