Zenn
Open6

WasmLinux: LKLをWebAssemblyにしたい

okuokuokuoku

とりあえずツールチェインを用意してそれを適当に使わせる作戦にする。ホストツール類は要らない(自分で作るので)。

Fedora

Fedoraはclangとllvmのパッケージが分かれているようなので、

# dnf install clang llvm lld

でそれぞれインストールする必要があった。これをLKLが要求する各種ツールにwrapしておく。

https://github.com/lkl/linux/blob/31a835e54b32bc76cde50f41aab41285ab9053d9/tools/lkl/Makefile.autoconf#L167-L177

(直接Makefile.autoconf内で対応しても良いんだけど細かい調整が面倒なので。。)

okuokuokuoku

make のコマンドライン作り

make -C tools/lkl ARCH=lkl CC=`pwd`/_warp/bin/warp-cc LD=`pwd`/_warp/bin/warp-ld \
AR=`pwd`/_warp/bin/warp-ar NM=`pwd`/_warp/bin/warp-nm OBJCOPY=llvm-objcopy \
KALLSYMS_EXTRA_PASS=1 CONFIG_OUTPUT_FORMAT=wasm32 CROSS_COMPILE=wasm32 V=1

今回ELFじゃないので KALLSYMS_EXTRA_PASS が無いと失敗するっぽい。また、LKLは未知の CONFIG_OUTPUT_FORMAT を弾いているので無視するように変更した。

https://github.com/okuoku/lkl-wasm/commit/a2049abe73a81e29aab07982dfe88d6f7c89160e

okuokuokuoku

syscall.c のビルドが通らない

arch/lkl/kernel/syscalls.c:25:35: note: expanded from macro '__SYSCALL'
#define __SYSCALL(nr, sym) [nr] = (syscall_handler_t)sym,
                                  ^~~~~~~~~~~~~~~~~~~~~~
<inline asm>:1:24: error: unknown flag
.section .syscall_defs,"a"
                       ^
<inline asm>:5:15: error: Expected ,, instead got:

.section .text
              ^
81 warnings and 2 errors generated.
make[3]: *** [scripts/Makefile.build:250: arch/lkl/kernel/syscalls.o] エラー 1
make[2]: *** [scripts/Makefile.build:500: arch/lkl/kernel] エラー 2
make[1]: *** [Makefile:1996: .] エラー 2
make: *** [Makefile:84: /home/oku/repos/linux/tools/lkl/lib/lkl.o] エラー 2

初手からか。。

https://github.com/okuoku/lkl-wasm/commit/e6a6b195a1388a5307beb5655aee8a5b2ce8c578

そもそも .section の仕様がLLVMのテストぐらいからしか分からない。tool-conventionに無いし。。

okuokuokuoku

Computed gotoが無い

In file included from kernel/rcu/srcutiny.c:14:
In file included from ./include/linux/rcupdate_wait.h:9:
In file included from ./include/linux/rcupdate.h:28:
./include/linux/bottom_half.h:31:20: error: WebAssembly hasn't implemented computed gotos
static inline void local_bh_enable(void)
                   ^
./include/linux/bottom_half.h:31:20: error: WebAssembly hasn't implemented computed gotos
fatal error: error in backend: Cannot select: 0xd6cf70: i32 = BlockAddress<@local_bh_enable, %1> 0
In function: local_bh_enable

そういやLinuxはclean Cじゃないんだった。。カーネルはシングルスレッドで動かすので、RCUを真面目にやる必要性は殆どない。というか専用のワークアラウンドが必要だろう。

kernel/panic.c:651:21: error: Non-Emscripten WebAssembly hasn't implemented __builtin_return_address
        __warn(file, line, __builtin_return_address(0), taint, NULL, &args);
                           ^
kernel/panic.c:644:22: error: Non-Emscripten WebAssembly hasn't implemented __builtin_return_address
                __warn(file, line, __builtin_return_address(0), taint,
                                   ^

これも同種だな。。正直これ無限にありそうなんだけど。。

https://github.com/okuoku/lkl-wasm/commit/0cb256b6923684cb5b94d8ce9f040c6baa95870f

okuokuokuoku

sys_ni.c でICE

これはちょっと厳しいもんがあるな。。いったん #if 0 。

$ clang --target=wasm32 -ffreestanding -Wp,-MMD,kernel/.sys_ni.o.d -nostdinc -I./arch/lkl/include -I./arch/lkl/include/generated -I./include -I./arch/lkl/include/uapi -I./arch/lkl/include/generated/uapi -I./include/uapi -I./include/generated/uapi -include ./include/linux/compiler-version.h -include ./include/linux/kconfig.h -include ./include/linux/compiler_types.h -D__KERNEL__ -Qunused-arguments -fmacro-prefix-map=./= -Wall -Wundef -Werror=strict-prototypes -Wno-trigraphs -fno-strict-aliasing -fno-common -fshort-wchar -fno-PIE -Werror=implicit-function-declaration -Werror=implicit-int -Werror=return-type -Wno-format-security -std=gnu11 --target=wasm32 -fintegrated-as -Werror=unknown-warning-option -Werror=ignored-optimization-argument -fno-builtin -fno-delete-null-pointer-checks -Wno-frame-address -Wno-address-of-packed-member -O0 -Wframe-larger-than=1024 -fno-stack-protector -Wno-gnu -Wno-unused-but-set-variable -Wno-unused-const-variable -fno-omit-frame-pointer -fno-optimize-sibling-calls -ftrivial-auto-var-init=zero -fno-stack-clash-protection -Wdeclaration-after-statement -Wvla -Wno-pointer-sign -Wcast-function-type -Wimplicit-fallthrough -fno-strict-overflow -fno-stack-check -Werror=date-time -Werror=incompatible-pointer-types -Wno-initializer-overrides -Wno-sign-compare -Wno-pointer-to-enum-cast -Wno-tautological-constant-out-of-range-compare -Wno-unaligned-access -Wno-cast-function-type-strict -g -gdwarf-4 -DKBUILD_MODFILE=\"kernel/sys_ni\" -DKBUILD_BASENAME=\"sys_ni\" -DKBUILD_MODNAME=\"sys_ni\" -D__KBUILD_MODNAME=kmod_sys_ni -c -o kernel/sys_ni.o kernel/sys_ni.c
PLEASE submit a bug report to https://github.com/llvm/llvm-project/issues/ and include the crash backtrace, preprocessed source, and associated run script.
Stack dump:
0.      Program arguments: clang --target=wasm32 -ffreestanding -Wp,-MMD,kernel/.sys_ni.o.d -nostdinc -I./arch/lkl/include -I./arch/lkl/include/generated -I./include -I./arch/lkl/include/uapi -I./arch/lkl/include/generated/uapi -I./include/uapi -I./include/generated/uapi -include ./include/linux/compiler-version.h -include ./include/linux/kconfig.h -include ./include/linux/compiler_types.h -D__KERNEL__ -Qunused-arguments -fmacro-prefix-map=./= -Wall -Wundef -Werror=strict-prototypes -Wno-trigraphs -fno-strict-aliasing -fno-common -fshort-wchar -fno-PIE -Werror=implicit-function-declaration -Werror=implicit-int -Werror=return-type -Wno-format-security -std=gnu11 --target=wasm32 -fintegrated-as -Werror=unknown-warning-option -Werror=ignored-optimization-argument -fno-builtin -fno-delete-null-pointer-checks -Wno-frame-address -Wno-address-of-packed-member -O0 -Wframe-larger-than=1024 -fno-stack-protector -Wno-gnu -Wno-unused-but-set-variable -Wno-unused-const-variable -fno-omit-frame-pointer -fno-optimize-sibling-calls -ftrivial-auto-var-init=zero -fno-stack-clash-protection -Wdeclaration-after-statement -Wvla -Wno-pointer-sign -Wcast-function-type -Wimplicit-fallthrough -fno-strict-overflow -fno-stack-check -Werror=date-time -Werror=incompatible-pointer-types -Wno-initializer-overrides -Wno-sign-compare -Wno-pointer-to-enum-cast -Wno-tautological-constant-out-of-range-compare -Wno-unaligned-access -Wno-cast-function-type-strict -g -gdwarf-4 -DKBUILD_MODFILE=\"kernel/sys_ni\" -DKBUILD_BASENAME=\"sys_ni\" -DKBUILD_MODNAME=\"sys_ni\" -D__KBUILD_MODNAME=kmod_sys_ni -c -o kernel/sys_ni.o kernel/sys_ni.c
1.      <eof> parser at end of file
2.      Code generation
 #0 0x00007f2e74ac350a llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) (/lib64/libLLVM-16.so+0xcc350a)
 #1 0x00007f2e74ac0e94 llvm::sys::RunSignalHandlers() (/lib64/libLLVM-16.so+0xcc0e94)
 #2 0x00007f2e749db204 (/lib64/libLLVM-16.so+0xbdb204)
 #3 0x00007f2e7385fbb0 __restore_rt (/lib64/libc.so.6+0x3dbb0)
 #4 0x00007f2e76379b38 (/lib64/libLLVM-16.so+0x2579b38)
 #5 0x00007f2e763761b4 (/lib64/libLLVM-16.so+0x25761b4)
 #6 0x00007f2e763086c0 llvm::MCAssembler::Finish() (/lib64/libLLVM-16.so+0x25086c0)
 #7 0x00007f2e7546d25d llvm::AsmPrinter::doFinalization(llvm::Module&) (/lib64/libLLVM-16.so+0x166d25d)
 #8 0x00007f2e74c3a3c1 llvm::FPPassManager::doFinalization(llvm::Module&) (/lib64/libLLVM-16.so+0xe3a3c1)
 #9 0x00007f2e74c33e51 llvm::legacy::PassManagerImpl::run(llvm::Module&) (/lib64/libLLVM-16.so+0xe33e51)
#10 0x00007f2e7d09eaf2 clang::EmitBackendOutput(clang::DiagnosticsEngine&, clang::HeaderSearchOptions const&, clang::CodeGenOptions const&, clang::TargetOptions const&, clang::LangOptions const&, llvm::StringRef, llvm::Module*, clang::BackendAction, std::unique_ptr<llvm::raw_pwrite_stream, std::default_delete<llvm::raw_pwrite_stream>>) (/lib64/libclang-cpp.so.16+0x1c9eaf2)
#11 0x00007f2e7d47281d (/lib64/libclang-cpp.so.16+0x207281d)
#12 0x00007f2e7becf95f clang::ParseAST(clang::Sema&, bool, bool) (/lib64/libclang-cpp.so.16+0xacf95f)
#13 0x00007f2e7e04f966 clang::FrontendAction::Execute() (/lib64/libclang-cpp.so.16+0x2c4f966)
#14 0x00007f2e7dfc8840 clang::CompilerInstance::ExecuteAction(clang::FrontendAction&) (/lib64/libclang-cpp.so.16+0x2bc8840)
#15 0x00007f2e7e0d28c4 clang::ExecuteCompilerInvocation(clang::CompilerInstance*) (/lib64/libclang-cpp.so.16+0x2cd28c4)
#16 0x000000000041655a cc1_main(llvm::ArrayRef<char const*>, char const*, void*) (/usr/bin/clang-16+0x41655a)
#17 0x0000000000412ded (/usr/bin/clang-16+0x412ded)
#18 0x00007f2e7dc1a4e6 (/lib64/libclang-cpp.so.16+0x281a4e6)
#19 0x00007f2e749daf44 llvm::CrashRecoveryContext::RunSafely(llvm::function_ref<void ()>) (/lib64/libLLVM-16.so+0xbdaf44)
#20 0x00007f2e7dc19dc7 clang::driver::CC1Command::Execute(llvm::ArrayRef<std::optional<llvm::StringRef>>, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>*, bool*) const (/lib64/libclang-cpp.so.16+0x2819dc7)
#21 0x00007f2e7dbe26be clang::driver::Compilation::ExecuteCommand(clang::driver::Command const&, clang::driver::Command const*&, bool) const (/lib64/libclang-cpp.so.16+0x27e26be)
#22 0x00007f2e7dbe2927 clang::driver::Compilation::ExecuteJobs(clang::driver::JobList const&, llvm::SmallVectorImpl<std::pair<int, clang::driver::Command const*>>&, bool) const (/lib64/libclang-cpp.so.16+0x27e2927)
#23 0x00007f2e7dbfe83a clang::driver::Driver::ExecuteCompilation(clang::driver::Compilation&, llvm::SmallVectorImpl<std::pair<int, clang::driver::Command const*>>&) (/lib64/libclang-cpp.so.16+0x27fe83a)
#24 0x00000000004124a1 clang_main(int, char**) (/usr/bin/clang-16+0x4124a1)
#25 0x00007f2e73849b8a __libc_start_call_main (/lib64/libc.so.6+0x27b8a)
#26 0x00007f2e73849c4b __libc_start_main@GLIBC_2.2.5 (/lib64/libc.so.6+0x27c4b)
#27 0x000000000040f185 _start (/usr/bin/clang-16+0x40f185)
clang-16: error: clang frontend command failed with exit code 139 (use -v to see invocation)
clang version 16.0.6 (Fedora 16.0.6-3.fc38)
Target: wasm32
Thread model: posix
InstalledDir: /usr/bin
clang-16: note: diagnostic msg:
********************

PLEASE ATTACH THE FOLLOWING FILES TO THE BUG REPORT:
Preprocessed source(s) and associated run script(s) are located at:
clang-16: note: diagnostic msg: /tmp/sys_ni-4dc779.c
clang-16: note: diagnostic msg: /tmp/sys_ni-4dc779.sh
clang-16: note: diagnostic msg:

********************

https://github.com/okuoku/lkl-wasm/commit/a2d87b55e1ba7801fb9decf58cc3d9e58e0600ff

okuokuokuoku

wasm-ld で --start-group --end-group が使えない

。。。マジで。

make -f ./scripts/Makefile.vmlinux_o
wasm-ld: error: unknown argument: --start-group
wasm-ld: error: unknown argument: --end-group
make[2]: *** [scripts/Makefile.vmlinux_o:61: vmlinux.o] エラー 1

というかldscriptも無いのか。。

make -f ./scripts/Makefile.vmlinux
wasm-ld: error: unknown argument: --build-id=sha1
wasm-ld: error: unknown argument: --script=./arch/lkl/kernel/vmlinux.lds
wasm-ld: error: unknown argument: --start-group
wasm-ld: error: unknown argument: --end-group

LKLとしてはどっちも要らないのでskipするようにした方が良いのかな。