iTranslated by AI
Is Go for Large-Scale Environments?
This article is a complete rewrite of a post I scribbled in Scraps around May 2021. The original Scraps post has already been made private (archived).
These types of articles (essays) around April or May are like seasonal traditions, so writing one in the middle of summer might feel a bit out of season. However, while ego-searching on Twitter, I thought, "Maybe I should summarize this as a blog post after all," so I'm writing it now, better late than never.
The starting tweet is below:
By the way, I can't give a simple "yes" or "no" to the question in the title. If asked in person, I would likely only be able to say, "It depends." Software is essentially "one-of-a-kind"; while you can speak in broad generalities, it becomes increasingly difficult the more you delve into the details. Choosing a programming language is a prime example of this—ultimately, it's often decided by existing software assets and the skills of the available engineers. Well, if you only reuse what you already have, you'll eventually wither away. Oh wait, is that what's happening in Japan right now? (lol)
Being in a position where you can choose the language during requirements definition is a fortunate thing (for a programmer).
Abstraction of Memory Management and Parallel Processing
A prominent feature of Go is that "memory management and parallel processing control are integrated into the language specification and runtime module."
For instance, a programmer does not need to worry about whether the actual instance of a local variable declared in a function is allocated on the stack or created on the heap.
var global *int
func f() {
var x int
x = 1
global = &x
}
When written this way, the instance of the local variable x will likely be created on the heap (since it is referenced outside the function scope).
func g() {
y := new(int)
*y = 1
}
When written this way, the instance of the local variable y might be allocated on the stack (or eliminated entirely by optimization lol). Even when an instance is created on the heap, its deallocation timing is determined by the GC (Garbage Collector).
Programmers only need to worry about minimizing the reference scope and duration of an instance; the rest can be left to the compiler and GC. Regarding GC performance, as of version 1.9, it claims:
The runtime.ReadMemStats function now takes less than 100µs even for very large heaps.
Regarding parallel processing, Go also abstracts "concurrency" through goroutines, so you don't need to worry about which CPU core, CPU thread, or OS thread it's running on[1]. The Go runtime module creates its own unique threads[2] for each goroutine—units even smaller than typical OS threads—and manages them with its own scheduler. This allows for the creation and execution of a massive number of goroutines, far exceeding the native limits of hardware or the OS.
From this perspective, it certainly reveals a "[fugo-esque](http://www.pitecan.com/fugo.html "Fugo-style programming")" (extravagant) mindset—the idea that "if local optimization is likely to introduce bugs or vulnerabilities, we should just scale up the platform to compensate for the performance gap." However, I think the causal relationship is slightly different from saying it was "specifically created to support development in large-scale environments with relatively abundant CPU and memory."
API Abstraction and POSIX Dependency
An article I recently found interesting is this one:
One of Go's features is its multi-platform support, and this characteristic can be seen as a benefit of such API abstraction. In particular, I think the abstraction of I/O via the [io](https://pkg.go.dev/io "io · pkg.go.dev").Reader/Writer interfaces is exceptionally well-executed.
Moreover, Go's standard packages are not just simple abstractions; they are of a quality comparable to general-purpose frameworks. For instance, even when building a web application, you can accomplish a great deal using only the standard packages.
On the other hand, these API abstractions are designed and implemented assuming POSIX-compliant systems. Consequently, they have the disadvantage of being difficult to use on platforms that deviate significantly from POSIX.
To be blunt, it is impossible to build kernels or device drivers themselves in Go. Furthermore, it is not suitable for strict real-time processing[3]. Let's leave those tasks to [Rust][4] (lol).
There's Always TinyGo
If the official Go depends on POSIX, then why not make a toolchain with less dependency on POSIX? That's the idea behind TinyGo.
TinyGo is an LLVM-based Go compiler for embedded use, and it works well with the popular WebAssembly. It's characterized by its ability to generate much smaller binaries compared to the official Go compiler, but in exchange, there are some restrictions on the features you can use.
By using TinyGo, you can develop small systems for the so-called IoT.
Build Fast, Refactor Fast
Go is a language that, perhaps unusually for a compiled language, is specialized for "building fast." By "building fast," I don't mean the amount of prior study required, compilation speed, or the amount of code written, but rather "whether you can safely build what you've thought of, exactly as you thought of it." Both the simplicity and the constraints built into the Go language specification exist for this purpose.
For example, Go is often compared to traditional object-oriented programming languages like Java, but it simply discards gimmicks like exception handling and inheritance, which only serve as noise when "writing what you've thought of as is." There are clear reasons why there is no priority among goroutines and why sync.Mutex is non-reentrant.
Another characteristic of Go is that it's a language that strongly supports refactoring. Because of its simple language specification, it's easy to modify. Since you can use interface types to keep relationships between objects "loose," it's relatively easy to reorganize relationships, separate highly reusable functions into separate packages, or even "replace a poorly performing package entirely."
In short, I think the biggest advantage of Go is being able to "build fast and fix (refactor) fast[5]." In fact, Go has excellent affinity with things like CI (Continuous Integration) and CD (Continuous Delivery).
If you want the system you're about to build to be refactorable, including Go as a candidate is not a bad idea.
References
-
To say that you don't need to worry about the details of parallel processing in Go means, in other words, that you should avoid imagining naive time-sharing in Go's concurrency, as that can be a source of bugs. ↩︎
-
The unique threads managed by Go on a per-goroutine basis are created with a small stack size of about 2KB and are dynamically resized as needed. Furthermore, from version 1.14 onwards, preemptive multitasking is supported. ↩︎
-
"Real-time processing" here refers to "completing divided jobs at a predetermined timing within a specified duration." ↩︎
-
Compared to Go, Rust increases code flexibility by minimizing the functionality (or responsibility) of the runtime module. Thus, Rust is well-suited for kernel and device driver development. However, "high code flexibility" also means that the responsibility is shifted to the developer. Whether a system can be built to be safe, refactorable, and performant depends on the programmer's skill. ↩︎
-
Reading "改す" as "naosu" (to fix/correct) is not dictionary-correct Japanese. Just for your information (lol). ↩︎
Discussion