Understanding APIs, ABIs, and System Calls

Introduction

This article is a brief overview to help you understand the following concepts in a Linux environment:

• What is an Application Programming Interface (API)?
• What is an Application Binary Interface (ABI)?
• What is the relationship between system calls, APIs, and ABIs?
• How do they differ from one another?

While plenty of information on these topics exists on Wikipedia and elsewhere if you search for it, such resources often contain too much detail for beginners to grasp, and the relationships between the concepts can be difficult to see. Therefore, rather than explaining each term in isolation, I have formatted this as a Q&A for common questions. Please note that terminology can vary slightly depending on the person; this is my interpretation. If you find any blatant errors, I would appreciate it if you could point them out.

What prompted me to write this was seeing this tweet a while ago. It made me realize that many modern languages hide how code and data are translated into binaries, so these kinds of topics aren't very well known anymore—even though knowing them is quite fun.

What is an API?

An API defines the specifications for functions (including methods in OOPLs) and data (including classes in OOPLs) at the source code level. It does not concern itself with how those functions or data are represented as binary data.

Recently, you might be more familiar with Web APIs—which define what kind of response you get when sending a specific request to a specific URL for a web service—rather than just programming language APIs.

Let's explain an API using source code. Suppose we write the following C code:

int plus(int x, int y) {
  return x + y;
}

int main(void) {
  int a = 1;
  int b = 2;
  ..
  plus(a, b):
  ..
}

In this case, the API for plus() is: "If you pass an int value as the first argument and an int value as the second argument, it returns an int value that is the sum of the two." To reiterate, it does not care about endianness or the number of bytes in an int, which depend on the CPU architecture or the C compiler implementation.

What is an ABI?

An ABI defines the specifications for functions (including methods in OOPLs) and data (including classes in OOPLs) at the binary level. It considers details such as endianness and data sizes. Furthermore, referring back to the source code in the previous section, the ABI also deals with the calling convention—how each argument is placed at specific addresses at the assembly language level before calling a particular instruction when plus() is invoked.

For those who have even a slight knowledge of assembly language (especially for the x86_64 architecture), the following information may provide a general understanding of calling conventions:

• Wikipedia page on "Calling convention"
• System V Application Binary Interface "3.2 Function Calling Sequence"

An ABI reveals itself in situations such as:

• When calling a function in a binary-format library from another application or library.
• When the byte order of a protocol used by a program is specified.

What does "hitting an API" mean?

It is slang that means "calling a function according to its API." You don't need to overthink it.

Differences between the POSIX API and system call invocations

The POSIX API is a set of C language function APIs defined in the POSIX standard, primarily created to enhance portability among UNIX-like operating systems. While Linux is not strictly POSIX-compliant, it generally provides the APIs defined in POSIX. On Ubuntu, you can install manpages-posix and run the command man 3 read to read the specification of read() as a POSIX API.

A system call is a method for a program to request the kernel to perform tasks such as hardware operations. For system calls, both the API (for example, running man 2 read to look up the definition of the read() system call) and the ABI are defined.

• System V Application Binary Interface A.2.1 Calling Conventions

Usually, it is invoked by writing the argument values into registers or specific locations in memory and then calling a special assembly instruction^[1].

Among POSIX APIs, functions that need to invoke the kernel must call a system call based on the ABI, but the function name and the system call name do not have to have a one-to-one correspondence. For example, in glibc, calling the fork() function internally invokes the clone() system call. As long as the specifications for fork() as a POSIX API are fulfilled, the implementation detail of which internal system call is used does not matter.

Can system calls only be called directly from C?

Depending on how you define "directly," my opinion is actually that "any language other than assembly language, including C, cannot directly call system calls."

Even in C, assembly language is used internally when calling a system call. For instance, in glibc, when a programmer calls read(), this function uses a feature called inline assembly to invoke the system call via assembly language. Alternatively, one could call a library that contains functions for invoking system calls written in assembly. Whether you consider these methods as "directly" calling them depends on the person, but as mentioned above, I do not think so.

Languages other than C that are said to be able to "directly call system calls" also use methods similar to the ones described above in the end.

• In the case of Rust
• In the case of Go

How do you call functions from one programming language in another?

You use some kind of mechanism that connects the caller language and the callee language. For example, here is how to connect C and Python:

• Extending Python with C or C++

Why is C often used for low-level programming?

Reasons for this vary from person to person, but here is what I think:

• Programmers can handle memory in an almost raw form (conversely, they "must handle it")^[2]
• High ABI compatibility between different compilers and different versions
• Because the language specification is small, implementations are relatively easy to create and memory consumption is low
• Many low-level API standards, such as the POSIX API, are designed for C
• C is often used as an intermediary to call one language from another (see the previous section)
• It is sufficiently mature and has many surrounding tools

Conclusion

If I feel like it, I might write a sequel covering the following topics:

• What is compatibility?
• Library versioning
• ELF symbol versioning

脚注

1. Implementations are conceivable where a system call could be invoked just by writing to specific memory without using assembly-level instructions, but we will set that aside for now.

2. Because of this, C is sometimes called a "high-level assembly language."