What comes after Serverless?

Introduction

This article is a re-edited version for public release of a piece I wrote for "Memo-okiba TecReport," a technical manual published irregularly by the doujin circle "Memo-okiba."

⇒ Memo-okiba TecReport 2023.12

In addition to this article, we cover topics like "I want to join SecHack365 too!" and "2023 Review and 2024 Technology Predictions" with keywords such as:

Mega clouds vs. specialized clouds / SoC-based hypervisors / Licensing and cloud vendors / Event-driven APIs / AI pair programming in the LLM era / GPUs and NPUs as daily necessities / Passkeys / Web accessibility / Resumption of real-world events

What Comes After Serverless?

Nearly eight years have passed since the technical movement known as "serverless" began to gain momentum. Cloud vendors' FaaS (Function-as-a-Service) offerings have entered a stable phase in terms of running relatively small programs called "functions," and their integration with container ecosystems is progressing. Furthermore, many cloud services that claim to be "serverless" have emerged, regardless of their actual underlying implementations.

In this article, I will review the current state of serverless technology and then look a bit further ahead from several perspectives.

Reviewing the Current State

FaaS, which carries the word "Function" in its name, is so called because the entry point for execution is a function or method. In essence, however, FaaS is a fully managed PaaS that allows you to run arbitrary programs under certain constraints.

In typical FaaS, the actual execution time is measured in milliseconds, and billing is based on resource consumption, typically calculated by multiplying this time by factors like memory usage.

This pay-as-you-go model is significant, and cloud vendors invest considerable effort into making it both viable and profitable. For example, to safely and efficiently run processes for different customers on a single physical machine, they must provide micro-VMs (like Firecracker for AWS). At the same time, to ensure functions can execute when many requests arrive simultaneously, they must manage the number of concurrently running processes and redirect traffic from load balancers. If the current physical machine isn't enough, they must add another machine and spin up new processes.

While cloud vendors handle these tasks "magically" (fully managed), developers are required to write applications that are "cloud-native." As the "Processes" factor in the long-standing "The Twelve-Factor App"^[1] implies, applications cannot save or share data in the file system or memory (they must be stateless and shared-nothing), requiring everything to be offloaded to external databases or cloud services^[2].

This is just the story of primitive FaaS, and various cloud services are realized through software built under these cloud-native constraints. Today, many services are branded as "serverless" by various cloud vendors.

As an author, I perceive "serverless" not as a single definition, but as a term retrospectively applied to services that possess certain characteristics within the evolutionary flow of cloud computing (and computer science). Therefore, I believe that whether something is serverless is not a clear binary 1/0 choice, but rather a gradient.

Characteristics of Serverless

The "definition of serverless"^[3] by Momento, which I introduce here, nicely captures the characteristics that define what makes something "serverless."

1. No provisioning or management required
2. Pay-as-you-go billing with no minimum fee
3. Available via a single API call
4. No planned downtime
5. No instances

If a cloud service satisfies all of these, it can be called "quite serverless," whereas those that satisfy only some are "a little bit serverless."

Considering reality, for example, to accept actual requests, processes that listen on specific ports must be running, and achieving a completely pay-as-you-go model is quite difficult. In other words, cloud vendors essentially "bear the burden" of certain resource consumption.

Even if provisioning is not required and management is fully delegated, there are servers somewhere in reality. If a customer's processes are allocated and auto-scaled on a per-server basis, the unit of measurement for pay-as-you-go billing inevitably becomes larger, counting by the machine.

Realizing a truly serverless-like service is difficult if we think in terms of virtual machines. From the beginning, as a multi-tenant cloud service, processing must be "scattered" and distributed thinly and widely across a massive resource pool.

But this is exactly the definition of cloud computing^[4] itself:

• On-demand self-service
• Broad network access
• Resource pooling
• Rapid elasticity
• Measured service

Therefore, if you can run cloud-native software in a cloud-native way, wouldn't that result in a very "serverless-like" architecture?

Changes in the Main Battlefield of Software Development

In the ancient days of CGI, web application logic was completed on the server side, while the browser side ran JavaScript and the like mainly for rendering HTML and CSS, and for minor decorations like making text glow or move.

Time passed, and the Ajax era, which began with Google Maps, and the web application framework era led by Ruby on Rails arrived. To develop frontend applications on browsers effectively, libraries that made DOM manipulation easy, such as Prototype and jQuery, were developed. Rails also aimed not only for HTML rendering but also for ease of writing REST APIs and integration with Ajax.

Today is the golden age of frameworks based on the Virtual DOM, such as React and Vue. On top of that, frameworks like Next.js have appeared. The main battlefield for web applications, which Rails and others once dominated, has now shifted to the frontend side.

While I will introduce this in detail later, the impact of Cloudflare Workers adopting JavaScript was significant, and a key point for future thinking is that it has further increased the opportunities for frontend developers to contribute to server-side ecosystems.

Frontend and Server-Side Development Productivity

To build a new feature, you need to develop both the frontend and server-side components and coordinate data between them.

The most straightforward way is to define an interface centered on a schema, such as OpenAPI, GraphQL, or gRPC. Various ecosystems are in place, including automatic code generation from schemas. This method is likely standard if you are using different languages for the frontend and server-side.

Alternatively, as a different approach, tools like tRPC allow you to share a single type definition between the frontend and server-side by running TypeScript on the server-side as well (via transpilation) in Node.js.

When the server-side is implemented with multiple microservices, a BFF (Backend for Frontend) is sometimes set up to accept requests from the frontend and bridge the specifications of the backend and the frontend. For example, AWS has a GraphQL BFF service called AppSync.

As seen here, development is required on both the frontend and server-side, and efforts are being made to reduce cognitive load and improve development productivity.

Intrusion of Client-Side Development Languages into the Server-Side

Frameworks like Next.js on the frontend have mechanisms that tightly integrate with the server-side, such as SSR (Server-Side Rendering) and ISR (Incremental Static Regeneration), for performance and other reasons. To leverage these frameworks, there are benefits to adopting the same language and framework on the server-side as well.

To utilize these mechanisms, new hosting services incorporating FaaS, such as Vercel and Netlify, have emerged. Furthermore, a trend is forming to leverage CDN edge computing services for use cases where performance is critical. This is where services like Cloudflare Workers fit in well.

New Approaches to FaaS

Traditional FaaS attempts to run software in various languages, so it performs security isolation (sandboxing) at the micro-VM layer beneath the language environment. However, even with micro-VMs, booting a process from scratch inevitably takes some time.

However, if the server-side can be written in JavaScript, you can utilize the sandboxing mechanisms already present in Node.js or V8 (the JavaScript runtime for Chrome) to execute functions both securely and at high speed. Cloudflare Workers and similar services leverage this V8 sandbox. Because of its high efficiency, they have adopted billing based on CPU time excluding I/O wait^[5].

Approaching from the exact opposite perspective but arriving at the same destination is CDN edge computing using WebAssembly (Wasm). Wasm is a new technology that allows applications compiled from various languages to run within a sandbox. Since Wasm also implements sandboxing as a fundamental specification, it can be executed lightweightly. Examples include Fastly Compute@Edge, which allows you to run applications using Wasm.

In this way, the execution environments for CDN edge computing are based on V8 or Wasm, which assume isolation via sandboxes. Which one will become more mainstream in the future remains to be seen.

FaaS Programming Models

Although they have entered a stable phase, FaaS from mega-cloud providers continues to evolve.

Azure Functions previously used a mostly common interface across all languages, but it is now shifting toward language-specific interfaces for Python and JavaScript. For Python^[6], it provides decorators, and for JavaScript^[7], it offers an interface similar to frameworks like Express.js.

Additionally, Hono, a framework that emerged for CDN edge computing, is also advancing support for AWS Lambda and Azure Functions through its Adapter abstraction.

CDN Edge Computing

CDN edge computing services, several of which have appeared here, have rapidly gained presence over the past few years.

The biggest difference between existing mega-clouds like AWS and Azure and CDN edge computing is that multi-tenancy and multi-region architectures are premises for the entire system. The very concept of a CDN as a content delivery network service started from statistical multiplexing effects through multi-tenancy.

Those who have used Lambda@Edge on AWS may know that although configuration is global, the concept of regions remains internally. When considering multi-tenancy, there was a challenge where the more effort put into ensuring security isolation, the higher the cost of cold starts. However, as already introduced, this is resolved by assuming JavaScript or Wasm sandboxes.

From the perspective of mega-cloud providers, such highly constrained approaches might look like a type of disruptive innovation.

Various Serverless Services

Up to this point, I have introduced FaaS and CDN edge computing environments as operating environments for running your own software, but various cloud services, including data stores, are also solving various challenges as distributed systems.

As per the "Characteristics of Serverless" introduced at the beginning of this article, there is a gradient, but some basic mechanisms are common.

One is the separation of the Compute Layer and the Storage Layer. For example, TiDB Serverless uses Amazon S3 as the final data storage destination^[8]. Snowflake and others share a similar architecture that separates Compute and Storage. WarpStream^[9] takes this to the extreme. It builds an Apache Kafka-compatible data streaming infrastructure on top of Amazon S3, allowing the sender (Producer) and receiver (Consumer) to access data on S3 directly, thereby achieving this without any resident applications running.

Since Compute writes all state to Storage, the remaining task is how to protect the Storage side. The basic patterns for this are also Replication and Sharding. Replication distributes a single piece of data across multiple machines. Sharding distributes the storage destination machine using a specific key contained in the data.

In this way, various data store cloud services are becoming serverless. I will conclude by introducing a few.

For RDBMS, there are Cloudflare D1 based on SQLite, TiDB Serverless mentioned earlier, Neon, PlanetScale, Vercel Postgres, etc. For caching, in addition to the recently popular Momento, Upstash provides Redis and Kafka.

Search systems are also a form of data store. There are services like Algolia and Meilisearch Cloud. Furthermore, there is the field of vector search; while not particularly new technology, it is something for which demand has suddenly emerged in the context of LLM RAG. New services such as Cloudflare's Vectorize and Momento Vector Index are entering the market. Graph DBs are also an old field, but they might attract attention for some reason.

Conclusion

I have introduced various ways of thinking to see a little further ahead into serverless.

As a recap, I will list a few keywords.

• Serverless means utilizing the cloud in a cloud-native way
• Thin and wide architectures premised on multi-tenancy
• CDN edge computing premised on multi-tenancy and multi-region
• Cold start reduction through lightweight language-based sandboxes
• Penetration into the server-side due to frontend convenience (SSR, ISR, etc.)
• Interfaces, schemas, and type management between frontend and server-side

Now, have a happy serverless life!

脚注

1. The Twelve-Factor App https://12factor.net/

2. I am setting aside the discussion of reusing temporary files or cache memory here.

3. https://www.gomomento.com/blog/fighting-off-fake-serverless-bandits-with-the-true-definition-of-serverless

4. The NIST Definition of Cloud Computing https://nvlpubs.nist.gov/nistpubs/Legacy/SP/nistspecialpublication800-145.pdf

5. https://blog.cloudflare.com/workers-pricing-scale-to-zero/

6. https://techcommunity.microsoft.com/t5/azure-compute-blog/azure-functions-v2-python-programming-model-is-generally/ba-p/3827474

7. https://techcommunity.microsoft.com/t5/apps-on-azure-blog/azure-functions-node-js-v4-programming-model-is-generally/ba-p/3929217

8. https://logmi.jp/tech/articles/329530

9. https://www.warpstream.com/blog/kafka-is-dead-long-live-kafka