iTranslated by AI
Tackling Stateful Challenges in Cloud Run: A Design Example Using Cloud Run and WebRTC
This article is the 6th entry of the NTT docomo Business Advent Calendar 2025.
Cloud Run is a service you would want to use for almost any use case, as it features fast instance startup and is convenient in many other ways. On the other hand, when trying to build stateful applications, you often eventually run into various constraints and end up shifting towards more flexible execution environments like GCE or GKE.
In this article, I will introduce an example of building a stateful chat service using only Cloud Run without using a message broker.
When trying to create a chat service on Cloud Run, you would normally use a message broker such as Redis or Cloud Pub/Sub to implement message routing between multiple instances. However, depending on the use case, you might be constrained by limitations derived from the message broker (such as maximum message size, throughput characteristics, and billing models). Therefore, this time, I will try an approach that directly connects Cloud Run instances using WebRTC without using these services.
Finally, I will include links to repositories for verifying the operation of each configuration, so please use them as a reference.
Using only Cloud Run Service
There are several types of Cloud Run, such as service, job, and (preview) worker pools. I will start by introducing the method of using only Cloud Run service.
The idea is simple: P2P-connect each instance of the Cloud Run service and route messages without a message broker. I use WebRTC for the P2P connection.
Preparations for establishing P2P connections via WebRTC on Cloud Run
Default Cloud Run Service instances cannot be directly P2P-connected via WebRTC. Therefore, I combine a VPC and Cloud NAT as follows:
- Connect the Cloud Run Service to a VPC (Serverless VPC Access)
- Configure Cloud NAT for the VPC
- Use a STUN server to establish a P2P connection (UDP) between instances
When configuring Cloud NAT, you do not necessarily need to reserve a static IP address yourself; automatically assigned NAT IPs are fine. This is because from the perspective of the STUN server, it is sufficient to know "which global IP is the traffic coming from."
If you use werift as the WebRTC library, you must specify the following option when creating the RTCPeerConnection within the Cloud Run instance:
const pc = new RTCPeerConnection({ iceUseLinkLocalAddress: true });
How to connect instances
Signaling is required for instances to establish a P2P connection via WebRTC. Here, I use Firestore as a substitute for a signaling server.
One might wonder, "Isn't Firestore essentially a message broker?" but I make an exception here because it is used strictly for signaling (only for exchanging connection information between instances).
When multiple instances are generated due to scale-out, each instance exchanges connection information (offer / answer) via Firestore to establish P2P connections.
As the number of instances increases, all instances exchange connection information with each other via Firestore, establishing a full-mesh P2P connection.
Chat flow
The message flow when actually chatting is as follows:
-
If clients are accommodated in the same Cloud Run Service instance:
- Simply route messages within the instance
-
If clients are accommodated in different Cloud Run Service instances:
- Forward messages via the WebRTC DataChannel between instances
Disadvantages
Here are some problems with this approach:
Bottlenecks during scaling
If all instances of a Cloud Run Service P2P-connect in a full-mesh, the number of connections increases as N * (N - 1) / 2 for N instances. As the number of instances grows, the number of connections becomes unrealistic, leading to a configuration where the overall load spikes every time a scale-out occurs.
To avoid this, for example, the following measures are needed:
- Build an overlay network and only make hub instances full-mesh
- Use a K-tier tree topology (a kind of ad-hoc SFU-like configuration)
In any case, the technical difficulty level increases significantly.
Lifecycle of Cloud Run Service instances
The lifetime of a Cloud Run Service instance is not clearly defined and varies, but since the maximum execution time for an HTTP request is one hour, it is not well-suited for use cases that require long-term persistence.
- If you maintain long-term P2P connections or persistent WebSocket connections, instance restarts/replacements can cause:
- Potential loss of messages in transit
-
Relatively frequent reconnection processing on the client side
resulting in degraded user experience.
To mitigate this problem, the application needs to implement the following measures:
- Protocol design that includes message retransmission control and ACK
- Client-side automatic reconnection logic, etc.
How to combine Cloud Run Service and Cloud Run Job
Next, to mitigate the two problems mentioned above:
- Bottlenecks during scaling
- Cloud Run Service instance lifetime
I will introduce a configuration that uses Cloud Run Job.
Comparison of the two patterns
First, let's briefly compare the two patterns covered so far.
| Aspect | Cloud Run Service only | Cloud Run Service + Job |
|---|---|---|
| Connection topology | Full-mesh P2P between instances (N^2 connections without ingenuity) | Hub-style P2P between clients and Job instances |
| Instance lifetime | Approx. 1 hour max per HTTP request | Job task timeout can be set up to 7 days (preview) |
| Scaling complexity | Connection count between instances tends to surge with each instance addition | Relatively easy to control if Job instance count and accommodation per instance are designed |
| Implementation difficulty | Full-mesh management and reconnection control tend to be complex | Job startup and allocation logic are required, but the topology itself is simpler |
| Assumed use case | Use cases with limited connection counts | Mid-to-large scale persistent chat |
From here, I will look at the details of the configuration using Cloud Run Job.
Here, roles are divided as follows:
- Cloud Run Service:
- Entry point for HTTP/WebSocket from clients
- Trigger for signaling and Job startup
- Cloud Run Job:
- Termination point for P2P connections with clients
- Message relay (in a sense, "something SFU-like")
Instead of making instances full-mesh, P2P connections are made between clients and Cloud Run Job instances, avoiding the connection explosion between instances during scaling. Also, since Cloud Run Job can set a maximum execution time of up to 7 days as a current preview feature, the instance lifetime problem can be significantly mitigated.
Note that Cloud Run Job also requires the same preparations as described in "Preparations for establishing P2P connections via WebRTC on Cloud Run" as Cloud Run Service.
How to connect clients and instances
In this configuration as well, Firestore is used for signaling for P2P connections.
The number of Cloud Run Jobs to launch is designed according to the use case (e.g., up to X concurrent connections per Job instance).
Since Cloud Run Job does not have an HTTP endpoint, it cannot be accessed directly from clients. However, if signaling for WebRTC can be established via a separate path (here, Cloud Run service + Firestore), the Job instance itself can behave as a normal WebRTC peer.
Chat flow
Simply relay messages using the Cloud Run Job instance as a relay node.
It is possible to design a system where one Job instance corresponds to a "room" or "channel," but this might waste the instance's computing resources, so ensure proper capacity design.
Cautionary points
Unlike VMs, Cloud Run Job has a lifetime and will terminate when the task timeout is exceeded. Therefore, you need to implement measures such as:
- Limiting session lifetimes to about half of the Job's maximum duration
- Notifying the client of the termination in advance when the session ends
- Incorporating reconnection logic that assumes Job rotation (restarts)
to allow for proper service out.
Disadvantages
Currently, there are the following concerns:
- Cloud Run Job is originally a feature intended for "batch processing," and using it for a continuously online relay server is somewhat of an unintended use case
- Jobs are billed for uptime, so assuming "24/7 persistent connections for 7 days" will result in significant costs, and it is often cheaper to run resident processes on VMs (GCE)
- Since it does not support auto-scaling, you need to design scale-out/in yourself
Comparison with VMs (GCE)
All the configurations seen so far were from the perspective of "how much can be done with only Cloud Run?", but in practice, you also compare it with the option of running resident processes on VMs.
Scaling and cost trade-offs
-
Cloud Run
- Can scale in to zero instances when idle
- Fast scale-out is a strength for short-term, sudden traffic
-
VM (GCE)
- Startup is not fast, so you need to keep a certain number of instances waiting, incurring constant costs even when idle
- On the other hand, if connection counts or traffic are consistently above a certain level, it is more advantageous in terms of unit price than Cloud Run's long-running jobs
Differences in operation and ecosystem
-
Cloud Run
- OS and runtime management are almost unnecessary, allowing focus on container image and application deployment
- Log collection and metric integration are GCP-natively integrated, making operation easy
-
VM (GCE)
- High degree of freedom in operation, such as OS updates, security patches, and installation of log/monitoring agents, also increases the complexity
- In return, it offers flexibility to "bring anything," such as kernel-level tuning, custom network stacks, or introduction of existing SFU/messaging products
To summarize roughly:
- Use cases where "connections are zero to few in normal times and surge during peak times" -> Room for creative use of Cloud Run
- Use cases where there is a consistent connection count and you would "be staying connected anyway" -> VMs are often more cost-advantageous
That is the general image.
Conclusion
The two patterns covered in this article are not about "this is the correct architecture," but rather design examples for exploring how far we can push into stateful challenges on an execution platform that assumes statelessness.
The configuration picked up this time can be considered for application in any workload requiring "bidirectional and continuous communication," not limited to chat.
In the use case of chat handled here, honestly adopting message brokers like Redis or Cloud Pub/Sub is likely more advantageous in practice from the perspectives of:
- Simplicity of implementation/operation
- Ease of fault isolation
- Abundance of existing knowledge
However, in situations where you want to handle use cases on Cloud Run where silver bullets like message brokers cannot be applied, there is room to consider solutions by combining ideas like:
- P2P connections between instances (WebRTC / DataChannel)
- Role division of Cloud Run services / jobs
I hope the content introduced in this article will be a hint when designing systems premised on Cloud Run or when facing stateful challenges on platforms intended for statelessness.
Appendix
Discussion