iTranslated by AI
re:Invent 2025: Architecting for Hypergrowth: How Skyscanner Scaled to 200 Million Monthly Users
Introduction
By transcribing various overseas lectures into Japanese articles, we aim to make hidden high-quality information more accessible. In this project, which proceeds with such a concept, here is the presentation we will take up this time!
For transcription articles of re:Invent 2025, information is summarized in this Spreadsheet. Please check it as well.
📖 re:Invent 2025: AWS re:Invent 2025 - Architecting for hypergrowth: Scaling to 200 million users w/ Skyscanner-ARC209
This video explains best practices and architectural guidance for scaling applications on AWS. It introduces how to start with a 3-tier web application in the early stages and leverage managed services like EKS Auto Mode and Aurora DSQL. The Skyscanner case study tells the story of their 10-year journey from hybrid cloud to ECS, and then to a cell-based EKS architecture, in order to handle nearly 200 million users per month. Specific technical choices and trade-offs are detailed, such as efficient Spot instance management using Karpenter, caching strategies using CloudFront and ElastiCache, and fault isolation through multi-region configurations. The importance of introducing CPS metrics for cost management and continuous improvement by platform engineering teams is also emphasized.
Note: This article is automatically generated while maintaining the content of the existing lecture as much as possible. Please be aware that there may be typos or incorrect information.
Main Content
The Beginning of the Scaling Journey: Build, Measure, Learn Mindset
Well, hello everyone. Are you enjoying re:Invent so far? Please raise your hands. How many of you came here today because you have an application idea but don't know where to start? Please raise your hands, don't be shy. I see. And how many of you are managing or building infrastructure on AWS and need to learn how to scale? That should be everyone in this room. Wonderful. Understood.
Well, I am really excited. No matter what stage of the scaling journey you are in, I am truly happy to be able to share with you today the best practices and architectural guidance for scaling on AWS. Skyscanner is also joining us today, and you will hear directly from them how they were able to scale globally to handle up to 200 million users. Let's get started.
As you know, it's okay if you don't have an architecture yet, right? Even if you just have an idea, there is no such thing as an architecture designed for high scalability from day one. But you do your best, right? So, it is really important to first get into a mindset of building, measuring, and learning. The architecture you start with on day one does not necessarily have to be part of your final production environment two or five years later, right? So, first of all, we want to build an initial architecture that can probably support tens of thousands of users, then measure while understanding the application's performance metrics, learn from that, collect feedback and evidence, and repeat that process. In other words, build, measure, and learn.
So, let's start from day one. There is no application yet. Perhaps the only application I know how to start with is a 3-tier web application, right? That is, a frontend, a backend, and a data store layer. This is what you can think of when you first start building your architecture. And there is also a cycle of continuous improvement of building, measuring, and learning. So, today we will start with a 3-tier web application. It might not exist a year from now, or it might not work out. We might change to a different type of architectural pattern, but who knows. And you will be able to see that Skyscanner didn't start with their first architecture from day one either. You can hear about how Skyscanner was able to scale over many years later, but for now, let's start from day one.
Day One Architectural Choices: Computing, API, and Data Store Foundations
So, as we begin with a 3-tier web application, we need something to host the frontend and backend, right? Something like compute. AWS has several different options for compute. There is EC2, which are virtual servers in the cloud. There is ECS, EKS, and Fargate, which are for container ecosystems. And finally, there is Lambda, which is a serverless compute option. Now, how should you choose? There are many different options, each with many different use cases. What is best?
You have the most control over settings with EC2, but when you move to ECS or EKS—the container ecosystems—there are still some decisions to consider. Think about things like network settings or security configurations. EKS helps you run Kubernetes at scale. ECS is for container management. And when you move to Fargate, it means you want to run containers without managing servers. And if you choose AWS Lambda, you only really need to worry about the application code and run it when necessary.
So, putting it all together, we can think of our various compute options as a spectrum. If you look at the top, there is AWS Lambda. You only really need to think about managing the application code, but as you move further down, think about Fargate. Then you take on a bit more decision-making responsibility. That is, you need to think about data integration, security configurations, and so on. As you move further down, you take on more responsibility. So, where is the best place to start? In my opinion, it is probably best to start somewhere in the middle layer, because from there, you can take on additional responsibilities. If you go to a lower layer, you can take on additional responsibilities, and this is something to consider when thinking about hosting your application.
Regarding Skyscanner, they used EKS in 2018.
Based on what we learned from Skyscanner, we will choose to start our initial architecture on EKS. EKS is a fully managed service for running Kubernetes clusters. It handles the installation, operation, and scaling of the Kubernetes control plane. This allows you to focus on your application rather than the underlying infrastructure. It is also compatible with Kubernetes tools and plugins.
Now, EKS Auto mode fully automates Kubernetes cluster management for compute, storage, and networking. Think of it as your own dedicated Kubernetes assistant that automatically provisions optimal compute resources and optimizes the infrastructure by scaling node groups based on workload demand. It also features multi-AZ for high availability.
Now that we have decided to run the frontend and backend on EKS, the next thing we want to consider is how to expose the business logic to the frontend. We now have two different pieces. Almost every customer I talk to today is building APIs. When building APIs, there are three different options.
The first is Amazon API Gateway. Think of this as something built specifically for REST APIs. Next, there is the Application Load Balancer. This is a Layer 7 proxy and is excellent for route mapping. And finally, there is AWS AppSync. This is perfect for hosting GraphQL-based APIs. Personally, as I work with startups, I see many building REST APIs with API Gateway, but all of these services are still very commonly used when exposing business logic to the frontend.
So, putting it all together, there is no wrong answer when selecting a service. Think of this as a sort of cheat sheet for choosing API fronting services. If you are looking for a complex API with multiple data sources, choose AppSync. If you need WebSockets, throttling, and usage tiers, and plan to scale to millions of requests per month, choose API Gateway. If you have a single API action or method and need billions of requests per day, choose an Application Load Balancer.
Database Strategy and Completing the Initial Architecture with Aurora DSQL
Now, I have spoken in detail about compute. When thinking about your application, you will need some kind of data store, right? When starting from scratch, one of the things to consider is whether to use a relational database or a non-relational database. And today, there are many different options for different types of databases, such as graph databases, document stores, and so on.
Should you go with a relational database, or something more specific, or a NoSQL database? My recommendation is to start with a SQL database. Now, some of you might be wondering why start with SQL. Let me explain why. SQL is an established, well-known technology. People learn SQL in university. There is a community, existing code, and existing documentation, so it is really easy to start with SQL.
Now, some of you may still not be convinced. You might think, I don't understand why I should start with SQL; I'm going to be handling massive amounts of data, and a SQL database surely won't be able to handle that. Many of you here likely feel this applies to you—you have massive data, you need to optimize for it today, and you don't think a relational database can work.
Let's dig a little deeper into this. What do we mean by massive amounts of data? Are you going to have multiple terabytes of data in the first year? Returning to the build, measure, and learn cycle, you can build now and improve later, right? So, if you don't have multiple terabytes of data right now, you might want to consider revisiting that in a year or two. In other words, those kinds of use cases are actually quite rare.
So, in what other cases would you need NoSQL? NoSQL, or non-relational databases, includes things like document stores and graph databases. However, applications with ultra-low latency requirements might be a case where you want to consider a non-relational database. For example, metadata-driven datasets. This might be something that exists years down the road, but on day one, it might not be something you encounter very often.
Again, if you are starting from zero, this won't apply to most of you, so please start with a SQL database. Amazon Aurora is our relational database service. It is durable, meaning data is stored across three Availability Zones. And it is fully managed, so you don't need to worry about the underlying infrastructure. You also don't need to worry about provisioning or management tasks like patching and backups.
Recently, in the last year or so, we released Amazon Aurora DSQL. This is our distributed SQL database service. It is PostgreSQL-compatible, fast, and familiar. Because it is serverless, you don't need to worry about underlying infrastructure or instances to manage. There is no downtime due to patching or maintenance windows, and it offers virtually unlimited scaling. So, what does that mean? Let's dive a little deeper into how Aurora DSQL scales.
With DSQL, this underlying infrastructure is actually invisible. All you really need to do is create a cluster endpoint in the console and connect to it. Then you can start using the database. But what is happening internally is that the internal architecture is decomposed and distributed. This means that these individual components—the compute layer, transaction logs, and storage layer—are completely decoupled and scale independently.
To summarize, this is our initial architecture: we are using EKS for compute and Aurora DSQL as the data store. Again, this will take us quite far without needing to spend a huge amount of effort upfront. EKS Auto Mode reduces operational overhead, and each of these has high availability and is configured for multi-AZ. So, this can take us quite far. However, with hundreds or certainly thousands of users—and I work with startups—if a startup grows rapidly to millions of users in the coming year or so, that is usually when you start to encounter problems.
Dealing with Growth Challenges: Observability and Kubernetes Auto-Scaling
It might be in your first year, your fifth year, or perhaps right now, but at some point, you will start encountering problems. You might notice different parts of your business impacting others, or database queries becoming slow due to large table sizes or index growth. Therefore, at some point, you will want to rethink or re-evaluate your architecture. Let's dig a little deeper, revisiting the frontend, backend, and data store. For each of these different layers, how do you want to scale and optimize performance?
Now, before we move on, you can't tune what you haven't measured—returning to the build, measure, and learn cycle, we are missing the measure and learn parts before we start building again. That is where Amazon CloudWatch comes in for observability. It is built natively on AWS. You can measure CPU utilization, latency, request rates, and more. There is something called Real User Monitoring, which allows you to collect and view client-side data regarding your web application.
And using the power of generative AI, there is something called CloudWatch Investigations, which surfaces relevant information to help you quickly investigate and resolve incidents. CloudWatch Investigations takes metrics, logs, traces, and other data to generate root cause hypotheses and actionable insights. Now that you have data at your fingertips, you can make data-driven decisions. You might see actually slow database queries or slow API requests that you can address by changing your architecture.
Now, let's learn more about the frontend layer.
This is also one of the ways Skyscanner scaled their frontend. They used CloudFront, which is our content delivery network, built on top of over 700 points of presence around the world. So, you want to cache content closer to your end users to reduce latency.
Now that we've covered the frontend, let's talk a little bit about the data layer. One thing you might be thinking about is going multi-region. When you create a multi-region cluster with DSQL, DSQL actually creates another cluster in a different region and links them. These linked regions ensure strongly consistent reads and writes. There is also a third region called a witness region, which essentially holds limited, encrypted transaction logs and is used to provide durability and availability for these multi-region clusters.
Now, the best database queries are the ones you don't need to run often. That is where caching comes in. With ElastiCache, it's a fully managed service, so you don't have to worry about managing the underlying hosts. You speed up reads by storing frequently accessed data in faster memory locations. Skyscanner actually uses Valkey and Redis clusters to help with caching.
Now, let's learn a bit more about the backend. We can dive a little deeper into Kubernetes auto-scaling. There are two main elements to EKS auto-scaling: node scaling and pod scaling. Pod scaling includes horizontal pod autoscaling and vertical pod autoscaling. Think of horizontal pod autoscaling as scale-out—meaning increasing the number. Think of vertical pod autoscaling as scale-up—meaning making the pods larger. However, if there is no available capacity in the nodes within the cluster, you need to think about cluster or node auto-scaling.
The strategy for Cluster Autoscaler centers on the use of EC2 Auto Scaling groups. Cluster Autoscaler assumes that the instance types within a node group are identical. So, if you have multiple different types of instances or different purchasing options like Spot and On-Demand, you would end up with multiple node groups to support multiple node types. That could actually lead to additional clusters spinning up, making management difficult, right? That is where Karpenter comes into play.
Karpenter does not use node groups or Auto Scaling groups. Instead, it manages EC2 instances directly. Karpenter is an open-source Kubernetes cluster autoscaler that provides dynamic, group-less provisioning of worker nodes. One way to improve efficiency and cost is to intelligently select instance types and consolidate pods to reduce compute costs. This was a very big deal for Skyscanner. When they used EKS in 2018, they managed it using Karpenter. And the best person to talk about this is Paul from Skyscanner.
Skyscanner's 10 Years: Scaling to 160 Million Monthly Users and AWS Utilization
Thanks, Christine. Hi everyone. This talk covers Skyscanner's scaling story over the past 10 years. We'll touch on the scaling context, the problems that actually drive the scale we operate at. I'll talk about our 10-year journey of our compute platform. We'll touch on some day ones during that journey. I'll talk about flight live pricing, which is one of the largest workloads running on our platform. And I'll also talk about the cultural and organizational scaling tactics we use.
Skyscanner is a global, trusted travel marketplace. Every day, we connect millions of people to trusted partners for flight, car rental, hotel, and package searches. We work with over 1,200 global partners. Visit Las Vegas is one of them. And at the beginning of this year, we had 160 million monthly users, and we've grown quite a bit throughout this year.
We support over 18 million unique flight routes. And in a single day, it's not unusual for 100 billion prices to come back from all these partners. At peak, there are 5,000 requests per second. And in a single month, there are about 1.1 billion searches. So, there is a huge amount of traffic, and we need to scale our architecture to support that.
Flight meta-search is what drives the scale we operate at. From a UX perspective, it looks simple, so you might wonder what's difficult about it. But this has a really high cardinality problem. There are billions of unique ticketable flights per year. There is a huge number of partners to integrate. There is petabytes of data ingestion coming in from these partners. And we see large traffic spikes due to seasonality, marketing campaigns, and other events. Therefore, our architecture must handle volume, volatility, and variability all at the same time.
We leverage AWS to help us scale. We position AWS as our technology partner. From our perspective, we value the global reach AWS provides, their focus on resilience, their ability to scale, and the breadth of services offered. Christina already talked about many of the services, didn't she? There is also an innovation pipeline behind those services. One thing that is clear in our architecture is that we are not dependent on a single region.
Skyscanner of yesterday was a .NET monolith on SQL Server on very heavy server boxes in our founder's loft. It's now sitting in our Edinburgh reception, so you can see how much we've evolved over these 14 years. Today's Skyscanner is a cloud-native system. It is composed of about 300 Java services. We use 4 active regions and 12 Availability Zones. We have 24 production Kubernetes clusters and run over 37,000 cores across 250+ different instance types daily. We handle 400,000 requests per second across the service mesh of all clusters and use hundreds of terabytes of cache per day.
We use many AWS services. You'll see a lot of familiar ones there. Christina already talked about some. We also use Bedrock, but this isn't a talk about AI, unfortunately. I won't touch on that, as there has been plenty of AI talk this week. The key point of this list isn't about individual services. It's about how we combine them, how we configure them for reliability, blast radius management, and cost.
Evolution of Compute Platform: From Hybrid Cloud to Cell Architecture
Now, let's talk about compute scaling. Compute is the backbone of our scaling story. I'll describe the four generations of our compute platform. We'll progress from a hybrid cloud using Auto Scaling Groups and EC2, to large-scale ECS clusters, large-scale Kubernetes clusters, and finally to a cellular EKS architecture.
Skyscanner V1, this was around 2014 to 2016. It was a very pragmatic use of AWS. It was for burst capacity for our partner scraping environment. We ran Classic Load Balancers, EC2 Auto Scaling Groups, and were using CloudFormation almost from the start. This architecture allowed us to scale up to about 100 services. Traffic to the data center went through HAProxy, which abstracted service endpoints and handled failover when Direct Connect had issues. The key point here is that we started from hybrid, and used AWS as a pressure valve for growth rather than a big-bang migration.
Now we move to V2. Containerization arrived, and by 2018, we had deployed about 300 services and hundreds of Lambda functions. These services were deployed to centrally managed ECS clusters. Around this time, we built our own continuous deployment system called Slingshot, which allowed us to scale up the number of deployments significantly. However, there was one problem with this architecture. Since it was the era before ALB, we needed an ELB for every service. This turned out to be quite costly, which was one of the drawbacks of this simple approach. Another lesson learned here was that using a large number of microservices can lead to unmanaged growth. A lot of complexity starts to creep into the architecture.
Now we move to V3. This is the era of Kubernetes. We primarily adopted Kubernetes to introduce Spot into our computing platform. Around 2019, we had about 500 services. We still had hundreds of Lambda functions, but we were performing about 10,000 deployments a month. We operated ECS and Kubernetes in parallel and used centrally managed clusters. By this point, we had migrated most of our flight-related workloads from the data center. We gained efficiency and speed, but the drawback was what we called "mega clusters."
In essence, they were huge failure domains. And in mid-2019, we reached a turning point.
We faced the problem of having two different container scheduler technologies. We had multiple mega clusters, and the number of significant outages was increasing. The clusters had become too large, essentially reaching a state where failure was not an option. Developer experience wasn't great either. There were too many ways to deploy, and platform operations were fragile. So, inspired by AWS's efforts on fault isolation, we considered blast radius and recognized the need to reduce the unit of failure. This is a great example of practicing "reinventing" by learning from AWS and being able to talk to their leaders and service teams.
Cellular Kubernetes Platform and Efficiency with Karpenter
Now, Skyscanner V4, which is our current architecture. This is about five years old. We have adopted a cellular Kubernetes platform. We use Istio's cross-cluster service mesh. We operate regional accounts and have multiple EKS cells per region. We manage the service and cluster lifecycle using GitOps and use Argo CD. We use standardized AMIs and container toolchains. We still perform about 10,000 deployments per month, but since the number of services is nearly half compared to 2019, the throughput is significantly higher. We have set upper limits on cluster size and use NLBs at the edge to send traffic into the mesh.
In concept, we traded mega clusters for a fleet of small, configurable cells. So why use cells? It's because it provides guarantees. If a failure occurs in one cluster, it only affects 1/N of the total capacity. We limit cluster sizes, perform upgrades incrementally, and apply an N+2 deployment policy to services. This means services are deployed to at least three clusters within a region, and for large services, it can be up to five. This allows us to lose availability without multiple cluster failures, and in most cases, handle it without relying on full regional failover. This model is designed for partial failure as the normal case, not an edge case.
In this cell architecture, compute is supplied by Spot. In a normal state, 95% is Spot compute, and 5% is On-Demand, used for critical functions like traffic ingress. We rely heavily on Spot features, such as rebalance recommendations, placement scores, and Karpenter. Karpenter turned out to be an amazing technology for us. It significantly reduces the complexity and effort of operating a large Spot fleet. The trick to Spot is aggressive diversification. We use over 250 different instance types across four regions and 12 AZs, which is why it works for using Spot at scale. This DevOps ART diagram shows 24 hours of Spot instances across the entire fleet.
We also learned painful lessons while operating the cell environment. In 2021, an improperly configured push resulted in the deletion of application namespaces across all 24 clusters in a matter of minutes. It was effectively an "RM minus RF" for Skyscanner. For those interested in the details, we have published a full report. This incident forced us to take the simplicity of the control plane, the scope of configuration impact, and operational drills seriously.
To rebuild resilience and trust, first, we minimized the deployment of global configurations that could be scoped to services or cells. Second, we conducted regular backup and restore exercises using new runbooks. At that time, we weren't using EKS, so we had to manage our own etcd database. Third, when adding logic to templates, we treated them as code since they were no longer just configuration; meaning unit tests, linting, and comprehensive reviews. And finally, we continued to invest in incident commanders. This allowed engineers to focus on fixing rather than coordinating.
The lesson learned from operating the cell environment is that the control plane is the most important system you will build, and you should not underinvest in it. We underestimated the effort required to migrate hundreds of services from legacy clusters. It is definitely a marathon, not a sprint. It took several years to complete. Cells introduce an over-provisioning trade-off. Small services can be over-replicated, and large services will end up being placed in over 20 clusters. This is one of the reasons we use Spot. And excellent observability and standardization are essential to prevent the cell architecture from becoming unmanageable.
Flights Live Pricing: Microservice Architecture Processing 5000 Requests Per Second
Next, let's talk about Flights Live Pricing, as this is one of the largest workloads we run in our cells environment. It processes about 5,000 searches per second.
This generates 100 billion prices per day, with 70 gigabits of data transfer occurring every minute in individual regions. In other words, a massive amount of activity is happening. If Flights Live Pricing is unavailable, users cannot search for flights, making it a P1 incident.
Scaling is not just a compute issue. To scale Flights Live Pricing, there are many additional support systems. I will talk about those now. As Christina mentioned, we use CloudFront. We use Route 53 to bring global traffic into our regions, which resolves to multiple CloudFront distributions. CloudFront is for CDN functionality, where we perform caching, header control, and URL rewrites. There is also edge security provided by Amazon WAF. These are stages before traffic reaches our regions.
Next, there is an additional layer of Route 53, which uses weighted DNS and health checks to shift traffic between regions. Within a region, an NLB directs traffic to cell clusters, from which Istio handles last-mile routing to individual pods. In 2015, we made a very pragmatic decision to operate our own NAT instances. This is because we have petabytes of data ingestion from partners. While Managed NAT is a great service, it was not economically viable for us.
If an issue occurs with a NAT instance, for example, if it becomes saturated or faces other problems, we fail over to Managed NAT. However, for all traffic between the EU and the US, we mainly run on Graviton with network-optimized EC2. We also use Bring Your Own IP, which makes it easier to manage IP ranges for partner APIs. The key point here is that you don't need to make everything managed. You can think pragmatically and choose your own path. For us, this is about cost management.
Caching is the largest data store use case at Skyscanner. We use hundreds of terabytes per region. Flights Live Pricing is one of the largest users among them. We operate multi-AZ Redis and Valkey clusters. We are currently in the process of migrating to Valkey, and the responsibility for these caches lies with the service teams themselves. There is no central management of these stores because they are part of the services themselves. Conceptually, we are caching partner quotes, constructor itineraries, reference prices, geographic data, and much more for flight searches.
Skyscanner is a data business. We send about 55 billion events per day to our data lake, which is under management of about 25 petabytes of data, with a large portion coming from Flights Live Pricing. There are regional Slipstream endpoints. This is a custom Go service operating in the cell environment that writes compressed micro-batches to multiple Kinesis streams. However, we learned that not all data is equal. Therefore, we use different streams with different SLAs and quality profiles.
On the consumer side, we have Spark Structured Streaming jobs running on Databricks that consume from all regions and write to Delta tables and S3. There, we use Intelligent Tiering for storage cost management. Summarizing these, Flights Live Pricing is a microservice architecture. While the diagram here is considerably simplified compared to the actual one, it should give you a hint of how everything works together. We have sessions, and a Flights Pricing Service. This is where traffic enters, and you can see the cache being used there. Basically, we are caching sessions for searches.
Itinerary Construction is one of the more computationally intensive services we have. It does a lot of the work in assembling itineraries and is what is actually presented to you. We also handle ranking. We have a series of services that manage quote retrieval, and the "Who to Ask" service is all about routes. It actually manages the routes that we search for you. Then there is an Egressor service, which holds all the logic for partner integrations, where NATs come into play to actually communicate with partners.
Scaling Culture and Organization: Open Specifications, Cost Management, and Platform Engineering
Raw technology alone does not scale. You also need to think about abstraction and culture. At Skyscanner, we have a strong principle of prioritizing open specifications. This has driven our adoption of Kubernetes, OpenTelemetry, Delta Lake, Istio, Karpenter, and other open-source projects. This gives us portable and well-understood abstraction layers at critical points of compute, networking, and data.
At our scale, cost is a top priority. We track costs using cost per 1,000 sessions. We call this CPS. This is our primary infrastructure metric. CPS has become a common language between engineering and finance, and we use it to set annual goals. These are reviewed monthly by all engineering groups, and each Skyscanner team owns their own spending.
Each team owns their own spending. Because this is managed through common tools and tagging policies, costs are delegated, not centralized.
We adopted platform engineering before it became popular. This was essentially started out of necessity in 2018. Today, 40 engineers operate and evolve our production platform. They cover cloud infrastructure, compute, global traffic routing, observability, CI/CD, and SRE enablement. Approximately 50% of their time is spent operating these systems, another 40% is spent on their improvement—essentially product work—and 10% on learning and development. In other words, we aren't just keeping the lights on in our production platform. We are continuously moving it forward every day.
To summarize, here are the lessons we've learned over the past decade. Have an opinion. You need to select a minimal set of technologies that meet your scaling and resilience needs, then standardize and harden them. Manage blast radius; explicitly decide how far you allow failures to propagate. Speak the language of business. Scaling isn't cheap, so become well-versed in costs and value so you can have real conversations, and make cost everyone's responsibility. Above all, be pragmatic. Even an imperfect architecture can handle a surprising amount of traffic. The art lies in continuously improving without stopping the world. This is how we grew from a server in a loft to a multi-region, cell-based platform serving nearly 200 million users. I'll pass it over to Fraser. He will talk about different scaling patterns and how they can be implemented using AWS services.
The Inflection Point at 1 Million Users: Migration to Microservices and Database Partitioning
Thank you, Paul. It's great to hear about Skyscanner's 10-year journey on the cloud. Now, let's circle back to some of what Christine covered. Ten years have passed, we have about 1 million users, and we are at a bit of an inflection point. First, needs for new features emerge. Business needs are changing. Look at Amazon.com; things like personalization are added, helping business growth. And along with that, new infrastructure needs arise. Your beloved database probably has a single writer instance, which is starting to become a real bottleneck and issue for operations. And finally, monolithic architecture is really bothering you. It's becoming a large single point of failure, slowing down development and operations. So, what can you do about this?
First, we'll look at microservice architecture. There are many sessions covering microservice architecture at re:Invent, but essentially it is the act of breaking down a monolithic or large-scale application into smaller components. There are two main ways to do this. The first is data domain mapping, which involves looking at all your data stores, data structures, schemas, etc., finding commonalities, and splitting along those lines. Alternatively, you can split by business function. As you do this, it becomes an inflection point to also re-examine your compute platform. Paul mentioned earlier that they started with ECS and then evolved to EKS. Those needs will change. Technology that was working very well at one point might start becoming a hindrance from another point, and that depends heavily on your application and architecture. Finally, when operating many microservices, you need to think about how to mesh them and how to couple them.
Next, we want to look at the database. Databases are truly critical to applications. They store all your very important data, but at some point they become too large and you need to start splitting them. When splitting, you will likely look at things by function or purpose. In this example, we have a forums database, users database, and products database. This doesn't help if you have very unoptimized queries or massive tables. Looking at those cases, often you are actually using the wrong technology. Initially, you needed a quick way to get analytics or data, and you might have done it with a relational database. It works well up to a point, but then it becomes a huge headache. So, think very carefully about the appropriate technology you are choosing. A custom-built data warehouse scales much better than a relational database.
Another thing to consider is NoSQL. Christine explained this in quite some detail earlier. One thing we hear from many customers is when to start considering NoSQL or DynamoDB. DynamoDB excels at large-scale scaling with very low latency. Really good use cases include key-value data stores and metadata. However, this is not a one-size-fits-all approach. We heard a lot about AI this week. AI will significantly change data needs. In how we do things, data will evolve in a massive way. And you will be getting much more data in both structured and unstructured forms, so think carefully about the database technology you choose and how it will grow.
Another thing to look at is how to split backend services. Broadly speaking, when doing this, look to reflect the data layer. Look at those data patterns and split that way. This is another inflection point in your compute platform as well. There is no right answer to this, so please make it work for your specific use case. Next, look at business logic—what truly makes your business function. And consider migrating from synchronous to asynchronous communication. We will cover this in the next few slides. Finally, look at technologies like queues, buses, and streams that help build event-driven architectures.
A really important thing to remember is that as you scale, everything becomes more difficult. Microservices add an extra layer of complexity on top. You have more to manage, more to think about, and more things existing. It becomes more operationally difficult, but you gain a lot of benefits from it.
Migration to Asynchronous Architecture: Leveraging Queues, Streams, and Buses
Now, let's talk about thinking asynchronously. Here is a diagram that helps explain it. On the left side, we have a synchronous command where a client calls Service A, which then calls Service B. If there is a problem with Service B, it doesn't necessarily return a response to the client, leaving it hanging. If you make it asynchronous, the client only accesses Service A, receives a response, and accesses Service B separately. In other words, if there is a problem between Service A and Service B—perhaps it's a network issue, or there could be some issue or delay with Service B—the client still receives a successful response.
Let's look at a more relevant example. Suppose you have an e-commerce application. You have an orders API, a client posts to it, and accesses an invoice generation service separately. Apart from that, the client can also access the invoice service directly, but even if a problem occurs with the invoice service, orders can still be placed. You can receive money, generate revenue, and run your business.
Migrating to an asynchronous architecture is an investment that requires more time. This is because you need to truly understand the data and the various commonalities associated with it. Understanding communication between things is also very important, as are the configuration changes you need to make. By doing this, you gain a deeper understanding of your application and its architecture.
As you aim for growth, you need to consider how to connect things and how to decouple them. For this, look at things like topics, streams, queues, and buses. Here are four services: Amazon Simple Notification Service (SNS), Amazon Simple Queue Service (SQS), Amazon EventBridge, and Amazon Kinesis Data Streams. These all do slightly different things and each has different use cases.
If you're wondering what to use, for example, if you have high data throughput, need ordering, might have multiple consumers, and need the ability to replay data, Kinesis Data Streams is very suitable. If it's one-to-one, without much fan-out, and you want to access targets directly, use SNS. If you need functionality to buffer requests in a queue and have them consumed, and you want to choose whether to order them or not, SQS is a very good service. And finally, if you have one-to-many fan-out with many different targets and schemas, EventBridge is a good choice. It is worth mentioning that you can use several of these in combination. This is not a one-size-fits-all approach, and there are very good integrations between these. It depends on your architecture.
Cell-Based and Multi-Region Architecture: Designing Failure Boundaries and Best Practices
Now, let's look at microservices. This is a fairly typical microservice architecture that we see often from customers. Don't be afraid. There are many lines and many services, but this is fairly typical. Let's walk through it. On the left side, we have Amazon Route 53, which provides a 100% SLA on the data plane to resolve DNS. It can also help with health checks and routing. We have CloudFront, which accelerates the end-user experience and caches static content. And then we enter the compute layer of EKS, deploying to various microservices and data stores from there.
Everyone's architecture is unique. Your choice of data store is unique, your choice of compute is unique, and how you use it is unique to your business. Skyscanner has achieved great success with EKS and Spot, realizing cost-efficient resilience. As I've said many times, it really depends on what you need.
Another concept Paul touched on was the concept of cell-based architecture. This is something we use quite frequently at AWS and Amazon for our own services. This involves deploying a full copy of the application into cells. Cells are failure boundaries. This is done to reduce the scope of impact from failures. You partition data, effectively sharding it, and achieve full separation between cells. There are solid bulkheads between them. On top, there is a routing layer, which will have high resilience and availability. Many people look at something like Route 53 because it has a 100% SLA on the data plane. However, at some point, you need to be able to scale this. You need to know how big they are. So, you need to balance the management overhead of having cells with how much failure you can tolerate if an incident occurs.
Another thing Paul touched on briefly was multi-region architecture. Many customers consider multi-region architecture for various reasons. It could be for regulatory reasons. If you are in a regulated industry, you might need a certain uptime availability. You might have another regulatory need to keep data in a specific country, or you might simply want to be closer to your customers and provide lower latency. There is one key point when having multiple regions: design your architecture considering regional independence. If a problem occurs in one region, it shouldn't affect another region. Keep them separated. Avoid mutual dependencies. Make writes idempotent. And like cells, this creates additional operational overhead. Recognize that before you start. Think carefully about the trade-offs you really need, what you need to comply with, and what your availability needs are. There are many talks on multi-region architecture that you can hear this week and watch online. If you are considering such a journey, I recommend checking them out.
Now, let me briefly talk about best practices that can be applied to both cell and multi-region architectures. First, when deploying code or updates, make sure to leverage regions or cells. Deploy very gradually, very incrementally. In other words, deploy only to a small number of areas, one cell, one region, a small number of users, and if a problem occurs, roll back quickly. Shrink the scope of impact. This is how AWS performs deployments; we do it very gradually. Also, conduct performance testing. This is an area where many customers struggle, but performance testing is a truly great way to understand scale. It also helps you know at what point you need to consider migrating to another cell or another region, what starts breaking, and what starts causing problems.
Christine mentioned observability earlier, and this is a truly critical element. Make sure to adjust it to fit your failure boundaries. If a problem occurs in one cell, how do you know what is happening in that cell? Make sure it is clearly identified, tagged, and marked. And finally, when performing tests or monitoring, focus on results. Do you really care that much if there is a problem with an individual EC2 instance? Probably not. What you care about are business results. Can users find products? Can they place orders? When performing tests, focus on these results. It's about what the final experience is for the end user.
The Essence of Scaling: Do Less and Focus on Resilience
Today, you've heard many things that will help you scale your architecture to the next level. Much of this depends heavily on your business needs, growth requirements, and how you intend to move forward. However, these things will enable you to really scale to the next level for a long time to come.
Finally, compared to five years ago, we are in a much better situation in terms of the availability of managed services that we can use right away and scale more efficiently. Across the entire stack, there are far more resources available. It has become far easier for you to do so. The best way to scale is to do less. As Christine mentioned, the best query is the query you don't run. Use caching so you don't actually have to do the work. Let it do the work. It is there to be hit. It is there to take on that heavy lifting. You can reduce the range in which your database is actually queried and processed.
Refactoring is a big investment. You need to spend a lot of time on it. It also comes with several challenges along the way. So, think carefully when executing, and consider various trade-offs. As you move forward, look for the optimal technology based on what you need and be flexible. Focus on application resilience and availability. We are in a digital world, and people expect to be able to do things on-demand when they want to. Any downtime can cause significant damage to your business or brand, creating even bigger problems. Design your architecture around these failure boundaries. Make sure to understand the scope of impact. If a failure occurs, what will be the scale of that failure?
Thank you for attending our session today. Christine, Paul, and I will be outside for a while after we finish, so if you have any questions, please feel free to ask. And please fill out the session survey in the mobile app. Thank you.
This article was automatically generated using Amazon Bedrock, maintaining the information from the original video as much as possible.
Discussion