iTranslated by AI
re:Invent 2025: Marketo's Large-Scale AWS Migration Case Study and the Journey to Data Center Closure
Introduction
By transcribing various overseas lectures into Japanese articles, we aim to make hidden, high-quality information more accessible. The presentation we are featuring in this project, based on that concept, is here!
For re:Invent 2025 transcription articles, information is compiled in this Spreadsheet. Please check it as well.
📖re:Invent 2025: AWS re:Invent 2025 - Marketo's Digital Transformation (ISV201)
This video explains the large-scale migration of Marketo, Adobe's B2B marketing automation solution. Marketo was acquired by Adobe in 2018 for $4.75 billion, and having acquired seven companies prior to that, it became a complex project to migrate a collection of different technology stacks to AWS. The project proceeded in stages, starting with a technical feasibility study in 2022, migrating a Hadoop cluster to Spark streaming on EKS, and moving 13 types of Mongo to Mongo Atlas. In 2024, 40 terabytes of data from the largest customer were migrated, and data center closure is planned for 2025. By migrating approximately 170 MySQL 5.7 clusters to Aurora and Solr 8, which ran on 5,000 VMs, to Solr 9 on EKS, performance improvement and scalability were achieved. The utilization of AWS Professional Services and the Migration Acceleration Program were key to the success.
- Please note that this article is automatically generated while maintaining the content of the existing lecture as much as possible. There may be typos or incorrect information.
Main Content
Marketo's Large-Scale Migration: The Challenge of a B2B Marketing Platform Integrating Multiple Acquired Companies
Hello, welcome, and thank you. First, if I could just have a show of hands, how many of you in here have used Acrobat for PDFs, Photoshop, Marketo, or any other Adobe products? Basically, everyone. We're a very close, tightly knit family. Thank you all for joining us. Let me ask another question. How many of you here are working on large, complex migrations for your own companies, for your customers, or for your partners? Excellent. And how many of you are working on ambitious agentic AI projects? About to start, or already operating large applications? Perfect.
Well, you're all in the right place. Over the next 20 minutes, Gaurav and David will share two core messages with you. One is, if you're running complex migrations, best practices, learnings, and how you should approach large, complex migrations. And two, that migrating to AWS is not only just relevant, but almost essential for us to succeed with our AI workloads. If you have any questions, we'll gladly stick around and answer questions on both these points.
But before we start, let me just briefly introduce what Marketo actually is. Marketo is Adobe's B2B marketing automation solution. Every single re:Invent email you've received, or if your marketing team sends out emails to customers and partners, it's very, very likely done through Marketo. Over 5,000 customers use Marketo across five data centers and three continents.
But what makes this entire migration truly special is this: if you look at Marketo, many people might think it's a product, but Marketo itself was fundamentally a company that Adobe acquired in 2018 for $4.75 billion. And before that acquisition, Marketo as a company had acquired seven additional companies, so this wasn't just migrating one application to AWS. It was migrating a collection of different companies' different technology stacks to AWS, and some of these challenges don't get solved in a week, or a month, or a year. It takes an entire journey to migrate a complex application like Marketo to AWS.
And to share that entire journey, and also to understand the key takeaways of how you can migrate faster and more securely, let me invite David to the stage. Actually, I'm quite surprised I'm here. I'm not surprised that they came to me a few days ago and said, David, you need to present, but when we were acquired, we actually had four data centers, not five. And the first thing Adobe did was build a fifth on-premise data center. So when Adobe acquired us, the idea of moving to the public cloud was completely off the table. We were a private cloud-based company, and that was it.
To further emphasize that point, when Adobe acquired us, we had just finished migrating to another public cloud platform. So, we weren't very enthusiastic about doing this. So, what changed? I'll tell you about that a little later, but here you can see what our journey looked like. In 2022, we conducted a technical feasibility study. In 2023, backend services started migrating little by little. In 2024, we actually migrated customers to run exclusively on AWS, and by 2025, we're ready to close one of our data centers and the rest over the next two to three years.
So what changed? Why did we do this? Well, let's start with the third point there, legacy infrastructure challenges. When we built that fifth data center, it was really hard. We hadn't built a data center in a long time. We had customized versions for specific things. Our biggest problem was Hadoop. Getting a Hadoop cluster up and running was a nightmare. I'll come back to that a little later.
Of course, other reasons include global scale and volume. Data centers are great for known workloads, but as your workloads change and your customers grow, data centers have fixed resources, and you're essentially stuck with them. On the other hand, with AWS, you can adjust things, you can improve them. That's one reason. Another reason is zero downtime, or very close to zero downtime. If AWS goes down, that's global news, right?
If Marketo goes down, unfortunately, it's not news. And the data center we closed this year actually experienced a catastrophic failure in 2023, so customers in that region were not happy at all. They were delighted to hear we were moving to AWS. And the fourth reason is truly looking to the future. With AWS, you have access to the entire agentic environment, and that opens up so many possibilities.
Phased Migration Strategy: From Hadoop to Spark on EKS, and Achieving a Fully AWS Environment
So let's get into the details. We didn't migrate everything at once. In fact, when we migrated the first things, nobody even noticed we were migrating to the public cloud. So, let's go back to my friend Hadoop. Time passed. Now we had five unwieldy Hadoop clusters, and our security team told us to upgrade them, but we couldn't. We talked to the vendor. They're right there. They couldn't either. They said they'd build a new one. And that was the last thing we wanted to do, which was to invest more hardware to build another customized one.
So we really dug deep and thought. What does Hadoop give us? And for us, the biggest use case was Spark streaming applications. So in 2022, we ran one of our proofs of concept. It was to run Spark streaming on EKS, specifically Adobe's custom version of EKS, known as Ethos. After proving that it worked, we migrated 20 Spark streaming jobs from our data centers to AWS. And nobody noticed. See, 20% of our workloads are now quietly running in the cloud.
Next was Mongo. We had 13 different types of Mongo for different applications, and we had two people managing them. That wasn't scalable, but fortunately, we had a good relationship with Mongo. Hi Lauren, and they convinced us that Atlas was the way to go. So over the last couple of years, we migrated all of our individual Mongos to Mongo Atlas. This is also running on AWS. So now about 40% of our applications are running in the cloud, but nobody really noticed anything. In fact, they felt the benefits, but they didn't know where it was coming from.
Moving forward to 2024, actually late 2023, one of our largest customers, who might even be at their conference right now, told us they wanted us to run Marketo on their public cloud platform. I thought, "What?" But then I realized, considering we were already running in a hybrid-bridged environment like it says here, it was just a matter of taking the next step. So we had to figure out what applications we needed to run entirely on AWS to support the main application. We did that work and spun up those applications in AWS and in our partner data center. And we built the pods and migrated the customer. This was a huge feat, really.
AWS was our largest customer by some metrics. The slide says 40 terabytes of data, but when we started the process, it was closer to 90. So just to migrate them, we had to reduce the database, and that took some convincing. But eventually, because the application ran faster, we were able to reduce it to 40 terabytes. And how do you move 40 terabytes from one place to another, with zero downtime? That required a lot of tooling. A lot of, okay, let's take it from Pure Storage here to a Pure Storage appliance on AWS, rebuild it on RDS, and start syncing data from the data center.
So months later, yes, it took months for the data to catch up, because that's how much activity AWS does daily. Once it caught up, we performed the cutover. We shut down the on-premise stuff, did a bunch of metadata transformations, and poof, AWS started running on AWS. We jumped for joy, wiped our brows and said, "What's next?" But we didn't actually do that. We wanted to leave it alone for a while.
And AWS ran on AWS for several months without issue. Performance improved, campaigns ran faster. I thought, well, how do we scale this? So we went back to the problematic data center. Because real-world considerations come into play here. The real world of data centers is that they're a big investment. And our APAC region was due for renewal in February 2026.
As you may have noticed, it's not February 2026 yet, but when you're migrating from something, you need to plan well in advance. So we started this work at the end of 2024, looking at what we needed to do to have a fully AWS-only data center. In these hybrid regions, some of our infrastructure, like load balancers, were running on-premise, and the databases in the pod were running on AWS, but here, we don't have a data center to support us. So we had to figure out all the other components and how to migrate them, and we did that over time.
Once everything was running in the AWS APAC region, we placed some test subscriptions there. Everything seemed fine. We migrated some test customers, and that was fine too. And then, we needed to migrate at scale. Now, the first time we migrated to AWS, it was a single instance, but now we're using pods. The core unit of Marketo is a pod, with a front end, a back end, a database cluster, and about 100 to 200 customers in one pod. So instead of migrating them one by one, we're migrating them in batches. The data movement is the same, but the metadata transformation is a little different. And we're moving into the future.
Leveraging Managed Services and Partnerships: Operational Efficiency with Aurora, ElastiCache, and Mongo Atlas
So the future is to decommission all data centers and be fully AWS native. That gives us all the wonderful things that AWS brings. This shows how we paired our data centers with AWS Regions. This was essential for our hybrid approach. If you look at Australia, that red dot should disappear, and it will. So, we've come this far. What has changed? Well, how have we leveraged managed services?
We have about 170 MySQL 5.7 clusters. Together, that's about 2 petabytes of data. We are migrating that to Aurora over time. And as I said earlier, we get much better performance from Aurora right out of the box. But what's great is that we scale up our database so that AWS can market to all of us and send out all the emails to make this event happen, like when re:Invent is taking place. When the event is over, we scale down. This is something we can't do on-premise, so that in itself is wonderful.
I talked about Hadoop, so I don't think I need to say much more about Spark on EKS. Solr 8 is our biggest footprint in the data center. We're running Solr on 5,000 physical VMs. Solr 9 can run on EKS, but it behaves a little differently. However, it runs better and faster, so this is a big win for customers because we use Solr to speed up search. Not surprising, right? For Redis, we were big fans of Redis, but not having to manage Redis and letting ElastiCache take care of it really lightened the load on my team.
And finally, I mentioned Mongo, and they are worth mentioning more than once, because the time and energy they saved my team, and the expertise and tools they provided when they practically performed the migration for us, make them an invaluable partner. Now, let me talk about another invaluable partner, AWS, because we couldn't have done it without their help. The first line is MAP funding, the Migration Acceleration Program. I won't talk about it, but feel free to ask the AWS folks how it works.
Again, it's a proven migration methodology. They've migrated people before. Certainly, what we did they hadn't done before, but they brought a lot of knowledge to the table. They brought the ability to say, oh, here's a bottleneck, let's resolve it. They are truly excellent partners. Much of that work was done in conjunction with AWS Professional Services. They did a lot of the heavy lifting in automating the migration from Solr 8 to Solr 9. They helped us create a software load balancer to replace F5. They did many other things. So, thank you, AWS Professional Services.
And one of the advantages, which doesn't actually exist in my world, but when I talk to the sales people, the ability to co-sell with AWS is fantastic. People want it, and being able to say, yes, your pod is on AWS, has made a huge difference in how we upsell.
Oh, what does this slide convey? This slide conveys that mine is in the top three. That's the application layer. The other two layers show what AWS brings to the table. The foundational layer is at the bottom, because it's foundational, but functionally, it actually belongs closer to the app. And the intelligence layer is all the add-ons that AWS brings to the table. The ability to use Bedrock for AI, EKS, all that stuff.
So you want to know how to get started, so my friend Gav is going to come back here and talk about how to get started. Thank you, David. You know, a successful mission from Earth to Mars usually takes about 6.5 months, but if you start wrong and do everything else right, it can take up to three years longer. So, the message is for all mission-critical things, your AI ambitions, start today and get off to a good start.
And to make it easier for you to get off to a good start, there are several resources that David was able to utilize, and these are available to all of you. There are several migration immersion days. As he mentioned, Aurora was very important. There are also database migration workshops, and as he mentioned, some input on the Migration Acceleration Program to get you started quickly. If you have any questions for us, we are here, and thank you for joining us today. Thank you.
- This article is automatically generated using Amazon Bedrock, maintaining the information from the original video as much as possible.
Discussion