iTranslated by AI
re:Invent 2025: Implementing a Video Sampling and Search System with ElastiCache and Multimodal Embeddings
Introduction
By transcribing various overseas lectures into Japanese articles, we aim to make hidden, high-quality information more accessible. This project covers the following presentation!
For notes on re:Invent 2025, please check the information summarized in this Spreadsheet.
📖re:Invent 2025: AWS re:Invent 2025 - Video sampling & search using ElastiCache & multimodal embeddings (DAT433)
In this video, Elad and Kevin, Principal Engineers at ElastiCache, demonstrate how to implement vector similarity search using Valkey. Through building a video sampling and search application, they explain mechanisms such as VSS capabilities for hash maps and JSON documents, asynchronous indexing using dedicated worker threads, and fan-out queries across shards. In the coding demonstration, they use Amazon Bedrock's Titan embedding models, connect to ElastiCache via the Valkey Glide client, define indexes using the FT.CREATE command, and configure the HNSW algorithm and cosine metrics. They detail the process of implementing the entire pipeline—from video frame deduplication and multimodal analysis to semantic search—using Lambda functions and Step Functions. As a scaling strategy, they also explain the differences between improving ingestion speed by adding shards and increasing query throughput by adding replicas.
Note: This article is automatically generated while maintaining the content of the existing lecture as much as possible. Please note that there may be typos or incorrect information.
Main Content
Basic Operations of Vector Similarity Search in Valkey
My name is Elad, and with me is Kevin. We are both Principal Engineers at ElastiCache. Kevin will show you the actual code for building a video sampling and search application in a moment. But first, I would like to dive a little deeper into how vector similarity search works in Valkey.
VSS works on two data types in Valkey: hash maps and JSON documents. We start by defining an index and a schema. When you modify one of these objects, the change happens immediately on the main key database, and then the search module receives a keyspace notification. It then uses a set of dedicated worker threads to begin indexing those changes asynchronously. This means the Valkey main thread can continue to handle other requests from other users. However, the specific user who sent the change will wait because we have a synchronous API—and the synchronous API is really, really useful here because you can read what you wrote. It's not like other databases where things happen asynchronously at some point in the future. Once indexing is complete, the user receives an OK response and can proceed.
Queries are processed immediately by the query engine. This work is also performed on dedicated worker threads. Queries are sent not just to the local index, but also to remote shards. This is because when searching for nearest neighbors, you don't necessarily know which shard they are on. So, it fans out, and the query is applied everywhere. When the results return, Valkey optionally complements them with data from the main database and, of course, sends them back to the user.
Scaling Strategy: Different Approaches for Ingestion and Search
Let's talk a little about scaling. For those who are Valkey users, it's a bit different from what you're used to. When ingesting information, you send changes to a specific primary within a specific shard. That primary handles ingestion, index building, and updates, and asynchronously updates the replicas. The replicas also index the information. This architecture means that the more you shard and scale out, the more data you can store. And the ingestion speed also improves. In other words, you can ingest billions and billions of items per second. Well, maybe billions per second is impossible, but the more you increase shards, the higher the support.
Search behaves differently. When you send a query, the receiving node must fan out to all shards as I mentioned earlier. This means that adding shards does not increase throughput or queries per second, because all work must be performed on every shard. So, if you need more throughput, more queries per second, what can you do? You can add replicas. As you can see, the query fans out to all shards, but it chooses either a replica or a primary. If you have many replicas, you can actually perform more work simultaneously. Another good option is to scale up the instance type. With more cores, those dedicated worker threads can actually do more work, and they are actually very, very efficient at utilizing cores.
Overall Architecture of the Video Sampling and Search Application
Now, let's talk about what Kevin is going to build. We are going to create a web application. This consists of two parts. The first is the ingestion part, where you upload a video. And the second is the query part, where you can execute queries using either text or images. Well, what connects these is actually Valkey. During ingestion, we take the image, analyze it, and use foundation models to convert it into embeddings. Basically, embeddings are vectors. We insert them, and when executing a query, we take the query. This is either text or an image. We convert that into embeddings as well, using a foundation model to turn it into a vector.
Then, we look for similar vectors or similar images in Valkey. If we find something, we can retrieve it from S3 and display it to the user.
So, in the ingestion pipeline, we take the video, break it down into frames, and then perform deduplication. After finding images that are sufficiently unique, we perform multimodal analysis. Delving deeper into deduplication, and what it means for the application we are going to build, it is triggered by an insertion into S3—that is, the creation of an object. We start by taking the image, converting it into embeddings, and looking for similar images in Valkey on ElastiCache.
If we find something sufficiently similar—that is, a near neighbor—the image is likely not unique enough and doesn't provide additional value or extra content. So, we can discard it and skip all the extra analysis. However, if it is sufficiently unique, we continue the pipeline, perform additional analysis, and finally save it. Now, multimodal analysis works by taking the image and running it through several analysis models. We use categorization, labeling, and summarization. We take all the outputs of these. This is basically text.
We take the text and the image, and put them all into a foundation model because we want to generate embeddings. That is what we are doing—embedding vectors. That's all there is to it. And we take the text and convert it into embeddings, but we also insert the text itself into ElastiCache alongside the generated embeddings. In other words, we get this JSON object containing multiple fields. We have the text of everything we analyzed, the vectors we obtained from the foundation model, and we will be able to search on all of that.
Now, search means the user inputs either text or an image. In either case, we take that, convert it into embeddings in the same way, and look for nearest neighbors. If we find something, we retrieve it from S3 and display it. Sounds good. With that, I'll hand it over to Kevin. Thanks, Elad. I'll switch over to the demo machine.
Demonstration of the Completed Web Application
First, before we actually write the code, I want to show you what the completed application looks like. This is the Web UI that Elad was talking about, and one video, an AWS advertisement, has been ingested here. I will show you what this system does; when you click this, it performs an analysis on all of these frames and provides some insights about the video. It includes various labels identified from Amazon Rekognition, text from Amazon Transcribe, and other things identified through individual frames.
This is all derived from a series of frames. This is set to a 1-second sampling interval, and yes, regarding the insights—about the labels, categories, labels, and text—how are they determined? Well, these come from the services Elad mentioned. For example, if you send individual frames to Amazon Rekognition, you can identify labels. For instance, things like 'home' and 'indoors'.
This might not be the best example, but you can see 'home' and 'indoors' hits in the 1-second interval, 8-second, 18-second, and 27-second frames. You can click these to actually seek to those frames. And while what was identified isn't displayed, these are part of the machine learning services provided by AWS that take images as input.
These frames are extracted, and you can see the frame extraction. Here, we go through frame by frame at a 1-second sampling interval. Each of these frames is sent to Claude in this demo to generate a summary. There is another view to see labels, labels and text, and you can also check the subtitles for each frame. As we go through, you can see it is a 1-second interval, but there is also a similarity score you can observe here. This is derived from the deduplication process. You can see that these frames have a threshold set at 0.24. Frames below that are deleted because they are sufficiently similar to another frame. All of this is information derived from individual frames.
Live Coding Begins: Clearing the Environment and Verifying the Valkey Cluster
Now, let's explain where Valkey fits into this and its components. This is what we will look at today. First, I want to clear the code that has already been written. To do that, I will simply execute CDK deploy to remove the information here. This is all driven by a CDK package, which should clear things out. Now, let's move to the Lambdas used for implementation and delete this video as well. We are going to clear everything.
The actual frames are stored in S3. The metadata, in this case, persistent metadata, is stored in DynamoDB, and the vectors themselves are stored within Valkey. I will open the Valkey CLI here to clear all information within Valkey so we can start from a blank slate. First, let me show you my Valkey. There are various configurations for ElastiCache, but what we are using this time is a provisioned Valkey cluster running version 8.2. This is the version that supports vector search. It has 3 shards, each with 3 replicas, for a total of 9 nodes in the entire cluster. We are running on an R7G XLarge node type, but it is supported on various configurable instance types.
I will execute this. There are 3 different shards. I will run flushall on all shards. I am just moving to different shards using GET here. For those who aren't familiar with Valkey, I'm basically just flushing everything. Indexes are defined in a slightly different way, so I'll call drop index. Let's do that. This allows us to start from zero, giving us a clean state to build upon.
Now, the first thing we need to do in the pipeline I mentioned earlier is to make deduplication and saving functional. These are all derived from the embeddings we are using today, which are derived from Amazon Titan. We are using two different embedding models. One is V2 for text, which is slightly better for text information. Then we are using this multimodal embedding model, which can generate embeddings for both text and images, so we can input those frames to get vectors. We will implement this through Bedrock. Now, let's refresh these. Any questions so far?
Implementing Embedding Generation using Amazon Bedrock
I can explain this Lambda. This application is split into a number of different Lambdas, each built for a specific purpose. Many of them are connected via Step Functions, which we can look at a little later. One of the core things is actually the function that generates embeddings, because we use it everywhere—in search, in ingestion, and in deduplication. At the moment, the Lambda doesn't do much. Let's see if it works. It shouldn't be doing much. Let's see what CDK deploy did. It worked.
So, to ensure it isn't cached, let's try opening it again. Let's see what happens. The file was last updated 6 hours ago. In the worst case, if the deployment here doesn't update it, I can clear it manually. Wait a second. Was it a region issue? Let's see. Let's start with this embedding part first. Wait a second. I flushed all the data, but the code wasn't all flushed. I see, it doesn't seem to want to be deleted, so I will push it manually. That's fine, I'll just copy and paste.
Now, from local to here, and then deploy. Let's see if the same thing happens with the others. The empty one is actually empty here, and this is the Lambda handler for validation, which currently doesn't return anything in the response. However, what it should do is accept text and image inputs (both are optional), and then accept an embedding type to call one of the models. First, I want to create a Bedrock client. I got a little help from Amazon Q, but I intend to use a Boto3 client. This is Python, and it connects to the Bedrock runtime. The autocomplete feels nice. And if you spot any typos, feel free to point them out. While I'm doing this, let's pair program to make it actually work.
Now, I want to handle the different embedding models here. If the embedding model is multimodal, I want to use the multimodal embedding type. Otherwise, I use the text model. This helps a little. And I want to generate a request, which will be JSON. I can do that, or make it simpler with just text input. If you want to check the API, the one for Bedrock isn't here, but it accepts a JSON document and input text. Or if there is an image input, we use the image. And for those who have never used Bedrock, it's very easy. In fact, you can have it autocompleted. Dump the request as a JSON document and call the correct model. Accept and content type aren't strictly necessary, but this will return a vector response. By loading the response body, you can get the body from the response, and from there, get the embedding. So it's a JSON with just one field called embedding, and I just return that to the body here. I could return the actual JSON blob, but I'm returning the vector embedding. I see, this looks fine. Let's see. Now, let's deploy and test it. Has anyone noticed an obvious mistake? This just shows how easy it is to interact with Bedrock.
Now, I'll create a simple test. I need an embedding type and text or an image. Let's do that. Set the embedding type to text, and create a test event. So, what was it? It wasn't text input and image input, it wasn't text image.
There is no text input, just image. Wait, text input is "Hello World", and what I expect is for a vector to be returned, and it actually was returned. It's a large vector, 1,024 in length. This is clear from the output vector size. Now, the question is, what is generating the vector? Yes, the question is whether the Lambda is generating the vector. In this case, the Lambda is not performing the vector generation. The Lambda is basically just calling the Bedrock service, and the Bedrock service is interacting with the Titan model.
There is an option to generate embeddings locally within a Lambda or an EC2 instance, since there are local embedding models. In this demo, I'm just interacting with Bedrock via the AWS CLI, and getting credentials locally via Lambda to call that remote service to get the embeddings. Yes, to the question of whether token IDs or credentials are required—that's correct. Since this is set up so credentials exist in the Lambda, the default SDK simply acquires them when it instantiates the client.
Saving Vectors and Indexing with the Valkey Glide Client
This is the foundation. It looks like it's working, but we haven't connected to Valkey yet, so if I try to ingest a video, nothing will happen. So the next step is to take the vector and, when we decimate the video, actually save that information within Valkey. For that, we have this vector saving function, which doesn't seem to have been cleared either. So, I can use the same trick as before: take the local empty one, overwrite it, and deploy.
That's good. This is the latter part of the path, where we've already retrieved the keyframes, and all the information is saved in DynamoDB and provided in this event context: S3 image, task ID, frames, and some configuration items. This frame is a JSON blob that we save in ElastiCache, which contains a lot of fields. I can show you, but it includes the S3 URI so it can be displayed in the UI, a timestamp, a lot of metadata needed to run vector searches, and the actual vector itself. These have already been input here.
Our job here is to initialize it. We have create Valkey client, but it doesn't do anything yet, so I want to be able to connect to Valkey. To do that, I use a client called Valkey Glide. This is supported by our team and is one of the official Valkey clients. It's a polyglot client, so it supports multiple different languages. Since all the Lambdas are written in Python, we use the Python version here.
Now, let's import some things and see if it understands. Probably not. From glide sync, we're just importing * here because there are many things we're going to use. In fact, glide sync is enough to create the client. To initialize the client, we need a domain endpoint and a port. These are retrieved from ElastiCache. We have a single configuration endpoint here and port 6379. Valkey works in cluster mode by discovering all the different nodes. I mentioned there are 9 nodes in the cluster. From this configuration endpoint, we can query any node to identify what the 9 nodes are, and communicate with them regarding data shards. In other words, it's a direct communication from the client to the nodes that hold all this data in memory.
So, let's initialize the client. We do this by returning a GlideClusterClient. Since cluster mode is enabled, we are using the cluster client. Call create, pass some cluster client configurations, and pass the list of addresses. In this case, there's only one address, which is the domain endpoint. And we pass the port. We have TLS enabled here, but authentication is not enabled.
Looking at this, encryption in transit is enabled, which is a toggleable option. But authentication is not enabled. Authentication is also available, but it's slightly simpler without it. For TLS only, if we set the use TLS flag to true, the client will perform the TLS negotiation. Then, we use Advanced Glide Client Configuration to set an increased timeout here. We set a 2-second timeout, which is perfect for this specific demo. This should actually allow us to create the client.
And actually, next we skip index creation, because it can be done asynchronously here. And I want to be able to save this frame. The basic data structure we are dealing with here is JSON in Valkey. This is one of the modules available in Valkey, it's included in the bundle, and Glide supports it using this Glide JSON wrapper. First, we need to import that. Then we can use it.
From glide sync, I think it's actually there by default. What's not there is full text. Glide JSON set. And we pass the EC client. Can we use this Lambda function? Regarding the question of whether there's a way to generate this code using agentic AI—that's a good question. The answer is yes. As you can see, Amazon Q is attempting this here, but it's not always successful. For this demo, I tried to do this using Q, and while there's a lot of background information on Bedrock calls, there's a bit less information in the dataset regarding the use of Valkey Glide. That's because it's a relatively new repository. However, I was able to have it generate code there as well.
In other words, this is an option for you to use this, or use something else to try code generation with large language models. The question is, what is Q? Well, I'm not using it right now. Ah, the thing that popped up here is, I believe, Amazon Q. Since it's running, I could pause it. But I don't think Q is part of the Lambda console here. I might be wrong. There is an Amazon Q tab here, but I think that's just for asking questions.
But if you use something like the Q I have here, you can actually generate code by chatting. The reason I'm not doing that today is not to have it generate something, but to explain the actual points. So, when I set it, I'm passing the client, and then I want to set it with a specific key index. Everything in Valkey is indexed by key name, and here we put frame as a prefix for the frame, and then append a unique frame ID. This is a common pattern when interacting with Valkey.
And since we're dealing with JSON, we replace the entire JSON document. With a dot, and then use JSON to dump the frame. This saves it inside Valkey, and you can interact with JSON. You don't need a vector index. That's just one of the supported data structures. But here I actually want to be able to create an index, so if you run this set, it will be automatically ingested and available for vector search.
This goes into the vector capability, the VSS capability. There are a lot of different options. Let me show you the Valkey command. Let's do FT.CREATE. Valkey is quite well documented regarding commands. The slightly difficult part is that there's a command here on the side, there's an FT.CREATE function. FT stands for full text, but currently, Valkey only supports vector search. Full text is coming soon. You can create a schema with various field types and attributes, so as I mentioned before, we do that to index two different vector fields: text and image.
So let's go back there. To actually create the index, you only need to create it once, and while this can be done manually, let's have it run as part of the Lambda. To do that, we can first test whether it exists. We actually need to import all full-text-related things. These are not included in GlideSync. These are separate commands. From GlideSync.sync commands, import FT to interact with the VSS module. And from GlideShared commands, the server modules—this is a bit long. It would be convenient to have Gen AI create it here. Import * from create options. Okay. Now we should be able to call FT.info. Pass it to the client, and then check our index. We have one index here. Check if it exists. Call it an index with video frames. This will throw an exception if it doesn't exist. except Exception as X. Okay. And if it doesn't exist, try to create it here.
Now, to create this, we first need to create a schema. And we need to define exactly what the fields we want to index are. So, let's call this a vector field. The name will be where to find it within the JSON field—we can talk about where the data comes from later—but as a data type, it's called mm_embedding within the JSON field, which stands for multimodal embedding. I alias this to mm_embedding so I don't have to deal with the dollar sign at the beginning. When you actually run a query, you can omit the dollar sign. We use the HNSW algorithm. Oh, look, it's kindly auto-filling it. Let's see if it's actually correct. It's a little bit off. VectorAlgorithm is the correct one. This is the pitfall of using Gen AI—you might get it mostly correct, but not 100 percent. So, let's make it HNSW. Let's pause this so it doesn't get in the way—er, it was down below. Good, this should be a little better. HNSW, it didn't work. So, let's see if it works this time. And I want to pass the HNSW attributes.
HNSW is a graph navigation scheme, and we need to pass what the vector field attributes are, what dimensions to expect, and how we want to generate the score. We can pass and tell it that these vectors have 1,024 dimensions, and I'd like to use the cosine metric instead of L2. So, metric type cosine. And everything in the vector field is float thirty-twos, so basically, this is a multimodal embedding, so I can copy and paste this for the text embedding, because that is—line 38, distance, yes, thank you. You have a sharp eye. Let's see if I can copy and paste this for the text embedding; it's inside this text embedding. Good, now we can index two different fields. Each has a different embedding. One represents only text, and the other represents both text and images.
This is how we create most of the schema, and the rest is—er, good, we need to complete the definition. And there are some options for telling it which fields to index. We could try to look at everything saved in Valkey and index all JSON documents, but here we narrow it down to things that are JSON.
Specifically, things that start with this frame prefix. That is to say, we can say that we'll inspect everything starting with frame to see if it can be inserted into the index. This allows us to save other things on the same Valkey server without having to index them or insert them into different indexes. And as I mentioned before, you can choose to put one object into multiple indexes.
And we can execute FT.CREATE to pass the video frame, index ID, and the schema and options. This creates the index. Let's wrap this in a try-catch, because multiple things could try to create it at the same time, and one of them might throw an error. Usually, we'd handle this more properly, but since we're just doing it for demo purposes, we'll ignore any errors thrown here and verify manually.
So, now we have the functionality to create a client. First, we check if the index exists. If it doesn't exist, we proceed with the creation, and then we go on to save this. We can also do it in the reverse order. It's not important to create the index before saving the data. The index will be backfilled, so you can drop and recreate it dynamically, and it will search the keyspace and reindex everything as needed. However, we chose to do it this way here.
And, if there are no other typos, let's deploy this. There's another typo. Exception on line 59. Good. We don't have a test for this, so theoretically we could test it within the Lambda, but since we need to generate proper context here, I'll test it through the application.
Implementing and Testing Search Functionality using the FT.SEARCH Command
While that's running, let's clear out the search Lambda function that we also plan to create. This wasn't cleared before, so let's deploy the empty version. Er, never mind. Okay. Deploy the empty search. Okay, let's go back to the application. It's stuck on deleting. That might be the reason things aren't working well. Okay. Ignore the things stuck on deleting. Because we've already manually cleared everything in ElastiCache.
Now, let's see if this works. Nothing should be created yet. Er, that's good. And there shouldn't be any keys either. Okay, this is a multi-shard cluster, so keys target specific nodes. Each frame could be placed on a different shard. However, if you create the index just once on one shard, it automatically propagates to the other shards, so everyone gets the same index.
So, let's ingest a video and see what happens. Let's ingest the same ad we tried earlier. There are a lot of settings here, and you can choose the sampling interval. Here, we're doing one per second. This is what I talked about earlier. For each image frame, we can run various detections here. Recognition detects labels and text, as well as moderation and celebrities. And summary is run from Haiku, and Transcribe does the subtitles. Let's leave them as default and upload.
This executes the Step Function. We'll look at that in a moment, but it runs transcription and then various analyses. As I said earlier, Valkey is perfect for ultra-low latency, real-time use cases. This isn't really a good showcase for real-time functionality. Because this is running at a bit of a lower level, or in an offline manner. In other words, we're running a workflow against this and displaying a web UI that allows for searching. So, this is really just an example application of how to connect, but Valkey really shines when performing real-time streaming.
When performing analysis or other things where much higher latency and throughput are required than this example application. Now, let's check the extraction flow. We're running a series of Lambda functions. Let's see if I can make this bigger. Yes, we're running a series of Lambda functions to get the video metadata. Can you maximize this? Yes. And zoom in.
Perform sampling, delete duplicate frames, deduplicate the sampled images, iterate through them, extract metadata, and actually save those vectors. And it looks like something went wrong. So, let's dig into what else went wrong. The vector is not defined. I see, it seems I made a typo when saving. Where did I write vector? Vector dot, vector type, yes. That's fine, let's try. Good, and rerun from the failure point. Let's see if it completes normally.
Still no good. Let me see if there are other typos. await and create index are missing from create options. Let's look at that. create options has a data type, and ah, prefixes. This is it. I wish Amazon Q would help me here, if I could just ingest the JSON syntax, but I have this. So, let's try again.
We're iterating through these in a loop, and I think the concurrency is quite low. So it takes a while. Again, if you're doing something in real-time, you can ingest while the frames are being streamed. After fixing all the typos, this completed normally. And in the one that doesn't delete, you should see the same UI we saw earlier. Different image frames are displayed.
And I should point out that there is one duplicate here. This specific image was taken from YouTube, but a similarity score is displayed. Frame 9 is—well, yes, we're going from frame 8 to frame 10 here, but if you look at what happens between 8 and 9, it feels like the same woman is moving around, so we were able to deduplicate that specific frame.
Now, let's see how this looks in Valkey. If you run keys star, there might be keys. There should be, because they're in different ones. The frame prefix followed by a timestamp is displayed. This is a UUID followed by a timestamp. You can run JSON.GET. This is a lot of vectors, but you can see there are things other than vectors as well. There is a task ID, and then two embeddings that we generated. And the actual metadata itself—text and labels. These are all things that are actually indexed within the vector. And let's see if I can make this bigger. Maybe not immediately. Ah, I can. Yes, I did.
And the S3 URI. These are all things that are indexed. If you list the vector indexes, there is one that performs similarity checks, and this video frame index is what we defined earlier, so you can execute FT.INFO to get information about the actual video frames and observability.
This generates a lot of output, but it shows the definition. I defined it with both text embeddings and multimodal embeddings. But the interesting part is that it also tells me the number of indexed documents. 7 different documents—there are 14 records in the index that can be searched on this specific node. This should correspond to the number of keys here. 7, yes. Each node holds a part of the index, and when a search is performed, it fans out, but that is transparent to the application.
If it cleared successfully, the search should be broken now. If I try to attempt a semantic search here, nothing is returned. It says "video not found" because it's not connected. So, the last thing I want to do is make this searchable. To do that, we look at the search function here. I created an index and populated the data. And now I want to search it. This is getting information from the Web UI when performing searches. It's preconfigured with the Valkey connection, but basically I need to implement a function to search for text embeddings. I'll do that first. This is text—a semantic search.
So, there's input text and a score threshold. I'll use these to find only relevant results. To do this, I use the FT.SEARCH command. I'll explain this briefly here. First of all, we have text as input. The first thing I should do is generate an embedding from the text. This get embedding function calls the lambda I defined earlier, and basically converts the input text into a vector embedding. I pass the input text, but I don't pass an image. And I need to query the given vector. To do this for the index, I use the syntax defined in the Valkey command FT.SEARCH. Syntactically it's a bit difficult, but I'll explain what this means.
First, the star mark means we're not filtering on vectors. You can also filter with different tags, but we won't touch that today. Perform a K-nearest neighbor search and pass the default K. In this example, it's 20, but I can also hardcode the number here. And we query this text embedding field within the index. And I show that I'm passing this text embedding field as a variable for the query vector, and return the actual score here. I want things above the score. By default, it only returns the score, but that's not very useful. Because I want some metadata, not just the key name which is just a UUID.
So, I can do this using vector search options. Return the fields stored in JSON—we'll do return field field identifier equals. I want the score, and I want other fields too. So, let me copy and paste this. I want the task ID. I want to be able to input this. I also want the image's S3 URI. That's in there too. I want the timestamp as well, and I also want the text itself so I can display the matched text. These are the fields to return. By default, it only returns 10 results regardless of the K value I pass. Therefore, I want to change this to return 20 results in this case. So, let's set it to EC default K.
And finally, I need to actually pass the vector I'm searching for. To do this, I define a query vector parameter that matches this string. And I pack it as a struct so it can be sent over the network. Encode it in a way Valkey can recognize. Pack the length of the embedding, and then get the actual raw embedding itself. Okay, the query looks good. And I just call the function itself. Run FT.SEARCH, pass the client, the index name to search, and the options I defined earlier. And there's a helper function here. It
formats the results, performs filtering, and returns them in the format I want to display in the UI. Execute format frame results on the response with the specified score threshold. This performs the filtering.
Now, this is easy to test. As soon as the deployment is complete, I can run the Web UI. Let me see if I made any mistakes here. I'll do the same thing. Search for AWS. Ah, I didn't remember this well enough. Let's find the FT search options. Can anyone see my mistake? If I can't find it, I'll go into the CloudWatch logs and check. Let's check what kind of error is being returned.
No items in the Set. Did I pass the wrong query options? Should I have passed a list instead of a set? Let's go back. Where am I using Set? There's a return field list. FT search vector said params—yes, that's it. A colon, thank you. And this is still correct, yes. Let's try. Thank you.
Good. 20 frames were returned from a single video. If you expand this, you can see that they're ordered from most relevant to least relevant. As you can see, when you run the search, it pulls up the ones with the lowest score here, and the ones that actually represent AWS appear. Claude was able to extract this text from here. Looking at a few others, AWS is included in the images at the bottom here. AWS exists here too, so the most relevant results are displayed first.
However, I haven't done multimodal search yet. But basically it's the same thing, so I can copy and paste a lot of this. You'll just be searching for something different. I could create a helper function, but I won't right now. So, I search multimodal, generate a multimodal embedding, pass the image too, and change this to MM embedding. That's all I think. You get the same details, but it's a different embedding. It's in the same index. Let's see what happens.
And now, this other dropdown search opens, and I can perform multimodal search. I can pass text or images to this. This time I'll search for "ogre". There's an ogre or some kind of monster in this video. Let's see if it's displayed correctly. Now, I get 20 frames again, and the top ones are the monster from the AWS video. And I can combine these, so I can search for "AWS" and "ogre". They feel like they're being placed alternately. Since I'm generating embeddings for both at the same time, I can see the most relevant results for both "ogre" and "AWS".
Q&A: Regarding Cost-Effectiveness and Caching Strategy
Good. I think that's everything I wanted to demonstrate today. I wanted to mention that this is an application that is widely available online in the AWS Solutions library. However, as a point of caution, this application was originally developed for OpenSearch, so it doesn't work with Valkey. The code running is for OpenSearch, but it is possible to replace it with Valkey. And within this AWS Labs, there is something called "Guidance for Media Extraction and Dynamic Content Policy Framework". This is a framework, so if you want to understand more deeply how the entire ingestion pipeline works, you can check it out at your own pace.
Now, I'd like to take your questions.
Yes, the question is regarding the high cost of the service I ran. You're putting it directly into ElastiCache, but wouldn't it make more sense to put it in something like DynamoDB and store the embeddings there? Yes, so that's a question about cost—whether this is a cost-effective way. Generally, yes, that's a good point. Information in ElastiCache is temporary. So, in cases like this where you want to keep it long-term, you're right. A persistent database would be more suitable for this specific use case. That's why, in real-time cases where you're actually doing something like security analysis, or where information is more temporary and you're just looking for specific threats before aging out or setting TTLs, those kinds of cases would be better use cases for vector search in ElastiCache.
You could also store embeddings in something like DynamoDB. DynamoDB doesn't have vector search capability, but after that, if it's just the core working set, you can migrate to Valkey and search easily. And age out the old ones. That's one method too. You can set a TTL on all these JSON documents so they're deleted after a certain period, and even then, you can keep the master in another persistent database. This is a pattern used by customers.
A question from the back. Is it possible to get your code, this implementation method using Valkey, on GitHub? Valkey content isn't currently available. I think we can consider making the Valkey part itself available. If you'd like, I can talk to you directly later and arrange to provide you with the information. Thank you.
There's a question here, too. Actually, it's not exactly a question, but I'm generally confused about using an embedding cache. Because you don't always get cache hits, and when that happens, you don't search persistent storage, right? In this case, we are using ElastiCache as the primary vector database. That is to say, there is no other vector database in this case. We've completely replaced OpenSearch or Postgres with ElastiCache. So, you get a cache hit whenever there's a relevant match.
Is the question about how to use ElastiCache as a vector database in front of another vector database, which is persistent storage? Yes, that's possible too, right? You can perform searches. In this case, it might be fine to just have the hot working set here, but in that case, you need to fall back to the database when a miss occurs. In other words, it handles cache hits, but doesn't necessarily handle cache misses, and in that case, you might need to fall back and bring that vector into Valkey. So, you could use a look-aside cache pattern for vectors as well. You'd have a two-layer cache for vector search.
But there might be closer, more similar vectors in persistent storage, but you wouldn't hit them because there's something in the cache, right? Yes, that depends on the situation. So, setting a score threshold becomes important. As you said, if only a part of the vectors are in the Valkey cache and the rest are in persistent storage, you might not get the most relevant result, but it might be a sufficiently good result. If you put everything in here, you'll get the most relevant result, so that might work better. In fact, we see many customers storing all their vector data here. Just keep in mind that if you do that, you have to deal with its temporary nature. Got it, thank you.
Good. Thank you. I'll take other questions offline. I'll be outside. Thank you very much.
- This article was created automatically using Amazon Bedrock, maintaining the information from the original video as much as possible.
Discussion