iTranslated by AI
Getting Started with Lambda #7: Connecting to RDS (Aurora) from Lambda - Part 2
Introduction
In the previous article, we covered setting up an Aurora DB instance within a VPC.
In this article, we will connect to the DB from a Lambda function located within the same VPC.
I will also introduce the setup for RDS Proxy, which is used for this purpose.
As a prerequisite, we assume that the RDS instance and VPC have already been created.
What is RDS Proxy?
Amazon RDS Proxy is a highly available, fully managed database proxy for Amazon Relational Database Service (RDS) that makes applications more scalable and more resilient to database failures, and more secure.
It is literally a proxy for AWS RDS.
The reason this is necessary is that when considering a connection from Lambda to RDS, Lambda starts up for every request and establishes a connection each time.
This is the so-called simultaneous connection issue; RDS Proxy sits in the middle and manages the connection pool, which resolves these problems.
Before RDS Proxy was introduced, the combination of Lambda and RDS seems to have been treated almost as an anti-pattern (see the link below).
RDS Proxy Configuration
Let's dive straight into configuring RDS Proxy.
Creating a Secret
RDS Proxy manages DB credentials using secrets.
Therefore, we use AWS Secrets Manager.
We will include the credentials of the RDS instance created last time (default is postgres) in the secret.
The name will be rds_proxy_secret.
For the value of KVS_RESOURCE in the content, use the value from "Key Management Service (KMS) > AWS managed keys > aws/secretsmanager" in the AWS console.
aws secretsmanager create-secret \
--name "rds_proxy_secret" \
--description "rds_proxy_secret" \
--region us-east-2 \
--secret-string '{"username":"postgres","password":"postgres"}'
--kms-key-id 【KVS_RESOURCE】
It is a success if a JSON response is returned.
Creating a Role
Create a role to be used this time.
To access RDS, prepare a JSON file like the following.
The name will be rds-proxy-policy.json.
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": "rds.amazonaws.com"
},
"Action": "sts:AssumeRole"
}
]
}
Create the role using the above.
The role name will be rds_proxy_role.
aws iam create-role --role-name rds_proxy_role \
--assume-role-policy-document file://rds-proxy-policy.json
Adding a Policy to Use the Secret
Add a policy to the created role.
This policy is for using the value of the secret we created earlier.
Create a secret-reader-policy.json as follows.
The value for KVS_RESOURCE is the same as the one used in create-secret.
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "VisualEditor0",
"Effect": "Allow",
"Action": "secretsmanager:GetSecretValue",
"Resource": [
"【SECRET_ARN】"
]
},
{
"Sid": "VisualEditor1",
"Effect": "Allow",
"Action": "kms:Decrypt",
"Resource": "arn:aws:kms:【REGION】:【USER_ID】:key/【KVS_RESOURCE】",
"Condition": {
"StringEquals": {
"kms:ViaService": "secretsmanager.【REGION】.amazonaws.com"
}
}
}
]
}
Add the policy to the role using this JSON.
aws iam put-role-policy --role-name rds_proxy_role \
--policy-name secret-reader-policy --policy-document file://secret-reader-policy.json
Using CMK
Use CMK to encrypt and decrypt the secret for use.
CMK is generated using AWS Key Management Service (hereafter KMS).
Prepare an rds-key.json like the following.
{
"Id":"rds-kms",
"Version":"2012-10-17",
"Statement":
[
{
"Sid":"Enable IAM User Permissions",
"Effect":"Allow",
"Principal":{"AWS":"arn:aws:iam::【USER_ID】:root"},
"Action":"kms:*","Resource":"*"
},
{
"Sid":"Allow access for Key Administrators",
"Effect":"Allow",
"Principal":
{
"AWS":
["arn:aws:iam::【USER_ID】:user/【USER_NAME】","arn:aws:iam::【USER_ID】:role/rds_proxy_role"]
},
"Action":
[
"kms:Create*",
"kms:Describe*",
"kms:Enable*",
"kms:List*",
"kms:Put*",
"kms:Update*",
"kms:Revoke*",
"kms:Disable*",
"kms:Get*",
"kms:Delete*",
"kms:TagResource",
"kms:UntagResource",
"kms:ScheduleKeyDeletion",
"kms:CancelKeyDeletion"
],
"Resource":"*"
},
{
"Sid":"Allow use of the key",
"Effect":"Allow",
"Principal":{"AWS":"arn:aws:iam::【USER_ID】:role/rds_proxy_role"},
"Action":["kms:Decrypt","kms:DescribeKey"],
"Resource":"*"
}
]
}
Create the key with the following command.
aws kms create-key --description "rds-key" --policy file://rds-key.json
Creating the Proxy
Finally, we will create the RDS Proxy.
Specify the ARN of the created secret and role, as well as the VPC subnets and security groups.
We will name the proxy test-rds-proxy.
aws rds create-db-proxy \
--db-proxy-name test-rds-proxy \
--engine-family POSTGRESQL \
--auth '[{"Description": "test_rds_proxy auth", "AuthScheme": "SECRETS", "SecretArn": "【SECRET_ARN】", "IAMAuth": "DISABLED"}]' \
--role-arn 【ROLE_ARN】 \
--vpc-subnet-ids '["【SUBNET_1】","【SUBNET_2】"]' \
--vpc-security-group-ids 【SECURITY_GROUP_ID】
If successful, a JSON response will be returned.
Linking the Proxy and DB
Next, link the created proxy with the database.
Assume the database cluster name is test-aurora-postgresql-cluster.
aws rds register-db-proxy-targets \
--db-proxy-name test-rds-proxy \
--db-cluster-identifiers test-aurora-postgresql-cluster
This is also successful if a JSON response is returned.
Checking the Status
Since linking the RDS Proxy and the database takes some time, check the status with the following command.
aws rds describe-db-proxy-targets --db-proxy-name test-rds-proxy
Refer to the TargetHealth value in the response.
It will likely show PENDING_PROXY_CAPACITY for a while, so please wait.
If AUTH_FAILURE is displayed, there is often an issue with the role configuration.
There are several other error indicators; check the link below for troubleshooting.
It is complete once the value eventually becomes AVAILABLE.
Preparing Lambda
Now we are ready to use RDS Proxy.
The Endpoint value in the response of the command used earlier for checking the status is the actual endpoint for the RDS Proxy.
In the previous article, we confirmed connectivity with the RDS instance using psql from a local environment.
However, since RDS Proxy can only be accessed from within a VPC (see below), this time we will access it from a Lambda function located within the VPC.
An RDS Proxy must be in the same virtual private cloud (VPC) as the database. The proxy can access the database, but the proxy isn't publicly accessible.
Lambda Content
Place a Lambda function like the following within the VPC where the RDS Proxy resides.
Various settings are configured in environment variables, and DB_HOST will contain the aforementioned Endpoint value.
Also, ensure that pg is available using layers or similar means.
We assume that the TASK table used for the query in this example has already been created in the previous article.
const {Client} = require('pg');
exports.handler = async (event, context) => {
try{
// client configuration
const client = new Client({
host: process.env.DB_HOST,
port: process.env.PORT,
user: process.env.USER,
password: process.env.PASSWORD,
});
// connection
await client.connect(function(err) {
if (err) {
console.error('connection error', err.stack)
}
});
// execute query
const res = await client.query('SELECT id, name FROM TASK');
console.log(res);
return {
statusCode: 200,
body: JSON.stringify(res.rows),
}
}catch (e) {
console.log("Error...");
console.log(e);
throw e;
}finally{
}
};
By executing this Lambda function, the following response was returned:
{
"statusCode": 200,
"body": "[{\"id\":0,\"name\":\"task1\"},{\"id\":1,\"name\":\"task2\"}]"
}
This confirms that the connection is working from the Lambda function via RDS Proxy.
Summary
In this article, we implemented a process to send queries from Lambda to an RDS instance via RDS Proxy.
Although this is a simplified implementation, by combining this content with API Gateway, you will be able to build a REST API that returns dynamic responses from a DB.
I hope this content was helpful.
Reference
Discussion