Multi-user Lambda Development Environments: Designing and Operating Environment Isolation with AWS CDK

Lambda-based serverless development is easy to start, but as the team size grows, you hit the wall of "development environment conflicts."

In the project I handled, we initially edited code directly in the AWS Management Console, but as we developed with multiple people, I felt the development experience declining.
At the time, we were still exploring what the best practices were, but eventually, we settled on a configuration using AWS CDK (Cloud Development Kit) to create independent environments for each developer.

In this article, I will share the background of the technical selection and the actual construction flow.

Background and Challenges: Limits of Console Development

At the beginning of the project, we wrote Lambda code directly on the AWS Console and changed API Gateway settings. However, as the number of development members increased, the following issues emerged:

• Environment setup is time-consuming:
  Manually setting up API Gateway routing and IAM role permissions to create an "environment for User X" was difficult. Moreover, because it was manual, settings were often "slightly different," which frequently led to bugs.
• Psychological burden during merging:
  When merging changes from Person A and Person B, we had to merge source code manually, which was a hassle.
• Unknown "Why this setting?":
  Configuring in the console leaves no change history. When trouble occurred, it was difficult to trace "when and who changed it," making troubleshooting difficult.

All of these stem from "infrastructure not being managed as code."
We decided to move forward with implementing IaC (Infrastructure as Code).

Comparison and Evaluation: Why CDK instead of SAM?

For the move to IaC, we compared AWS SAM and AWS CDK.

AWS SAM vs AWS CDK

For Lambda-centric configurations, SAM—which allows writing in YAML—is a standard choice. However, I chose AWS CDK (TypeScript) this time. There are two main reasons:

1. Affinity with the team's tech stack:
   Our team members were familiar with programming languages such as C++. Therefore, we felt that CDK, which allows defining infrastructure using "programming logic" like loops and conditional branches, was more intuitive to understand than writing declaratively in YAML. It also made sense in terms of minimizing learning costs.

2. Concerns about future scalability:
   Currently, the setup consists only of Lambda and API Gateway, but there was a possibility of adding WAF or making the DynamoDB configuration more complex in the future. I had a hunch that "YAML descriptions would become bloated and painful to manage."

Implementation and Solution: Dynamic Environment Generation via Context

The core of this design is "being able to provision a dedicated stack (environment) with a single command."
To achieve this, we utilized CDK's Context (cdk.json).

1. Project Structure

We centralized the Lambda source code in the lambda/ directory and configured the CDK side to reference it. The actual business logic is written in lambda/lambda_code.py.

The entire directory is managed as a single Git repository.

my-cdk-app/
├── bin/
│   └── my-cdk-app.ts       # Entry point
├── lib/
│   └── my-cdk-stack.ts     # Resource definition
├── lambda/                 # Lambda function code
│   └── FuncA/              # Per-function directory
│       └── lambda_code.py  # Source code (Python)
├── cdk.json                # Environment configuration
└── package.json

2. Defining Environment Variables with Context

We define parameters for each developer in cdk.json. We add a suffix to avoid resource name duplication errors ("Already Exists").

{
  "context": {
    "dev-userA": {
      "stackName": "MySystemStack-UserA",
      "suffix": "-userA"
    },
    "dev-userB": {
      "stackName": "MySystemStack-UserB",
      "suffix": "-userB"
    }
  }
}

3. Dynamic Loading on the Stack Side

Setting values are read using the argument passed during deployment as a key.

cdk deploy -c config=dev-userA

First, the environment settings are retrieved in the entry point (bin/my-cdk-app.ts) and applied as the stack name. This allows different stacks to coexist within the same AWS account.

// bin/my-cdk-app.ts
const app = new cdk.App();

// Get environment key from arguments (-c config=dev-userA)
const envKey = app.node.tryGetContext('config'); 
const envConfig = app.node.tryGetContext(envKey);

// Apply stackName from cdk.json when creating the stack
new MySystemStack(app, envConfig.stackName, {
  /* props... */
});

Next, the individual resource settings are reflected inside the stack (lib/my-cdk-stack.ts).

// lib/my-cdk-stack.ts
import * as cdk from 'aws-cdk-lib';
import { Construct } from 'constructs';
import * as lambda from 'aws-cdk-lib/aws-lambda';

export class MySystemStack extends cdk.Stack {
  constructor(scope: Construct, id: string, props?: cdk.StackProps) {
    super(scope, id, props);

    // Get information from the arguments passed during deployment
    const envKey = this.node.tryGetContext('config'); 
    const envConfig = this.node.tryGetContext(envKey);

    // Example: Lambda function definition
    new lambda.Function(this, 'MyHandler', {
      runtime: lambda.Runtime.PYTHON_3_12,
      code: lambda.Code.fromAsset('lambda/FuncA'),
      handler: 'lambda_code.handler', 
      functionName: `my-function${envConfig.suffix}`, // Create a unique resource name for each environment
      environment: {
        'ENV_NAME': envKey,
      }
    });
  }
}

4. Git Branching and Development Flow with CDK

By keeping the source code and infrastructure definitions in the same repository, the usual Git workflow can now be applied directly to infrastructure management.

1. Branch Creation: Create a feature/xxx branch from main and start working.
2. Local Development: Perform logic changes in lambda_code.py and necessary infrastructure changes (such as adding environment variables) as a set.
3. Operation Check: Run cdk deploy -c config=dev-userA to test in your own dedicated environment. Since there's no risk of breaking someone else's environment, you can experiment freely.
4. Conflict Resolution: Incorporate changes from main before merging. Even if infrastructure settings conflict, they can be resolved simply as text differences on Git.
5. Review: Create a Pull Request. Reviewers can check not only the code but also "how the infrastructure changes."
6. Merge: If okay, merge into main.

By introducing this flow, the challenges mentioned at the beginning were resolved as follows:

• Elimination of "Manual Operation Costs":
  Even when a new member joins, they can get their own dedicated development environment just by cloning the repository and running cdk deploy. Worries about missing configurations have disappeared.
• Standardization of Conflict Resolution:
  Since infrastructure settings are also code (text), even complex API Gateway configuration changes can now be compared for differences on Git and merged.
• Ensuring "Visibility and Evidence of Infrastructure Change History":
  By managing the infrastructure configuration with Git, logs clearly show "when," "who," and "why" a setting was changed. Not only can you trace the history of past decisions, but it also provides peace of mind because you can use Git features to revert to a previous state in the event of a configuration error.

Retrospective and Remaining Challenges

By migrating to this configuration, conflicts between developers were eliminated, and the development experience improved. However, not everything has been resolved, and some challenges still remain.

• Deployment is still manual:
  Currently, deployments are done from each individual's local machine. Ideally, it should be integrated into a CI/CD pipeline and automatically deployed on a merge trigger. If time permits, I would like to proceed with this alongside infrastructure improvements, such as setting up GitLab Runners.

Conclusion

While the introduction of CDK involved some trial and error, the biggest takeaway was that it fundamentally reduced "team development stress" more than just being a tool introduction.

I hope this provides some hints for those struggling with serverless development in a similar way.

References

• Setting up an AWS CDK development environment on a local Windows environment | DevelopersIO
• Switching configuration parameters for each deployment environment with AWS CDK | DevelopersIO