iTranslated by AI

The content below is an AI-generated translation. This is an experimental feature, and may contain errors. View original article
💭

Getting Started with LocalStack on WSL2 Ubuntu 20.04

に公開
AWS
Docker
Ubuntu
WSL 2
LocalStack
tech

Introduction

LocalStack is a solution for running AWS applications and Lambda in a local environment.
The open-source version has limitations, but the Pro version supports many services.[1]
In this article, I tried running LocalStack by adding Docker to Ubuntu 20.04 on WSL2.

↓ I also wrote about a method to simplify WSL2 setup using cloud-init.
https://zenn.dev/s_ryuuki/articles/46c6d9d8d34404

Environment

Host OS

Guest OS (WSL2)

  • Ubuntu 20.04
    • Docker: 20.10.11
    • Python: 3.8.10
    • docker-compose: 1.29.2
    • LocalStack: 0.13.1
    • aws-cli: 2.4.5

Preparing Ubuntu 20.04

Installing and Starting Ubuntu 20.04

Display the list of distributions.

PS C:\Users\hoge> wsl --list --online
インストールできる有効なディストリビューションの一覧を次に示します。
'wsl --install -d <Distro>' を使用してインストールします。

NAME            FRIENDLY NAME
Ubuntu          Ubuntu
Debian          Debian GNU/Linux
kali-linux      Kali Linux Rolling
openSUSE-42     openSUSE Leap 42
SLES-12         SUSE Linux Enterprise Server v12
Ubuntu-16.04    Ubuntu 16.04 LTS
Ubuntu-18.04    Ubuntu 18.04 LTS
Ubuntu-20.04    Ubuntu 20.04 LTS

Install Ubuntu 20.04.

PS C:\Users\hoge> wsl --install -d Ubuntu-20.04
ダウンロード中: Ubuntu 20.04 LTS
インストール中: Ubuntu 20.04 LTS
Ubuntu 20.04 LTS はインストールされました。
Ubuntu 20.04 LTS を起動しています...

[ex] Uninstalling Ubuntu 20.04.

PS C:\Users\hoge> wsl --shutdown
PS C:\Users\hoge> wsl --unregister Ubuntu-20.04

Configuring Ubuntu 20.04

Set the username and password.[2]

Installing, this may take a few minutes...
Please create a default UNIX user account. The username does not need to match your Windows username.
For more information visit: https://aka.ms/wslusers
Enter new UNIX username: hoge
New password:****
Retype new password:****
passwd: password updated successfully
Installation successful!
To run a command as administrator (user "root"), use "sudo 
 〜中略〜 
This message is shown once once a day. To disable it please create the
/home/hoge/.hushlogin file.
hoge@AA:~$

Stop the beep sound in Bash and Vim.

hoge@AA:~$ echo "set bell-style none" >> ~/.inputrc
hoge@AA:~$ echo "set visualbell t_vb=" >> ~/.vimrc

Update the package index.

hoge@AA:~$ sudo apt -y update
[sudo] password for hoge:
 〜中略〜 
Reading package lists... Done
Building dependency tree
Reading state information... Done
251 packages can be upgraded. Run 'apt list --upgradable' to see them.

Update the packages.

hoge@AA:~$ sudo apt -yV upgrade
Reading package lists... Done
Building dependency tree
Reading state information... Done
Calculating upgrade... Done
The following NEW packages will be installed:
   distro-info (0.23ubuntu1)
 〜中略〜 
Updating certificates in /etc/ssl/certs...
0 added, 0 removed; done.
Running hooks in /etc/ca-certificates/update.d...
done.

Installing Tools

Install to handle zip files.

hoge@AA:~$ sudo apt -y install zip

Install to check LocalStack's JSON responses.

hoge@AA:~$ sudo apt -y install jq

Restarting

hoge@AA:~$ exit

Check the status.

PS C:\Users\hoge> wsl -l -v
  NAME            STATE           VERSION
* Ubuntu-20.04    Running         2

Stop the distribution.

PS C:\Users\hoge> wsl -t Ubuntu-20.04

Start the distribution.

PS C:\Users\hoge> wsl ~
hoge@AA:~$

It is a good idea to create a snapshot if necessary.[3]

Preparing Docker

Installation

Add the necessary software.

hoge@AA:~$ sudo apt -y install \
apt-transport-https \
ca-certificates \
curl \
gnupg \
lsb-release

Add the GPG key.

hoge@AA:~$  curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg

Add the Docker package repository to apt.

hoge@AA:~$ echo \
"deb [arch=amd64 signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/ubuntu \
$(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null

Install Docker Engine.

hoge@AA:~$ sudo apt update
hoge@AA:~$ sudo apt -y install docker-ce docker-ce-cli containerd.io

Install docker-compose.

hoge@AA:~$ sudo curl -L "https://github.com/docker/compose/releases/download/1.29.2/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
hoge@AA:~$ sudo chmod +x /usr/local/bin/docker-compose
hoge@AA:~$ docker-compose --version
docker-compose version 1.29.2, build 5becea4c

Starting the Service

Start the Docker daemon.

hoge@AA:~$ sudo service docker start

[ex] Starting the service at login[4]

Open the sudo configuration file.

hoge@AA:~$ sudo visudo

Add the Docker settings to the last line. Change "hoge" to your own username.

hoge ALL=NOPASSWD: /usr/sbin/service docker start, /usr/sbin/service docker stop, /usr/sbin/service docker restart

Save and exit. Ctrl(control)+x, y, Enter.

Add the settings to .bashrc.

echo 'service docker status > /dev/null 2>&1
if [ $? = 1 ]; then
    sudo service docker start
fi' >> ~/.bashrc

Test if it starts at login.

hoge@AA:~$ sudo service docker stop
 * Stopping Docker: docker               [ OK ]
hoge@AA:~$ exit
PS C:\Users\hoge> wsl ~
 * Starting Docker: docker               [ OK ]

Setting Up for User Operation

Add the user to the docker group. Replace "hoge" with your configured username.

hoge@AA:~$ sudo usermod -aG docker hoge
hoge@AA:~$ exit
PS C:\Users\hoge> wsl ~

Running a Startup Test

Confirm if hello-world can be executed with Docker.[5]

hoge@AA:~$ docker run hello-world
Unable to find image 'hello-world:latest' locally
latest: Pulling from library/hello-world
2db29710123e: Pull complete
Digest: sha256:cc15c5b292d8525effc0f89cb299f1804f3a725c8d05e158653a563f15e4f685
Status: Downloaded newer image for hello-world:latest

Hello from Docker!
〜中略〜
Share images, automate workflows, and more with a free Docker ID:
 https://hub.docker.com/

For more examples and ideas, visit:
 https://docs.docker.com/get-started/

Preparing LocalStack

Prepare docker-compose.yml

Download LocalStack from GitHub and unzip it.[6]

hoge@AA:~$ curl -L -O https://github.com/localstack/localstack/archive/refs/tags/v0.13.0.zip
hoge@AA:~$ unzip v0.13.0.zip

Edit docker-compose.yml

hoge@AA:~$ vi localstack-0.13.0/docker-compose.yml

Changes:
Fixed the version to v0.13.1. Removed settings for the Pro version. Removed the IP binding (127.0.0.1:)[7][8]. Added LAMBDA_EXECUTOR for Lambda settings.[9]

docker-compose.yml (State after editing)
version: "3.8"

services:
  localstack:
    container_name: "${LOCALSTACK_DOCKER_NAME-localstack_main}"
    image: localstack/localstack:0.13.1
    network_mode: bridge
    ports:
      - "4566:4566"
      - "4571:4571"
    environment:
      - SERVICES=${SERVICES- }
      - DEBUG=${DEBUG- }
      - DATA_DIR=${DATA_DIR- }
      - LAMBDA_EXECUTOR=${LAMBDA_EXECUTOR- }
      - LOCALSTACK_API_KEY=${LOCALSTACK_API_KEY- }  # only required for Pro
      - HOST_TMP_FOLDER=${TMPDIR:-/tmp/}localstack
      - DOCKER_HOST=unix:///var/run/docker.sock
      - LAMBDA_EXECUTOR=docker-reuse
    volumes:
      - "${TMPDIR:-/tmp}/localstack:/tmp/localstack"
      - "/var/run/docker.sock:/var/run/docker.sock"

Save and exit editing docker-compose.yml. (In vi, press the Esc key. Type ":wq" and press Enter.)

Running

Start using docker-compose.[10]

hoge@AA:~$ cd localstack-0.13.0
hoge@AA:~/localstack-0.13.0$ docker-compose up -d
Pulling localstack (localstack/localstack:)...
a10c77af2613: Pull complete
 〜中略〜 
Digest: sha256:3fcef2507cd96b64a5efd94ab25e95fa58b46e97c1217e3c0e3cec352d8376ee
Status: Downloaded newer image for localstack/localstack:0.13.1
Creating localstack_main ... done

Check the startup status.

hoge@AA:~/localstack-0.13.0$ docker-compose ps
     Name               Command          State                                              Ports
---------------------------------------------------------------------------------------------------------------------------------------------
localstack_main   docker-entrypoint.sh   Up      0.0.0.0:4566->4566/tcp,:::4566->4566/tcp, 0.0.0.0:4571->4571/tcp,:::4571->4571/tcp, 5678/tcp

[ex] Stopping using docker-compose.

hoge@AA:~/localstack-0.13.0$ docker-compose down

Checking the Service Startup Status

Check the startup status of services available in LocalStack.[11]

hoge@AA~/localstack-0.13.0$  curl -s http://localhost:4566/health | jq
{
  "features": {
    "initScripts": "initialized"
  },
  "services": {
    "acm": "available",
    "apigateway": "available",
    "cloudformation": "available",
    "cloudwatch": "available",
    "config": "available",
    "dynamodb": "available",
    "dynamodbstreams": "available",
    "ec2": "available",
    "es": "available",
    "events": "available",
    "firehose": "available",
    "iam": "available",
    "kinesis": "available",
    "kms": "available",
    "lambda": "available",
    "logs": "available",
    "redshift": "available",
    "resource-groups": "available",
    "resourcegroupstaggingapi": "available",
    "route53": "available",
    "route53resolver": "available",
    "s3": "available",
    "secretsmanager": "available",
    "ses": "available",
    "sns": "available",
    "sqs": "available",
    "ssm": "available",
    "stepfunctions": "available",
    "sts": "available",
    "support": "available",
    "swf": "available"
  }
}

Accessing LocalStack

AWS CLI Version 2 Setup

Download and install awscli.[12]

hoge@AA:~$ cd
hoge@AA:~$ curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
hoge@AA:~$ unzip awscliv2.zip
hoge@AA:~$ sudo ./aws/install
hoge@AA:~$ aws --version
aws-cli/2.4.5 Python/3.8.8 Linux/5.10.16.3-microsoft-standard-WSL2 exe/x86_64.ubuntu.20 prompt/off
hoge@AA:~$ rm -fr aws
hoge@AA:~$ rm awscliv2.zip

Install awscli-local.

hoge@AA:~$ sudo apt -y install python3-pip
hoge@AA:~$ pip3 install awscli-local

Add the path so that the command can be executed.

hoge@AA:~$ echo 'export PATH="$PATH:~/.local/bin"' >> ~/.bashrc
hoge@AA:~$ source ~/.bashrc

Create dummy settings for LocalStack connection.

hoge@AA:~$ aws configure --profile=localstack
AWS Access Key ID [None]: dummy
AWS Secret Access Key [None]: dummy
Default region name [None]: ap-northeast-1
Default output format [None]: json

S3

Created buckets and added files will disappear when you run "docker-compose down".[13]

Create the hoge-bucket bucket.

hoge@AA:~$ awslocal s3 mb s3://hoge-bucket --profile=localstack
make_bucket: hoge-bucket

Confirm that the hoge-bucket bucket has been created.

hoge@AA:~$ awslocal s3 ls --profile=localstack
2020-09-02 22:36:16 hoge-bucket

Add a file to the hoge-bucket bucket and verify it.

hoge@AA:~$ echo 'Test Hoge Hoge!' > hoge.txt
hoge@AA:~$ awslocal s3 cp hoge.txt s3://hoge-bucket --profile=localstack
upload: ./hoge.txt to s3://hoge-bucket/hoge.txt
hoge@AA:~$ awslocal s3 ls s3://hoge-bucket --profile=localstack
2020-09-02 22:38:45         16 hoge.txt

Lambda

Preparing a Lambda Function

Create the code.

hoge@AA:~$ vi myFunction.py

The code assumes that the "hoge-bucket" bucket created in S3 already exists.[14]

myFunction.py
import os
import boto3
from boto3.session import Session
from datetime import datetime

session = Session(
    aws_access_key_id='dummy',
    aws_secret_access_key='dummy',
    region_name='ap-northeast-1'
)

if os.getenv('LOCALSTACK_HOSTNAME') is None:
    endpoint = 'http://localhost:4566'
else:
    endpoint=f"http://{os.environ['LOCALSTACK_HOSTNAME']}:4566"

s3 = session.resource(
    service_name='s3', 
    endpoint_url=endpoint
)

def lambda_handler(event, context):

    bucket = 'hoge-bucket'
    key = datetime.now().strftime('%Y-%m-%d-%H-%M-%S') + '.txt'
    file_contents = 'Python Lambda Save File'

    s3.Bucket(bucket).put_object(Key=key, Body=file_contents)

    return 'create file '+ key

Save the edits to myFunction.py and exit. (In vi, press the Esc key, type ":wq", and press Enter.)

Compress it into zip format.

hoge@AA:~$ zip lambda.zip myFunction.py

Registering and Executing with Lambda

Register the Function.[15]

hoge@AA:~$ awslocal lambda create-function --function-name myFunction  --runtime python3.8 --handler myFunction.lambda_handler --role r1  --zip-file fileb://lambda.zip --profile=localstack
{
    "FunctionName": "myFunction",
    "FunctionArn": "arn:aws:lambda:us-east-1:000000000000:function:myFunction",
    "Runtime": "python3.8",
    "Role": "r1",
    "Handler": "myFunction.lambda_handler",
    "CodeSize": 592,
    "Description": "",
    "Timeout": 3,
    "LastModified": "2020-10-03T08:20:15.032+0000",
    "CodeSha256": "SuHNSoLMS/zBPGAKiezjN/qL10Nntx2b3o5Z3SSivWY=",
    "Version": "$LATEST",
    "VpcConfig": {},
    "TracingConfig": {
        "Mode": "PassThrough"
    },
    "RevisionId": "a8380dc0-349f-4be4-949c-065a90431750",
    "State": "Active",
    "LastUpdateStatus": "Successful",
    "PackageType": "Zip",
    "Architectures": [
        "x86_64"
    ]
}

Execute the Function.[16]

hoge@AA:~$ awslocal lambda invoke --cli-binary-format raw-in-base64-out --function-name myFunction --profile=localstack result.log
hoge@AA:~$ cat result.log | jq
"create file2020-10-02-08-35-49.txt"

Confirm that the file has been added to S3.

hoge@AA:~$ awslocal s3 ls s3://hoge-bucket --profile=localstack
2020-10-02 08:35:49         23 2020-10-02-08-35-49.txt

[ex] Running LocalStack using Docker only

Run it without using docker-compose.[17]
You must have already completed adding the user to the docker group, installing python3-pip, and adding the path to ~/.local/bin.

Install LocalStack.

hoge@AA:~$ pip3 install localstack==0.13.1

Delete the files created when you ran docker-compose, as they will cause an error.[18]

hoge@AA:~$ sudo rm -fr /tmp/localstack/

Run LocalStack.
It will take some time for the first run as the Docker image needs to be downloaded.

hoge@AA:~$ LAMBDA_EXECUTOR=docker-reuse SERVICES=s3,lambda DEBUG=1 localstack start

Once it starts, you will be able to access S3 and Lambda.
Stop it with Ctrl(control)+c.

References

WSL2

Docker

LocalStack

脚注

  1. AWS Service Feature Coverage | LocalStack Docs ↩︎

  2. It will open in the command prompt. It is better to work in Windows Terminal after the setup. ↩︎

  3. How to redo "Ubuntu on WSL2" when it gets messy - Qiita ↩︎

  4. Might not work well if the environment was imported? ↩︎

  5. An error will occur if the user has not been correctly added to the group -> docker: Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Post "http://%2Fvar%2Frun%2Fdocker.sock/v1.24/containers/create": dial unix /var/run/docker.sock: connect: permission denied. ↩︎

  6. Since only the docker-compose.yml is needed, other methods of acquisition are also fine. ↩︎

  7. You need to access via IP address instead of localhost. You can check the IP address in the shell with ip addr. ↩︎

  8. Additionally, if you want to access from another PC, port forwarding and firewall settings are required. Accessing servers started on WSL2 from external devices – GUNMA GIS GEEK ↩︎

  9. Setting LAMBDA_DOCKER_NETWORK=host causes a "listen tcp :9001: bind: address already in use" error. ↩︎

  10. You can check the logs by running docker-compose up without the -d flag. ↩︎

  11. It may take some time for the services to start, and you might encounter errors. In that case, wait a little while before accessing. ↩︎

  12. Installing, updating, and uninstalling the AWS CLI version 2 on Linux - AWS Command Line Interface ↩︎

  13. There is also a configuration for persistence. Persisting data saved in LocalStack - CLOVER🍀 ↩︎

  14. When accessing S3 or DynamoDB from a Lambda function, you need to obtain the hostname using os.environ['LOCALSTACK_HOSTNAME'] and set the endpoint. ↩︎

  15. The --handler myFunction.lambda_handler must be specified as "filename.function_name". ↩︎

  16. It will take a little while to start the first time because the Docker image needs to be downloaded. ↩︎

  17. If it is currently running, stop it with docker-compose down first. ↩︎

  18. PermissionError: [Errno 13] Permission denied: '/tmp/localstack/cache' ↩︎

Discussion