Zenn
🦖

Dynamips/Dynagenラボ構築

2023/02/12に公開
Ansible
dynamips
dynagen
tech

本エントリについて

時代遅れ感がありますが Dynamips/Dynagen を利用したラボ環境を構築します。
Cisco機器についての基本的な学習のためにはDynagenを利用する環境でも充分かと思います。

  • 環境はAWSのEC2インスタンスに作ります。
    勉強するとき以外は使わない環境なので、自宅PCを奇麗に保つため、自宅PCリソースや電気代等の節約のため、EC2インスタンスを利用します。
    考え方次第ですが、自宅PCでKVM等の上で作ってもよいかもしれません。
  • コスト削減のため、スポットインスタンスを利用します。
    不必要な時に停止することで、さらにコストを押さえます。
  • 環境構築は terraform を利用します。
    CloudFormation でもいいかもしれませんが、趣味で。

構築資材の準備

事前に Terraformの利用環境は構築済みで、AWS のアカウント情報も利用可能となっていることを想定しています。

tf ファイル

およそ以下のような tf ファイルを準備します。

provider "aws" {
  region = "ap-northeast-1"
}

# variables
variable "ami" {}
variable "instance_type" {}

variable "key_name" {
  default = "testkey"
}

variable "userdata" {
  default = "userdata.sh"
}

data "aws_cloudformation_export" "public_subnet" {
  name = "mystack-PublicSubnet1"
}

data "aws_cloudformation_export" "security_group" {
  name = "mystack-SecurityGroupPub"
}

# ec2 spot instance request
resource "aws_spot_instance_request" "spot_instance_dynamips_labo" {
  spot_type                      = "persistent"
  instance_interruption_behavior = "stop"

  ami           = var.ami
  instance_type = var.instance_type
  key_name      = var.key_name
  subnet_id     = data.aws_cloudformation_export.public_subnet.value
  vpc_security_group_ids = [
    data.aws_cloudformation_export.security_group.value
  ]
  user_data = file(var.userdata)

  associate_public_ip_address = "true"
  tags = {
    Name  = "spot_instance_dynamips_labo",
    Built = "terraform"
  }
}

# output
output "spot_instance_requestid" {
  value = aws_spot_instance_request.spot_instance_dynamips_labo.id
}
  • VPC、サブネット、セキュリティグループ、ssh鍵は作成済みのものを利用します。
    サブネットとセキュリティグループは、別に CloudFormation で作成していたものを利用しています。
  • インスタンスタイプとAMIは、構築時に指定します。AMIは Debian Buster (amd64) の最新版を利用します。

ユーザデータファイル

ユーザデータファイルを準備します。

#! /bin/sh

version=0.2.22
sed -i.bak 's/ main$/ main contrib non-free/' /etc/apt/sources.list \
&& apt update && apt upgrade -y \
&& DEBIAN_FRONTEND=noninteractive apt install -y \
 vpcs \
 openvswitch-switch \
 snmp snmptrapd \
 freeradius \
 postfix mailutils \
 tcpdump \
 tftpd-hpa tftp-hpa \
 telnet \
 tmux \
 python3-pip sshpass python3-paramiko \
&& curl -fsSL https://toolbelt.treasuredata.com/sh/install-debian-buster-td-agent4.sh | sudo sh \
&& wget https://github.com/GNS3/dynamips/archive/refs/tags/v${version}.tar.gz \
&& tar zxf v${version}.tar.gz \
&& rm v${version}.tar.gz \
&& cd dynamips-${version} \
&& apt install -y gcc cmake libelf-dev libpcap-dev make musl-dev linux-headers-amd64 \
&& mkdir -p build \
&& cd build \
&& cmake .. \
&& make \
&& make install \
&& rm -rf dynamips-${version} \
&& apt remove -y gcc cmake make musl-dev linux-headers \
&& apt autoremove -y \
&& apt install -y dynagen \
&& pip3 install ansible \
&& ovs-vsctl add-br br0 \
&& ip link set br0 up \
&& ip tuntap add dev tap0 mode tap \
&& ovs-vsctl add-port br0 tap0 \
&& ip link set tap0 up \
&& touch /home/admin/startup.sh \
&& chmod +x /home/admin/startup.sh \
&& cat << END > /home/admin/startup.sh
#! /bin/sh
ip address add 10.2.0.1/24 brd + dev br0
ip tuntap add dev tap0 mode tap
ip link set tap0 up
ip link set br0 up
ip route add 10.2.0.0/16 dev br0
ip addr show br0
ovs-vsctl show
dynamips -H 7200 &
ps -ef | grep dynamips | grep -v grep
END
  • dynamips はソースコードからビルドします。
  • 各種検証に必要と思われるツール類もインストールします。
  • 仮想スイッチは、Open vSwitch を利用してみます。
  • 仮想ブリッジのアドレスはOSを起動するたびに毎回割り当てることにし、そのための簡易スクリプトを用意します。

Makefile

およそ以下のようなMakefile を用意します。

instance_type=t3.nano
plan=terraform.plan

ami=$(shell aws ssm get-parameters --names /aws/service/debian/release/buster/latest/amd64 --region ap-northeast-1 --query Parameters[].Value --output text)
sir=$(shell terraform output -json | jq -r ".spot_instance_requestid.value")
terraform_opt=-var ami=$(ami) -var instance_type=$(instance_type)
instance_id=$(shell aws ec2 describe-instances --filter "Name=spot-instance-request-id,Values=$(sir)" --query 'Reservations[].Instances[].InstanceId' --output text)

terraform-init:
        terraform init
terraform-plan:
        terraform validate
        terraform plan $(terraform_opt) -out $(plan)
terraform-plan-destroy:
        terraform plan -out $(plan)
terraform-apply:
        terraform apply $(plan)
terraform-clean:
        rm -rf .terraform/providers $(plan)

list-instance:
        aws ec2 describe-spot-instance-requests --query 'SpotInstanceRequests[?Tags[?Key == `Name`].Value | [0] == `spot_instance_dynamips_labo`].{"SpotInstanceRequestId":SpotInstanceRequestId, "InstanceId":InstanceId, "Status":Status , "State":State}'
        aws ec2 describe-instances --filter "Name=spot-instance-request-id,Values=$(sir)" --query 'Reservations[].Instances[].{"State":State, "InstanceId":InstanceId, "PublicDnsName":PublicDnsName}'

stop-instance:
        aws ec2 stop-instances --instance-id $(instance_id)
start-instance:
        aws ec2 start-instances --instance-id $(instance_id)

cancel-request:
        aws ec2 terminate-instances --instance-id $(instance_id)
        aws ec2 cancel-spot-instance-requests --spot-instance-request-ids $(sir)
  • インスタンスタイプは、ここでは最低限の t3.nano を指定していますが、実際は適切なサイズを指定します。
  • AMIはパブリックパラメータからAMIは Debian Buster (amd64) の最新版のAMI IDを取得します。

環境構築

$ make terraform-init
terraform init

Initializing the backend...

Initializing provider plugins...
- Reusing previous version of hashicorp/aws from the dependency lock file
- Installing hashicorp/aws v4.51.0...
- Installed hashicorp/aws v4.51.0 (signed by HashiCorp)

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.

$ make terraform-plan | cat                                                                                                                                                                                                [105/493]
terraform validate
Success! The configuration is valid.

terraform plan -var ami=ami-041e370f47f9b619e -var instance_type=t3.nano -out terraform.plan
data.aws_cloudformation_export.public_subnet: Reading...
data.aws_cloudformation_export.security_group: Reading...
data.aws_cloudformation_export.security_group: Read complete after 0s [id=cloudformation-exports-ap-northeast-1-mystack-SecurityGroupPub]
data.aws_cloudformation_export.public_subnet: Read complete after 0s [id=cloudformation-exports-ap-northeast-1-mystack-PublicSubnet1]

Terraform used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # aws_spot_instance_request.spot_instance_dynamips_labo will be created
  + resource "aws_spot_instance_request" "spot_instance_dynamips_labo" {

(省略)

    }

Plan: 1 to add, 0 to change, 0 to destroy.

Changes to Outputs:
  + spot_instance_requestid = (known after apply)

─────────────────────────────────────────────────────────────────────────────

Saved the plan to: terraform.plan

To perform exactly these actions, run the following command to apply:
    terraform apply "terraform.plan"

エラーがなければ apply します。

-bash-4.2$ make terraform-apply | cat
terraform apply terraform.plan
aws_spot_instance_request.spot_instance_dynamips_labo: Creating...
aws_spot_instance_request.spot_instance_dynamips_labo: Creation complete after 2s [id=sir-5rapkqtj]

Apply complete! Resources: 1 added, 0 changed, 0 destroyed.

Outputs:

spot_instance_requestid = "sir-5rapkqtj"

起動したスポットインスタンスの情報を確認します。

$ make list-instance
aws ec2 describe-spot-instance-requests --query 'SpotInstanceRequests[?Tags[?Key == `Name`].Value | [0] == `spot_instance_dynamips_labo`].{"SpotInstanceRequestId":SpotInstanceRequestId, "InstanceId":InstanceId, "Status":Status , "State":State}'
[
    {
        "InstanceId": "i-0bbd73d8079d1cdfd",
        "Status": {
            "Message": "Your spot request is fulfilled.",
            "Code": "fulfilled",
            "UpdateTime": "2023-01-22T07:03:18.000Z"
        },
        "State": "active",
        "SpotInstanceRequestId": "sir-5rapkqtj"
    }
]
aws ec2 describe-instances --filter "Name=spot-instance-request-id,Values=sir-5rapkqtj" --query 'Reservations[].Instances[].{"State":State, "InstanceId":InstanceId, "PublicDnsName":PublicDnsName}'
[
    {
        "InstanceId": "i-0bbd73d8079d1cdfd",
        "State": {
            "Code": 16,
            "Name": "running"
        },
        "PublicDnsName": "ec2-54-250-21-140.ap-northeast-1.compute.amazonaws.com"
    }
]

作成したインスタンスの PublicDnsName に対して ssh で入り、下記を確認します。

  • ユーザデータが正常に実行できたこと
  • 仮想スイッチが構築できること
admin@ip-10-0-0-144:~$ tail -f /var/log/cloud-init-output.log
Unpacking python-configobj (5.0.6-3) ...
Selecting previously unselected package dynagen.
Preparing to unpack .../dynagen_0.11.0-7_all.deb ...
Unpacking dynagen (0.11.0-7) ...
Setting up python-configobj (5.0.6-3) ...
Setting up dynamips (0.2.14-1) ...
Setting up dynagen (0.11.0-7) ...
Processing triggers for man-db (2.8.5-2) ...
Cloud-init v. 20.2 running 'modules:final' at Sun, 22 Jan 2023 07:03:31 +0000. Up 8.74 seconds.
Cloud-init v. 20.2 finished at Sun, 22 Jan 2023 07:06:56 +0000. Datasource DataSourceEc2Local.  Up 213.28 seconds

$ sudo ./startup.sh
RTNETLINK answers: File exists
ioctl(TUNSETIFF): Device or resource busy
RTNETLINK answers: File exists
4: br0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000
    link/ether 3e:96:7d:04:10:4f brd ff:ff:ff:ff:ff:ff
    inet 10.2.0.1/24 brd 10.2.0.255 scope global br0
       valid_lft forever preferred_lft forever
    inet6 fe80::3c96:7dff:fe04:104f/64 scope link
       valid_lft forever preferred_lft forever
709cf55a-b676-4863-86b4-913bbeca30dc
    Bridge "br0"
        Port "tap0"
            Interface "tap0"
        Port "br0"
            Interface "br0"
                type: internal
    ovs_version: "2.10.7"
Cisco Router Simulation Platform (version 0.2.22-amd64/Linux stable)
Copyright (c) 2005-2011 Christophe Fillot.
Build date: Jan 22 2023 07:06:18

ILT: loaded table "mips64j" from cache.
ILT: loaded table "mips64e" from cache.
ILT: loaded table "ppc32j" from cache.
ILT: loaded table "ppc32e" from cache.
Hypervisor TCP control server started (port 7200).
root     21525 21515  0 07:42 pts/1    00:00:00 dynamips -H 7200

インスタンスの停止

persist を指定したスポットインスタンスは、インスタンスを停止することができます。
また、停止したインスタンスは、必要な時に再開することができます。
EIPアドレスを割り当てていないので、停止している間の課金は、ディスク領域に関わる料金だけになります。

-bash-4.2$ make stop-instance
aws ec2 stop-instances --instance-id i-0bbd73d8079d1cdfd
{
    "StoppingInstances": [
        {
            "InstanceId": "i-0bbd73d8079d1cdfd",
            "CurrentState": {
                "Code": 64,
                "Name": "stopping"
            },
            "PreviousState": {
                "Code": 16,
                "Name": "running"
            }
        }
    ]
}
-bash-4.2$

インスタンスの再開

停止していたインスタンスを再開します。

-bash-4.2$ make start-instance
aws ec2 start-instances --instance-id i-0bbd73d8079d1cdfd
{
    "StartingInstances": [
        {
            "InstanceId": "i-0bbd73d8079d1cdfd",
            "CurrentState": {
                "Code": 0,
                "Name": "pending"
            },
            "PreviousState": {
                "Code": 80,
                "Name": "stopped"
            }
        }
    ]
}

PublicDnsName を確認します。

-bash-4.2$ make list-instance
aws ec2 describe-spot-instance-requests --query 'SpotInstanceRequests[?Tags[?Key == `Name`].Value | [0] == `spot_instance_dynamips_labo`].{"SpotInstanceRequestId":SpotInstanceRequestId, "InstanceId":InstanceId, "Status":Status , "State":State}'
[
    {
        "InstanceId": "i-0bbd73d8079d1cdfd",
        "Status": {
            "Message": "Your Spot request is fulfilled.",
            "Code": "fulfilled",
            "UpdateTime": "2023-01-22T07:57:00.000Z"
        },
        "State": "active",
        "SpotInstanceRequestId": "sir-5rapkqtj"
    }
]
aws ec2 describe-instances --filter "Name=spot-instance-request-id,Values=sir-5rapkqtj" --query 'Reservations[].Instances[].{"State":State, "InstanceId":InstanceId, "PublicDnsName":PublicDnsName}'
[
    {
        "InstanceId": "i-0bbd73d8079d1cdfd",
        "State": {
            "Code": 16,
            "Name": "running"
        },
        "PublicDnsName": "ec2-54-95-70-193.ap-northeast-1.compute.amazonaws.com"
    }
]

スポットインスタンス契約の解除

長期間使用しないときは、AMIを保存して、スポットインスタンス契約を解除します。
スポットインタンス契約解除の前にAMIを保存しておき、環境を再利用する時は、AMIから起動します。

AMI 保存

-bash-4.2$ make list-instance
aws ec2 describe-spot-instance-requests --query 'SpotInstanceRequests[?Tags[?Key == `Name`].Value | [0] == `spot_instance_dynamips_labo`].{"SpotInstanceRequestId":SpotInstanceRequestId, "InstanceId":InstanceId, "Status":Status , "State":State}'
[
    {
        "InstanceId": "i-0bbd73d8079d1cdfd",
        "Status": {
            "Message": "Your Spot instance is marked for stop due to user-initiated stop.",
            "Code": "marked-for-stop",
            "UpdateTime": "2023-01-22T08:09:17.000Z"
        },
        "State": "active",
        "SpotInstanceRequestId": "sir-5rapkqtj"
    }
]
aws ec2 describe-instances --filter "Name=spot-instance-request-id,Values=sir-5rapkqtj" --query 'Reservations[].Instances[].{"State":State, "InstanceId":InstanceId, "PublicDnsName":PublicDnsName}'
[
    {
        "InstanceId": "i-0bbd73d8079d1cdfd",
        "State": {
            "Code": 80,
            "Name": "stopped"
        },
        "PublicDnsName": ""
    }
]

-bash-4.2$ make create-ami
aws ec2 create-image --description "Backup AMI for Dynamips Labo environment from sir-5rapkqtj" --instance-id i-0bbd73d8079d1cdfd --name "Dynamips Labo environment 20230122081059"
{
    "ImageId": "ami-0314fbc3211babfa8"
}

-bash-4.2$ aws ec2 describe-images --image-id ami-0314fbc3211babfa8
{
    "Images": [
        {
            "VirtualizationType": "hvm",
            "Description": "Backup AMI for Dynamips Labo environment from sir-5rapkqtj",
            "PlatformDetails": "Linux/UNIX",
            "EnaSupport": true,
            "Hypervisor": "xen",
            "State": "pending",
            "SriovNetSupport": "simple",
            "ImageId": "ami-0314fbc3211babfa8",
            "UsageOperation": "RunInstances",
            "BlockDeviceMappings": [
                {
                    "DeviceName": "/dev/xvda",
                    "Ebs": {
                        "SnapshotId": "snap-0133e49aae5f72d0b",
                        "DeleteOnTermination": true,
                        "VolumeType": "gp2",
                        "VolumeSize": 8,
                        "Encrypted": false
                    }
                }
            ],
            "Architecture": "x86_64",
            "ImageLocation": "xxxxxxxxxxxx/Dynamips Labo environment 20230122081059",
            "RootDeviceType": "ebs",
            "OwnerId": "xxxxxxxxxxxx",
            "RootDeviceName": "/dev/xvda",
            "CreationDate": "2023-01-22T08:11:02.000Z",
            "Public": false,
            "ImageType": "machine",
            "Name": "Dynamips Labo environment 20230122081059"
        }
    ]
}

スポットインスタンス解約

-bash-4.2$ make cancel-request
aws ec2 terminate-instances --instance-id i-0bbd73d8079d1cdfd
{
    "TerminatingInstances": [
        {
            "InstanceId": "i-0bbd73d8079d1cdfd",
            "CurrentState": {
                "Code": 48,
                "Name": "terminated"
            },
            "PreviousState": {
                "Code": 80,
                "Name": "stopped"
            }
        }
    ]
}
aws ec2 cancel-spot-instance-requests --spot-instance-request-ids sir-5rapkqtj
{
    "CancelledSpotInstanceRequests": [
        {
            "State": "cancelled",
            "SpotInstanceRequestId": "sir-5rapkqtj"
        }
    ]
}

-bash-4.2$ make list-instance
aws ec2 describe-spot-instance-requests --query 'SpotInstanceRequests[?Tags[?Key == `Name`].Value | [0] == `spot_instance_dynamips_labo`].{"SpotInstanceRequestId":SpotInstanceRequestId, "InstanceId":InstanceId, "Status":Status , "State":State}'
[
    {
        "InstanceId": "i-0bbd73d8079d1cdfd",
        "Status": {
            "Message": "Your Spot request is canceled.",
            "Code": "request-canceled",
            "UpdateTime": "2023-01-22T08:14:12.000Z"
        },
        "State": "cancelled",
        "SpotInstanceRequestId": "sir-5rapkqtj"
    }
]
aws ec2 describe-instances --filter "Name=spot-instance-request-id,Values=sir-5rapkqtj" --query 'Reservations[].Instances[].{"State":State, "InstanceId":InstanceId, "PublicDnsName":PublicDnsName}'
[
    {
        "InstanceId": "i-0bbd73d8079d1cdfd",
        "State": {
            "Code": 48,
            "Name": "terminated"
        },
        "PublicDnsName": ""
    }
]
-bash-4.2$

スポットインスタンスのState が cancelled になったら、Terraform 上でも設定を削除します。

-bash-4.2$ mv spot_request.tf spot_request.tf.destroy
-bash-4.2$
-bash-4.2$ make terraform-plan-destroy
terraform plan -out terraform.plan
aws_spot_instance_request.spot_instance_dynamips_labo: Refreshing state... [id=sir-5rapkqtj]

Changes to Outputs:
  - spot_instance_requestid = "sir-5rapkqtj" -> null

You can apply this plan to save these new output values to the Terraform state, without changing any real infrastructure.

────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────

Saved the plan to: terraform.plan

To perform exactly these actions, run the following command to apply:
    terraform apply "terraform.plan"
-bash-4.2$
-bash-4.2$ make terraform-apply
terraform apply terraform.plan

Apply complete! Resources: 0 added, 0 changed, 0 destroyed.


-bash-4.2$
GitHubで編集を提案

Discussion