💨
Ubuntu24.04 で Open vSwitch を試す
環境
$ sudo ovs-vsctl --version
ovs-vsctl (Open vSwitch) 3.3.0
DB Schema 8.5.0
$ cat /etc/os-release
PRETTY_NAME="Ubuntu 24.04.2 LTS"
NAME="Ubuntu"
VERSION_ID="24.04"
VERSION="24.04.2 LTS (Noble Numbat)"
VERSION_CODENAME=noble
ID=ubuntu
ID_LIKE=debian
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
UBUNTU_CODENAME=noble
LOGO=ubuntu-logo
Cisco Switchとの間でトランクポートを試す
Open vSwitchの設定
利用する tun デバイスを作成
sudo ip tuntap add dev tap00 mode tap
sudo ip tuntap add dev tap11 mode tap
sudo ip tuntap add dev tap12 mode tap
sudo ip tuntap add dev tap21 mode tap
sudo ip tuntap add dev tap22 mode tap
OVSの設定
sudo ovs-vsctl add-port br0 tap00 trunks=10,20
sudo ovs-vsctl add-port br0 tap11 tag=10
sudo ovs-vsctl add-port br0 tap12 tag=10
sudo ovs-vsctl add-port br0 tap21 tag=20
sudo ovs-vsctl add-port br0 tap22 tag=20
確認
$ sudo ovs-vsctl show
(略)
Bridge br0
Port tap11
tag: 10
Interface tap11
Port tap00
trunks: [10, 20]
Interface tap00
Port tap21
tag: 20
Interface tap21
Port tap0
Interface tap0
Port tap12
tag: 10
Interface tap12
Port br0
Interface br0
type: internal
Port tap22
tag: 20
Interface tap22
ovs_version: "3.3.0"
リンクアップさせる
sudo ip link set tap00 up
sudo ip link set tap11 up
sudo ip link set tap12 up
sudo ip link set tap21 up
sudo ip link set tap22 up
確認
$ ip link
(略)
18: tap11: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel master ovs-system state UP mode DEFAULT group default qlen 1000
link/ether fe:f1:e4:b4:2a:af brd ff:ff:ff:ff:ff:ff
19: tap00: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel master ovs-system state UP mode DEFAULT group default qlen 1000
link/ether e2:ab:0c:23:49:c8 brd ff:ff:ff:ff:ff:ff
20: tap12: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel master ovs-system state UP mode DEFAULT group default qlen 1000
link/ether a2:3d:50:c9:3c:a7 brd ff:ff:ff:ff:ff:ff
21: tap21: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel master ovs-system state UP mode DEFAULT group default qlen 1000
link/ether 82:13:2f:69:6a:2a brd ff:ff:ff:ff:ff:ff
22: tap22: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel master ovs-system state UP mode DEFAULT group default qlen 1000
link/ether 0e:bb:62:f8:55:6c brd ff:ff:ff:ff:ff:ff
Ciscoスイッチの設定
VLAN作成
vlan 10
name vlan0010
vlan 20
name vlan0020
アクセスポート設定
int range fa1/0 - 3
switchport mode access
switchport access vlan 10
int range fa1/4 - 7
switchport mode access
switchport access vlan 20
トランクポート設定
int fa1/15
switchport trunk encapsulation dot1q
switchport mode trunk
確認
ESW1#show vlan-s
VLAN Name Status Ports
---- -------------------------------- --------- -------------------------------
1 default active Fa1/8, Fa1/9, Fa1/10, Fa1/11
Fa1/12, Fa1/13, Fa1/14
10 vlan0010 active Fa1/0, Fa1/1, Fa1/2, Fa1/3
20 vlan0020 active Fa1/4, Fa1/5, Fa1/6, Fa1/7
1002 fddi-default act/unsup
1003 token-ring-default act/unsup
1004 fddinet-default act/unsup
1005 trnet-default act/unsup
VLAN Type SAID MTU Parent RingNo BridgeNo Stp BrdgMode Trans1 Trans2
---- ----- ---------- ----- ------ ------ -------- ---- -------- ------ ------
1 enet 100001 1500 - - - - - 1002 1003
10 enet 100010 1500 - - - - - 0 0
20 enet 100020 1500 - - - - - 0 0
1002 fddi 101002 1500 - - - - - 1 1003
1003 tr 101003 1500 1005 0 - - srb 1 1002
1004 fdnet 101004 1500 - - 1 ibm - 0 0
1005 trnet 101005 1500 - - 1 ibm - 0 0
ESW1#show int trunk
Port Mode Encapsulation Status Native vlan
Fa1/15 on 802.1q trunking 1
Port Vlans allowed on trunk
Fa1/15 1-4094
Port Vlans allowed and active in management domain
Fa1/15 1,10,20
Port Vlans in spanning tree forwarding state and not pruned
Fa1/15 1,10,20
仮想マシン設定
仮想マシンは Deib1qn12 を利用
# cat /etc/os-release
PRETTY_NAME="Debian GNU/Linux 12 (bookworm)"
NAME="Debian GNU/Linux"
VERSION_ID="12"
VERSION="12 (bookworm)"
VERSION_CODENAME=bookworm
ID=debian
HOME_URL="https://www.debian.org/"
SUPPORT_URL="https://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/"
debian1
ip address add 192.168.10.1/24 dev ens3
ip address add 192.168.20.1/24 dev ens4
ip link set ens3 up
ip link set ens4 up
確認
$ ip a
2: ens3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
link/ether 0c:8f:23:a9:00:00 brd ff:ff:ff:ff:ff:ff
altname enp0s3
inet 192.168.10.1/24 scope global ens3
valid_lft forever preferred_lft forever
inet6 fe80::e8f:23ff:fea9:0/64 scope link
valid_lft forever preferred_lft forever
3: ens4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
link/ether 0c:8f:23:a9:00:01 brd ff:ff:ff:ff:ff:ff
altname enp0s4
inet 192.168.20.1/24 scope global ens4
valid_lft forever preferred_lft forever
inet6 fe80::e8f:23ff:fea9:1/64 scope link
valid_lft forever preferred_lft forever
debian2
ip address add 192.168.10.2/24 dev ens3
ip address add 192.168.20.2/24 dev ens4
ip link set ens3 up
ip link set ens4 up
疎通確認
Ciscoスイッチ側のVPCと、仮想マシンのLinuxの間で疎通確認を実施する。
vpc1
ip 192.168.10.11 255.255.255.0 192.168.10.254
vpc2
ip 192.168.20.11 255.255.255.0 192.168.20.254
Debian の Bonding(Active-backup) を試す
OVS の設定
不要な設定を削除
sudo ovs-vsctl remove br0 tap11 tag=10
sudo ovs-vsctl remove br0 tap12 tag=10
sudo ovs-vsctl remove br0 tap21 tag=20
sudo ovs-vsctl remove br0 tap22 tag=20
sudo ovs-vsctl del-port br0 tap11
sudo ovs-vsctl del-port br0 tap12
sudo ovs-vsctl del-port br0 tap21
sudo ovs-vsctl del-port br0 tap22
確認
$ sudo ovs-vsctl show
(略)
Bridge br0
Port tap00
trunks: [10, 20]
Interface tap00
Port tap0
Interface tap0
Port br0
Interface br0
type: internal
ovs_version: "3.3.0"
Bonding Active-backup トランクポート設定
sudo ovs-vsctl add-bond br0 bond1 tap11 tap12
確認
$ sudo ovs-vsctl show
Bridge br0
Port tap00
trunks: [10, 20]
Interface tap00
Port tap0
Interface tap0
Port bond1
Interface tap11
Interface tap12
Port br0
Interface br0
type: internal
ovs_version: "3.3.0"
$ sudo ovs-appctl bond/show bond1
---- bond1 ----
bond_mode: active-backup
bond may use recirculation: no, Recirc-ID : -1
bond-hash-basis: 0
lb_output action: disabled, bond-id: -1
updelay: 0 ms
downdelay: 0 ms
lacp_status: off
lacp_fallback_ab: false
active-backup primary: <none>
active member mac: fe:f1:e4:b4:2a:af(tap11)
member tap11: enabled
active member
may_enable: true
member tap12: enabled
may_enable: true
仮想マシン設定
必要なパッケージをインストール
apt update && apt install ifenslave ethtool tcpdump
Active-backup モードで bonding 設定
/etc/network/interfaces 抜粋
auto bond0
iface bond0 inet static
address 10.2.0.254
netmask 255.255.255.0
network 10.2.0.0
#gateway x.x.x.x
bond-slaves ens3 ens4
bond-mode active-backup
bond-miimon 100
bond-downdelay 200
bond-updelay 200
設定反映
systemctl restart networking
確認
# lsmod | grep bond
bonding 221184 0
tls 135168 1 bonding
# ethtool bond0
Settings for bond0:
Supported ports: [ ]
Supported link modes: Not reported
Supported pause frame use: No
Supports auto-negotiation: No
Supported FEC modes: Not reported
Advertised link modes: Not reported
Advertised pause frame use: No
Advertised auto-negotiation: No
Advertised FEC modes: Not reported
Speed: 10000Mb/s
Duplex: Full
Auto-negotiation: off
Port: Other
PHYAD: 0
Transceiver: internal
Link detected: yes
# ip a show bond0
4: bond0: <BROADCAST,MULTICAST,MASTER,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 0c:8f:23:a9:00:00 brd ff:ff:ff:ff:ff:ff
inet 10.2.0.254/24 brd 10.2.0.255 scope global bond0
valid_lft forever preferred_lft forever
inet6 fe80::e8f:23ff:fea9:0/64 scope link
valid_lft forever preferred_lft forever
疎通確認
(割愛)
Debian の Bonding(LACP)との接続を試す
OVS 設定
sudo ovs-vsctl set port bond1 \
lacp=active \
bond_mode=balance-tcp \
other_config:lacp-time=fast \
other_config:bond-hash-policy=layer2+3
確認
$ sudo ovs-appctl bond/show bond1
---- bond1 ----
bond_mode: balance-tcp
bond may use recirculation: yes, Recirc-ID : 1
bond-hash-basis: 0
lb_output action: disabled, bond-id: -1
updelay: 0 ms
downdelay: 0 ms
next rebalance: 3883 ms
lacp_status: configured
lacp_fallback_ab: false
active-backup primary: <none>
active member mac: 00:00:00:00:00:00(none)
member tap11: disabled
may_enable: false
member tap12: disabled
may_enable: false
仮想マシン設定
LACP設定
auto bond0
iface bond0 inet static
address 10.2.0.254
netmask 255.255.255.0
network 10.2.0.0
#gateway x.x.x.x
bond-slaves ens3 ens4
bond-mode 802.3ad
bond-miimon 100
bond-downdelay 200
bond-updelay 200
bond-lacp-rate 1
bond-xmit-hash-policy layer2+3
設定反映
systemctl restart networking
確認
# ip a show bond0
6: bond0: <BROADCAST,MULTICAST,MASTER,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 0c:8f:23:a9:00:00 brd ff:ff:ff:ff:ff:ff
inet 10.2.0.254/24 brd 10.2.0.255 scope global bond0
valid_lft forever preferred_lft forever
inet6 fe80::e8f:23ff:fea9:0/64 scope link
valid_lft forever preferred_lft forever
# ethtool bond0
Settings for bond0:
Supported ports: [ ]
Supported link modes: Not reported
Supported pause frame use: No
Supports auto-negotiation: No
Supported FEC modes: Not reported
Advertised link modes: Not reported
Advertised pause frame use: No
Advertised auto-negotiation: No
Advertised FEC modes: Not reported
Speed: 10000Mb/s
Duplex: Full
Auto-negotiation: off
Port: Other
PHYAD: 0
Transceiver: internal
Link detected: yes
OVS側で確認すると、lacp_status の値が configured から negotiated へ変化する。
$ sudo ovs-appctl bond/show bond1
---- bond1 ----
bond_mode: balance-tcp
bond may use recirculation: yes, Recirc-ID : 1
bond-hash-basis: 0
lb_output action: disabled, bond-id: -1
updelay: 0 ms
downdelay: 0 ms
next rebalance: 2508 ms
lacp_status: negotiated
lacp_fallback_ab: false
active-backup primary: <none>
active member mac: fe:f1:e4:b4:2a:af(tap11)
member tap11: enabled
active member
may_enable: true
hash 218: 16777215 kB load
member tap12: enabled
may_enable: true
Debian の Bonding(LACP) トランクポートとの接続を試す
OVS
OVSの各ポートをトランクポートに変更する。
sudo ovs-vsctl set port bond1 trunks=10,20
確認
$ sudo ovs-vsctl show
Bridge br0
Port tap00
trunks: [10, 20]
Interface tap00
Port tap0
Interface tap0
Port bond1
trunks: [10, 20]
Interface tap11
Interface tap12
Port br0
Interface br0
type: internal
ovs_version: "3.3.0"
仮想マシン設定
auto bond0
iface bond0 inet manual
#address 10.2.0.254
#netmask 255.255.255.0
#network 10.2.0.0
#gateway x.x.x.x
bond-slaves ens3 ens4
bond-mode 802.3ad
bond-miimon 100
bond-downdelay 200
bond-updelay 200
bond-lacp-rate 1
bond-xmit-hash-policy layer2+3
up ifconfig bond0 0.0.0.0 up
auto bond0.10
iface bond0.10 inet static
address 10.2.10.254
netmask 255.255.255.0
vlan-raw-device bond0
auto bond0.20
iface bond0.20 inet static
address 10.2.20.254
netmask 255.255.255.0
vlan-raw-device bond0
systemctl restart networking 実行したら、原因不明だが以下のメッセージが出た
Job for networking.service failed because the control process exited with error code.
See "systemctl status networking.service" and "journalctl -xeu networking.service" for details.
このときのシスログ
Jul 21 10:26:17 debian ifup[1261]: ifup: failed to bring up bond0.20
Jul 21 10:26:17 debian systemd[1]: networking.service: Main process exited, cod>
確認
# ip a show bond0
7: bond0: <BROADCAST,MULTICAST,MASTER,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 0c:8f:23:a9:00:00 brd ff:ff:ff:ff:ff:ff
inet6 fe80::e8f:23ff:fea9:0/64 scope link
valid_lft forever preferred_lft forever
# ip a show bond0.10
8: bond0.10@bond0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 0c:8f:23:a9:00:00 brd ff:ff:ff:ff:ff:ff
inet 10.2.10.254/24 brd 10.2.10.255 scope global bond0.10
valid_lft forever preferred_lft forever
inet6 fe80::e8f:23ff:fea9:0/64 scope link
valid_lft forever preferred_lft forever
# ip a show bond0.20
9: bond0.20@bond0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether 0c:8f:23:a9:00:00 brd ff:ff:ff:ff:ff:ff
inet 10.2.20.254/24 brd 10.2.20.255 scope global bond0.20
valid_lft forever preferred_lft forever
以下で対処
# ip link set bond0.20 up
# ip link show bond0.20
9: bond0.20@bond0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000
link/ether 0c:8f:23:a9:00:00 brd ff:ff:ff:ff:ff:ff
疎通確認
vpc1
PC1> ip 10.2.10.2 255.255.255.0
Checking for duplicate address...
PC1 : 10.2.10.2 255.255.255.0
PC1> ping 10.2.10.254
84 bytes from 10.2.10.254 icmp_seq=1 ttl=64 time=1.463 ms
84 bytes from 10.2.10.254 icmp_seq=2 ttl=64 time=1.284 ms
84 bytes from 10.2.10.254 icmp_seq=3 ttl=64 time=1.316 ms
84 bytes from 10.2.10.254 icmp_seq=4 ttl=64 time=1.233 ms
84 bytes from 10.2.10.254 icmp_seq=5 ttl=64 time=1.238 ms
vpc2
PC2> ip 10.2.20.2 255.255.255.0
Checking for duplicate address...
PC2 : 10.2.20.2 255.255.255.0
PC2> ping 10.2.20.254
84 bytes from 10.2.20.254 icmp_seq=1 ttl=64 time=1.421 ms
84 bytes from 10.2.20.254 icmp_seq=2 ttl=64 time=1.016 ms
84 bytes from 10.2.20.254 icmp_seq=3 ttl=64 time=1.194 ms
84 bytes from 10.2.20.254 icmp_seq=4 ttl=64 time=0.963 ms
84 bytes from 10.2.20.254 icmp_seq=5 ttl=64 time=1.181 ms
後片付け
sudo ovs-vsctl clear port bond1 trunks
sudo ovs-vsctl clear port bond1 lacp bond_mode other_config
sudo ovs-vsctl del-port br0 bond1
sudo ovs-vsctl clear port tap00 trunks
sudo ovs-appctl bond/show bond1
sudo ovs-vsctl show
Discussion