OpenvSwitch初探 - VLAN篇

发表于 2018-04-01   |   分类于 技术

传统的交换机设备都是支持VLAN隔离的，OpenvSwitch作为一个主流的虚拟交换机，自然也是支持VLAN相关的实现的。OVS交换机内部也可以通过VLAN ID来隔离交换机的各个端口。

例如：如果一个OVS交换机的端口设置了tag标签（该端口处于access模式），数据包从该端口进入到交换机内部时，该数据包就会被打上对应的tag，于是该数据包也就只能从设置了相同tag的端口发出，而在出交换机的时候，数据包上的tag会被删除。这样就实现了交换机内部的一个隔离。

private vlan tag.png

目前OVS的端口支持四种VLAN模式，分别是trunk、access、native-tagged、native-untagged，刚创建的端口默认是不设置任何的VLAN模式。

对这几个VLAN模式的介绍如下：

模式说明
默认在默认模式下（VLAN_mode没被设置），如果指定了端口的tag属性，那么这个端口就工作在access模式，并且其trunk属性的值应该保持为空。否则，这个port就工作在trunk模式下，如果trunk被指定，则使用指定的trunk值。
trunktrunk模式的端口允许传输所有在其trunk属性中指定的那些VLAN对应的数据包。其他VLAN的数据包就会被丢弃。从trunk模式的端口中进入的数据包其VLAN ID不会发生变化。如果进入的数据包不含有VLAN ID，则该数据包进入交换机后的VLAN为0。从trunk模式的端口出去的数据包，如果VLAN ID不为空，则依然保持该VLAN ID，如果VLAN ID为空，则出去后不再包含802.1Q头部
accessaccess模式的端口只允许不带VLAN的数据包进入，不管数据包的VLAN ID是否与其tag相同，只要含有VLAN ID，这个数据包都会被端口drop。数据包进入access端口后会被打上和端口tag相同的VLAN，而再从access端口出去时，数据包的VLAN会被删除，也就是说从access的端口出去的数据包和进来时一样是不带VLAN的。
native-taggednative-tagged端口类似于trunk端口，它们之间的区别是如果进入native-tagged端口的数据包不含有802.1Q头部，即没有指定VLAN，那么该数据包会被当作native VLAN，其VLAN ID由端口的tag指定。
native-untaggednative-untagged端口类似于native-tagged端口，不同点是native VLAN中的数据包从native-untagged端口出去时，会被去掉802.1Q头部。

使用docker容器动手实践

理论知识了解后就能动手来实践了，由于环境资源的限制，只能用更轻量级的docker容器来做实验。

环境拓扑

docker_ovs_env.png

使用docker创建四个不带网卡的容器

docker run -t -i -d --name vm01 --net=none --privileged sshd-centos7 /bin/bash
docker run -t -i -d --name vm02 --net=none --privileged sshd-centos7 /bin/bash
docker run -t -i -d --name vm03 --net=none --privileged sshd-centos7 /bin/bash
docker run -t -i -d --name vm04 --net=none --privileged sshd-centos7 /bin/bash

创建OVS网桥

ovs-vsctl add-br vswitch0
ovs-vsctl add-br vswitch1

使用ovs-docker工具给容器添加网卡到ovs网桥

ovs-docker add-port vswitch0 eth0 vm01 --ipaddress=192.168.1.2/24
ovs-docker add-port vswitch1 eth0 vm02 --ipaddress=192.168.1.3/24
ovs-docker add-port vswitch0 eth0 vm03 --ipaddress=192.168.1.4/24
ovs-docker add-port vswitch1 eth0 vm04 --ipaddress=192.168.1.5/24

连接vswitch0和vswitch1

网桥与网桥的连接需要依靠一对patch类型的端口

$ ovs-vsctl add-port vswitch0 patch_to_vswitch1
ovs-vsctl: Error detected while setting up 'patch_to_vswitch1'.  See ovs-vswitchd log for details.

$ ovs-vsctl add-port vswitch1 patch_to_vswitch0
ovs-vsctl: Error detected while setting up 'patch_to_vswitch0'.  See ovs-vswitchd log for details.


$ ovs-vsctl set interface patch_to_vswitch1 type=patch
$ ovs-vsctl set interface patch_to_vswitch0 type=patch

$ ovs-vsctl set interface patch_to_vswitch0 options:peer=patch_to_vswitch1
$ ovs-vsctl set interface patch_to_vswitch1 options:peer=patch_to_vswitch0

创建完这对patch类型的端口后，两个交换机就可以相互连通了。ovs端口默认是trunk模式，且可以trunk所有的VLAN tag，所以在这对patch类型的端口上可以通过打上了任意tag的流量。

查看ovs网桥的所有端口

$ ovs-vsctl show
270e167e-2f5e-4b46-be62-2c9c142cdd9f
    Bridge "vswitch1"
        Port "patch_to_vswitch0"
            Interface "patch_to_vswitch0"
                type: patch
                options: {peer="patch_to_vswitch1"}
        Port "vswitch1"
            Interface "vswitch1"
                type: internal
        Port "6eef85c4718a4_l"
            Interface "6eef85c4718a4_l"
        Port "39c0765e1c964_l"
            Interface "39c0765e1c964_l"
    Bridge "vswitch0"
        Port "patch_to_vswitch1"
            Interface "patch_to_vswitch1"
                type: patch
                options: {peer="patch_to_vswitch0"}
        Port "vswitch0"
            Interface "vswitch0"
                type: internal
        Port "2720669d9c064_l"
            Interface "2720669d9c064_l"
        Port "f4e1ae64ef704_l"
            Interface "f4e1ae64ef704_l"
    ovs_version: "2.5.0"

测试容器的连通性

使用vm01 ping其他几个容器

# docker attach vm01
[root@ffa59e57d97f /]# ping 192.168.1.3 -c 3  
PING 192.168.1.3 (192.168.1.3) 56(84) bytes of data.
64 bytes from 192.168.1.3: icmp_seq=1 ttl=64 time=0.230 ms
64 bytes from 192.168.1.3: icmp_seq=2 ttl=64 time=0.057 ms
64 bytes from 192.168.1.3: icmp_seq=3 ttl=64 time=0.062 ms

--- 192.168.1.3 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 1999ms
rtt min/avg/max/mdev = 0.057/0.116/0.230/0.080 ms
[root@ffa59e57d97f /]# ping 192.168.1.4 -c 3   
PING 192.168.1.4 (192.168.1.4) 56(84) bytes of data.
64 bytes from 192.168.1.4: icmp_seq=1 ttl=64 time=0.241 ms
64 bytes from 192.168.1.4: icmp_seq=2 ttl=64 time=0.053 ms
64 bytes from 192.168.1.4: icmp_seq=3 ttl=64 time=0.058 ms

--- 192.168.1.4 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 1999ms
rtt min/avg/max/mdev = 0.053/0.117/0.241/0.087 ms
[root@ffa59e57d97f /]# ping 192.168.1.5 -c 3   
PING 192.168.1.5 (192.168.1.5) 56(84) bytes of data.
64 bytes from 192.168.1.5: icmp_seq=1 ttl=64 time=0.209 ms
64 bytes from 192.168.1.5: icmp_seq=2 ttl=64 time=0.054 ms
64 bytes from 192.168.1.5: icmp_seq=3 ttl=64 time=0.055 ms

--- 192.168.1.5 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2000ms
rtt min/avg/max/mdev = 0.054/0.106/0.209/0.072 ms

vm01可以和vm02、vm03、vm04相互通信。

为port设置VLAN tag

为vm01、vm02对应的ovs端口设置tag为100，为vm03、vm04对应的ovs端口设置tag为200

# ovs-vsctl list interface 6eef85c4718a4_l | grep container_id
external_ids        : {container_id="vm04", container_iface="eth0"}
# ovs-vsctl set port 6eef85c4718a4_l tag=200

# ovs-vsctl list interface 39c0765e1c964_l | grep container_id
external_ids        : {container_id="vm02", container_iface="eth0"}
# ovs-vsctl set port 39c0765e1c964_l tag=100

# ovs-vsctl list interface 2720669d9c064_l | grep container_id
external_ids        : {container_id="vm03", container_iface="eth0"}
# ovs-vsctl set port 2720669d9c064_l tag=200
 
# ovs-vsctl list interface f4e1ae64ef704_l | grep container_id               
external_ids        : {container_id="vm01", container_iface="eth0"}
# ovs-vsctl set port f4e1ae64ef704_l tag=100

测试容器连通性

设置完tag后，理论上，设置了相同tag的端口之间才能相互通信，即vm01能和vm02通信，vm03能和vm04通信，而vm01、vm02无法和vm03、vm04通信。

先测试vm01

# docker attach vm01
[root@ffa59e57d97f /]# ping 192.168.1.3 -c 1
PING 192.168.1.3 (192.168.1.3) 56(84) bytes of data.
64 bytes from 192.168.1.3: icmp_seq=1 ttl=64 time=0.211 ms
^C
--- 192.168.1.3 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.211/0.211/0.211/0.000 ms


[root@ffa59e57d97f /]# ping 192.168.1.4 -c 1  
PING 192.168.1.4 (192.168.1.4) 56(84) bytes of data.
From 192.168.1.2 icmp_seq=1 Destination Host Unreachable
^C
--- 192.168.1.4 ping statistics ---
1 packets transmitted, 0 received, +1 errors, 100% packet loss, time 0ms


[root@ffa59e57d97f /]# ping 192.168.1.5 -c 1 
PING 192.168.1.5 (192.168.1.5) 56(84) bytes of data.
From 192.168.1.2 icmp_seq=1 Destination Host Unreachable
^C
--- 192.168.1.5 ping statistics ---
1 packets transmitted, 0 received, +1 errors, 100% packet loss, time 0ms

vm01只能ping通设置了相同tag的vm02，其他容器则无法ping通。

再测试vm03

# docker attach vm03
[root@c17ff55307d7 /]# ping 192.168.1.4
PING 192.168.1.4 (192.168.1.4) 56(84) bytes of data.
64 bytes from 192.168.1.4: icmp_seq=1 ttl=64 time=0.033 ms
64 bytes from 192.168.1.4: icmp_seq=2 ttl=64 time=0.040 ms
^C
--- 192.168.1.4 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 999ms
rtt min/avg/max/mdev = 0.033/0.036/0.040/0.007 ms


[root@c17ff55307d7 /]# ping 192.168.1.5
PING 192.168.1.5 (192.168.1.5) 56(84) bytes of data.
64 bytes from 192.168.1.5: icmp_seq=1 ttl=64 time=0.269 ms
64 bytes from 192.168.1.5: icmp_seq=2 ttl=64 time=0.057 ms
^C
--- 192.168.1.5 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 999ms
rtt min/avg/max/mdev = 0.057/0.163/0.269/0.106 ms


[root@c17ff55307d7 /]# ping 192.168.1.3
PING 192.168.1.3 (192.168.1.3) 56(84) bytes of data.
^C
--- 192.168.1.3 ping statistics ---
2 packets transmitted, 0 received, 100% packet loss, time 1000ms

[root@c17ff55307d7 /]#

vm03也只能ping通设置了相同tag的vm04

设置网桥之间的patch端口只trunk100

无论是vm01与vm02通信，还是vm03与vm04通信都需要经过两个网桥之间的patch类型的端口，默认是trunk所有的tag的，现在设置它们只trunk tag100。

ovs-vsctl set port patch_to_vswitch1 VLAN_mode=trunk
ovs-vsctl set port patch_to_vswitch0 VLAN_mode=trunk

ovs-vsctl set port patch_to_vswitch0 trunk=100
ovs-vsctl set port patch_to_vswitch1 trunk=100

测试容器之间的连通性

使用vm01 ping vm02

# docker attach vm01
[root@ffa59e57d97f /]# ping 192.168.1.3
PING 192.168.1.3 (192.168.1.3) 56(84) bytes of data.
64 bytes from 192.168.1.3: icmp_seq=1 ttl=64 time=0.222 ms
64 bytes from 192.168.1.3: icmp_seq=2 ttl=64 time=0.051 ms
^C
--- 192.168.1.3 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 999ms
rtt min/avg/max/mdev = 0.051/0.136/0.222/0.086 ms

由于vm01、vm02对应端口的tag为100，在patch端口的trunk范围内，所以vm01依然可以ping通vm02

使用vm03 ping vm04

# docker attach vm03
[root@c17ff55307d7 /]# ping 192.168.1.5
PING 192.168.1.5 (192.168.1.5) 56(84) bytes of data.
^C
--- 192.168.1.5 ping statistics ---
2 packets transmitted, 0 received, 100% packet loss, time 999ms

vm03和vm04对应端口的tag为200，不在patch端口的trunk范围内，所以vm03已经无法ping通vm04

查看flow table

$ ovs-ofctl dump-flows vswitch0
NXST_FLOW reply (xid=0x4):
 cookie=0x0, duration=2504.456s, table=0, n_packets=292, n_bytes=36188, idle_age=410, priority=0 actions=NORMAL
 
$ ovs-ofctl dump-flows vswitch1
NXST_FLOW reply (xid=0x4):
 cookie=0x0, duration=46151.880s, table=0, n_packets=155, n_bytes=10998, idle_age=63, priority=0 actions=NORMAL

没有转发用的flow table存在，所以猜测普通VLAN tag的转发是不需要依靠flow的，但是支持open flow又是ovs比较核心的东西，所以在后面的文章中会和大家一起分享我对flow相关的学习内容。

附一些OVS配置命令

设置VLAN mode

ovs-vsctl set port <port name> VLAN_mode=trunk|access|native-tagged|native-untagged

设置VLAN tag

ovs-vsctl set port <port name> tag=<tag值>

设置VLAN trunk

ovs-vsctl set port <port name> trunk=100,200

删除port的配置

ovs-vsctl remove port <port name> <config item> <config value>
# 例如
ovs-vsctl remove port <port name> tag 100
ovs-vsctl remove port <port name> trunk 200

查看port的属性

ovs-vsctl list interface <port name>
openvswitch vlan

已有 9 条评论


  1. scw

    谢谢你的文章。

    scw May 30th, 2018 at 10:28 am回复
  2. Schenk

    受益了，谢谢你。

    Schenk October 8th, 2018 at 03:51 pm回复
    1. 志平

      不客气，共同进步

      志平 October 23rd, 2018 at 07:04 pm回复
  3. sunznx

    有个疑问 trunk 和 native-untagged 好像是一样的

    sunznx February 17th, 2019 at 11:54 pm回复
    1. sunznx

      搞错了，抱歉啊 - -

      sunznx February 18th, 2019 at 12:09 am回复
  4. 老廖

    感谢作者的分享, 非常详细

    老廖 December 26th, 2019 at 03:25 pm回复
  5. cu

    执行docker run -t -i -d --name vm01 --net=none --privileged sshd-centos7 /bin/bash有报错，提示没有这个镜像
    Unable to find image 'sshd-centos7:latest' locally
    Trying to pull repository docker.io/library/sshd-centos7 ...
    /usr/bin/docker-current: repository docker.io/sshd-centos7 not found: does not exist or no pull access.
    See '/usr/bin/docker-current run --help'.
    sshd-centos7这个镜像哪里有？

    cu October 20th, 2020 at 10:07 pm回复
    1. cu

      使用其他镜像搞定

      cu October 20th, 2020 at 11:06 pm回复
  6. yue

    想问一下作者，在同一个ovs交换机上，两个端口打上了vlan的tag=100，如何下发流表呢

    yue December 8th, 2020 at 11:16 am回复

发表新评论

© 2017 Powered by Typecho
苏ICP备15035969号-3