나의 전반적인 의도는 별도의 네트워크 네임스페이스에서 하이퍼바이저 인스턴스를 실행하는 것입니다. 유망해 보이는 것이 있었지만 iptables와 함께 작동하도록 더 깊이 시도했을 때 정상적인 상태 저장 방화벽 작업을 수행하지 못하게 하는 이상한 동작을 발견했습니다.
호스팅 환경 설정은 다음과 같습니다.
# Set up packet forwarding on physical interface.
sysctl -w net.ipv6.conf.enp0s31f6.forwarding=1
# Set up ipvlan in namespace
ip netns add vm0
ip link add ipvlan0 link enp0s31f6 type ipvlan mode l3
ip link set dev ipvlan0 netns vm0
ip netns exec vm0 ip link set ipvlan0 up
ip netns exec vm0 ip link set lo up
# note: physical interface has addr 2a01:4f9:2b:35a::2/64
ip netns exec vm0 ip addr add dev ipvlan0 2a01:4f9:2b:35a::3/128
# Set up tap
ip netns exec vm0 ip tuntap add dev tap0 mode tap user fdr
# set MAC just for convenience to keep link local addresses the same between attempts
ip netns exec vm0 ip link set address ca:9d:46:13:9b:64 dev tap0
ip netns exec vm0 ip link set dev tap0 up
Tap에 연결된 vm0에서 가상 머신을 시작합니다. 주소와 라우팅을 설정하려면 다음 명령을 실행하세요.
export HOST_TAP_LLADDR=fe80::c89d:46ff:fe13:9b64
sudo ip addr add 2a01:4f9:2b:35a::3/128 dev ens4
sudo ip link set ens4 up
sudo ip route add default via $HOST_TAP_LLADDR dev ens4
좋아, 이제 VM에서 ping을 시작하고 연결을 시작하면 tcpdump -lenvi tap0ICMP가 켜져 있는 것을 볼 수 있지만 네임스페이스 내에서는 트래픽을 tap0가져오기 위해 아무 것도 하지 않기 때문에 예상대로 진행되지 않습니다.tap0ipvlan0
# tcpdump -lenvi tap0
tcpdump: listening on tap0, link-type EN10MB (Ethernet), snapshot length 262144 bytes
20:32:02.332080 IP6 (flowlabel 0x60b7e, hlim 64, next-header ICMPv6 (58) payload length: 64) 2a01:4f9:2b:35a::3 > 2a00:1450:400f:80c::200e: [icmp6 sum ok] ICMP6, echo request, id 2, seq 527
20:32:03.356084 IP6 (flowlabel 0x60b7e, hlim 64, next-header ICMPv6 (58) payload length: 64) 2a01:4f9:2b:35a::3 > 2a00:1450:400f:80c::200e: [icmp6 sum ok] ICMP6, echo request, id 2, seq 528
20:32:04.380079 IP6 (flowlabel 0x60b7e, hlim 64, next-header ICMPv6 (58) payload length: 64) 2a01:4f9:2b:35a::3 > 2a00:1450:400f:80c::200e: [icmp6 sum ok] ICMP6, echo request, id 2, seq 529
20:32:05.404096 IP6 (flowlabel 0x60b7e, hlim 64, next-header ICMPv6 (58) payload length: 64) 2a01:4f9:2b:35a::3 > 2a00:1450:400f:80c::200e: [icmp6 sum ok] ICMP6, echo request, id 2, seq 530
^C
# tcpdump -lenvi ipvlan0
tcpdump: listening on ipvlan0, link-type EN10MB (Ethernet), snapshot length 262144 bytes
<nothing>
^C
좋아, 뭔가 조치를 취해보자. 패킷을 꺼내서 tap0넣는 것이 ipvlan0더 쉽습니다 . 라우팅을 사용할 수 있습니다.
# Note: I can't seem to get routing to work by enabling forwarding on
# individual interfaces in the namespace, which is weird.
sysctl -w net.ipv6.conf.all.forwarding=1
ip route add default via fe80::1 dev ipvlan0 proto static
tap0tcpdump를 사용하여 이를 확인할 수 있습니다. ICMP가 전송되는 것만 볼 수 있으며 , ipvlan0두 패킷이 모두 전송되고 ICMP 응답이 수신됩니다.
root@Ubuntu-2204-jammy-amd64-base ~ # tcpdump -lenvi ipvlan0
tcpdump: listening on ipvlan0, link-type EN10MB (Ethernet), snapshot length 262144 bytes
20:42:08.542989 4c:52:62:0e:05:d3 > 4c:52:62:0e:05:d3, ethertype IPv6 (0x86dd), length 118: (flowlabel 0x60b7e, hlim 63, next-header ICMPv6 (58) payload length: 64) 2a01:4f9:2b:35a::3 > 2a00:1450:400f:80c::200e: [icmp6 sum ok] ICMP6, echo request, id 2, seq 1119
20:42:08.549416 d0:07:ca:8d:19:31 > 4c:52:62:0e:05:d3, ethertype IPv6 (0x86dd), length 118: (flowlabel 0x60b7e, hlim 119, next-header ICMPv6 (58) payload length: 64) 2a00:1450:400f:80c::200e > 2a01:4f9:2b:35a::3: [icmp6 sum ok] ICMP6, echo reply, id 2, seq 1119
20:42:09.567009 4c:52:62:0e:05:d3 > 4c:52:62:0e:05:d3, ethertype IPv6 (0x86dd), length 118: (flowlabel 0x60b7e, hlim 63, next-header ICMPv6 (58) payload length: 64) 2a01:4f9:2b:35a::3 > 2a00:1450:400f:80c::200e: [icmp6 sum ok] ICMP6, echo request, id 2, seq 1120
20:42:09.573468 d0:07:ca:8d:19:31 > 4c:52:62:0e:05:d3, ethertype IPv6 (0x86dd), length 118: (flowlabel 0x60b7e, hlim 119, next-header ICMPv6 (58) payload length: 64) 2a00:1450:400f:80c::200e > 2a01:4f9:2b:35a::3: [icmp6 sum ok] ICMP6, echo reply, id 2, seq 1120
20:42:10.590995 4c:52:62:0e:05:d3 > 4c:52:62:0e:05:d3, ethertype IPv6 (0x86dd), length 118: (flowlabel 0x60b7e, hlim 63, next-header ICMPv6 (58) payload length: 64) 2a01:4f9:2b:35a::3 > 2a00:1450:400f:80c::200e: [icmp6 sum ok] ICMP6, echo request, id 2, seq 1121
20:42:10.597439 d0:07:ca:8d:19:31 > 4c:52:62:0e:05:d3, ethertype IPv6 (0x86dd), length 118: (flowlabel 0x60b7e, hlim 119, next-header ICMPv6 (58) payload length: 64) 2a00:1450:400f:80c::200e > 2a01:4f9:2b:35a::3: [icmp6 sum ok] ICMP6, echo reply, id 2, seq 1121
^C
root@Ubuntu-2204-jammy-amd64-base ~ # tcpdump -lenvi tap0
tcpdump: listening on tap0, link-type EN10MB (Ethernet), snapshot length 262144 bytes
20:42:14.686993 2e:db:57:5c:90:28 > ca:9d:46:13:9b:64, ethertype IPv6 (0x86dd), length 118: (flowlabel 0x60b7e, hlim 64, next-header ICMPv6 (58) payload length: 64) 2a01:4f9:2b:35a::3 > 2a00:1450:400f:80c::200e: [icmp6 sum ok] ICMP6, echo request, id 2, seq 1125
20:42:15.710995 2e:db:57:5c:90:28 > ca:9d:46:13:9b:64, ethertype IPv6 (0x86dd), length 118: (flowlabel 0x60b7e, hlim 64, next-header ICMPv6 (58) payload length: 64) 2a01:4f9:2b:35a::3 > 2a00:1450:400f:80c::200e: [icmp6 sum ok] ICMP6, echo request, id 2, seq 1126
20:42:16.735027 2e:db:57:5c:90:28 > ca:9d:46:13:9b:64, ethertype IPv6 (0x86dd), length 118: (flowlabel 0x60b7e, hlim 64, next-header ICMPv6 (58) payload length: 64) 2a01:4f9:2b:35a::3 > 2a00:1450:400f:80c::200e: [icmp6 sum ok] ICMP6, echo request, id 2, seq 1127
^C
하지만 이제 우리는 막혔습니다: 어떻게 ipvlan0패킷을 에서 로 가져오는가 tap0? tc패킷이 성공적으로 미러링되었으며 대상 MAC 주소가 다시 작성되었음을 확인했습니다 .
# Move frames between tap and ipvlan, changing MAC addresses so the
# receiving side knows the frame is for them, and filtering on IP
# addresses in event of oddly addressed traffic coming in or going
# out.
tc qdisc add dev ipvlan0 handle ffff: ingress
export VM_MAC=2e:db:57:5c:90:28
tc filter add dev ipvlan0 parent ffff: protocol ipv6 \
u32 match ip6 dst 2a01:4f9:2b:35a::3 \
action skbmod set dmac $VM_MAC \
action mirred egress redirect dev tap0
마침내 우리의 작업은 성과를 거두었고 pingtcpdump를 검토할 수 있습니다 tap0:
tcpdump -lenvi tap0
tcpdump: listening on tap0, link-type EN10MB (Ethernet), snapshot length 262144 bytes
20:46:09.471859 2e:db:57:5c:90:28 > ca:9d:46:13:9b:64, ethertype IPv6 (0x86dd), length 118: (flowlabel 0x60b7e, hlim 64, next-header ICMPv6 (58) payload length: 64) 2a01:4f9:2b:35a::3 > 2a00:1450:400f:80c::200e: [icmp6 sum ok] ICMP6, echo request, id 3, seq 5
20:46:09.478344 d0:07:ca:8d:19:31 > 2e:db:57:5c:90:28, ethertype IPv6 (0x86dd), length 118: (flowlabel 0x60b7e, hlim 119, next-header ICMPv6 (58) payload length: 64) 2a00:1450:400f:80c::200e > 2a01:4f9:2b:35a::3: [icmp6 sum ok] ICMP6, echo reply, id 3, seq 5
20:46:10.473455 2e:db:57:5c:90:28 > ca:9d:46:13:9b:64, ethertype IPv6 (0x86dd), length 118: (flowlabel 0x60b7e, hlim 64, next-header ICMPv6 (58) payload length: 64) 2a01:4f9:2b:35a::3 > 2a00:1450:400f:80c::200e: [icmp6 sum ok] ICMP6, echo request, id 3, seq 6
20:46:10.479936 d0:07:ca:8d:19:31 > 2e:db:57:5c:90:28, ethertype IPv6 (0x86dd), length 118: (flowlabel 0x60b7e, hlim 119, next-header ICMPv6 (58) payload length: 64) 2a00:1450:400f:80c::200e > 2a01:4f9:2b:35a::3: [icmp6 sum ok] ICMP6, echo reply, id 3, seq 6
이를 도입한 후에는 라우팅을 제거하고 다른 방향(from to)에서 동일한 작업을 수행하여 모든 것을 단순화 tc할 수 있습니다 .tap0ipvlan0
# Undo routing stuff
sysctl -w net.ipv6.conf.all.forwarding=0
ip route del default via fe80::1 dev ipvlan0
# Use tc much the same, but in reverse:
tc qdisc add dev tap0 handle ffff: ingress
# ipvlan shares host MAC with the physical interface
tc filter add dev tap0 parent ffff: protocol ipv6 \
u32 match ip6 src 2a01:4f9:2b:35a::3 \
action skbmod set dmac 4c:52:62:0e:05:d3 \
action mirred egress redirect dev ipvlan0
패킷이 다시 흐릅니다.
# tcpdump -lenvi tap0
tcpdump: listening on tap0, link-type EN10MB (Ethernet), snapshot length 262144 bytes
20:52:25.641314 2e:db:57:5c:90:28 > ca:9d:46:13:9b:64, ethertype IPv6 (0x86dd), length 118: (flowlabel 0x60b7e, hlim 64, next-header ICMPv6 (58) payload length: 64) 2a01:4f9:2b:35a::3 > 2a00:1450:400f:80c::200e: [icmp6 sum ok] ICMP6, echo request, id 3, seq 377
20:52:25.647845 d0:07:ca:8d:19:31 > 2e:db:57:5c:90:28, ethertype IPv6 (0x86dd), length 118: (flowlabel 0x60b7e, hlim 119, next-header ICMPv6 (58) payload length: 64) 2a00:1450:400f:80c::200e > 2a01:4f9:2b:35a::3: [icmp6 sum ok] ICMP6, echo reply, id 3, seq 377
^C
이제 이 시점에서 나는 모든 것이 괜찮다고 생각했지만 iptables가 매우 이상하게 동작하고 있다는 것을 발견했습니다. 예를 들어 로깅을 활성화하려면 다음을 수행합니다.
ip6tables -I INPUT 1 -j LOG
ip6tables -I FORWARD 1 -j LOG
ip6tables -I OUTPUT 1 -j LOG
호스트 네임스페이스에서 직접 ping을 수행하면 예상대로 하나의 OUT레코드를 얻은 다음 하나의 레코드를 얻습니다.IN
Feb 09 20:54:37 Ubuntu-2204-jammy-amd64-base kernel: IN= OUT=enp0s31f6 SRC=2a01:04f9:002b:035a:0000:0000:0000:0002 DST=2a00:1450:400f:080c:0000:0000:0000:200e LEN=104 TC=0 HOPLIMIT=64 FLOWLBL=471262 PROTO=ICMPv6 TYPE=128 CODE=0 ID=3 SEQ=1
Feb 09 20:54:37 Ubuntu-2204-jammy-amd64-base kernel: IN=enp0s31f6 OUT= MAC=4c:52:62:0e:05:d3:d0:07:ca:8d:19:31:86:dd SRC=2a00:1450:400f:080c:0000:0000:0000:200e DST=2a01:04f9:002b:035a:0000:0000:0000:0002 LEN=104 TC=0 HOPLIMIT=60 FLOWLBL=471262 PROTO=ICMPv6 TYPE=129 CODE=0 ID=3 SEQ=1
OUT그러나 tcpdump를 통해 VM이 패킷을 수신하고 있고 ping이 성공했음을 알고 있음에도 불구하고 트래픽이 흐르고 있는 경우에도 VM의 트래픽 기록 만 얻을 수 있습니다 .
Feb 09 20:58:53 Ubuntu-2204-jammy-amd64-base kernel: IN= OUT=enp0s31f6 SRC=2a01:04f9:002b:035a:0000:0000:0000:0003 DST=2a00:1450:400f:080c:0000:0000:0000:200e LEN=104 TC=0 HOPLIMIT=64 FLOWLBL=396158 PROTO=ICMPv6 TYPE=128 CODE=0 ID=3 SEQ=764
Feb 09 20:58:54 Ubuntu-2204-jammy-amd64-base kernel: IN= OUT=enp0s31f6 SRC=2a01:04f9:002b:035a:0000:0000:0000:0003 DST=2a00:1450:400f:080c:0000:0000:0000:200e LEN=104 TC=0 HOPLIMIT=64 FLOWLBL=396158 PROTO=ICMPv6 TYPE=128 CODE=0 ID=3 SEQ=765
Feb 09 20:58:55 Ubuntu-2204-jammy-amd64-base kernel: IN= OUT=enp0s31f6 SRC=2a01:04f9:002b:035a:0000:0000:0000:0003 DST=2a00:1450:400f:080c:0000:0000:0000:200e LEN=104 TC=0 HOPLIMIT=64 FLOWLBL=396158 PROTO=ICMPv6 TYPE=128 CODE=0 ID=3 SEQ=766
또한 iptables를 통해 무엇이든 기록하도록 네임스페이스를 가져오려고 시도했지만 vm0성공하지 못했습니다. 호스트만 이 로그를 생성하고 IN항목이 누락되었습니다. 이는 상태 저장 방화벽을 수행할 수 있는 능력이 존재하지 않는 것 같다는 것을 의미합니다.
무슨 일이죠? 내가 나무를 잘못 찾고 있는 걸까?