Hi Benjamin, sorry for the late reply.
I'm happy to hear that someone will try to run scionlab in Kubernetes. I
hope we can make it work.
Let me try to help you with your setup, by first removing dead ends:
- can you try to run scionlab in a normal VM behind your router? It will
probably work, and then we are sure that the router is forwarding the port.
- that ICMP from the attachment point is the attachment point saying that
there is nobody listening on port 50195. The ICMP is being sent to you
because the attachment point received a packet to that port from you. I
checked the AP right now and I don't see anybody connecting to that port
from a public IP address. Can you tell us your ISD-AS id?
Best regards,
Juan A.
On Wed, Jul 7, 2021 at 4:35 PM Benjamin Pereto <benjamin(a)sandchaschte.ch>
wrote:
> Hi,
>
> I'm currently trying to get the scionlab setup work on kubernetes. I'm
> doing this to better
> understand how SCION works and improving my skills in kubernetes.
>
> For the development, containers with docker-compose is already used, and I
> assume, it should also
> work on kubernetes.
>
> The setup is similar to
> https://docs.scionlab.org/content/config/setup_endhost.html and using the
> scionlab packages for ubuntu with a ubuntu container image.
>
> My Setup is behind a FW/NAT and a port-forwarding is configured. So, this
> **should** work.
>
> A little overview of the topology:
> SCION(17-ffaa:0:1107) -> 50000:router (NAT port-forward) -> k8s
> Service:50000/udp -> pod:50000/udp
>
> Starting with the border-router, the startup seems as expected until:
> Failed to read batch {"err": "read udp 10.88.0.230:50000: recvmmsg:
> connection refused"}
>
> TCPdump on my router shows:
> * outgoing UDP packets from the border-router
> * incoming ICMP unreachable message from the gateway 192.33.93.195 of the
> attachement point 17-
> ffaa:0:1107 as defined in the scionlab config/topology.json
>
> 16:20:52.043441 IP (tos 0xc0, ttl 38, id 8924, offset 0, flags [none],
> proto ICMP (1), length 148)
> 192.33.93.195 > 77.109.191.151: ICMP 192.33.93.195 udp port 50195
> unreachable, length 128
> IP (tos 0x0, ttl 37, id 58046, offset 0, flags [DF], proto UDP
> (17), length 120)
> 77.109.191.151.55039 > 192.33.93.195.50195: [udp sum ok] UDP, length 92
>
> metrics also showing o router interface up:
> curl -sfS http://10.88.0.230:30401/metrics | grep router_interface_up
> # HELP router_interface_up Either zero or one depending on whether the
> interface is up.
> router_interface_up{interface="1",isd_as="17-ffaa:1:55",neighbor_isd_as="17-ffaa:0:1107"}
> 0
>
> I'm aware, that this is not a supported setup, but eventually someone has
> a hint where I can debug?
> What does the icmp unreachable from the scionlab gateway mean?
> Is the dispatcher service needed for the border router?
>
> Best regards,
> Benjamin
>
> [1] Logs of border-router: https://hastebin.com/dotovazoco.go
>
>
> _______________________________________________
> SCION mailing list
> SCION(a)lists.inf.ethz.ch
> https://lists.inf.ethz.ch/mailman/listinfo/scion
>
Hi,
I'm currently trying to get the scionlab setup work on kubernetes. I'm doing this to better
understand how SCION works and improving my skills in kubernetes.
For the development, containers with docker-compose is already used, and I assume, it should also
work on kubernetes.
The setup is similar to https://docs.scionlab.org/content/config/setup_endhost.html and using the
scionlab packages for ubuntu with a ubuntu container image.
My Setup is behind a FW/NAT and a port-forwarding is configured. So, this **should** work.
A little overview of the topology:
SCION(17-ffaa:0:1107) -> 50000:router (NAT port-forward) -> k8s Service:50000/udp -> pod:50000/udp
Starting with the border-router, the startup seems as expected until:
Failed to read batch {"err": "read udp 10.88.0.230:50000: recvmmsg: connection refused"}
TCPdump on my router shows:
* outgoing UDP packets from the border-router
* incoming ICMP unreachable message from the gateway 192.33.93.195 of the attachement point 17-
ffaa:0:1107 as defined in the scionlab config/topology.json
16:20:52.043441 IP (tos 0xc0, ttl 38, id 8924, offset 0, flags [none], proto ICMP (1), length 148)
192.33.93.195 > 77.109.191.151: ICMP 192.33.93.195 udp port 50195 unreachable, length 128
IP (tos 0x0, ttl 37, id 58046, offset 0, flags [DF], proto UDP (17), length 120)
77.109.191.151.55039 > 192.33.93.195.50195: [udp sum ok] UDP, length 92
metrics also showing o router interface up:
curl -sfS http://10.88.0.230:30401/metrics | grep router_interface_up
# HELP router_interface_up Either zero or one depending on whether the interface is up.
router_interface_up{interface="1",isd_as="17-ffaa:1:55",neighbor_isd_as="17-ffaa:0:1107"} 0
I'm aware, that this is not a supported setup, but eventually someone has a hint where I can debug?
What does the icmp unreachable from the scionlab gateway mean?
Is the dispatcher service needed for the border router?
Best regards,
Benjamin
[1] Logs of border-router: https://hastebin.com/dotovazoco.go
