UE cannot connect to the internet

[marcohol1]

Hello everyone,

I want to deploy a docker container of eUPF and used the docker-compose template here (https://github.com/edgecomllc/eupf/blob/main/docs/install.md/#deploy-with-docker-compose). I use Open5GS core and the services are also deployed in docker containers. I also have an actual gNB and UE for testing. I have already made it worked with the normal UPF of Open5GS. However, when I swap out the UPF to the eUPF, I've noticed that the UE sends DNS queries but it does not go to the internet. I tried to ping 8.8.8.8 from the UE and it cannot go through the internet. However, when I try to ping eUPF using the UE, I see in Wireshark that it receives the ping and sends a reply but the reply is not received by the UE.

Here is part of the docker-file where eUPF is deployed:

 eupf:
    image: ghcr.io/edgecomllc/eupf:main
    privileged: true
    volumes:
      - /sys/fs/bpf:/sys/fs/bpf
    environment:
      - GIN_MODE=release
      - UPF_INTERFACE_NAME=eth0
      - UPF_XDP_ATTACH_MODE=generic
      - UPF_API_ADDRESS=:8081
      - UPF_PFCP_ADDRESS=:8805
      - UPF_METRICS_ADDRESS=:9091
      - UPF_PFCP_NODE_ID=172.18.0.20
      - UPF_N3_ADDRESS=172.18.0.20
    ulimits:
      memlock: -1
    cap_add:
      - NET_ADMIN
      - SYS_ADMIN
    ports:
      - 2152:2152/udp
      - 8805:8805/udp
      - 8080:8080
      - 9090:9090
    restart: unless-stopped
    networks:
      mynet1:
        ipv4_address: 172.18.0.20
    sysctls:
      - net.ipv4.conf.all.forwarding=1

  net-tools:
    image: praqma/network-multitool:alpine-extra@sha256:47b259d4463950f5c10d9c0bf63d9e71ec456618f5549a414afa0c04392e0ac1
    network_mode: host
    privileged: true
    restart: unless-stopped
    command:
      - /bin/sh
      - -c
      - |
        ip ro add 10.45.0.0/16 via 172.18.0.20
        echo "done"
        tail -f /dev/null

Thank you very much and have a nice day!

Best regards,
Marco

[howardswe499]

Hi Marco @marcohol1 ,

It seems to me there could possibly be an issue of NAT configuration and/or missing sysctl -w net.ipv4.ip_forward=1 config on your Docker host? If these settings are not configured properly, UE ping Internet would not work due to following reasons:

1. Docker host can't forward packets to another interface
2. If it could forward packets, the packet can't return back to Docker host (because packets sent out would have source IP 172.18.0.20)

However, I'm not sure why pinging eUPF would not work. Perhaps it's actually sending an ICMP unreachable message back to UE?

To fix, on Docker host, try running following commands:

# Enable packet forwarding (for forwarding traffic to another interface)
sudo sysctl -w net.ipv4.ip_forward=1

# Insert rule in iptables to allow packets to be forwarded
sudo iptables -I FORWARD 1 -j ACCEPT

# Configure NAT
sudo iptables -t nat -A POSTROUTING -o <internet-interface-name> -j MASQUERADE

If that still doesn't work, we might need you to provide some packet capture & system info.

First of all, I'd like to check your Docker host network settings when eUPF is running:

ip a s

For packet capture, when pinging Internet, try monitoring the outbound (i.e. to internet) interface of the eUPF host device. You could use the command:

sudo tcpdump -ni <interface-name> 

to check real-time packet info from a specific device. Note however that packets that enter eUPF's XDP packet forwarder will not appear on the packet capture, so that's any packets that enter/exit the N3, N4, N6 interface of eUPF.

If you would like to provide a packet capture file (pcap) or at least a screenshot of the pcap for analysis, you can use the following command to save the packet capture into a .pcap file.

sudo tcpdump -i <interface-name> -w <file-name>

Best of Luck!
Howard

[pirog-spb]

Hi @marcohol1,

If your question is still opened, I'd recommend to enable dataplane logs(ebpf trace pipe) and verify packet processing pipeline.

Usually the most common issue with packet forwarding is wrong routing table configuration.

In order to enable logging you should build eUPF with specific option:

docker build -t local/eupf:latest --build-arg BPF_ENABLE_LOG=1 .

To see logs check cat /sys/kernel/debug/tracing/trace_pipe