Skip to content

Commit

Permalink
Update Introduction_of_gtp5g_and_some_kernel_concepts.md
Browse files Browse the repository at this point in the history
  • Loading branch information
ianchen0119 authored Sep 20, 2023
1 parent ea84f89 commit 0e65bf0
Showing 1 changed file with 47 additions and 44 deletions.
91 changes: 47 additions & 44 deletions docs/blog/Introduction_of_gtp5g_and_some_kernel_concepts.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,21 +6,23 @@
---

## Overview
GTP, or the General Packet Radio System Tunneling Protocol, is a group of IP-based communication protocols used to transport General Packet Radio Services (GPRS) within LTE, 5G NR, and other networks. GTP can be broken down into several components: GTP-C, GTP-U, and GTP prime. GTP-C and GTP-U are responsible for the control plane and user plane, respectively, while GTP prime is utilized for transmitting charging data through the Ga interface as defined in the 3GPP GPRS Core Network.
GTP (General Packet Radio System Tunneling Protocol) is a group of IP-based communication protocols used to transport General Packet Radio Services (GPRS) within LTE, 5G NR, and other networks. GTP can be broken down into several components: GTP-C, GTP-U, and GTP prime. GTP-C and GTP-U are responsible for the control plane and user plane, respectively, while GTP prime is utilized for transmitting charging data through the Ga interface as defined in the 3GPP GPRS Core Network.

In the context of free5GC, the UPF network function combines the GTP-C control part to correctly instruct the routing path for any packet passing through the core network. GTP-U, on the other hand, is managed by [gtp5g](https://github.com/free5gc/gtp5g), which transports packets using kernel modules generated by gtp5g.
This article will introduce how gtp5g assists free5GC in handling packets and some kernel-related concepts.

Let's start the journey!

Additional information:

- Linux kernel version is 5.4.0-159-generic in article. According to other versions, some of content would be different, but the main concept is the same.
- Gtp5g version is v0.8.2
- UPF version is v1.2.0

## Netlink, Generic Netlink and Rtnetlink
### Netlink
Before we continue, i need to introduce [Netlink](https://man7.org/linux/man-pages/man7/netlink.7.html) first. What is Netlink? Netlink is an IPC (Inter Process Communicate) protocol which can connect kernel space and user space processes by socket. Traditionally, it used three methods: **Ioctl**, **sysfs**, or **procfs**, to facilitate communication between the kernel and user space. However, it can only be initiated from user space, not from kernel space. Netlink can support not only **initiated from kernel and user space** but also:

- Bidirectional transmission, asynchronous communication.
- Standard socket API used in user space.
- Specialized API used in kernel space.
Expand Down Expand Up @@ -75,7 +77,7 @@ The following figure is Generic Netlink structure:
- The ***Generic Netlink controller*** which is part of the kernel and is responsible for dynamically allocating Generic Netlink communication channels and other management tasks. The Generic Netlink controller is implemented as a standard Generic Netlink user, however, it listens on a special, pre-allocated Generic Netlink channel.
- The kernel socket API. Generic Netlink sockets are created with the PF_NETLINK domain and the NETLINK_GENERIC protocol values.
### Rtnetlink
### RtNetlink
The last one is [rtnetlink](https://man7.org/linux/man-pages/man7/rtnetlink.7.html), it also known as Netlink protocol type NETLINK_ROUTE, user space program could read and alter kernel's routing table or create new network device.
Expand Down Expand Up @@ -114,7 +116,7 @@ type Driver interface {
}
```

UPF use rtnl to create device (interface) name 'upfgtp'. User can observe it while executing **run.sh**.
UPF use rtnl to create device (interface) named `upfgtp`. User can observe it while executing **run.sh**.
```golang
func OpenGtp5gLink(mux *nl.Mux, addr string, mtu uint32, log *logrus.Entry) (*Gtp5gLink, error) {
g := &Gtp5gLink{
Expand Down Expand Up @@ -201,7 +203,7 @@ func OpenGtp5gLink(mux *nl.Mux, addr string, mtu uint32, log *logrus.Entry) (*Gt
}
```

Connect UPF Driver functions and gtp5g_genl_ops.
Connect UPF Driver functions and `gtp5g_genl_ops`.
```golang
// internl/forwarder/buffnetlink/server.go
func OpenServer(wg *sync.WaitGroup, client *nl.Client, mux *nl.Mux) (*Server, error) {
Expand Down Expand Up @@ -237,12 +239,12 @@ func OpenServer(wg *sync.WaitGroup, client *nl.Client, mux *nl.Mux) (*Server, er
## GTP5G
- Gtp5g utilizes a Linux kernel module to manage packet traffic. A Linux kernel module can be thought of as a small piece of code that is inserted into the Linux kernel, allowing users to customize the program according to the current hardware device
- In gtp5g, the primary function is **gtp5g_init** in **gtp5g.c**; it exposes most of the components and techniques provided by gtp5g. This article will choose the following concepts to investigate further:
- Network device -> net_device_ops
- Rtnetlink -> gtp5g_link_ops
- Generic Netlink -> gtp5g_genl_family
- Network device -> `net_device_ops`
- Rtnetlink -> `gtp5g_link_ops`
- Generic Netlink -> `gtp5g_genl_family`
- Additionally, the article will present two functions in detail:
- rtnl_link_register()
- genl_register_family()
- `rtnl_link_register()`
- `genl_register_family()`

```cpp
// src/gtp5g.c
Expand Down Expand Up @@ -286,7 +288,7 @@ static int __init gtp5g_init(void)
}
```
### net_device_ops
### `net_device_ops`
It is defined in **dev.h** and referenced in **dev.c**. The structure net_device_ops encompasses all operations related to network device, and free5GC inherits some of these operations to implement self-made netdev ops.
```cpp
// include/dev.h
Expand All @@ -307,9 +309,10 @@ const struct net_device_ops gtp5g_netdev_ops = {
};
```
According to [/include/linux/netdevice.h](https://elixir.bootlin.com/linux/latest/source/include/linux/netdevice.h#L1422), you can find the definition of hooks of **net_device_ops**:
- .ndo_init: This function is called once when a network device is registered. The network device can use this for any late stage initialization or semantic validation. It can fail with an error code which will be propagated back to register_netdev.
- .ndo_uninit: This function is called when device is unregistered or when registration fails. It is not called if init fails.
- .ndo_start_xmit: Called when a packet needs to be transmitted. Returns NETDEV_TX_OK. Can return NETDEV_TX_BUSY, but you should stop the queue before that can happen; it's for obsolete devices and weird corner cases, but the stack really does a non-trivial amount of useless work if you return NETDEV_TX_BUSY. Required; cannot be NULL.
- `.ndo_init`: This function is called once when a network device is registered. The network device can use this for any late stage initialization or semantic validation. It can fail with an error code which will be propagated back to register_netdev.
- `.ndo_uninit`: This function is called when device is unregistered or when registration fails. It is not called if init fails.
- `.ndo_start_xmit`: Called when a packet needs to be transmitted. Returns NETDEV_TX_OK. Can return NETDEV_TX_BUSY, but you should stop the queue before that can happen; it's for obsolete devices and weird corner cases, but the stack really does a non-trivial amount of useless work if you return NETDEV_TX_BUSY. Required; cannot be NULL.
```cpp
struct net_device_ops {
int (*ndo_init)(struct net_device *dev);
Expand Down Expand Up @@ -427,7 +430,7 @@ static netdev_tx_t gtp5g_dev_xmit(struct sk_buff *skb, struct net_device *dev)
```
### gtp5g_link_ops
### `gtp5g_link_ops`
Ignore headder file here, structure defines Rtnetlink operations.
```cpp
Expand All @@ -447,16 +450,16 @@ struct rtnl_link_ops gtp5g_link_ops __read_mostly = {
```

Definition is in [/include/net/rtnetlink.h](https://elixir.bootlin.com/linux/v5.4/source/include/net/rtnetlink.h#L59):
- .kind: Identifier
- .maxtype: Highest device specific netlink attribute number
- .policy: Netlink policy for device specific attribute validation
- .priv_size: sizeof net_device private space
- .setup: net_device setup function
- .validate: Optional validation function for netlink/changelink parameters
- .newlink: Function for configuring and registering a new device
- .dellink: Function to remove a device
- .get_size: Function to calculate required room for dumping device specific netlink attributes
- .fill_info: Function to dump device specific netlink attributes
- `.kind`: Identifier
- `.maxtype`: Highest device specific netlink attribute number
- `.policy`: Netlink policy for device specific attribute validation
- `.priv_size`: sizeof net_device private space
- `.setup`: net_device setup function
- `.validate`: Optional validation function for netlink/changelink parameters
- `.newlink`: Function for configuring and registering a new device
- `.dellink`: Function to remove a device
- `.get_size`: Function to calculate required room for dumping device specific netlink attributes
- `.fill_info`: Function to dump device specific netlink attributes

```cpp
struct rtnl_link_ops {
Expand Down Expand Up @@ -548,7 +551,7 @@ static void gtp5g_link_setup(struct net_device *dev)
}
```

### rtnl_link_register()
### `rtnl_link_register()`
- Definition is in [/net/core/rtnetlink.c](https://elixir.bootlin.com/linux/v5.4/source/net/core/rtnetlink.c#L372)
- This function should be used by drivers that create devices during module initialization. It must be called before registering the devices
- Using the **kind** property of rtnl_link_ops to search for the existence of ops in link_ops. If ops do not exist, then inserted it at the end of link_ops
Expand Down Expand Up @@ -593,15 +596,15 @@ struct genl_family gtp5g_genl_family __ro_after_init = {
};
```
Definition is in [/include/net/genetlink.h](https://elixir.bootlin.com/linux/v5.4.159/source/include/net/genetlink.h#L46):
- .name: name of family (exclusive)
- .version: protocol version (usually is 1)
- .hdrsize: length of user specific header in bytes
- .maxattr: maximum number of attributes supported
- .netnsok: set to true if the family can handle network namespaces and should be presented in all of them
- .ops: the operations supported by this family
- .n_ops: number of operations supported by this family
- .mcgrps: multicast groups used by this family
- .n_mcgrps: number of multicast groups
- `.name`: name of family (exclusive)
- `.version`: protocol version (usually is 1)
- `.hdrsize`: length of user specific header in bytes
- `.maxattr`: maximum number of attributes supported
- `.netnsok`: set to true if the family can handle network namespaces and should be presented in all of them
- `.ops`: the operations supported by this family
- `.n_ops`: number of operations supported by this family
- `.mcgrps`: multicast groups used by this family
- `.n_mcgrps`: number of multicast groups

```cpp
struct genl_family {
Expand Down Expand Up @@ -629,7 +632,7 @@ struct genl_family {
};
```
Gtp5g defines gtp5g_genl_ops, all operations are one-to-one match to Driver functions in UPF part:
Gtp5g defines `gtp5g_genl_ops`, all operations are one-to-one match to Driver functions in UPF part:
```cpp
// src/genl/genl.c
static const struct genl_ops gtp5g_genl_ops[] = {
Expand Down Expand Up @@ -659,13 +662,13 @@ static const struct genl_ops gtp5g_genl_ops[] = {
}
```

### genl_register_family()
### `genl_register_family()`
- Definition is in [/net/netlink/genetlink.c](https://elixir.bootlin.com/linux/v5.4.159/source/net/netlink/genetlink.c#L322)
- Registers the specified family after validating it first. Only one family may be registered with the same family name or identifier. The family's ops, multicast groups and module pointer must already be assigned. Return 0 on success or a negative error code
- Three functions within this method hold greater significance (all of them in /net/netlink/genetlink.c):
- genl_validate_ops()
- genl_family_find_byname()
- genl_validate_assign_mc_groups()
- `genl_validate_ops()`
- `genl_family_find_byname()`
- `genl_validate_assign_mc_groups()`

```cpp
int genl_register_family(struct genl_family *family)
Expand Down Expand Up @@ -735,7 +738,7 @@ errout_locked:
EXPORT_SYMBOL(genl_register_family);
```
#### genl_validate_ops()
#### `genl_validate_ops()`
This function will verify if there is defined function for the operations and will also compare whether any operation is reused by a command. Using gtp5g as an example, command can be considered as action such as add, del, modify PDR rules.
```cpp
static int genl_validate_ops(const struct genl_family *family)
Expand All @@ -762,7 +765,7 @@ static int genl_validate_ops(const struct genl_family *family)
}
```

#### genl_family_find_byname()
#### `genl_family_find_byname()`
Function would check every entry in genl_fam_idr whether exists the same family name.

```cpp
Expand All @@ -779,14 +782,14 @@ static const struct genl_family *genl_family_find_byname(char *name)
}
```
Here is the macro of idr_for_each_entry():
Here is the macro of `idr_for_each_entry()`:
```cpp
#define idr_for_each_entry(idr, entry, id) \
for (id = 0; ((entry) = idr_get_next(idr, &(id))) != NULL; id += 1U)
```

#### genl_validate_assign_mc_groups()
#### `genl_validate_assign_mc_groups()`
This changes the number of multicast groups that are available when **ntensok** is ture:

```cpp
Expand Down Expand Up @@ -884,5 +887,5 @@ Jimmy Chang
- https://www.cnblogs.com/ssyfj/p/16230540.html
- https://www.twblogs.net/a/5b81e5852b71772165aedd7a
- https://www.cnblogs.com/ssyfj/p/16230540.html
- [IT blog by Lan Chen](https://ithelp.ithome.com.tw/articles/10302887)
- [IT blog by Ian Chen](https://ithelp.ithome.com.tw/articles/10302887)
- [IT blog by 0xff07](https://ithelp.ithome.com.tw/articles/10243519)

0 comments on commit 0e65bf0

Please sign in to comment.