Skip to content

Commit

Permalink
kvm_watcher:优化代码并完善说明文档 (#787)
Browse files Browse the repository at this point in the history 
* add kvmexit watcher

* Update kvmexit.py

* VMexit

* 修改action

* 修改action

* 修改makefile

* update makefile

* update yml

* 调整代码格式

* vm exit

* 优化代码

* modify sort_bug

* modify sort_bug

* add fun

* vcpu_load

* optimize code

* modify sortFun

* Revert "modify sortFun"

This reverts commit 3bb0058.

* fix bug
  • Loading branch information
@Monkey857
Monkey857 authored May 10, 2024
1 parent 439159c commit ca9488a
Show file tree
Hide file tree
Showing 2 changed files with 116 additions and 95 deletions.
39 changes: 37 additions & 2 deletions eBPF_Supermarket/kvm_watcher/docs/kvm_vcpu.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

## 概述

kvm watcher中的kvm vcpu子功能模块是设计用于监控和分析虚拟化环境中虚拟 CPU (VCPU) 活动的工具，它通过精确记录 VCPU 的唤醒/挂起事件、halt poll 时间的变化，以及 KVM 虚拟机在热迁移过程中产生的脏页信息，为优化虚拟机性能、提高系统响应速度、提供数据支持。
kvm watcher中的kvm vcpu子功能模块是设计用于监控和分析虚拟化环境中虚拟 CPU (VCPU) 活动的工具，它通过精确记录 VCPU 的唤醒/挂起事件、halt poll 时间的变化，虚拟cpu进调度和出调度的时间记录，以及 KVM 虚拟机在热迁移过程中产生的脏页信息，为优化虚拟机性能、提高系统响应速度、提供数据支持。

## 原理介绍

Expand All @@ -22,6 +22,9 @@ KVM 暂停轮询系统是 KVM 内的一项功能，其在某些情况下可以

在虚拟化环境中，脏页指的是自上次同步以来已经被修改的内存页。特别是在虚拟机热迁移过程中，源虚拟机上的内存页在复制到目标虚拟机的同时仍然处于活动状态，任何在此过程中对这些页的修改都会导致脏页的产生。监控这些脏页对于优化热迁移过程至关重要，因为过多的脏页生成可能会导致迁移延迟，甚至影响到虚拟机的运行性能。此监控功能特别适用于虚拟机热迁移的场景，其中脏页的精确监控和管理可以显著优化迁移过程。

### load_vcpu

加载虚拟 CPU（vCPU）是虚拟化环境中的一个重要步骤，它涉及为虚拟机创建和配置虚拟 CPU 实例，并将其加载到虚拟化平台中的运行环境中。加载 vCPU 将为客户机提供计算资源，允许其执行操作系统和应用程序代码。每个 vCPU 实例代表了一个独立的处理器核心，可以并行执行客户机指令。虚拟化环境中的多个 vCPU 可以并发执行客户机代码，从而支持虚拟机中的多任务和并发执行。加载多个 vCPU 允许虚拟机同时运行多个线程或进程。通过合理地分配物理资源并加载 vCPU，KVM可以优化虚拟机的性能和资源利用率。
## 挂载点

### wakeup
Expand All @@ -43,6 +46,12 @@ KVM 暂停轮询系统是 KVM 内的一项功能，其在某些情况下可以
| :----- | :---------------------- |
| kprobe | mark_page_dirty_in_slot |

### load_vcpu

| 类型 | 名称 |
| :----- | :----------------------|
| kprobe | vmx_vcpu_load |
| kprobe | vmx_vcpu_put |
## 示例输出

### wakeup
Expand Down Expand Up @@ -128,7 +137,22 @@ PID GFN REL_GFN SLOT_ID COUNTS
630632 f0122 122 3 61
....
```
### load_vcpu

```
sudo ./kvm_watcher -o
TIME:15:23:05
pid tid total_time max_time min_time counts vcpuid pcpuid
------------ ------------ ------------ ------------ ------------ ------------ ------------ ------------
49214 49224 1.2344 0.3670 0.0346 9 1 2
49214 49225 0.5978 0.2171 0.1820 3 2 0
49062 49072 2.4008 0.2656 0.0806 14 0 4
49214 49223 4.5310 0.2794 0.0217 66 0 7
54504 54514 6.9243 0.8872 0.0235 34 1 10
49145 49166 2.8076 1.1653 0.0689 10 1 9
54504 54513 3.8860 0.3099 0.0210 30 0 0
49145 49165 1.4737 0.4106 0.0690 8 0 12
```
## 参数介绍

### dirty page
Expand All @@ -140,4 +164,15 @@ PID GFN REL_GFN SLOT_ID COUNTS
- **REL_GFN**: 相对于 GFN 的偏移量。
- **NPAGES**: 内存槽的页面数量。
- **USERSPACE_ADDR**: 触发脏页的用户空间地址。
- **SLOT_ID**: 内存插槽标识，指示哪个内存区域包含了脏页。
- **SLOT_ID**: 内存插槽标识，指示哪个内存区域包含了脏页。

### load_vcpu

- **TIME(ms)**: 事件发生的时间，以毫秒为单位。
- **PID/TID**: 进程或线程的标识符。
- **total_time**: 2秒内调度vcpu的总时间。
- **max_time**: 2秒内调度vcpu的最大时间。
- **min_time**: 2秒内调度vcpu的最小时间。
- **counts**: 2秒内调度vcpu的次数。
- **vcpuid**: 虚拟CPU号。
- **pcpuid**: vCPU对应绑定的物理CPU号。
172 changes: 79 additions & 93 deletions eBPF_Supermarket/kvm_watcher/src/kvm_watcher.c
View file
Original file line number Diff line number Diff line change
Expand Up @@ -864,6 +864,80 @@ const char *getCurrentTimeFormatted() {
return ts; // 返回指向静态字符串的指针
}

// In order to sort vm_exit maps
int sort_by_key(int fd, struct exit_key *keys, struct exit_value *values) {
int err = 0;
struct exit_key lookup_key = {};
struct exit_key next_key = {};
struct exit_value exit_value;
int first = 1;
int i = 0, j;
int count = 0;
while (!bpf_map_get_next_key(fd, &lookup_key, &next_key)) {
count++;
if (first) {
first = 0;
bpf_map_lookup_elem(fd, &next_key, &exit_value);
keys[0] = next_key;
values[0] = exit_value;
i++;
lookup_key = next_key;
continue;
}
err = bpf_map_lookup_elem(fd, &next_key, &exit_value);
if (err < 0) {
fprintf(stderr, "failed to lookup exit_value: %d\n", err);
return -1;
}
// insert sort
j = i - 1;
struct exit_key temp_key = next_key;
struct exit_value temp_value = exit_value;
while (j >= 0 &&
(keys[j].pid > temp_key.pid || (keys[j].tid > temp_key.tid))) {
keys[j + 1] = keys[j];
values[j + 1] = values[j];
j--;
}
i++;
keys[j + 1] = next_key;
values[j + 1] = temp_value;
// Move to the next key
lookup_key = next_key;
}
return count;
}

// clear the specific map
int clear_map(void *lookup_key, void *next_key, enum EventType type, int fd) {
int err;
switch (type) {
case HYPERCALL:
memset(lookup_key, 0, sizeof(struct hc_key));
break;
case TIMER:
memset(lookup_key, 0, sizeof(struct timer_key));
break;
case VCPU_LOAD:
memset(lookup_key, 0, sizeof(struct load_key));
break;
case EXIT:
memset(lookup_key, 0, sizeof(struct exit_key));
break;
default:
return -1;
}
while (!bpf_map_get_next_key(fd, lookup_key, next_key)) {
err = bpf_map_delete_elem(fd, next_key);
if (err < 0) {
fprintf(stderr, "failed to cleanup map: %d\n", err);
return -1;
}
lookup_key = next_key;
}
return 1;
}

int print_hc_map(struct kvm_watcher_bpf *skel) {
int fd = bpf_map__fd(skel->maps.hc_map);
int count_fd = bpf_map__fd(skel->maps.hc_count);
Expand Down Expand Up @@ -896,24 +970,8 @@ int print_hc_map(struct kvm_watcher_bpf *skel) {
// // Move to the next key
lookup_key = next_key;
}
memset(&lookup_key, 0, sizeof(struct hc_key));
while (!bpf_map_get_next_key(fd, &lookup_key, &next_key)) {
err = bpf_map_delete_elem(fd, &next_key);
if (err < 0) {
fprintf(stderr, "failed to cleanup hc_map: %d\n", err);
return -1;
}
lookup_key = next_key;
}
memset(&lookup_key, 0, sizeof(struct hc_key));
while (!bpf_map_get_next_key(count_fd, &lookup_key, &next_key)) {
err = bpf_map_delete_elem(count_fd, &next_key);
if (err < 0) {
fprintf(stderr, "failed to cleanup hc_count: %d\n", err);
return -1;
}
lookup_key = next_key;
}
clear_map(&lookup_key, &next_key, HYPERCALL, fd);
clear_map(&lookup_key, &next_key, HYPERCALL, count_fd);
return 0;
}

Expand Down Expand Up @@ -948,64 +1006,10 @@ int print_timer_map(struct kvm_watcher_bpf *skel) {
// Move to the next key
lookup_key = next_key;
}

// Clear the timer_map
memset(&lookup_key, 0, sizeof(struct timer_key));
while (!bpf_map_get_next_key(fd, &lookup_key, &next_key)) {
err = bpf_map_delete_elem(fd, &next_key);
if (err < 0) {
fprintf(stderr, "failed to cleanup timer_map: %d\n", err);
return -1;
}
lookup_key = next_key;
}
clear_map(&lookup_key, &next_key, TIMER, fd);
return 0;
}

// In order to sort vm_exit maps
int sort_by_key(int fd, struct exit_key *keys, struct exit_value *values) {
int err = 0;
struct exit_key lookup_key = {};
struct exit_key next_key = {};
struct exit_value exit_value;
int first = 1;
int i = 0, j;
int count = 0;
while (!bpf_map_get_next_key(fd, &lookup_key, &next_key)) {
count++;
if (first) {
first = 0;
bpf_map_lookup_elem(fd, &next_key, &exit_value);
keys[0] = next_key;
values[0] = exit_value;
i++;
lookup_key = next_key;
continue;
}
err = bpf_map_lookup_elem(fd, &next_key, &exit_value);
if (err < 0) {
fprintf(stderr, "failed to lookup exit_value: %d\n", err);
return -1;
}
// insert sort
j = i - 1;
struct exit_key temp_key = next_key;
struct exit_value temp_value = exit_value;
while (j >= 0 &&
(keys[j].pid > temp_key.pid || (keys[j].tid > temp_key.tid))) {
keys[j + 1] = keys[j];
values[j + 1] = values[j];
j--;
}
i++;
keys[j + 1] = next_key;
values[j + 1] = temp_value;
// Move to the next key
lookup_key = next_key;
}
return count;
}

int print_vcpu_load_map(struct kvm_watcher_bpf *skel) {
int fd = bpf_map__fd(skel->maps.load_map);
int err;
Expand Down Expand Up @@ -1040,16 +1044,7 @@ int print_vcpu_load_map(struct kvm_watcher_bpf *skel) {
load_value.vcpu_id, load_value.pcpu_id);
lookup_key = next_key;
}
// clear the maps
memset(&lookup_key, 0, sizeof(struct load_key));
while (!bpf_map_get_next_key(fd, &lookup_key, &next_key)) {
err = bpf_map_delete_elem(fd, &next_key);
if (err < 0) {
fprintf(stderr, "failed to cleanup counters: %d\n", err);
return -1;
}
lookup_key = next_key;
}
clear_map(&lookup_key, &next_key, VCPU_LOAD, fd);
return 0;
}

Expand Down Expand Up @@ -1109,16 +1104,7 @@ void __print_exit_map(int fd, enum NameType name_type) {
getName(keys[i].reason, name_type));
}
}
// clear the maps
memset(&lookup_key, 0, sizeof(struct exit_key));
while (!bpf_map_get_next_key(fd, &lookup_key, &next_key)) {
int err = bpf_map_delete_elem(fd, &next_key);
if (err < 0) {
fprintf(stderr, "failed to cleanup counters: %d\n", err);
return;
}
lookup_key = next_key;
}
clear_map(&lookup_key, &next_key, EXIT, fd);
}
int print_exit_map(struct kvm_watcher_bpf *skel) {
int exit_fd = bpf_map__fd(skel->maps.exit_map);
Expand Down

0 comments on commit ca9488a

Please sign in to comment.