Add first golang-only test #1037
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,73 @@ | ||
| # Design decisions | ||
|
|
||
| This document is a collection of high-level decisions, that have been made in the project. Why do we have a section about how things use to be? In case if we want to revisit certain concept it will be useful to know why we chose one approach over another. | ||
|
|
||
| ## Using Domain Specific Language (DSL) vs writing native golang tests | ||
|
|
||
| When talking about Eden tests till version 0.9.12, we are talking about `escript`, a Domain Specific Language (DSL) which describes test case and uses Eden to setup, environment. Escript looks like this | ||
|
|
||
| ```bash | ||
| # 1 Setup environment variables | ||
| {{$port := "2223"}} | ||
| {{$network_name := "n1"}} | ||
| {{$app_name := "eclient"}} | ||
|
|
||
| # ... | ||
|
|
||
| # 2 run eden commands | ||
| eden -t 1m network create 10.11.12.0/24 -n {{$network_name}} | ||
|
|
||
| # 3 run escript commands | ||
| test eden.network.test -test.v -timewait 10m ACTIVATED {{$network_name}} | ||
|
|
||
| eden pod deploy -n {{$app_name}} --memory=512MB {{template "eclient_image"}} -p {{$port}}:22 --networks={{$network_name}} | ||
|
|
||
| # 4 execute shell script which are defined inside escript file | ||
| exec -t 5m bash ssh.sh | ||
| stdout 'Ubuntu' | ||
|
|
||
| # 5 overwrite configuration | ||
| -- eden-config.yml -- | ||
| {{/* Test's config. file */}} | ||
| test: | ||
| controller: adam://{{EdenConfig "adam.ip"}}:{{EdenConfig "adam.port"}} | ||
| eve: | ||
| {{EdenConfig "eve.name"}}: | ||
| onboard-cert: {{EdenConfigPath "eve.cert"}} | ||
| serial: "{{EdenConfig "eve.serial"}}" | ||
| model: {{EdenConfig "eve.devmodel"}} | ||
|
|
||
| -- ssh.sh -- | ||
| EDEN={{EdenConfig "eden.root"}}/{{EdenConfig "eden.bin-dist"}}/{{EdenConfig "eden.eden-bin"}} | ||
| for i in `seq 20` | ||
| do | ||
| sleep 20 | ||
| # Test SSH-access to container | ||
| echo $i\) $EDEN sdn fwd eth0 {{$port}} -- {{template "ssh"}} grep Ubuntu /etc/issue | ||
| $EDEN sdn fwd eth0 {{$port}} -- {{template "ssh"}} grep Ubuntu /etc/issue && break | ||
| done | ||
| ``` | ||
|
|
||
| So you can | ||
|
|
||
| 1) setup some environment variables | ||
| 2) run eden commands | ||
| 3) run escript commands with test | ||
| 4) execute user-defined shell scripts | ||
| 5) overwrite eden configuration | ||
|
|
||
| Escript file is fed as input to eden test command, which parses the variables using golang templates to substitute some variables using templates in golang, then it's going to read line by line and execute it via os.exec call in golang. test is actually a compiled golang binary, we compile it inside eden repo and then execute it, under the hood it is a manually forked version of standard golang test package with some added commands. | ||
|
|
||
| For more information on specific topics refer to following documents: | ||
|
|
||
| - Escript test structure [doc](./escript/test-anatomy-sample.md) | ||
| - Writing eden tasks for escript [doc](./escript/task-writing.md) | ||
| - Running tests [doc](./escript/test-running.md) | ||
|
|
||
| So there are several problems with that approach: | ||
|
|
||
| **You have to be an escript expert**: when new person comes to a project, they will have some industry skills, like C++ expertise, Rust, Golang, Computer Networking, etc. but if you never worked on this project, you never heard about escript, its' sole purpose was to be part of Eden and be useful language to describe tests for Eden. One might argue, but that's just bash on steroids. Well, true, but do you know what `!` means? It's not equal parameter in escript. So it is like bash, but not exactly it, and it might take time to figure out other hidden features, that could be solved by proper documentation. But before spiraling down that conversation lets think about tools? Imagine, that you wrote new fancy eden test and it doesn't work. Usual thing, you would say. Test Driven Development is all about writing failing tests first and then making them work. | ||
|
|
||
| **But how do you debug that test?** Because you're running an interpreter, which runs bash commands via os.exec plus you execute golang program, which is written and compiled inside the repository. Debug printing would work, but I find debuggers much more useful and efficient most of the times. It is a matter of taste, but better to debugger at your disposal, than not to have it in this case. Even worse, how do you debug escript problem? You need to know it's internals, there's no Stackoverflow or Google by your side, nor there are books about it. | ||
|
|
||
| **Bumping golang is actually manual action**: last but not least, we have to maintain custom test files which a basically copy-paste with one added function, bumping golang turns into some weird dances in that case. |
File renamed without changes.
File renamed without changes.
File renamed without changes.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,98 @@ | ||
| # How to write tests with Eden | ||
|
|
||
| The eden golang test SDK for eve consists of two categories of functions: | ||
|
|
||
| 1. `openevec`: infrastructure manager. `openevec` enables you to deploy a controller, a backend for that controller, deploy an edge node, etc. It does not change anything on an individual edge node. | ||
| 2. `evetestkit`: provides useful collection of functions to describe expected state of the system (controller, EVE, AppInstances) | ||
|
|
||
| And those functions are used in context of standard golang test library. We do a setup in TestMain (for more info check [this](https://pkg.go.dev/testing#hdr-Main)) and then write test functions which interact with the environment we created. Setting up environment takes couple of minutes, so it makes sense to do it once and run tests within that environment. | ||
|
|
||
| Source for the example below can be found [here](../tests/sec/sec_test.go) | ||
|
|
||
| In order to test a feature in Eden you need to | ||
|
|
||
| ## 1. Create a configuration file and describe your environment | ||
|
|
||
| The glue between `openevec` and `evetestkit` is a configuration structure called `EdenSetupArgs`. It contains all the necessary information to setup all the components of the test: controller, EVE, etc. You can fill in the structure manually, however `openevec` provides you with a convenient function to create default configuration structure providing project root path: | ||
|
|
||
| ```go | ||
| // ... | ||
| func TestMain(m *testing.M) { | ||
| currentPath, err := os.Getwd() | ||
| if err != nil { | ||
| log.Fatal(err) | ||
| } | ||
| twoLevelsUp := filepath.Dir(filepath.Dir(currentPath)) | ||
|
|
||
| cfg := openevec.GetDefaultConfig(twoLevelsUp) | ||
|
|
||
| if err = openevec.ConfigAdd(cfg, cfg.ConfigName, "", false); err != nil { | ||
| log.Fatal(err) | ||
| } | ||
| // ... | ||
| } | ||
| ``` | ||
|
|
||
| Due to backward compatibility with escript as of time of writing, the project root path should be Eden repository folder. So if you add tests in the `eden/tests/my-awesome-test` you need to go two levels up, will be removed with escript | ||
| Also we need to write configuration file to file system, because there are components (like changer) which read configuration from file system, will be removed with escript | ||
|
|
||
| ## 2. Initialize `openevec` Setup, Start and Onboard EVE node | ||
|
|
||
| When configuration you need `openevec` to create all needed certificates, start backend. For that we create `openevec` object based on a configuration provided, it is just a convinient wrapper. | ||
|
|
||
| ```go | ||
| // ... | ||
| func TestMain(m *testing.M) { | ||
| // ... | ||
| evec := openvec.CreateOpenEVEC(cfg) | ||
|
|
||
| evec.SetupEden(/* ... */) | ||
| evec.StartEden(/* ... */) | ||
| evec.OnboardEve(/* ... */) | ||
|
|
||
| // ... | ||
| } | ||
| ``` | ||
|
|
||
| ## 3. Initialize `evetestkit` and run test suite | ||
|
|
||
| `evetestkit` provides an abstraction over EveNode, which is used to describe expected state of the system. Each EveNode is running within a project You can create a global object within one test file to use it across multiple tests. Note that EveNode is not threadsafe, since controller is stateful, so tests should be run consequently (no t.Parallel()) | ||
|
|
||
| ```go | ||
| const projectName = "security-test" | ||
| var eveNode *evetestkit.EveNode | ||
|
|
||
| func TestMain(m *testing.M) { | ||
| // ... | ||
| node, err := evetestkit.InitializeTestFromConfig(projectName, cfg, evetestkit.WithControllerVerbosity("debug")) | ||
| if err != nil { | ||
| log.Fatalf("Failed to initialize test: %v", err) | ||
| } | ||
|
|
||
| eveNode = node | ||
| res := m.Run() | ||
| os.Exit(res) | ||
| } | ||
| ``` | ||
|
|
||
| ## 4. Write your test | ||
|
|
||
| Below is an example of test, which check if AppArmor is enabled (specific file on EVE exists). It uses `EveReadFile` function from `evetestkit` | ||
|
|
||
| ```go | ||
| const appArmorStatus = "/sys/module/apparmor/parameters/enabled" | ||
|
There was a problem hiding this comment. It would be great if this (and many other files) were parametrized, so they could be a property of some library. But that is beyond the scope of here. |
||
| // ... | ||
| func TestAppArmorEnabled(t *testing.T) { | ||
| log.Println("TestAppArmorEnabled started") | ||
| defer log.Println("TestAppArmorEnabled finished") | ||
|
|
||
| out, err := eveNode.EveReadFile(appArmorStatus) | ||
| if err != nil { | ||
| t.Fatal(err) | ||
| } | ||
|
|
||
| exits := strings.TrimSpace(string(out)) | ||
| if exits != "Y" { | ||
| t.Fatal("AppArmor is not enabled") | ||
| } | ||
| ``` | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks like you disable setup in workflows and move its logic here. Do you have any reason for that?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Will we keep our collect-info and other debugging logic in case of failure on onboarding stage, which is quite predictable?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hey @giggsoff , nice seeing you here :) So I did it purely from backward compatibility. For some reason escript tests didn't want to onboard through test.
Yes, we keep all collect-info logic there, no changes in that regard