Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a workflow that checks if all existing ruletypes have tests #71

Merged
merged 1 commit into from
Apr 1, 2024
Merged

Add a workflow that checks if all existing ruletypes have tests #71

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
84 changes: 84 additions & 0 deletions .github/workflows/ensure-tests-for-all-rules.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
name: Ensure we have tests for all available rule types
on:
pull_request:
schedule:
- cron: '0 0 * * *' # Every day at midnight
workflow_dispatch:

jobs:
# --------------------------------------------------------------------------------------------------
# Job to check if we have tests for all available rule types
# --------------------------------------------------------------------------------------------------
run:
name: Run
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@9bb56186c3b09b4f86b1c65136769dd318469633
- name: Compare the available rule types with the tests
env:
SMOKE_TESTS_LIST: "smoke-tests-list.yml"
GH_TOKEN: ${{ secrets.GH_PAT_READ_SMOKE_TESTS }}
SKIPPED_SMOKE_TESTS_LIST: "skipped-smoke-tests-list.yml"
shell: bash
run: |
AVAILABLE_RULES=$(find ./rule-types/github/ -type f -name "*.yaml" -exec basename {} .yaml \; | sort)

temp_dir=$(mktemp -d)
gh repo clone stacklok/minder-smoke-tests $temp_dir -- -q
pushd $temp_dir

# Get the list of existing rule types smoke tests
EXISTING_TESTS=$(grep -v '^ *#' $SMOKE_TESTS_LIST | grep "rules/" | sed 's/#.*$//' | sed 's/",\?$//' | sed 's/^"//' | awk -F'/' '{print $NF}' | sort)
SKIPPED_TESTS=$(cat $SKIPPED_SMOKE_TESTS_LIST)

echo -e "\e[93m**********************************************************************************************\e[0m"
echo -e "\e[93m* The following rule types are available *\e[0m"
echo -e "\e[93m**********************************************************************************************\e[0m"
echo -e "$AVAILABLE_RULES"

echo -e "\e[93m**********************************************************************************************\e[0m"
echo -e "\e[93m* The following rule types have smoke tests *\e[0m"
echo -e "\e[93m**********************************************************************************************\e[0m"
echo -e "$EXISTING_TESTS"

if [ -n "$SKIPPED_TESTS" ]; then
echo -e "\e[93m**********************************************************************************************\e[0m"
echo -e "\e[93m* The following rule types are skipped *\e[0m"
echo -e "\e[93m**********************************************************************************************\e[0m"
echo -e "The following rule tests are explicitly skipped:"
echo -e "$SKIPPED_TESTS"
fi

# Initialize a flag to track missing tests
MISSING_TESTS=false

echo -e "\e[93m**********************************************************************************************\e[0m"
echo -e "\e[93m* Start comparing... *\e[0m"
echo -e "\e[93m**********************************************************************************************\e[0m"

# Check if we have tests for all available rule types
for rule in $AVAILABLE_RULES; do
if [[ $SKIPPED_TESTS =~ $rule ]]; then
echo -e "\e[93m$rule (skipped)\e[0m"
elif [[ ! $EXISTING_TESTS =~ $rule ]]; then
echo -e "\e[91m$rule\e[0m"
MISSING_TESTS=true
else
echo -e "$rule"
fi
done

# Check the flag after looping through all rules
if [ "$MISSING_TESTS" = true ]; then
echo -e "\e[91m**********************************************************************************************\e[0m"
echo -e "\e[91m* FAILURE *\e[0m"
echo -e "\e[91m**********************************************************************************************\e[0m"
echo -e "One or more rule types are missing tests (highlighted in RED)"
exit 1
else
echo -e "\e[92m**********************************************************************************************\e[0m"
echo -e "\e[92m* SUCCESS *\e[0m"
echo -e "\e[92m**********************************************************************************************\e[0m"
echo -e "All rule types have tests!"
fi
Loading