Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add files api #360

Merged
merged 34 commits into from
Nov 30, 2023
Merged
Changes from 1 commit
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
041e051
move api from sda-pipeline
MalinAhlberg Oct 16, 2023
2f1393a
adapt code to new repo
MalinAhlberg Oct 16, 2023
82fb61c
move in api db functions from sda-pipeline
MalinAhlberg Oct 16, 2023
9c13a40
refactor jwt verification
MalinAhlberg Oct 17, 2023
2fe97ad
refactor: use common auth
MalinAhlberg Oct 26, 2023
f45d7a7
fix: db query for get files
MalinAhlberg Oct 31, 2023
6872913
fix: avoid panic if validation token is not setup properly
MalinAhlberg Oct 31, 2023
95ece28
clean up and simplify api tests
MalinAhlberg Oct 20, 2023
c2b4906
refactor: break out auth header parsning and add test
MalinAhlberg Oct 31, 2023
2cc4ab1
refactor: merge api test files
MalinAhlberg Nov 8, 2023
e84581a
refactor: move mock authentication to helper
MalinAhlberg Nov 8, 2023
d06e0fe
fix api tests
MalinAhlberg Nov 9, 2023
4bcf9a6
update dependencies
MalinAhlberg Nov 9, 2023
3e0fef1
add api integration tests
MalinAhlberg Nov 9, 2023
ff410e0
update dependencies
MalinAhlberg Nov 9, 2023
358aa1c
fix linter complaints
MalinAhlberg Nov 9, 2023
08f2846
break out api test
MalinAhlberg Nov 13, 2023
b20b3e6
tests: update env var to integration tests
MalinAhlberg Nov 13, 2023
466645e
Update sda/cmd/api/api.go
MalinAhlberg Nov 14, 2023
70a334a
refactor: prettify, clean up etc
MalinAhlberg Nov 14, 2023
6636457
fix: update test conf for mq
MalinAhlberg Nov 14, 2023
983c328
refactor: clean up go mod file
MalinAhlberg Nov 14, 2023
7e18b71
remove newline
MalinAhlberg Nov 14, 2023
010c7e8
Add api readme
MalinAhlberg Nov 14, 2023
2b29bda
Update sda/cmd/api/api.md
MalinAhlberg Nov 23, 2023
fb710c3
fix: list all relevant files in files api
MalinAhlberg Nov 23, 2023
c371b79
Update sda/cmd/api/api.md
MalinAhlberg Nov 24, 2023
3670a72
Update sda/cmd/api/api.go
MalinAhlberg Nov 27, 2023
9ca18d6
Update sda/cmd/api/api.go
MalinAhlberg Nov 27, 2023
8d10f96
purge all containers if oidc fails
MalinAhlberg Nov 27, 2023
153ab99
refactor: make all tests part of testsuite
MalinAhlberg Nov 27, 2023
f384cdd
feat: add file size to files api
MalinAhlberg Nov 27, 2023
ece5d66
add test for db getuserfiles
MalinAhlberg Nov 27, 2023
61b4cb1
Revert "feat: add file size to files api"
MalinAhlberg Nov 30, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
add api integration tests
MalinAhlberg committed Nov 30, 2023
commit 3e0fef151fdf3f74f7fba61024ed6f6c1b7bd28e
27 changes: 25 additions & 2 deletions .github/integration/sda-s3-integration.yml
Original file line number Diff line number Diff line change
@@ -276,6 +276,27 @@ services:
- ./sda/oidc.py:/oidc.py
- shared:/shared

api:
command: [ sda-api ]
container_name: api
depends_on:
credentials:
condition: service_completed_successfully
postgres:
condition: service_healthy
oidc:
condition: service_healthy
environment:
- DB_PASSWORD=download
- DB_USER=download
image: ghcr.io/neicnordic/sensitive-data-archive:PR${PR_NUMBER}
ports:
- "8090:8080"
restart: always
volumes:
- ./sda/config.yaml:/config.yaml
- shared:/shared

integration_test:
container_name: tester
command:
@@ -298,7 +319,9 @@ services:
sync-api:
condition: service_started
verify:
condition: service_started
condition: service_started
api:
condition: service_started
environment:
- PGPASSWORD=rootpasswd
- STORAGETYPE=s3
@@ -313,4 +336,4 @@ volumes:
minio_data:
postgres_data:
rabbitmq_data:
shared:
shared:
9 changes: 9 additions & 0 deletions .github/integration/tests/sda/20_ingest-verify_test.sh
Original file line number Diff line number Diff line change
@@ -68,3 +68,12 @@ until [ "$(curl -su guest:guest http://rabbitmq:15672/api/queues/sda/verified/ |
fi
sleep 2
done

# Test the API files endpoint
token="$(curl http://oidc:8080/tokens | jq -r '.[0]')"
curl -k -L "http://api:8080/files" -H "Authorization: Bearer $token"
response="$(curl -k -L "http://api:8080/files" -H "Authorization: Bearer $token" | jq -r 'sort_by(.inboxPath)|.[-1].fileStatus')"
if [ "$response" != "uploaded" ]; then
echo "API returned incorrect value, expected ready got: $response"
exit 1
fi