Add files api

[MalinAhlberg]

This PR closes #435. It is based on neicnordic/sda-pipeline#540

• merges api from repo sda-pipeline into sda
• adds the endpoint files to the API service for retrieving information about the files a user has submitted
• moves userauth from cmd to internal, for reuse in api
• updates and refactoring of the api code to use the common code

Testing:
When running the containers from sda-s3-integration.yml, this should work:

 # call to the oidc server 
token="$(curl http://localhost:8080/tokens | jq -r '.[0]')" 
 # call to the api
curl -k -L "http://localhost:8090/files" -H "Authorization: Bearer $token" 

If the integration tests (docker compose -f ~/NBIS/sloth/sensitive-data-archive/.github/integration/sda-s3-integration.yml run integration_test) are run, the curl response should be
[{"inboxPath":"requester_demo.org/data/file1.c4gh","fileStatus":"uploaded","createAt":"2023-11-13T10:12:43.144242Z"}]

Updates of the status of the user's file in the psql db should be reflected in the output of the call, so that only the last event is shown.

fixes #444
fixes #436

[MalinAhlberg]

Thanks for the very detailed review, @jbygdell 👍 ! I fixed almost all of the comments, except for the one in config.go, since the pattern

err = ....
if err != nil {

is used for all other cases in that section.

[jbygdell]

A short Readme.md file needs to be created

# Api

short one line description

## Service Description

More in depth description

[MalinAhlberg]

A short Readme.md file needs to be created

In sda/cmd/api?

[jbygdell]

A short Readme.md file needs to be created

In sda/cmd/api?

Yes, name it api.md so that it matches the other readme files

[MalinAhlberg]

A short Readme.md file needs to be created

See be12ee5

[dbampalikis]

Really nice! Tested it manually (without using the integration test) and it works great! Also tested with another user and the result is an empty json array, which feels correct. Not sure if we would like to add a message such as no files found or something, but that's just an extra thing I guess Updated the review after Alex's finding

[aaperis]

I made a test with two files. I uploaded the first one:

./sda-cli upload -config s3cfg --encrypt-with-key c4gh.pub.pem test_file_demo

curl -sk -L "http://localhost:8090/files" -H "Authorization: Bearer $token" | jq

and the files api responded nicely:

[
  {
    "inboxPath": "test_dummy.org/test_file_demo.c4gh",
    "fileStatus": "uploaded",
    "createAt": "2023-11-17T14:07:48.825563Z"
  }
]

Then, I uploaded another file:

./sda-cli upload -config s3cfg test_file_demo.c4gh -targetDir asd/testtest.c4gh

./sda-cli list -config s3cfg 
The provided token expires in less than 24 hours
Consider renewing the token.
1.00MB   asd/testtest.c4gh/test_file_demo.c4gh 
1.00MB   test_file_demo.c4gh 

Then

curl -sk -L "http://localhost:8090/files" -H "Authorization: Bearer $token" | jq

but the files api returns only info on the last uploaded file:

[
  {
    "inboxPath": "test_dummy.org/asd/testtest.c4gh/test_file_demo.c4gh",
    "fileStatus": "uploaded",
    "createAt": "2023-11-17T14:11:47.143238Z"
  }
]

We sat down with @dbampalikis and tested a correction to the db query, see suggestion in Dimi's comments :-)

[dbampalikis]

Really nice, apart for the comment about the database.
Also tested with another user and the result is an empty json array, which feels correct. Not sure if we would like to add a message such as no files found or something, but that's just an extra thing.
Finally, maybe we should have tests for multiple files?

[codecov-commenter]

Codecov Report

All modified and coverable lines are covered by tests ✅

❗ No coverage uploaded for pull request base (main@b159870). Click here to learn what that means.

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #360   +/-   ##
=======================================
  Coverage        ?   60.89%           
=======================================
  Files           ?       19           
  Lines           ?     3445           
  Branches        ?        0           
=======================================
  Hits            ?     2098           
  Misses          ?     1207           
  Partials        ?      140           

Flag	Coverage Δ
unittests	60.89% <0.00%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[MalinAhlberg]

Thanks @aaperis and @dbampalikis , really good that you spotted this 🙈 🙏 ! I commited your fix along with a test case for this, see dd0e91f.

[MalinAhlberg]

(force pushed to rebase on main)

[aaperis]

Nice!

[pontus]

Looks good.

I have some thoughts about the TLS-configuration but I think I'll make an issue for general overhaul at some point instead.