Skip to content

Setting up an S3 storage

Jump to bottom
L. Le Meur edited this page Nov 18, 2020 · 5 revisions

Some integrators have been unable to setup an S3 storage for encrypted publications. S3 storage in the LCP server is managed via the official AWS SDK stored on Github. This document is a quick "how-to", extracted from the AWS SDK documentation.

Create a bucket.

Make files publicly readable

source: https://docs.aws.amazon.com/AmazonS3/latest/dev/WebsiteAccessPermissionsReqd.html

  • Select the bucket
  • Go to Permissions
  • Add this bucket strategy:
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "PublicReadGetObject",
            "Effect": "Allow",
            "Principal": "*",
            "Action": "s3:GetObject",
            "Resource": "arn:aws:s3:::nameOfYourBucket/*"
        }
    ]
}

warning : change nameOfYourBucket for the name of your bucket.

Get your credentials

source: https://docs.aws.amazon.com/sdk-for-go/v1/developer-guide/setting-up.html

  • Open the IAM console.
  • create a user "lcp-server" + programmatic access
  • create a group "s3-client" + strategy "AmazonS3FullAccess"
  • add the user to the group
  • no optional key
  • download the csv -> access key + secret access key
  • store it securely.

ex. https://s3.console.aws.amazon.com/s3/buckets/edrlab-lcp-storage?region=eu-west-3&tab=objects

Set environment variables

source: https://docs.aws.amazon.com/sdk-for-go/v1/developer-guide/configuring-sdk.html

The AWS Region needs to be provided in the AWS shared config or on the environment variable as AWS_REGION. Credentials also must be provided; they will default to shared config file, but can be loaded from the environment if provided.

The AWS SDK for Go requires credentials (an access key and secret access key) to sign requests to AWS. You can specify your credentials in several different locations, depending on your particular use case. 

Access keys consist of an access key ID and secret access key, which are used to sign programmatic requests that you make to AWS. If you don’t have access keys, you can create them by using the AWS Management Console. We recommend that you use IAM access keys instead of AWS root account access keys. IAM lets you securely control access to AWS services and resources in your AWS account.

ex. on MacOS and Linux

$ export  AWS_REGION=YOUR_REGION
$ export AWS_ACCESS_KEY_ID=YOUR_AKID
$ export AWS_SECRET_ACCESS_KEY=YOUR_SECRET_KEY

warning: make sure these environment variables are set permanently.

note: alternatively, the access key and secret can be set in a shared credential file, read the doc.

Test

Go to https://github.com/aws/aws-sdk-go/

Copy/paste the sample in s3test.go in a gospace/src/s3test folder (replace "gospace" by your own go space folder name). This cli uses bucket and object key as parameters.

Copy an epub file (e.g. moby-dick.epub) into this test folder.

$ cd /Users/laurentlemeur/work/gospace/src/s3test
$ go run s3test.go -b edrlab-lcp-storage -k moby-dick.epub -d 10m < moby-dick.epub

-> is now fetchable via something like https://edrlab-lcp-storage.s3.eu-west-3.amazonaws.com/moby-dick.epub (the exact URL is found in the AWS console, object screen).

Configure properly the LCP server

Read carefully the LCP Server configuration page.

A few S3 parameters are used in the codebase but not described in the configuration page, because they don't seem to be used in practice.

This is the case for the token parameter, described in AWS Temporary Security Credentials in IAM and potentially used if credentials are defined in the configuration file, by a call to NewsStaticCredentials.

This is also the case for the disable_ssl and path_style parameters, for which some explanations can be found here and here.

Clone this wiki locally