This project is an emulator for several AWS services, à la Localstack. README in progress!
Currently supported services (see below for full support details):
- DynamoDB - highly experimental, only enough for Kinesis Client Library to work. Not recommended to use!
- Kinesis
- KMS
- S3
- SQS
Aws-in-a-box runs on HTTP (not HTTPS) but supports HTTP2 upgrade with h2c (HTTP without TLS).
- High-performance; no overhead from docker or proxies
- Single statically-linked 7MB native binary. No interpereter/runtime hell. (There are also 3MB compressed docker images if you prefer)
- Simple persistence scheme using atomic file writes. When using the native binary, no more broken persistent docker volumes
The easiest way to consume this project is from pre-built artifacts on the release page or the docker image. You can also build from source with either go native tooling or with Bazel, see Development section.
-addr string
Address to run on (default "localhost:4569")
-enableKMS
Enable Kinesis service (default true)
-enableKinesis
Enable Kinesis service (default true)
-enableSQS
Enable SQS service (default true)
-experimental_enableDynamoDB
Enable DynamoDB service (default true)
-experimental_enableS3
Enable S3 service (default true)
-kinesisDefaultDuration duration
How long to retain messages. Can be used to control memory usage. After creation, retention can be adjusted with [Increase/Decrease]StreamRetentionPeriod (default 24h0m0s)
-kinesisInitialShardsPerStream int
How many shards to create for each stream listed in -kinesisInitialStreams (default 2)
-kinesisInitialStreams string
Streams to create at startup. Example: stream1,stream2,stream3
-kinesisStreamCreateDuration duration
How long a new Kinesis stream stays in CREATING status (default 5s)
-kinesisStreamDeleteDuration duration
How long a deleted Kinesis stream stays in DELETING status (default 5s)
-logLevel string
debug/info/warn/error (default "debug")
-persistDir string
Directory to persist data to. If empty, data is not persisted.
-s3InitialBuckets string
Buckets to create at startup. Example: bucket1,bucket2,bucket3
You can use either native go tooling or Bazel.
go run .bazel run //:aws-in-a-box
go test ./...bazel test //...
Most of Kinesis is implemented, including the Consumer APIS. Remaining work:
- KMS integration not wired up
- A few List/Describe APIs are missing
- Shard split/merge
There is no persistence for Kinesis data.
Click to expand the detailed support table
| API | Support Status | Caveats/Notes |
|---|---|---|
| AddTagsToStream | ✅ Supported | |
| CreateStream | ✅ Supported | |
| DecreaseStreamRetentionPeriod | ✅ Supported | |
| DeleteStream | ✅ Supported | |
| DeregisterStreamConsumer | ✅ Supported | |
| DescribeLimits | ❌ Unsupported | |
| DescribeStream | ❌ Unsupported | This API is discouraged by AWS |
| DescribeStreamConsumer | ✅ Supported | |
| DescribeStreamSummary | ✅ Supported | |
| DisableEnhancedMonitoring | ❌ Unsupported | Cloudwatch not implemented |
| EnableEnhancedMonitoring | ❌ Unsupported | Cloudwatch not implemented |
| GetRecords | ✅ Supported | |
| GetShardIterator | ✅ Supported | |
| IncreaseStreamRetentionPeriod | ✅ Supported | |
| ListShards | ✅ Supported | |
| ListStreamConsumers | ❌ Unsupported | |
| ListStreams | ✅ Supported | |
| ListTagsForStream | ✅ Supported | |
| MergeShards | ❌ Unsupported | No support for merging/splitting yet. |
| PutRecord | ✅ Supported | |
| PutRecords | ❌ Unsupported | Use PutRecord for single records instead. |
| RegisterStreamConsumer | ✅ Supported | |
| RemoveTagsFromStream | ✅ Supported | |
| SplitShard | ❌ Unsupported | No support for merging/splitting yet. |
| StartStreamEncryption | ❌ Unsupported | |
| StopStreamEncryption | ❌ Unsupported | |
| SubscribeToStream | ✅ Supported | |
| UpdateShardCount | ❌ Unsupported | No support for merging/splitting yet. |
| UpdateStreamMode | ❌ Unsupported |
Most of KMS is implemented. Remaining work:
- Key deletion/key rotation is missing
- Grants are missing
- Key policies are missing
KMS data is fully persisted.
Click to expand the detailed support table
| API | Support Status | Caveats/Notes |
|---|---|---|
| CancelKeyDeletion | ❌ Unsupported | |
| ConnectCustomKeyStore | ❌ Unsupported | |
| CreateAlias | ✅ Supported | |
| CreateCustomKeyStore | ❌ Unsupported | |
| CreateGrant | ❌ Unsupported | |
| CreateKey | ✅ Supported | ECC_SECG_P256K1 and SM2 not supported |
| Decrypt | ✅ Supported | |
| DeleteAlias | ✅ Supported | |
| DeleteCustomKeyStore | ❌ Unsupported | |
| DeleteImportedKeyMaterial | ❌ Unsupported | |
| DescribeCustomKeyStores | ❌ Unsupported | |
| DescribeKey | ✅ Supported | Lots of metadata properties missing |
| DisableKey | ✅ Supported | |
| DisableKeyRotation | ❌ Unsupported | |
| DisconnectCustomKeyStore | ❌ Unsupported | |
| EnableKey | ✅ Supported | |
| EnableKeyRotation | ❌ Unsupported | |
| Encrypt | ✅ Supported | |
| GenerateDataKey | ✅ Supported | |
| GenerateDataKeyPair | ✅ Supported | ECC_SECG_P256K1 not supported |
| GenerateDataKeyPairWithoutPlaintext | ✅ Supported | ECC_SECG_P256K1 not supported |
| GenerateDataKeyWithoutPlaintext | ✅ Supported | |
| GenerateMac | ✅ Supported | |
| GenerateRandom | ✅ Supported | |
| GetKeyPolicy | ❌ Unsupported | |
| GetKeyRotationStatus | ❌ Unsupported | |
| GetParametersForImport | ❌ Unsupported | |
| GetPublicKey | ❌ Unsupported | |
| ImportKeyMaterial | ❌ Unsupported | |
| ListAliases | ✅ Supported | |
| ListGrants | ❌ Unsupported | |
| ListKeyPolicies | ❌ Unsupported | |
| ListKeys | ✅ Supported | |
| ListResourceTags | ✅ Supported | |
| ListRetirableGrants | ❌ Unsupported | |
| PutKeyPolicy | ❌ Unsupported | |
| ReEncrypt | ✅ Supported | |
| ReplicateKey | ❌ Unsupported | |
| RetireGrant | ❌ Unsupported | |
| RevokeGrant | ❌ Unsupported | |
| ScheduleKeyDeletion | ❌ Unsupported | |
| Sign | ✅ Supported | |
| TagResource | ✅ Supported | |
| UntagResource | ✅ Supported | |
| UpdateAlias | ✅ Supported | |
| UpdateCustomKeyStore | ❌ Unsupported | |
| UntagResource | ✅ Supported | |
| UpdateKeyDescription | ✅ Supported | |
| UpdatePrimaryRegion | ❌ Unsupported | |
| Verify | ✅ Supported | |
| VerifyMac | ✅ Supported |
Most common operations of S3 are implemented. Remaining work:
- Versioning
- A bunch of metadata/usage APIs
- Policy/ACL is missing
S3 blocks are persisted, but metadata is not. This will be fixed in the future.
Click to expand the detailed support table
| API | Support Status | Caveats/Notes |
|---|---|---|
| AbortMultipartUpload | ✅ Supported | |
| CompleteMultipartUpload | ✅ Supported | |
| CopyObject | ✅ Supported | |
| CreateBucket | ✅ Supported | |
| CreateMultipartUpload | ✅ Supported | |
| DeleteBucket | ✅ Supported | |
| DeleteBucketAnalyticsConfiguration | ❌ Unsupported | |
| DeleteBucketCors | ❌ Unsupported | |
| DeleteBucketEncryption | ❌ Unsupported | |
| DeleteBucketIntelligentTieringConfiguration | ❌ Unsupported | |
| DeleteBucketInventoryConfiguration | ❌ Unsupported | |
| DeleteBucketLifecycle | ❌ Unsupported | |
| DeleteBucketMetricsConfiguration | ❌ Unsupported | |
| DeleteBucketOwnershipControls | ❌ Unsupported | |
| DeleteBucketPolicy | ❌ Unsupported | |
| DeleteBucketReplication | ❌ Unsupported | |
| DeleteBucketTagging | ✅ Supported | |
| DeleteBucketWebsite | ❌ Unsupported | |
| DeleteObject | ✅ Supported | |
| DeleteObjects | ✅ Supported | |
| DeleteObjectTagging | ✅ Supported | |
| DeletePublicAccessBlock | ❌ Unsupported | |
| GetBucketAccelerateConfiguration | ❌ Unsupported | |
| GetBucketAcl | ❌ Unsupported | |
| GetBucketAnalyticsConfiguration | ❌ Unsupported | |
| GetBucketCors | ❌ Unsupported | |
| GetBucketEncryption | ❌ Unsupported | |
| GetBucketIntelligentTieringConfiguration | ❌ Unsupported | |
| GetBucketInventoryConfiguration | ❌ Unsupported | |
| GetBucketLifecycle | ❌ Unsupported | Discouraged by AWS |
| GetBucketLifecycleConfiguration | ❌ Unsupported | |
| GetBucketLocation | ❌ Unsupported | |
| GetBucketLogging | ❌ Unsupported | |
| GetBucketMetricsConfiguration | ❌ Unsupported | |
| GetBucketNotification | ❌ Unsupported | Discouraged by AWS. no longer used |
| GetBucketNotificationConfiguration | ❌ Unsupported | |
| GetBucketOwnershipControls | ❌ Unsupported | |
| GetBucketPolicy | ❌ Unsupported | |
| GetBucketPolicyStatus | ❌ Unsupported | |
| GetBucketReplication | ❌ Unsupported | |
| GetBucketRequestPayment | ❌ Unsupported | |
| GetBucketTagging | ✅ Supported | |
| GetBucketVersioning | ❌ Unsupported | |
| GetBucketWebsite | ❌ Unsupported | |
| GetObject | ✅ Supported | |
| GetObjectAcl | ❌ Unsupported | |
| GetObjectAttributes | ❌ Unsupported | |
| GetObjectLegalHold | ❌ Unsupported | |
| GetObjectLockConfiguration | ❌ Unsupported | |
| GetObjectRetention | ❌ Unsupported | |
| GetObjectTagging | ✅ Supported | |
| GetObjectTorrent | ❌ Unsupported | |
| GetPublicAccessBlock | ❌ Unsupported | |
| HeadBucket | ✅ Supported | |
| HeadObject | ✅ Supported | |
| ListBucketAnalyticsConfigurations | ❌ Unsupported | |
| ListBucketIntelligentTieringConfigurations | ❌ Unsupported | |
| ListBucketInventoryConfigurations | ❌ Unsupported | |
| ListBucketMetricsConfigurations | ❌ Unsupported | |
| ListBuckets | ✅ Supported | |
| ListMultipartUploads | ❌ Unsupported | implement me! |
| ListObjects | ❌ Unsupported | implement me! |
| ListObjectsV2 | ✅ Supported | |
| ListObjectVersions | ❌ Unsupported | |
| ListParts | ✅ Supported | |
| PutBucketAccelerateConfiguration | ❌ Unsupported | |
| PutBucketAcl | ❌ Unsupported | |
| PutBucketAnalyticsConfiguration | ❌ Unsupported | |
| PutBucketCors | ❌ Unsupported | |
| PutBucketEncryption | ❌ Unsupported | |
| PutBucketIntelligentTieringConfiguration | ❌ Unsupported | |
| PutBucketInventoryConfiguration | ❌ Unsupported | |
| PutBucketLifecycle | ❌ Unsupported | Deprecated |
| PutBucketLifecycleConfiguration | ❌ Unsupported | |
| PutBucketLogging | ❌ Unsupported | |
| PutBucketMetricsConfiguration | ❌ Unsupported | |
| PutBucketNotification | ❌ Unsupported | No longer used |
| PutBucketNotificationConfiguration | ❌ Unsupported | |
| PutBucketOwnershipControls | ❌ Unsupported | |
| PutBucketPolicy | ❌ Unsupported | |
| PutBucketReplication | ❌ Unsupported | |
| PutBucketRequestPayment | ❌ Unsupported | |
| PutBucketTagging | ✅ Supported | |
| PutBucketVersioning | ❌ Unsupported | |
| PutBucketWebsite | ❌ Unsupported | |
| PutObject | ✅ Supported | |
| PutObjectAcl | ❌ Unsupported | |
| PutObjectLegalHold | ❌ Unsupported | |
| PutObjectLockConfiguration | ❌ Unsupported | |
| PutObjectRetention | ❌ Unsupported | |
| PutObjectTagging | ✅ Supported | |
| PutPublicAccessBlock | ❌ Unsupported | |
| RestoreObject | ❌ Unsupported | |
| SelectObjectContent | ❌ Unsupported | |
| UploadPart | ✅ Supported | |
| UploadPartCopy | ❌ Unsupported | |
| WriteGetObjectResponse | ❌ Unsupported |
SQS support is in in-progress.
Click to expand the detailed support table
| API | Support Status | Caveats/Notes |
|---|---|---|
| AddPermission | ❌ Unsupported | |
| CancelMessageMoveTask | ❌ Unsupported | |
| ChangeMessageVisibility | ✅ Supported | |
| ChangeMessageVisibilityBatch | ✅ Supported | |
| CreateQueue | ✅ Supported | |
| DeleteMessage | ✅ Supported | |
| DeleteMessageBatch | ✅ Supported | |
| DeleteQueue | ✅ Supported | |
| GetQueueAttributes | ✅ Supported | |
| GetQueueUrl | ✅ Supported | |
| ListDeadLetterSourceQueues | ❌ Unsupported | |
| ListMessageMoveTasks | ❌ Unsupported | |
| ListQueues | ✅ Supported | |
| ListQueueTags | ✅ Supported | |
| PurgeQueue | ❌ Unsupported | |
| ReceiveMessage | ✅ Supported | wait not supported |
| RemovePermission | ❌ Unsupported | |
| SendMessage | ✅ Supported | |
| SendMessageBatch | ❌ Unsupported | |
| SetQueueAttributes | ✅ Supported | not all attributes |
| StartMessageMoveTask | ❌ Unsupported | |
| TagQueue | ✅ Supported | |
| UntagQueue | ✅ Supported |