Assemble and merge multiple YAML sources into a single document.
The goal is to weakly couple YAML documents together for final assembly. For example, a CloudFormation template for a three-tier architecture might contain definitions for individual containers:
Resources:
ECSTaskDefinition:
Type: AWS::ECS::TaskDefinition
Properties:
ContainerDefinitions:
- # Frontend container
Image: !Ref ReactImage
PortMappings:
- ContainerPort: 8080
HostPort: 80
Protocol: tcp
- # Backend container
Image: !Ref FlaskImage
PortMappings:
- ContainerPort: 8080
HostPort: 1080
Protocol: tcp
- # MongoDB container
Image: !Ref MongoDBImage
MountPoints:
- ContainerPath: /opt/mongodb
SourceVolume: mongodb
This works ok if you're keeping the infrastructure, frontend, backend, and database bits in the same repository. Or you could break everything into separate CloudFormation stacks and hope you get all of the cross-dependencies right.
Neither approach felt right for a simple website being developed by a small but plural number of developers. I wanted to keep the corresponding frontend, backend, and database source for code and infrastructure together and assemble them into a master template (with a few other support pieces in an infrastructure repository).
The above would become four separate documents maintained in separate repositories:
Infrastructure::cfn.yml | React::cfn.yml |
---|---|
Resources:
ECSTaskDefinition:
Type: AWS::ECS::TaskDefinition
Properties:
ContainerDefinitions:
!Transclude ContainerDefinitions
|
!Assembly ContainerDefinitions:
- # Frontend container
Image: !Ref ReactImage
PortMappings:
- ContainerPort: 8080
HostPort: 80
Protocol: tcp |
Flask::cfn.yml | MongoDB::cfn.yml |
!Assembly ContainerDefinitions:
- # Backend container
Image: !Ref FlaskImage
PortMappings:
- ContainerPort: 8080
HostPort: 1080
Protocol: tcp |
!Assembly ContainerDefinitions:
- # MongoDB container
Image: !Ref MongoDBImage
MountPoints:
- ContainerPath: /opt/mongodb
SourceVolume: mongodb |
Assemyaml provides two local tags, !Transclude
and !Assembly
.
The !Transclude
tag specifies a transclusion point -- where another document may specify
one or more YAML collections (sequence or mapping). It takes a string name used as a label
for the transclusion point. The transclusion is a mapping key; if the value is not null,
it is used as an assembly for that transclusion point.
The mapping containing the !Transclude
tag must not contain any other elements.
The !Assembly
tag specifies an assembly -- one or more YAML collections to be injected
into a corresponding transclusion point. It takes a string specifying the transclusion label.
If multiple documents provide the same assembly, the collection must be the same type;
you cannot mix sequences and mappings. If the assemblies are mappings, they must
have unique keys.
One document is designated the template. This document is written to the output, with all
!Transclude
mappings replaced by the assembled values. The other documents are called resources.
Template document | Resource 1 | Resource 2 | Result |
---|---|---|---|
Hello:
!Transclude values:
- Alpha
- Bravo |
!Assembly values:
- Charlie
- Delta |
!Assembly values:
- Echo
- Foxtrot |
Hello:
- Alpha
- Bravo
- Charlie
- Delta
- Echo
- Foxtrot |
Template document | Resource 1 | Resource 2 | Result |
---|---|---|---|
Hello:
!Transclude values:
Alpha: 1
Bravo: 2 |
!Assembly values:
Charlie: 3
Delta: 4 |
!Assembly values:
Echo: 5
Foxtrot: 6 |
Hello:
Alpha: 1
Bravo: 2
Charlie: 3
Delta: 4
Echo: 5
Foxtrot: 6 |
If you're using local tags !Transclude
or !Assembly
for another purpose (or if local tags
offend you), you may tell Assemyaml to use global tags instead:
Template document | Resource document |
---|---|
%TAG !assemyaml! tag:assemyaml.nz,2017:
---
Hello:
!assemyaml!Transclude values:
- Alpha |
%TAG !assemyaml! tag:assemyaml.nz,2017:
---
!assemyaml!Assembly values:
- Bravo |
assemyaml [options] template-document resource-documents...
assemyaml [options] --template template-document resource-documents...
Options:
--format json|yaml
- Write output in this format. (Only YAML is supported on input.)--no-local-tag
- Ignore!Transclude
and!Assembly
local tags and use global tags only.--output filename
- Write output to filename instead of stdout.
First, create a Lambda function from the Assemyaml ZIP file. Here are three ways of getting the ZIP file:
- Upload a named-version of Assemyaml from S3: https://s3.amazonaws.com/assemyaml.nz/assemyaml-lambda-0.2.1.zip for example.
- Upload the latest version of Assemyaml from S3: https://s3.amazonaws.com/assemyaml.nz/assemyaml-lambda-latest.zip
- Build your own version locally (you'll need Docker):
-
git clone https://github.com/dacut/assemyaml.git
-
cd assemyaml
-
./lambda-build
-
- You'll have a ZIP file named
lambda.zip
in the current directory.
- You'll have a ZIP file named
When used as a Lambda invocation stage in CodePipeline, UserParameters is a JSON object with the following syntax:
{ "TemplateDocument": "input-artifact::filename", "ResourceDocuments": ["input-artifact::filename", ...], "DefaultInputFilename": "filename", "OutputFilename": "filename", "LocalTag": true|false, "Format": "yaml|json" }
All parameters are optional.
TemplateDocument
specifies the input artifact and the filename within the artifact to use as the template document.
ResourceDocuments
specifies the input artifacts and filename within each artifact to use as resource documents. Any input artifacts not referenced in either TemplateDocuments
or ResourceDocuments
are appended to ResourceDocuments
as artifact::DefaultInputFilename
.
The DefaultInputFilename
key is used for an input artifact filename if an input artifact is not referenced in either TemplateDocument
or ResourceDocuments
. It defaults to assemble.yml
.
OutputFilename
specifies the filename to write in the output artifact. It defaults to assemble.yml
.
LocalTag
specifies whether the !Transclude
and !Assembly
local tags are allowed. It defaults to true.
Format
specifies the output template format. It defaults to yaml
.
If TemplateDocument
or ResourceDocument
is not specified, the following behavior applies:
Options specified | Input artifacts: `[A, B, C]` |
---|---|
{
"TemplateDocument": "B::f2",
"ResourceDocuments": [ "A::f1", "C::f3" ]
} | {
"TemplateDocument": "B::f2",
"ResourceDocuments": [ "A::f1", "C::f3" ]
} |
{
"TemplateDocument": "B::f2"
} | {
"TemplateDocument": "B::f2",
"ResourceDocuments": [ "A::assemble.yml", "C::assemble.yml" ]
} |
{
"ResourceDocuments": [ "A::f1", "C::f3" ]
} | {
"TemplateDocument": "B::assemble.yml",
"ResourceDocuments": [ "A::f1", "C::f3" ]
} |
{
"ResourceDocuments": [ "C::f3" ]
} | {
"TemplateDocument": "A::assemble.yml",
"ResourceDocuments": [ "C::f3", "B::assemble.yml" ]
} |
{
"TemplateDocument": "A::assemble.yml",
"ResourceDocuments": [ "B::assemble.yml", "C::assemble.yml" ]
} |