The PDK Templates is the default templates repository for use with the Puppet Development Kit, within which we have defined all the templates for the creation and configuration of a module. Look into these directories to find the templates:
moduleroottemplates get deployed onnew module,convertandupdate; use them to enforce a common boilerplate for central files.moduleroot_inittemplates get only deployed when the target module does not yet exist; use them to provide skeletons for files the developer needs to modify heavily.object_templatestemplates are used by the variousnew ...commands for classes, defined types, etc.
PDK builds the configuration of the module by reading a set of default configuration from config_defaults.yml and merging it with the contents of .sync.yml from module's root directory, if exists. Top-level keys of the resulting hash correspond to target files or represent a global configuration. Global configuration will be merged with the configuration hash for a particular target file. This allows a module developer to override/amend the configuration by putting new values into .sync.yml. A knockout prefix may be applied to elements within .sync.yml to remove elements from the default set. Target files are created by rendering a corresponding template which is referring its configuration via the @configs hash. The data for a target file may include delete: true or unmanaged: true in order to have the particular file removed from the module or left unmanaged, respectively.
Templates like this one can be used in conjunction with the PDK. As default the PDK itself uses the templates within this repository to render files for use within a module. Templates can be passed to the PDK as a flag for several of the commands.
pdk convert --template-url https://github.com/puppetlabs/pdk-templatesPlease note that the template only needs to be passed in once if you wish to change it, every command run on the PDK will use the last specified template. For more on basic usage and more detailed description of the PDK in action please refer to the PDK documentation.
The following is a description and explanation of each of the keys within config_defaults. This will help clarify the default settings we choose to apply to pdk modules.
A namespace for settings that affect multiple templates
| Key | Description |
|---|---|
| honeycomb | While enabled by default, honeycomb's use can be turned off: enabled: false |
EditorConfig helps maintain consistent coding styles for multiple developers working on the same project across various editors and IDEs.
Please see the EditorConfig site for details on how to install it as a plugin (or if your text editor or IDE includes support natively) and how to configure it.
To enable adding a .editorconfig:
.editorconfig:
unmanaged: falseA .gitattributes file in your repo allows you to ensure consistent git settings.
| Key | Description |
|---|---|
| include | Defines which extensions are handled by git automatic conversions (see the gitattributes documentation). The default configuration helps to keep line endings consistent between windows and linux users. |
A .gitignore file in your repo allows you to specify intentionally untracked files to ignore.
| Key | Description |
|---|---|
| required | The default list of files or paths for git to ignore or untrack that are commonly specified in a module project. |
| paths | Defines any additional files or paths for git to ignore or untrack. (see the gitignore documentation). |
Gitlab CI is a continuous integration platform that is free for all open source projects hosted on Github and Gitlab.com, it also has a self-hosted option that is free as well. We can trigger automated pipelines with ever change to our code base in the main branch, other branches, tags, or additional triggers. Gitlab CI uses a .gitlab-ci.yml file in the root of your repository tell Gitlab CI what jobs to run when in the pipeline.
| Key | Description |
|---|---|
| override | Defines whether your local .sync.yml will ignore the default values in pdk-templates. Defaults to false |
| defaults/custom | The defaults and custom keys are special keys used to denote when configuration is coming from config_defaults.yml or .sync.yml. While it is possible for users to extend the defaults provided by PDK, it's suggested that the user should only use the custom key to separate their overrides/extended configuration from the PDK provided defaults. |
| custom_stages | Defines a custom job stage for when the CI/CD jobs will be executed in the pipeline. By default syntax and unit are defined unless override: true. |
| beaker | Defines if you want the default, Docker-in-Docker acceptance job added. Can be set to true to enable the default acceptance job, or you can specify the variables and tags subkeys. These subkeys function the same as the global_variables option and the tags subkey found in the ruby_versions option. |
| global_variables | Allows you to set any global environment variables for the gitlab-ci pipeline. Currently includes setting the Puppet gem version. |
| cache | If this setting exists, it expects a single sub-key called paths. paths is an array of paths that will be cached for each subsequent job. Defaults to ['vendor/bundle'] |
| tags | If this setting exists, it expects an array of tags that will be added to each job. Not set by default. |
| bundler_args | Define any arguments you want to pass through to bundler. The default is --without system_tests --path vendor/bundle --jobs $(nproc) which avoids installing unnecessary gems while installing them to the `vendor/bundler. |
| ruby_versions | Define a list of ruby_versions to test against. Each version can have a series of sub-keys that are options. checks is the rake command(s) to run during the job. puppet_version sets the PUPPET_GEM_VERSION environment variable. allow_failure is an array of checks where you want to allow failures. tags is an array of Gitlab CI Runner tags. code_quality is a boolean that when enabled adds Code Quality reports to jobs that run puppet-lint (version 3.2.0 onwards). |
| ruby_versions\{job}\except/only | Basic except/only is an hash of checks with array of references of conditions for the checks:ruby_versions: Advanced except/only is an hash of checks with hash using 4 keywords 'variables', 'refs', 'changes', 'kubernetes' each with it's own array of references or conditions for the checks:ruby_versions:https://docs.gitlab.com/ce/ci/yaml/README.html#onlyexcept-advanced |
| custom_jobs | Define custom Gitlab CI jobs that will be executed. It is recommended that you use this option if you need customized Gitlab CI jobs. Please see the .gitlab-ci.yml docs for specifics. |
| rubygems_mirror | Use a custom rubygems mirror url |
| image | Define the Docker image to use, when using the Docker runner. Please see the Using Docker images docs for specifics. |
| custom_before_steps | Allows you to pass additional steps to the GitLab CI before_script. Please see the .gitlab-ci.yml docs for specifics. If given as an array, the steps are executed before the default steps of the before_script. If you want to have more control over the execution order of the steps, you can define a hash with a before and/or after key holding an array of steps to be executed before and after the default_script respectively. |
| default_before_script | If false, removes the default before_script section. Useful if you need a customised Bundler install, or to remove Bundler entirely. If the key is unset the default behaviour is to add before_script. |
| use_litmus | By default it is disabled. Set to true to configure travis to use Litmus testing tool for acceptance testing jobs with default values. |
| litmus | Allows you to update default config values. Its sub keys are provision_list, puppet_collection, ruby_version, install_wget which are detailed below. |
| litmus\puppet_collection | Allows you to specify the puppet version under test. Default test are ran on puppet6. |
| litmus\provision_list | Allows you to specify the platforms list under test. Default test are ran on platformes defined in provision.yaml file under travis_deb and travis_el |
| litmus\ruby_version | Allows you to specify the ruby version under test. Default it is set to 2.5.7 |
| litmus\install_wget | Allows you to enable automatic installation of wget on the platform under test. We need this when installing agent on travis_deb platforms. Default it is disabled. |
| litmus\complex\collection | Allows you to specify multiple collections of puppet_collection and provision_list, allowing you to set certain OS to only run on certain Puppet versions. |
If you are using Gitpod you will need to opt-in and enable gitpod support for pdk-templates. To do this simple set the following configurations.
.gitpod.Dockerfile:
unmanaged: false
.gitpod.yml:
unmanaged: falseThese workflows are depending on puppet-internal resources and are currently not suited for public consumption. Feel free to take them as inspiration how to run some tests on Github Actions. Please let us know at [email protected] what you come up with!
The auto release workflows uses the PDK experimental command pdk release prep to prepare a module release PR. By default the workflow can be triggered manually when a release preparation PR needs to be created, however it allows setting a cron based trigger that can run automatically.
To set up the automated release cron you can add a configuration to your .sync.yml file that matches the following example:
release_schedule:
cron: '0 3 * * 6'In this example the automated release prep workflow is triggered every Saturday at 3 am.
A .pdkignore file in your repo allows you to specify files to ignore when building a module package with
pdk build.
| Key | Description |
|---|---|
| required | The default list of files or paths for PDK to ignore when building a module package. |
| paths | Defines additional files or paths for PDK to ignore when building a module package. |
Travis CI is a hosted continuous integration platform that is free for all open source projects hosted on Github. We can trigger automated builds with every change to our code base in the main branch, other branches or even a pull request. Travis uses a .travis.yml file in the root of your repository to learn about your project and how you want your builds to be executed. | Key | Description | | :------------- |:--------------| | os | Set to an array of operating systems to test. See the TravisCI documentation for more details. | | dist | If specified, it will set the dist attribute. See the TravisCI documentation for more details. | | simplecov |Set to
trueto enable collecting ruby code coverage.| | ruby_versions |Define the ruby versions on which you want your builds to be executed.| | bundler_args |Define any arguments you want to pass through to bundler. The default is--without system_testswhich avoids installing unnecessary gems.| | env |Allows you to add new travis job matrix entries based on the included environment variables, one perenventry; for example, for adding jobs with specificPUPPET_GEM_VERSIONand/orCHECKvalues. See the Travis Environment Variables documentation for details.| | global_env |Allows you to set global environment variables which will be defined for all travis jobs; for example,PARALLEL_TEST_PROCESSORSorTIMEOUT. See the Travis Global Environment Variables documentation for details.| |docker_sets |Allows you to configure sets of docker to run your tests on. For example, if I wanted to run on a docker instance of Ubuntu I would addset:docker/ubuntu-14.04to my docker_sets attribute. The docker_sets is a hash that supports the 'set', 'testmode', and 'collection' keys. | |docker_sets['set']| This should reference the docker nodeset that you wish to run. | |docker_sets['testmode']| This configures theBEAKER_TESTMODEto use when testing the docker instance. The two options areapplyandagentif omittedapplyis used by default. | |docker_sets['collection]| This configures theBEAKER_PUPPET_COLLECTIONto use when testing the docker instance. The default ispuppet6. |docker_defaults |Defines what values are used as default when using thedocker_setsdefinition. Includes ruby version, sudo being enabled, the distro, the services, the env variables and the script to execute.| |stages |Allows the specification of order and conditions for travis-ci build stages. See Specifying Stage Order and Conditions.| |before_install_pre |Add install steps to the start ofbefore_install. | |before_install_post |Add install steps to the end ofbefore_install. | |includes |Ensures that the .travis file includes the following checks by default: Rubocop, Puppet Lint, Metadata Lint.| |remove_includes |Allows you to remove includes set inconfig_defaults.yml.| |branches |Allows you to specify the only branches that travis will run builds on. The default branches aremainand/^v\d/. | |branches_except |Allows you to specify branches that travis will not build on.| |remove_branches |Allows you to remove default branches set in config_defaults.yml.| |notifications |Allows you to specify the notifications configuration in the .travis.yml file.| |remove_notifications |Allows you to remove default branches set in config_defaults.yml.| |deploy_to_forge|Allows you to change the automatic deployment of modules to the forge. Sub keys areenabledandtag_regexwhich are detailed below| |deploy_to_forge\enabled|Allows you to enable or disable automatic forge deployments. Default is true| |deploy_to_forge\tag_regex|Allows you to use a regular expression to define which tags will trigger a deployment. The default is^v\d| |before_deploy|An array which can allow a user to specify the commands to run before kicking off a deployment. See [https://docs.travis-ci.com/user/deployment/releases/#setting-the-tag-at-deployment-time].| |use_litmus| By default it is disabled. Set totrueto configure travis to use Litmus testing tool for acceptance testing jobs with default values.| |litmus|Allows you to update default config values. Its sub keys areprovision_list,puppet_collection,rvm,install_wgetwhich are detailed below.| |litmus\puppet_collection|Allows you to specify the puppet version under test. Default test are ran on puppet 5 and puppet 6.| |litmus\provision_list|Allows you to specify the platforms list under test. Default test are ran on platformes defined in provision.yaml file under travis_deb and travis_el| |litmus\rvm|Allows you to specify the ruby version under test. Default it is set to 2.5.7| |litmus\install_wget|Allows you to enable automatic installation of wget on the platform under test. We need this when installing agent on travis_deb platforms. Default it is disabled. | |litmus\complex\collection|Allows you to specify multiple collections ofpuppet_collectionandprovision_list, allowing you to set certain OS to only run on certain Puppet versions. | |user|This string needs to be set to the Puppet Forge user name. To enable deployment the secure key also needs to be set.| |secure|This string needs to be set to the encrypted password to enable deployment. See https://docs.travis-ci.com/user/encryption-keys/#usage for instructions on how to encrypt your password.|
YARD is a documentation generation tool for the Ruby programming language. It enables the user to generate consistent, usable documentation that can be exported to a number of formats very easily, and also supports extending for custom Ruby constructs such as custom class level definitions.
| Key | Description |
|---|---|
| markup | Specifies the markup formatting of your documentation. Default is markdown. |
| optional | Define any additional arguments you want to pass through to the yardoc command. |
AppVeyor is a hosted, distributed continuous integration service used to build and test projects hosted on GitHub by spinning up a Microsoft Windows virtual machine. AppVeyor is configured by adding a file named appveyor.yml, which is a YAML format text file, to the root directory of the code repository.
| Key | Description |
|---|---|
| appveyor_bundle_install | Defines the bundle install command for the appveyor execution run. In our case we use bundle install --without system_tests as default, therefore avoiding redundant gem installation. |
| install_pre | Add install steps to the start of install. |
| install_post | Add install steps to the end of install. |
| environment | Defines any environment variables wanted for the job run. In our case we default to the latest Puppet 4 gem version. |
| matrix | This defines the matrix of jobs to be executed at runtime. Each defines environment variables for that specific job run. In our defaults we have a Ruby version specfied, followed by the check that will be run for that job. |
| simplecov | Set to true to enable collecting ruby code coverage. |
| test_script | This defines the test script that will be executed. For our purposes the default is set to bundle exec rake %CHECK%. As appveyor iterates through the test matrix as we defined above, it resolves the variable CHECK and runs the resulting command. For example, our last test script would be executed as bundle exec rake spec, which would run the spec tests of the module. |
| use_litmus | Configures Appveyor to be able to use Litmus for acceptance testing jobs |
| remove_includes | Allows you to remove includes set in config_defaults.yml. |
Rake is a Make-like program implemented in Ruby. Tasks and dependencies are specified in standard Ruby syntax within the Rakefile, present in the root directory of the code repository. Within modules context Rake tasks are used quite frequently, from ensuring the integrity of a module, running validation and tests, to tasks for releasing modules.
| Key | Description |
|---|---|
| requires | A list of hashes with the library to 'require', and an optional 'conditional'. |
| changelog_user | Sets the github user for the change_log_generator rake task. Optional, if not set it will read the author from the metadata.json file. |
| changelog_project | Sets the github project name for the change_log_generator rake task. Optional, if not set it will parse the source from the metadata.json file |
| changelog_since_tag | Sets the github since_tag for the change_log_generator rake task. Required for the changelog rake task. |
| changelog_max_issues | Sets the github max_issues for the change_log_generator rake task. Optional to limit max issues. |
| changelog_version_tag_pattern | Template how the version tag is to be generated. Defaults to 'v%s' which eventually align with tag_pattern property of puppet-blacksmith, thus changelog is referring to the correct version tags and compare URLs. |
| github_site | Override built-in default for public GitHub. Useful for GitHub Enterprise and other. (Example: github_site = https://git.domain.tld) |
| github_endpoint | Override built-in default for public GitHub. Useful for GitHub Enterprise and other. (Example: github_endpoint = https://git.domain.tld/api/v4) |
| gitlab | Allows to work with GitLab (Pending gitlab = true # Boolean default is false) |
| default_disabled_lint_checks | Defines any checks that are to be disabled by default when running lint checks. As default we disable the --relative lint check, which compares the module layout relative to the module root. Does affect .puppet-lint.rc. |
| extra_disabled_lint_checks | Defines any checks that are to be disabled as extras when running lint checks. No defaults are defined for this configuration. Does affect .puppet-lint.rc. |
| extras | An array of extra lines to add into your Rakefile. As an alternative you can add a directory named rakelib to your module and files in that directory that end in .rake would be loaded by the Rakefile. Example, to add syntax check exclusion paths for plan directories:
|
| linter_options | An array of options to be passed into linter config. Does affect .puppet-lint.rc. |
| linter_fail_on_warnings | A boolean indicating if the linter should exit non-zero on warnings as well as failures. Does affect .puppet-lint.rc. |
| linter_exclusions | An array of paths that the linter should exclude from running against. |
RuboCop is a Ruby static code analyzer. We use Rubocop to enforce a level of quality and consistency within Ruby code. Rubocop can be configured within .rubocop.yml which is located in the root directory of the code repository. Rubocop works by defining a sanitized list of cops that'll cleanup a code base without much effort, all of which support autocorrect and that are fairly uncontroversial across wide segments of the Community.
| Key | Description |
|---|---|
| include_todos | Allows you to use rubocop's "TODOs" to temporarily skip checks by setting this to true. See rubocop's --auto-gen-config option for details. Defaults to false. |
| selected_profile | Allows you to define which profile is used by default, which is set to strict, which is fully defined within the profiles section. |
| default_configs | Allows you to define the default configuration of which cops will run. Includes the full name of the cop followed by a description of it and an enforced style. Can also make use of the key excludes to exclude any files from that specific cop. |
| cleanup_cops | Defines a set of cleanup cops to then be included within a rubocop profile. Cops are defined by their full name, and further configuration can be done by specifying secondary keys. By default we specify a large amount of cleanup cops using their default configuration. |
| profiles | Defines the profiles that can be enabled and used within rubocop through the selected_profile option. By default we have set up three profiles: cleanups_only, strict, hardcore and off. |
A Gemfile is a file we create which is used for describing gem dependencies for Ruby programs. All modules should have an associated Gemfile for installing the relevant gems. As development and testing is somewhat consistant between modules we have used the template to define a set of gems relevant to these processes.
| Key | Description |
|---|---|
| required | Allows you to specify gems that are required within the Gemfile. Gems can be defined here within groups, for example we use the :development gem group to add in several gems that are relevant to the development of any module and the :system_tests gem group for gems relevant only to acceptance testing. |
| optional | Allows you to specify additional gems that are required within the Gemfile. This key can be used to further configure the Gemfile through assignment of a value in the .sync.yml file. |
Within each Gem group defined using the options above one or more gem item definitions may be listed in an array. Each item in that array must be a gem item hash.
| Gem Item Hash Keys | Description |
|---|---|
| gem | Required option specifying the gem name. |
| version | Required option to specify version or range of versions required using RubyGem version syntax. |
| platforms | Defines an array of platforms for which the Gem should be included. See the Gemfile platform guide for a list of valid platforms. |
| git | If required, specify a specific Git repository in which this Gem is located. See the Bundler docs for details. |
| branch | Optionally specify a branch to use if using the git option. Defaults to main. |
| ref | Optionally specify an arbitrary valid Git reference to use for the module version. |
| source | Specify an alternate Rubygems repository to load the gem from. |
| from_env | Specifies an environment variable containing either a Rubygem version specification indicating the version to use OR a URL indicating the location from which to load the gem. |
| condition | An optional string containing a Ruby-code conditional controlling if this gem will be processed in the Gemfile. |
The spec/default_facts.yml file contains a list of facts to be used by default when running rspec tests
| Key | Description |
|---|---|
| concat_basedir | Overrides the concat_basedir fact's value in the base template. Defaults to "/tmp". |
| ipaddress | Overrides the ipaddress fact's value in the base template. Defaults to "172.16.254.254". |
| is_pe | Overrides the is_pe fact's value in the base template. Defaults to false. |
| macaddress | Overrides the macaddress fact's value in the base template. Defaults to "AA:AA:AA:AA:AA:AA". |
| extra_facts | List of extra facts to be added to the default_facts.yml file. They are in the form: "name of fact: value of fact" |
The spec/spec_helper.rb file contains setup for rspec tests
| Key | Description |
|---|---|
| default_facter_version | Sets the default_facter_version rspec-puppet-facts parameter. |
| facter_implementation | Sets the facter_implementation rspec-puppet parameter. Current available options are rspec and facter, with facter being the default. The rspec implementation offers a significant performance improvement and results in faster unit tests. |
| hiera_config | Sets the hiera_config rspec-puppet parameter. |
| hiera_config_ruby | Sets the hiera_config rspec-puppet parameter. A ruby expression returning the path to your hiera.yaml file. hiera_config takes precedence if both values hiera_config and hiera_config_ruby are specified. |
| mock_with | Defaults to ':rspec'. If you need the previous default of "some mocha and some rspec-mocks", set mock_with: ~ (tilde symbol, the YAML null value). |
| spec_overrides | An array of extra lines to add into your spec_helper.rb. Can be used as an alternative to spec_helper_local |
| strict_level | Defines the Puppet Strict configuration parameter. Defaults to :warning. Other values are: :error and :off. :error provides strictest level checking and is encouraged. |
| strict_variables | Defines the Puppet Strict Variables configuration parameter. Defaults to true however due to puppetlabs_spec_helper forced override (https://github.com/puppetlabs/puppetlabs_spec_helper/blob/070ecb79a63cb8fa10f46532c413c055e2697682/lib/puppetlabs_spec_helper/module_spec_helper.rb#L71). Set to false to align with true default or with STRICT_VARIABLES=no environment setting. |
| coverage_report | Enable rspec-puppet coverage reports. Defaults to false |
| minimum_code_coverage_percentage | The desired code coverage percentage required for tests to pass. Defaults to 0 |
While we provide a basic template it is likely that it will not match what you need exactly, as such we allow it to be altered or added to through the use of the
.sync.ymlfile.
Values can be added to the data passed to the templates by adding them to your local .sync.yml file, thus allowing you to make changes such as testing against additional operating systems or adding new rubocop rules.
To add a value to an array simply place it into the .sync.yml file as shown below, here I am adding an additional unit test run against Puppet 4:
.travis.yml:
includes:
- env: PUPPET_GEM_VERSION="~> 4.0" CHECK=parallel_spec
rvm: 2.1.9Values can be removed from the data passed to the templates using the knockout prefix --- in .sync.yml.
To remove a value from an array, prefix the value ---. For example, to remove
2.5.1 from the ruby_versions array in .travis.yml:
.travis.yml:
ruby_versions:
- '---2.5.1'To remove a key from a hash, set the value to ---. For example, to remove the
ipaddress fact from spec/default_facts.yml:
spec/default_facts.yml:
ipaddress: '---'To add a custom internal puppet-lint plugin served from an internal Rubygems source, add
an entry similar to the following in .sync.yml file and run pdk update.
Gemfile:
optional:
':development':
- gem: 'puppet-lint-my_awesome_custom_module'
version: '>= 2.0'
source: 'https://myrubygems.example.com/'Please note that the early version of this template contained only a 'moduleroot' directory, and did not have a 'moduleroot_init'. The PDK 'pdk new module' command will still work with templates that only have 'moduleroot', however the 'pdk convert' command will fail if the template does not have a 'moduleroot_init' directory present. To remedy this please use the up to date version of the template.
As explained in Use GitHub actions at your own risk, when running github actions from outside the organisation, there is a risk that symbolic references get taken over by malicious actors. Similar things happened before in other ecosystems and other packaging registries. The blog post goes on to suggest pinning to specific SHAs and provides some tooling to do so. The downsides for us are that the tooling doesn't work well with our ERB templating, and the additional cost of updating the SHAs across all modules. Instead we fork at-risk actions into the puppetlabs namespace and use them from there. This allows us to consume updates at our pace and deploy changes across all modules without delay, while avoiding actions that surreptitiously change while we're not looking.
Since this still has some overhead, we exclude some "big-name" action maintainers:
- Anything maintained by Github, e.g. https://github.com/actions
- Anything maintained as part of a bigger OSS project we're using, like https://github.com/ruby/setup-ruby
- Anything maintained by a Puppet employee
To keep efforts low when updating actions, we list all forked actions here.
To keep confusion to a minimum, the version we use is always on a pdk-templates-v1 branch.
This way we can update (git fetch/git push) forked repositories with no prejudice, test out the changes, and only then update the pdk-templates-v1 branch.
That said, the branches used in pdk-templates should only contain upstream code and changes already in an upstream PR to minimize the diff we're carrying.
If we later need to support multiple versions of an action as we roll out changes, we can increment the -v1 part in the branch name to manage multiple versions.
- kvrhdn/gha-buildevents ➡️ puppetlabs/kvrhdn-gha-buildevents
- Gamesight/slack-workflow-status ➡️ puppetlabs/Gamesight-slack-workflow-status
The repos have restricted access only to @modules team members.