Skip to content

Schema Standards

Stuart Shelton edited this page Nov 4, 2016 · 2 revisions

Orientation

myway is an enhanced Flyway-compatible* set of tools which may be used to manage database schema migrations. Functionality offered by Flyway is further extended by the addition of support for enhanced metadata, provision of backup and recovery functionality, support for multiple schema types, and implementation of more comprehensive and reliable MySQL support (compared to Flyway 3.x).

The main known discrepancies are that the myway tools support only MySQL and Vertica at this point in time (whereas Flyway is largely database-agnostic), and that the checksum field in the Flyway-compatible schema_version table is always set to zero when a schema is deployed via myway: in this case, because the value Flyway uses is derived from Java's internal data-structures in a way which would be highly impractical to reproduce in any other language.

Database deployments and updates are performed from FlywayDB-compatible schema files with (broadly optional) additional metadata statements expressed in the form of SQL comments which can be placed alongside the applicable files (for dedicated Stored Procedure definitions) or as in-line comments at the top of schema files.

  • myway inherits Flyway's schema-naming conventions and uses the same in-database metadata table, but has a completely different command-line interface.

Schema Naming & Directory Layout Standards

The suggested layout for a myway-compatible directory hierarchy is as follows:

schema/
    <database_name>/
        <version>__<schema_file_description>.ddl.sql
        <version>__<schema_file_description>.dml.sql
        <version>__<schema_file_description>.dcl.sql
        <version>__<schema_file_description>.environment.dml.sql
        <version>__<schema_file_description>.not-environment.dml.sql
        ...

Stored Procedures may be stored in one of two ways. The suggested method - albeit one which doesn't work well when using git on Windows due to symlink-handling deficiencies - is to store versioned directories, as follows:

procedures/
    <database_name>-<version>[-r<revision>]/
        <database_name>.metadata
        <procedure_name>.sql
        ...

... or:

procedures/
    <database_name>/
        <version>[-r<revision>]/
            <database_name>.metadata
            <procedure_name>.sql
            ...

... the latter making the (optional) symlinks clearer by omitting the name of the database from the path.

This approach allows for a large number of identically-versioned Stored Procedure definitions to exist, whilst also allowing for data to be duplicated only when updated. For example:

procedures/
    mydatabase/
        v1.0/
            adduser.sql
            dropuser.sql
        v1.0-r1/
            adduser.sql
            dropuser.sql -> ../v1.0/dropuser.sql
        v1.1/
            adduser.sql -> ../v1.0-r1/adduser.sql
            dropuser.sql -> ../v1.0/dropuser.sql
            changeuser.sql
        v1.2/
            adduser.sql -> ../v1.0-r1/adduser.sql
            dropuser.sql -> ../v1.0/dropuser.sql
            changeuser.sql -> ../v1.1/changeuser.sql
            permuser.sql

... where, for each version or revision, a given Stored Procedure file is either a symlink to the last concrete text-file of the same name, or a new text-file which differs from the file of the same name in the immediately prior version directory. This has the significant benefit of providing an at-a-glance view of when a given file was last materially changed - from the v1.2 directory above, we can see that the dropuser.sql file has not been altered since its creation at v1.0, that permuser.sql is a new addition in v1.2, etc.

Alternatively, where symlink management is undesirable then the following 'squashed' layout can be used - this entails storing only the latest Stored Procedure versions and relying on a version-control system (such as git) to maintain non-current versions:

procedures/
    <database_name>/
         <database_name>.metadata
         <procedure_name>[-r<revision>].sql
         ...

... although this makes it significantly more difficult to visualise what changes have occurred over time, and moves away from the Flyway schema-migration concept of having a sequence of schema files which can be applied to migrate from any point to any later point - and is only possible because Stored Procedures do not themselves contain data which would be lost if they are dropped and re-created.

Schema Files

Schema files consist of SQL statements which directly affect the database, sub-divided into DDL changes (e.g. changes to the structure of the tables which make up the database), DML changes (e.g. changes only to the data within the tables of the database), and DCL changes (e.g. user-level privilege grants, etc.). In addition there are 'baseline' or 'initialiser' schema files (which may be used to create the initial database from scratch) and 'placeholder' or 'migration' schema files, which may be used to advanced the current metadata version to a a specified higher value.

A baseline schema file MAY take the form of unaltered mysqldump output, and can therefore contain both DML and DDL statements - although this is approach is now discouraged, with the restore option below providing a much better alternative. If creating a baseline schema-file from an existing database, then the mysqldump "--no-data"/"-d" option SHOULD be used to output only DDL without associated content. Maintaining this separation of DML and DDL allows for a fresh database schema to be created and migrated so as to be current without the requirement to simultaneously insert data, which may slow the process (when the quantity of data is large) and not always be necessary when performing migration on databases with pre-existing data.

Schema File Naming

Schema files MUST be named in a Flyway-compatible manner:

V<version>__<description>[.[not-]environment][.(dml|ddl)].sql

Except for the initial 'V' character, file names SHOULD consist solely of lower-case ASCII characters matching class '[a-z0-9-_]' and they MUST NOT contain spaces or double-underscores, other than the double-underscore delimiter following a version number.

Multiple version numbers (e.g. 'V<version>__V<version>__<description>.sql') are permitted only for placeholder/migration files with no SQL content - these files MUST contain version metadata in an initial comment, and MUST specify exactly two versions.

Metadata allows for schema-files to be targeted at a specific environment, and the name of this environment MAY be incorporated for cosmetic clarity, but is not further processed.

The '<version>' component SHOULD consist solely of digits from zero to nine, with full-stops as separators (although any string which results in the intended ordering when version-sorted and which doesn't contain the double-underscore sequence '__' is technically valid). By convention, existing schema use semantic version numbers starting at '0', then moving to '1.1' and incrementing the second digit thereafter (with '1.9' progressing to '1.10'), whilst fixes to an existing schema can break into further 'hotfix' digits, such as '1.8.1' (the first hotfix to the eighth step required to attain overall database schema-version one).

There are two approaches to naming - the suggested scheme is to omit trailing numerical components with value zero, whereas legacy naming includes all numerical components to result in broadly fixed-width display.

All legacy schema files SHOULD attempt to maintain an equal number of digits - however, placeholder files and base initialisers for which a hot-fix does not make sense MAY omit this value; thus the base initialiser file would be expected to be '0.0.0' or '0.0.0.0'. The former representation (with the hot-fix element removed) is the preferred form, as this ensures that metadata tables (which also do not specify a hot-fix value unless one is present) are consistent. For new schema-files, it is recommended that version-strings omit trailing sub-versions and hotfix-versions when these values would be zero.

The major-version of a schema file conventionally denotes the code-level required to interact with the contained change, or zero for code-agnostic schema updates. Once a non-zero code-level is specified, all following schema files MUST specify an equal or greater value.

The second component is a simple sequence-counter, and SHOULD increment sequentially from zero. The sequence does not need to be contiguous - gaps are allowed. However, placeholder files SHOULD be inserted if sequence-numbers which were previously present are removed due to reorganisation.

Where, due to DDL/DML splitting, a given database change must be separated into multiple files, the third component denotes this sub-sequence and increments from zero. This number MUST be present (as zero) if the change consists of a single file.

The fourth component, which is optional, is a hot-fix value which MAY be used if a revision to an existing file is required after it has already been applied to a (pre-)production branch, or similar. Such hot-fix files will be applied, in sequence, whenever they are present even if a higher major-version file has since been integrated. Placeholder files should specify a code-level value and a sequence-number value, but should omit any sub-sequence value or hot fix value from each specified version.

Once a given schema file has been released to a (pre-)production branch it MUST NOT then subsequently ever be changed, as the contents should be assumed to have been applied to a production database - and therefore any future schema files must take this into account. However, if required, additional hot-fix versions can be introduced to patch any issues in-place, if this cannot be achieved with a leaf schema file. Old schema-files MAY be removed or rolled-up into a single consolidated file (with an incremented, unique version number) - although in this case the resultant roll-up SHOULD follow the advice above in regards to splitting DDL and DML statements.

The 'description' field SHOULD be a short summary of the change the schema file introduces, and MAY be overridden by in-file metadata where a fuller description is warranted. Filename-based descriptions SHOULD NOT contain spaces, but space characters may be substituted for (ASCII) underscores or (ASCII) minus/hyphen signs as appropriate. It is suggested that underscores SHOULD be used universally unless context results in a hyphen being more readable.

A valid set of legacy schema file names would therefore be:

V<code version>.<change number>.<step>.<hotfix>__<description>[.<type>][.[not-]<environment>].sql
V22.39.0.0__Update_content.dml.sql
V22.40.0__V22.41.0__Placeholder.sql
V23.42.0.0__Update_my_table.ddl.sql
V23.42.1.0__Update_my_table_content.dml.sql
V23.42.1.1__my_table_content_hotfix.dml.sql
V23.43.0.0__Update_my_other_table.ddl.sql
V23.43.1.0__Update_my_other_table_content.dml.sql
V23.44.0.0__Update_my_other_table_content.production.dml.sql
V23.44.0.0__Update_my_other_table_content.not-production.dml.sql

... noting that, under this scheme, the major-version (the second component, which starts at 39 and counts upwards) is not reset when the code/compatibility indicator (the first component) is incremented.

The suggested approach (with trailing sub-versions and hot-fix versions removed) for the same sequence would be as follows:

V<code version>.<change number for version>[.<step>[.<hotfix>]]__<description>[.<type>][.[not-]<environment>].sql
V22.39__Update_content.dml.sql
V22.40__V22.41__Placeholder.sql
V23.1.1__Update_my_table.ddl.sql
V23.1.2__Update_my_table_content.dml.sql
V23.1.2.1__my_table_content_hotfix.dml.sql
V23.2.1__Update_my_other_table.ddl.sql
V23.2.2__Update_my_other_table_content.dml.sql
V23.3__Update_my_other_table_content.production.dml.sql
V23.3__Update_my_other_table_content.not-production.dml.sql

... where the required code-version required to successfully migrate beyond the 41st schema update 23, and the only change involved after the 39th schema update is to the metadata within the database without any associated DDL or DML statements. The first update which requires code version 23 (legacy schema change number 42) consists of both a DDL update followed by a DML update. Note that non-legacy changes count from 1 rather than 0, and that the final hot-fix digit is optional in all cases if its value would have been 0.

Please note that code-versions are for reference, and are not verified by the 'myway' package. For non-legacy schema files, the sequence (second) number SHOULD be reset to one(*) when the code-version (the first component) is changed. Maintaining a contiguous sequence for legacy schema makes migrating to non-legacy scheme much simpler.

(*) The sequence-number zero should only be used for initialisation, rather than for operations which modify an existing database.

Schema File Contents

Schema files MUST contain only comments, valid SQL statements, or a mixture of the two. Supported comments are C-style multi-line comments opening with '/*' and closing with '*/' or single-line comments starting with '--'. Shell-style single-line comments starting with '#' are parsed, but not fully supported and their use is deprecated. The first comment in a file, which SHOULD be a multi-line comment appearing at the very top of the file, has special meaning and is parsed to provide schema-file metadata. Metasyntactic variables should appear at the start of a new line within the initial multi-line comment - preceded only by whitespace and/or the '--' comment-symbol - and are interpreted as follows:

  • 'Description:' - The value following a whitespace separator is used to populate the Flyway schema description field, in place of the filename of the schema file (with underscores converted to spaces);
  • 'Engine:' - This optional field specifies the database engine which the back-end database must conform to in order for this schema file to be applied, and SHOULD be specified to ensure correct syntax and application;
  • 'Database:' - This optional field specifies the database which the schema file is intended to be applied to, and SHOULD be used when files are distributed singly, without a folder-structure to provide context;
  • 'Schema:' - This optional field specifies the Vertica schema within which the schema file MUST only be used when 'Engine' is also specified and set to 'vertica';
  • 'Previous version:' - The value following a whitespace separator is used to ensure that any prior prerequisite versions have already been applied. Values '0', '0.0', 'n/a', and 'NA' are all equivalent. If this declaration is not present then a warning is issued and the install-chain is not verified;
  • 'Target version:' - The value following a whitespace separator is entered into the database metadata to track the current change-number of the database, and is used for confirming that the prior install-chain is intact. If this declaration is not present then a warning is issued and the version from the file-name is used;
  • 'Environment:' - The value following a whitespace separator is used to control schema-file application: if 'myway' is invoked with the optional environment parameter, then schema-files specifying a non-matching environment in their metadata will not be applied, but the requested action will still either continue to later files or exit successfully - so long as one file named for the given change does include a matching Environment: directive. This allows for multiple files with otherwise identical names and metadata but differing environments to co-exist, and the appropriate file be applied without failures from non-matching environments halting the migration process. This logic may be inverted (e.g. apply schema-file to all environments except for the one specified) by inserting an exclamation-mark immediately before the environment name.
  • 'Restore:' - The specified argument MUST consist of a relative or absolute path to a 'mysqldump'-produced database backup, which will be required to be successfully restored before the contents of the schema-file (which may be otherwise blank) will be applied. As the restore is carried-out by piping the contents of the dump file directly into a MySQL client instance, no further processing is applied. This means that any backup to be restored MUST already contain valid metadata tables which are synchronised with the data in the backup. If this is not the case, then migrating through schema-files to reach the point where the backup was taken is the preferred solution to deploying fresh installations.

... following this comment-block, further comments have no further special meaning and are not passed to the database - except for database hints which are shown and also sent to the database for processing.

The following contraints apply to schema files:

  • Schema files MUST follow the UNIX convention of terminating a line with a single newline ("\n") character. Windows-style files which use "\r\n" may be converted with the 'dos2unix' command;
  • Each line MUST be blank, starting- ending- or within- a comment block, a valid single-line comment, or valid SQL.
  • Although multiple forms of comment will be correctly parse, the only valid SQL-compliant comment is a single-line comment starting with "--" or a C-style block of "/*" followed by "*/". MySQL alone also supports single-line comments starting with "#", but this is non-standard and will often fail to syntax-highlight in most editors. Standard SQL states that "{"/"}" braces can be used to contain comments, but MySQL does not support this. Double-hyphens and C-style blocks are the only permissible forms of comment delimiter, for consistency, and other styles MUST NOT be used. Comments will be shown in myway output ahead of the statement within which they appear, if verbose output is enabled;
  • Output SHOULD be formatted in a way which is readable on an eighty-column page, and comments MUST NOT extend beyond the eightieth column;
  • SQL statements MUST NOT include "USE <database>" statements, as these will not be processed. The one exception to this rule is that a base initialiser schema file MAY include a "USE <database>" line for clarity, but the statement will still not be processed;
  • SQL statements SHOULD NOT include 'LOCK' statements, as these will not be processed. All statements are executed by 'myway' occur with locks or within a transaction regardless. Note that locks provided in database hints are acted upon;
  • All identifiers MUST be quoted appropriately: \databases`and`tables`with back-ticks,'values' with single-quotes, etc;
  • All SQL keywords and non-user-defined function names MUST be expressed in ALL CAPS. Functions invocations MUST be followed by parens even if not taking parameters;
  • Unless referencing a database other than that to which the schema file is being applied, database references MUST be omitted. For example, specify "ALTER TABLE \jobs`" rather than "ALTER TABLE `database`.`jobs``". This ensures that the schema is portable to different databases (or can be split in order to apply to multiple databases) without large re-writes.
  • The database name is implicit from the schema path (if provided), and 'myway' will issue a warning if there appears to be any inconsistency;
  • Field types (such as "CHAR()", "VARCHAR()", "INT", etc.) MUST be expressed in ALL CAPS;
  • The field-type "INT" SHOULD be used in preference to "INTEGER", for consistency;
  • To avoid ambiguity, integer field-types SHOULD NOT include a length parameter unless also accompanied by the modifier "ZEROFILL", as the parameter is ignored by the database otherwise. This also applies to "TINYINT(1)" fields, for which the synonym "BOOLEAN" is a much more obvious alternative;
  • Database-defined constants which are not expressed consistently by the database itself (such as "CHARACTER SET" and "COLLATE" parameters) SHOULD be single-quoted and expressed in ALL CAPS. Similarly, numeric values should also be single-quoted in order to differentiate values and arguments from other forms of data;
  • The statement "ALTER TABLE MODIFY <attribute>" SHOULD be used in preference to "ALTER TABLE CHANGE <attribute> <attribute>" if the name '<attribute>' is not being updated, in order to avoid the potential confusion of having to specify the attribute-name twice;
  • If specifying an alternate name as an alias in an expression, the alias SHOULD be preceded by the "AS" keyword, where syntax allows, in order to avoid ambiguity. Aliases SHOULD be used where possible if a given table-name appears more than once in a given statement, and alias names SHOULD be single-quoted on definition and unquoted on use (unless specific quoting is necessary, in which case back-ticks can be used);
  • There SHOULD NOT be any whitespace characters immediately following an unquoted opening paren, or preceding a closing paren, unless the character following the opening paren or preceding the closing paren is another bracket - in which case spaces should be inserted to balance the opening bracket if necessary. Comma-separated lists SHOULD always include a space character after each comma. Parens following function-calls (such as "NOW()") SHOULD NOT be separated by whitespace, parens following keywords (such as "INSERT INTO ... VALUES (...)") SHOULD always be separated by whitespace. There SHOULD NOT be multiple adjacent spaces unless used to align columns with previous lines or within quoted strings. There SHOULD NOT be any whitespace characters before semi-colon line terminators (unless for column-formatting purposes), and there SHOULD NOT be any trailing whitespace at the end of any line;
  • Where statements form lengthy lines, they SHOULD be split into shorter lines and indented so as that distinct blocks form complete sub-sections. Leading commas are preferred, as it is much clearer when they are present or missing than when trailing at line-ends:
  INSERT INTO `projects` (
        `project_id`
      , `owner_id`
      , `description`
  )
       SELECT
             UUID()
          , `user_id`
          , 'Project'
       FROM
            `users`
  ;
  UPDATE `keys` AS 'k' JOIN `projects` AS 'p' ON  k.`user_id` = p.`owner_id`
      SET k.`project_id` = p.`project_id`
  ;
  • When the target database is Vertica, all quotation characters MUST be double-quotes, and schema prefixes or a correct 'SEARCH_PATH' MUST be provided;
  • Other than base-initialiser schema-files and placeholder schema-files, all schema-files lacking an 'Environment' metasyntactic variable or with a matching 'Environment' metasyntactic variable value MUST contain at least one valid SQL statement. Schema-files containing all commented-out code will be treated as failure cases;
  • For the protection of production environments, all "DROP TABLE" and "DROP DATABASE" statements will only be executed if 'myway' is invoked with the '--allow-unsafe' option. "DROP TEMPORARY TABLE" is allowed.

Example metadata headers

products/V00.0001.8.2__update_skus.dml.sql (a legacy schema file providing stage two of a hot fix to the 8th step of change '0001'):

/*-------------------------------------------------------------------------------
  -- Description:      Update SKUs
  -- Previous Version: 00.0001.8.1
  -- Target Version:   00.0001.8.2
  -- Environment:      production
  -------------------------------------------------------------------------------*/

accounts/V18.1__privilege_grants.dcl (a non-legacy schema file migrating from V17, which was a single-step change):

/*-------------------------------------------------------------------------------
  -- Description:      Apply user privileges
  -- Previous Version: 17.1
  -- Target Version:   18.1
  -- Engine:           vertica
  -------------------------------------------------------------------------------*/

admin/V00.000.0__V33.0000.0__restore_dump.sql (a legacy initialisation schema file which restores a file named admin.sql from the schema-file directory and checks that the database is at change V33 once complete):

/*-------------------------------------------------------------------------------
  -- Description:      Restore admin database dump
  -- Previous Version: 00.0000.0
  -- Target Version:   33.0000.0
  -- Restore:          admin.sql
  -------------------------------------------------------------------------------*/

Stored Procedures

  • Stored procedures must be collected into a parent directory named for the database against which the procedures are intended to be applied, and for the version/release number of the collection.
  • Each separate Procedure should be stored in a file consisting of the name of the Procedure being defined, followed by an optional revision number, followed by a '.sql' extension.
    • Each file SHOULD include a metadata section formatted as a comment before any other data, but only the 'Description:' field will be processed.
    • The specified '<name>' MUST match the name of the file.
    • Following the comment, files MUST consist of 'DROP PROCEDURE IF EXISTS <name>' immediately followed by the statements required to create the procedure.
    • There MUST NOT be any housekeeping code in these files - solely a DROP statement followed by the single statement needed to define the procedure.
    • The formatting of the Stored Procedures SHOULD match the examples above.
  • There MUST be a further metadata file named for the database the procedures will be loaded into. This mechanism provides the ability for stored procedures from one database to be loaded into another if required, whilst still maintaining an obvious link to the original parent database.
    • This metadata file MUST also commence with a metadata comment - but in this case, it is 'Previous version:' and 'Target version:' which are considered.
    • Whilst stored procedures should be entirely self-contained (as any procedure being defined is first dropped if it exists), the version chain established by the metadata is used to ensure that any clean-up between versions is carried out correctly. For example, a procedure which exists in version 1.1 but is dropped in version 1.2 should be removed in the version 1.2 directory's metadata. To this end, the metadata file alone MAY contain any housekeeping or cleanup routines needed between different releases. Equally, it is permissible for a given metadata file to contain no 'Previous version:' information is there has never been any prior cleanup required, allowing the procedures to be applied at an arbitrary point in time. Please note that, due to the non-interlocking requirement that Stored Procedures can be applied at any version and are always re-created from scratch, 'myway' does not support the back-up and restoration of Stored Procedures in isolation. Please note that this is NOT possible if the version number of a procedure is encoded within the procedure's name, as old versions will need to be removed at some point in the future. On the other hand, having versioned names allows for procedures with API-breaking changes to co-exist in the database with older, legacy-compatible versions. Due to this lock-step nature, it may be desirable for the version 1.3 metadata to contain cleanup for the version 1.1 procedures, allowing clean upgrades from version 1.2.
    • It is strongly RECOMMENDED that a complete copy of all stored procedures not be version-bumped for every change in a single constituent procedure - excessively high version numbers could quickly be reached. Version numbers should ideally only be updated when an API-breaking change is required. To support hot-fixes and non API-breaking changes, a revision should be appended to the file name, again dropping and re-defining the procedure in question. This will be applied after the original file, causing the latest revision to be the version present in the database as a result. The versions present can be determined by examining the Stored Procedure contents of the \myway_stored_procedures`` metadata table, which is created the first time Stored Procedures are loaded into the database.

Deploying schema files

Schema-files are automatically deployed, via myway.pl, by the applyschema.sh shell-script. This is turn is configured by schema.conf, which is searched for in /etc/dbtools/, /etc/, and $HOME, but can also be specified on the command-line. applyschema.sh has built-in help to document all options, but the key parameters are --dry-run which can be used to simulate a migration without actually updating the target database, and the --only <database> option which will only migrate the specified database rather than all known databases.

schema.conf follows an "ini"-style format, with section headings denoted by square-brackets, and then key/value pairs separated by whitespace-insignificant equals signs or keys alone. Variables may either appear under a "[DEFAULT]" section to indicate the settings which will be active unless overridden, or beneath individual database section headings in order to apply to that particular database only. The "[CLUSTERHOSTS]" section contains '<cluster-name> = <primary-host>' pairs which specify the host to connect to in order to provision to a given database host or cluster. The "[DATABASES]" section defines the names of the available databases (each of which must then have a dedicated section of their own, and finally there should be a section per database named identically to the database key specified under "[DATABASES]" which MUST contain a 'cluster = <cluster-name>' key/value pair, and MAY also specify any database-specific options:

[DEFAULT]
path = /srv/deployment data/database # Where can the top-level of the schema-files directory be found

environment = vagrant   # Optionally used for filtering

dbadmin = "user"
passwd = "password"

managed = true          # We expect to have schema files to load into listed databases

procedures = false      # Should we also look for stored-procedures?
#procedures.marker = '`<<VERSION>>' # ... and, if so, should we string-replace the marker with the version from the metadata?

options.force = no      # Continue in the face of errors - DANGEROUS!
options.verbose = no    # Provide a large quantity of very detailed output
options.warn = no       # Show additional debug-level output when failures occur
options.debug = no      # Produce copious developer-level debug output
options.quiet = yes     # Try to reduce the amount of output
options.silent = no     # Output only essential information or failures

mysql.compat = no       # Reduce functionality to operate on MySQL <5.6.4
mysql.relaxed = no      # Disable MySQL strict mode - DANGEROUS!

backups = no            # Don't take backups before making changes
#backups.compress = xz  # Compress backups with specified scheme
#backups.transactional  # Perform backup operations within a transaction
#backups.lock = no      # ... or lock tables during backup
#backups.keeplock = no  # ... or lock the entire database during backup - DANGEROUS!
#backups.separate = yes # Write each backed-up database to a separate file
#backups.skipmeta = yes # Don't include myway metadata in backup
#backups.extended = yes # Use extended insert to speed restores
#backups.keep = no      # Remove backups if migration process was successful

#parser.trustname = no  # Parse file in order to use metadata for versioning
#parser.allowdrop = no  # Disallow execution of DROP TABLE/DROP DATABASE statements

#version.max = 00.0000.0.0 # If specified, migration will fail if the end result does not match the stated version

[CLUSTERHOSTS]
core = database-write-master
vertica = vertica-write-master

[DATABASES]
accounts
admin
products

[products]
cluster = core
procedures = true

[admin]
cluster = core
#procedures = false

[accounts]
syntax = vertica
cluster = vertica
dbadmin = dbadmin
passwd = dbpassword
# DSN Database, ServerName, UserName, and Password will be populated from other variables unless overridden in the DSN string below...
# A fully-qualified DSN reads as follows:
#dsn = Driver=/opt/vertica/lib64/libverticaodbc.so;ServerName=vertica-write-master;Port=5433;Database=Vertica;UserName=dbadmin;Password=dbpasswd
# Alternatively, a DSN defined in /etc/odbc.ini can be referenced...
dsn = Driver=VerticaDSN
schema = myschema

Future Extensions

Triggers should sit alongside schema-files, whilst Functions should be treated as Stored Procedures. Add support and documentation for these two types of object.