2.0.0-alpha1

This is an early alpha release of Tokaido 2.0, which includes a whole stack of changes designed to bring Tokaido up to date and make it a bit easier for us to maintain in the future.

🚨 This is an alpha release and contains bugs. 🚨

Using this release

If you want to test out this release, please follow these instructions:

1. Download the tok-macos binary and save it on your system:
   curl -sSLo /usr/local/bin/tok-alpha https://github.com/ironstar-io/tokaido/releases/download/v2.0.0-alpha1/tok-macos
2. Make it executable:
   chmod +x /usr/local/bin/tok-alpha
3. Run it
   tok-alpha up

If you're an existing Tokaido 1 user stuck on Docker versions before 4.3.0, you need to upgrade your Docker before you can use Tokaido 2.0. This is because of a change Docker made to how container names are formatted in docker-compose.

Rolling back to Tokaido 1

If you encounter any major issues, you can revert to Tokaido 1 by simple downgrading to Docker < 4.3.0 again and going back to using tok instead of tok-alpha

Giving Feedback

If you encounter any bugs, please raise a GitHub Issue being sure to include:

1. A copy of any debug output available by using the -d flag (for example, tok-alpha up -d)
2. Details about your system (especially the processor type you are using: Silicon or Intel)
3. Detailed steps we can use to try and reproduce the issue on our end.

Without this information, we may not be able to help you resolve your problem.

Changes

This release focuses on bring Tokaido up-to-date and also removing some functionality which has made it harder for us to maintain the project in the past. Our hope is with this simplified feature set, we'll be better able to serve the vast majority of our users who never made use of those features:

• Removed support for Linux and Windows
• Removed support for tok test
• Removed support for PHP 7.2 and 7.3
• Removed support for MySQL - all projects now use MariaDB 10.8
• Removed support for Docker < 4.3.0
• Removed support for NodeJS 12
• Removed support for the Unison filesystem agent
• Removed the syslog container - all logs now go to a shared /app/logs folder
• Added support for PHP 8.0 and 8.1 (note PHP 7.4 support will be removed in December after it goes end of life)
• Improved logging format to match those used by Ironstar
• Renamed the 'drush' container to 'ssh'
• Renamed the 'tok' user to 'app' (again to match what we do at Ironstar)
• Enabled Memcache by default for all new projects
• Moved to using a universal binary for macOS
• Set up code signing for the macOS binary

Things we know are broken in this alpha

• The first time you run tok-alpha up you'll need to download new images. This happens during the "Starting your Drupal environment" phase and can take a significantly long time with no feedback.
• In certain circumstances, tok-alpha up may report errors starting containers even if all containers started fine. You can verify the true status of your containers by running docker-compose -f docker-compose.tok.yml ps and check that all containers are "Runnings"
• Solr and xdebug are not currently working
• tok new is not up to date and may not be working for some configurations
• When using Tokaido, you'll release warnings that you aren't on the latest version. This is because the built-in version check operates on stable releases, not pre-releases. You can safely ignore this warning.

1.15.0

In 1.15.0, we remove the "haproxy" and "varnish" containers, as they added a fair bit of weight to the project without adding very much value. Haproxy was used for SSL termination, and Varnish was used for local caching. In 1.15, Tokaido will enable SSL in the Nginx container and you'll connect directly to that when you run tok open. All in all, this will mean less Tokaido images to downloader, slightly faster local development, and less moving parts where things can go wrong.

1.15 also migrates from the old /tokaido folder to a more generic /app folder, but /tokaido is a symlink back to /app so the change is backwards compatible. This change is part of much larger work we're doing to bring our Drupal and Laravel hosting into a single unified platform, although no plans for the Tokaido CLI itself to support Laravel exist at this time.

Finally, 1.15 removes nvm in favour of straight-up nodejs installs using the current LTS release. While we tried to ship nodejs using nvm previously, it was quite a hassle since nvm is designed to be installed per-user, but Tokaido supports multiple users in production. It is also much more work for us to maintain multiple version of NodeJS rather than just accepting a single version. As such, Tokaido 1.15 includes NodeJS 12.19.0 and no support for other NodeJS versions.

1.15.0-beta1 - Streamlined Tokaido contains and improved NodeJS support

In 1.15.0, we remove the "haproxy" and "varnish" containers, as they added a fair bit of weight to the project without adding very much value. Haproxy was used for SSL termination, and Varnish was used for local caching. In 1.15, Tokaido will enable SSL in the Nginx container and you'll connect directly to that when you run tok open. All in all, this will mean less Tokaido images to downloader, slightly faster local development, and less moving parts where things can go wrong.

1.15 also migrates from the old /tokaido folder to a more generic /app folder, but /tokaido is a symlink back to /app so the change is backwards compatible. This change is part of much larger work we're doing to bring our Drupal and Laravel hosting into a single unified platform, although no plans for the Tokaido CLI itself to support Laravel exist at this time.

Finally, 1.15 removes nvm in favour of straight-up nodejs installs using the current LTS release. While we tried to ship nodejs using nvm previously, it was quite a hassle since nvm is designed to be installed per-user, but Tokaido supports multiple users in production. It is also much more work for us to maintain multiple version of NodeJS rather than just accepting a single version. As such, Tokaido 1.15 includes NodeJS 12.19.0 and no support for other NodeJS versions.

PHP and Linux upgrades

1.14 introduces a major re-work of how we structure our Docker images and especially the PHP image.

For a variety of reasons we've moved from using pre-built PHP binaries in our releases to compiling PHP ourselves from source. This will give us more control over timing when getting out patches, and also streamlines the process of giving our hosted customers access to build custom images. In the mix with all this is a move from Debian Stretch to Debian Buster as the base operating system for all containers.

What all this means is that there's a higher than usual risk that this release includes some bugs in the way certain applications use PHP. We've done our best to test this as thoroughly as we can, but if you're in the middle of an important task we recommend holding off before applying this release by running tok upgrade

Here's a full rundown of the changes:

General

• All images have been upgraded from Debian Stretch to Debian Buster
• Several default and unnecessary users and groups have been removed from the base image, such as 'news' and 'lp'
• 'postfix' has replaced 'sendmail'

Admin/SSH Image

• 'multitail' and 'google-cloud-sdk' have been removed
• 'less' and 'screen' have been added
• node v12, v14 have been added (v12 will become default in Tokaido 1.15)
• ssh ciphers are updated to the latest recommended

PHP

• PHP 7.4 is now the default version for new projects
• PHP is now compiled from source, instead of using pre-compiled binaries.
• FPM logs have been moved to /tokaido/logs instead of /tokaido/logs/fpm
• We've removed support for Tokaido's runtime variables from .tok/config.yml.
• wkhtmltopdf has been updated to 0.12.6 (was 0.12.5)
• ghostscript has been updated to 9.52 (was 9.26)
• Default PHP memory limit increased from 256M to 380M
• APCu is now enabled by default with 96MB memory limit
• the uploadprogress module is now installed

1.13.0 - Drupal 9 Support and more

Tokaido 1.13 includes some performance and stability fixes, and officially adds support for Drupal 9 (which worked already, but everything said "Drupal 8" which was a bit confusing).

• Added support for Drupal 9
• Set the default PHP version to PHP 7.3 (was 7.2)
• Removed PHP 7.1 support
• Added 'cached' mode for Docker filesystem mounts, which offers a 2x performance increase on MacOS
• Removed the 'proxy' service, which was more trouble than it was worth

Existing Tokaido uses on 1.12.1 can upgrade by running tok upgrade

Minor fixes to SSL certificate generation and self-upgrade

This patch fixes two minor issues:

• Fixes an issue with new https://*.local.tokaido.io TLS certificates not having a unique serial number, and thus not being trusted on MacOS systems where the certificate is being re-generated from an older version.

• Fixes an issue with the self-upgrade tool, where it wasn't able to resolve the Github download URL properly. This means that upgrading from 1.12.0 to 1.12.1 will require a new install, but from 1.12.2 onwards the simpler tok upgrade command should work.

1.12.0 - Windows Support and Self-Installer

Windows Support

We're delighted to announce that Tokaido now runs on Windows 10 Pro with Docker Desktop for Windows. You can use Git Bash (officially supported) or WSL1 (experimental). We will add support for WSL2 when it becomes generally available.

Check out the Windows Install Guide for more detail. Please let us know if you find any bugs on Windows (or whatever else!)

Self-installer

Previously, Tokaido was available via Homebrew on MacOS and as a direct binary download on Linux. From 1.12, we've chosen to use a self-installer that is built-in to the Tokaido binary.

This means that all three platforms now have a consistent install (tok install) and upgrade (tok upgrade) mechanism. 1.11 is our last release to be shipped via Homebrew, so to upgrade you'll need to brew uninstall tokaido and then follow the install instructions for MacOS

Tokaido on a USB

Tokaido is now available on a self-installing USB that we will be giving our at trade shows and meetups. This USB contains everything you need to get Tokaido running, including Docker Destkop for Mac and Windows, the Tokaido Docker Images, and a Composer cache for a starting Drupal installation. If you're at DrupalSouth this year, come find the guys in Ironstar tshirts to get your free USB stick!

Image Updates and other Minor Changes

• PHP upgraded to 7.1.33, 7.2.24, and 7.3.11
• Redis CLI upgraded to 5.0.6
• tok snapshot new now dumps the snapshot with DROP DATABASE included, so that restores are holistic
• Fixed an issue where the Proxy service wouldn't start on Linux
• Fixed an issue where the image Makefile was referencing older base images

1.11.0 - MacOS Catalina Support and TLS Updates

The focus of this release has been around improving start-up time for Tokaido. In fact, with all these smaller changes taken together, most Tokaido projects will now start-up in around half the time!

New TLS Certificate Engine

In this release, we've completely replaced the engine that Tokaido uses to generate a local Certificate Authority. This also includes some updates to support MacOS Catalina's more stringent TLS requirements (specifically, no more 10 year certs).

The new TLS engine uses native Go coding rather than our previous adoption of the CloudFlare SSL library. This removes a huge chunk of code from Tokaido.

Proxy Improvements

The Tokaido proxy service, which runs at https://local.tokaido.io:5154, has had it's networking model flipped around. Previously, The Tokaido Proxy container would join the networks of each Tokaido project on your system.

Now, your various projects join the Proxy network. The key difference of this model is that we don't have to restart the Tokaido proxy container when each new Tokaido project is started. This makes start-up time much faster.

Pinned Images

Starting from this release, we will be tagging the upstream Docker images for each Tokaido component with the same version number of Tokaido.

For example, Tokaido 1.11.0 will run images tagged 1.11.0. This will be the "edge" release and these contains will also be deployed to the Ironstar non-production cluster the same week as the Tokaido release.

This guarantees that so long as you don't update your Tokaido version, you won't find your images suddenly updated either. This should help some uses who need to stay with older images because they need depreciated functionality.

The concept of Stability Release Sets remains, and the "stable" set will pin Tokaido to the "newest patch release of the previous minor version".

As an example, when Tokaido 1.12.0 is released, uses with that Tokaido version will default to running 1.12.0 containers. If they want to run the new Tokaido CLI with the older containers, they can switch their Stability Release Set to "stable", and it will use Tokaido 1.11.x (where x is the last patch release of Tokaido 1.11, such as 1.11.5).

However, a user on Tokaido 1.13.0 won't be able to access Tokaido 1.11.x images. This will be part of our commitment moving forward to support only the two most recent Minor releases of Tokaido.

🚨🚨🚨 End of PHP 7.1 Support

PHP will cease providing security support for PHP 7.1 from 1 Dec 2019.

Tokaido will similarly cease providing the ability to run PHP 7.1 environments from the 1.13.0 release, which should come out some time in early December.

It's time to update your app from PHP 7.1 to 7.3. Thankfully, Tokaido makes this incredibly easy to test. Just run:

tok config-set tokaido phpversion 7.3 followed by tok up

If things go wrong, you can downgrade just as easily:

tok config-set tokaido phpversion 7.1 followed by tok up

1.10.0 - Updated `tok test` functionality

Welcome to Tokaido 1.10! This release introduces the a refactored version of tok test where we've moved from using Nightwatch to Testcafe.

The New "tok test"

In older versions of Tokaido, running tok test would bootstrap the Nightwatch tests that ship as part of Drupal Core (8.6 or higher). After resolving that there was a lack of support and community interest in Nightwatch for Drupal, we've moved to using TestCafe with a small number of out-of-the-box tests that we've written in our own npm module.

Adopting TestCafe introduces a number of really awesome features that weren't previously available:

• Adding new tests beyond the basics that we include is easier, with more examples and documentation available
• Test artifacts like screenshots and videos work out-of-the-box
• Tests are a bit faster (although still a bit slower due to Docker Volume instances)

Static Port for the Database

You can now specify a static port that the database container will always listen on for your project:

tok config-set global project database port XXXX

Container Updates

• [ssh] Bumped from Node 8 LTS to Node 10 LTS
• [ssh] Removed the 'show-logs' script. This drops some Node dependencies to make the container a bit lighter, and because the script wasn't very good or useful.
• [php] bump PHP 7.1.29 to 7.1.31
• [php] bump PHP 7.2.18 to 7.2.21
• [php] bump PHP 7.3.5 to 7.3.8

Minor Fixes

Here's a quick run-down of all the little things that also made it into Tokaido 1.10:

• Fixed (hopefully) an issue where certain Terminals were unable to wrap text properly #186
• Tuned how snapshots are taken so that the full database is included #177
• Fixed an issue where user-specific settings were leaking into the shared project config.yml #185

Thanks as always for using Tokaido and a huge thanks to our friends at Technocrat for all their assistance helping with testing for this release.

Pass Four - The new `tok test` now using testcafe!

Shift to use an absolute path for SQL dump and restore

---

Fixed compatibility issue with Drush 8

Same as the first cut, however this time moving the generated test files out of the Drupal docroot and into {PROJECT}/.tok/testcafe

Working for me after running tok new for the drupal-standard and drupal-umami templates, but should undergo a bit more testing.

Main command tok test
Flags

-d for debug mode (might be useful if the setup is failing for you)
--use-existing-db Run the tests with the test DB from the previous run. Can save about 40s on a new test run but the DB will carry over state from the previous test.