magento-docker

magento-docker is Docker environment for easy to setup, configure, debug Magento2 + Storefront instance with varnish, elasticsearch, redis, rabbit and mftf tests.

Requirements

• Git

• Docker

• Docker Compose

• Setup SSH-keys on your github account. (see docs for help)

• (optional - for Mutagen installation only) Install Mutagen docs

• Ensure you do not have dnsmasq installed/enabled locally (will be auto-installed if you've use Valet+ to install Magento)

How to install

Steps

1. Create a directory where all repositories will be cloned (used in your IDE)

   Proposed structure:

    ~/projects/storefront/magento-docker    # this repo
    ~/projects/storefront/repos             # directory with repositories

2. Update MAGENTO_PATH in .env with recently created directory path

3. Add $MAGENTO_DOMAIN from .env to hosts, e.g.:

    sudo -- sh -c "echo '127.0.0.1 magento.test >> /etc/hosts"

4. Add Magento authentication keys to access the Magento Commerce repository

• copy the contents of etc/php/auth.json.dist to new etc/php/auth.json file and replace placeholders with your credentials

You have two options to install the project

docker-compose based installation

This approach will use only docker-compose to install Storefront project. Files between the host and guest are synced with "delegated" options (see FILE_SYNC in .env)

• RUN git checkout docker-compose.yml - optional: reset changes if any present in docker-compose.yml
• RUN bash ./init_project - clone repos (if RECLONE=YES in .evn)
• RUN docker-compose up -d - up services
• RUN bash ./reinstall - install/reinstall Magento with repositories provided in INSTALLED_REPOS (see .env)

mutagen based installation

This approach will use mutagen service to share code between the host and guest. Mutagen will significantly speedup work of Magento instance

Before run, set MUTAGEN_INSTALLATION=YES in .env

4. RUN mutagen project start

Note, for the first installation (when you don't have cloned repositories yes) please change settings "RECLONE" to "yes" in ".env" file

Configuration

MAGENTO_PATH=/magento/magento-docker-install    # local directory to clone repos into
RECLONE=no                                      # flag indicate whether do re-clon of all repos or no
MAGENTO_EDITION=EE                              # EE|B2B
Notices:

gRPC performance testing

Preconditions:

• Product with id "1" is created
• Verify DOCKER_PATH in the .env file Ro run gRPC benchmark run mutagen project run grpc-run-bench or docker-compose -f bundles/grpc-testing.yml up Change test scenario if needed in magento-docker/etc/grpc/bench-test-config.json

Troubleshooting

1. Add MAGENTO_PATH path to Docker sharing folders (Docker preferences) in case docker-error

Scenarios

1. Enter container

• Run docker-compose exec app bash.

2. Relaunch container

• Run docker-compose scale <container_name>=0 && docker-compose scale <container_name>=1. For example: docker-compose scale app=0 && docker-compose scale app=1.

3. Run tests

1. docker-compose exec app magento prepare_tests
2. docker-compose exec app bin/magento dev:tests:run (unit, integration)
3. docker-compose exec app bash
4. cd dev/tests/acceptance/ and vendor/bin/codecept run (mftf)
5. cd dev/tests/functional/ and vendor/bin/phpunit run (mtf)
6. vnc://localhost:5900 pass:secret (mftf or mtf)

4. Enable/disable Xdebug

• Enable: mutagen project run xdebug-enable
• Disable: mutagen project run xdebug-disable

⚠️ Enabled Xdebug may slow your environment.

5. Magento (Re)-Installation

• mutagen project run reinstall

6. Optimization host

1. Redis optimization

   docker run -it --rm --privileged ubuntu /bin/bash
   echo never | tee /sys/kernel/mm/transparent_hugepage/enabled
   echo never | tee /sys/kernel/mm/transparent_hugepage/defrag
   

2. Optimization for MacOS

FAQ

1. If docker containers do not go up, check errors in console, run docker-compose down, fix issue and run docker-compose up again.
2. If Overwrite the existing configuration for db-ssl-verify?[Y/n] prompts in console, type Y.
3. If magento installation fails, run docker-compose exec app magento reinstall

TODO list

https://github.com/duhon/magento-docker/projects

gRPC set up:

As currently we don't have ability to generate Storefront API on fly - we put magento.proto file to the root of catalog-storefront:develop branch. It's needed to run gRPC server and client.

Steps for manual setup

1. PHP image should be built with "grpc" extension and rr-grpc
   • see details in ./build/php/fpm-grpc
2. enter into a container and run magento CLI command bin/magento storefront:grpc:init \\Magento\\CatalogStorefrontApi\\Api\\CatalogProxyServer \\Magento\\CatalogStorefrontApi\\Api\\VariantServiceProxyServer
3. gRPC server can be executed now: ./vendor/bin/grpc-server

Automated setup

1. Run mutagen project run grpc-server-start command to execute etc/php/tools/grpc script which does the following:

   • replaces magento.proto with a file from catalog-storfront if it's a link
   • runs magento CLI command bin/magento storefront:grpc:init \\Magento\\CatalogStorefrontApi\\Api\\CatalogProxyServer \\Magento\\CatalogStorefrontApi\\Api\\VariantServiceProxyServer that does the following:
     • copies certain files to vendor/bin (if they don't exist)
     • creates list of gRPC services and puts it to ./generated/code/grpc_services_map.php file (if not created yet)
   • runs gRPC server via executing of ./vendor/bin/grpc-server
   • Please NOTE: Port 9001 should be opened to allow external connections to the server.

2. Run gRPC client (can be executed from any instance which has access to app:9001):

   • Uncomment following code in docker-compose.yml:

        grpcui:
          image: wongnai/grpcui
          ports:
            - "8080:8080"
          volumes:
            - code:/var/www
          entrypoint: ["grpcui", "-plaintext", "-proto", "magento.proto", "-port", "8080", "-bind", "0.0.0.0", "-import-path", "/var/www/magento2ce", "app:9001"]

   • Make sure that paths to Magento root in entry point are correct (/var/www/magento2ce) - if no, replace them by correct paths.
   • Port 8080 should be opened to allow external connections to the client.
   • Run grpcui container: docker-compose up grpcui

Standalone services setup

Catalog Storefront contains services that can be run separately. For example storefront-message-broker is being started as separate application after Magento Monolith installed. Other services are optional (pricing, search, reviews) and can be started after Magento launched. For example pricing service can be started by the following steps:

1. Make sure container with pricing is available. and not commented before install.
2. In reinstall file uncomment catalog-pricing installation command or run this command if Magento already installed. docker-compose exec app-pricing magento reinstall_storefront_pricing
3. To enter the container run the following command: docker-compose exec app-pricing bash

To start grpc-server within pricing application do the following:

1. Navigate to .env file and set GRPC_SERVICE_CLASS to the value mentioned in the example:

GRPC_SERVICE_CLASS="\\Magento\\PricingStorefrontApi\\Api\\PriceBookServiceProxyServer"

2. From docker project root start the following command: docker-compose exec app-pricing magento grpc
3. To send requests to pricing application start grpcui. Uncomment grpcui-pricing section in docker-composer.yml and start via docker-compose: docker-compose up grpcui-pricing
4. Open http://0.0.0.0:8082/ in browser. magento.pricingStorefrontApi.proto.PriceBookService service must be present in services list.

Q/A

1. Unable to start grpcui.

If project uses symlinks then magento.proto in root folder will not work correctly with grpcui. The following errors can occur: Attaching to storefront_grpcui_1 grpcui_1 | Failed to process proto source files.: /var/www/magento2ce/magento.proto does not reside in any import path storefront_grpcui_1 exited with code 1 Connect to project, unlink proto and copy from catalog-storefront, then start container again: - unlink magento.proto - cp ../catalog-storefront/magento.proto magento.proto