Restart reinstalls Aegir

[webfed]

Very nice work, hope to use this for development.
When I restart the hostmaster container on OSX with Kitematic Aegir gets reinstalled. How can I prevent this?

[jonpugh]

Are you sure it's fully reinstalling hostmaster?

The entry point script for the container must run drush hostmaster-install, but aegir does not overwrite the database if it detects an existing install. It still runs through the paces of site verify, etc.

The existing install should be persistent between restarts, because we put all of /var/aegir in a volume, and the db container puts it's data in a volume too.

Can you try the special development environment I created: https://github.com/aegir-project/development

I'll do some more testing on kitematic.

[helmo]

I think we could add a README section about re-starting a container.

I remember getting access denied errors from provision when resting it a few days later.
Could the container be re-created with a different IP? That might relate to https://www.drupal.org/node/2794915

[JSzaszvari]

I'm seeing the same thing,

If I have a persistant data volume for both /var/aegir/ and /var/lib/mysql/

Upon rebooting the hostmaster I get the following in the console:

11/7/2016 7:39:03 AMÆGIR | -------------------------
11/7/2016 7:39:03 AMÆGIR | Enabling Hosting Task Queue...
11/7/2016 7:39:03 AMCommand pm-enable needs a higher bootstrap level to run - you will   �[31;40m�[1m[error]�[0m
11/7/2016 7:39:03 AMneed to invoke drush from a more functional Drupal environment to run
11/7/2016 7:39:03 AMthis command.
11/7/2016 7:39:03 AMThe drush command 'en hosting_queued' could not be executed.         �[31;40m�[1m[error]�[0m
11/7/2016 7:39:03 AMDrush was not able to start (bootstrap) the Drupal database.         �[31;40m�[1m[error]�[0m
11/7/2016 7:39:03 AMHint: This may occur when Drush is trying to:
11/7/2016 7:39:03 AM * bootstrap a site that has not been installed or does not have a
11/7/2016 7:39:03 AMconfigured database. In this case you can select another site with a
11/7/2016 7:39:03 AMworking database setup by specifying the URI to use with the --uri
11/7/2016 7:39:03 AMparameter on the command line. See `drush topic docs-aliases` for
11/7/2016 7:39:03 AMdetails.
11/7/2016 7:39:03 AM * connect the database through a socket. The socket file may be
11/7/2016 7:39:03 AMwrong or the php-cli may have no access to it in a jailed shell. See
11/7/2016 7:39:03 AMhttp://drupal.org/node/1428638 for details.
11/7/2016 7:39:03 AM
11/7/2016 7:39:03 AMDrush was attempting to connect to: 
11/7/2016 7:39:03 AM Drupal version        :  7.51                                      
11/7/2016 7:39:03 AM Site URI              :  daniel-hostmaster.mydomain.com     
11/7/2016 7:39:03 AM Database driver       :  mysql                                     
11/7/2016 7:39:03 AM Database hostname     :  database                                  
11/7/2016 7:39:03 AM Database port         :  3306                                      
11/7/2016 7:39:03 AM Database username     :  danielhostmaster                          
11/7/2016 7:39:03 AM Database name         :  danielhostmaster                          
11/7/2016 7:39:03 AM PHP configuration     :  /etc/php5/cli/php.ini                     
11/7/2016 7:39:03 AM PHP OS                :  Linux                                     
11/7/2016 7:39:03 AM Drush script          :  /usr/local/bin/drush                      
11/7/2016 7:39:03 AM Drush version         :  8.1.7                                     
11/7/2016 7:39:03 AM Drush temp            :  /tmp                                      
11/7/2016 7:39:03 AM directory                                                          
11/7/2016 7:39:03 AM Drush configuration   :  /var/aegir/hostmaster/sites/daniel-hostma 
11/7/2016 7:39:03 AM                          ster.mydomain.com/drushrc.php           
11/7/2016 7:39:03 AM                          /var/aegir/hostmaster/sites/all/drush/dru 
11/7/2016 7:39:03 AM                          shrc.php                                  
11/7/2016 7:39:03 AM Drush alias files     :  /var/aegir/.drush/hm.alias.drushrc.php    
11/7/2016 7:39:03 AM                          /var/aegir/.drush/platform_hostmaster.ali 
11/7/2016 7:39:03 AM                          as.drushrc.php                            
11/7/2016 7:39:03 AM                          /var/aegir/.drush/server_master.alias.dru 
11/7/2016 7:39:03 AM                          shrc.php                                  
11/7/2016 7:39:03 AM                          /var/aegir/.drush/server_database.alias.d 
11/7/2016 7:39:03 AM                          rushrc.php                                
11/7/2016 7:39:03 AM                          /var/aegir/.drush/hostmaster.alias.drushr 
11/7/2016 7:39:03 AM                          c.php                                     
11/7/2016 7:39:03 AM Drupal root           :  /var/aegir/hostmaster                     
11/7/2016 7:39:03 AM Drupal Settings       :  sites/daniel-hostmaster.mydomain.com/se 
11/7/2016 7:39:03 AM File                     ttings.php                                
11/7/2016 7:39:03 AM Site path             :  sites/daniel-hostmaster.mydomain.com  
11/7/2016 7:39:03 AM
11/7/2016 7:39:03 AM

The only way that I can find that resolves it is to delete all the persistant data from mysql and /var/aegir/

[Yusadolat]

Please can someone tell me where am wrong. After running command docker compose up I got the following error

ERROR: for hostmaster Cannot start service hostmaster: failed to create endpoint aegirdockerfile_hostmaster_1 on network aegirdockerfile_default: Error starting userland proxy: listen tcp 0.0.0.0:80: listen: address already in use
ERROR: Encountered errors while bringing up the project.

[helmo]

@Yusadolat thanks for creating a separate issue in #23

[fuzzy76]

@JSzaszvari I would have tried diagnosing by exec a bash inside the container and see if I could connect to the "danielhostmaster" database. It sure looks like the database is down. Another thing to try would be looking at the logs of your database server (wherever it is running).

[jonpugh]

Restarting a container may alter the IP that is detected by Aegir, which is used for the database grants.

There's many d.o issues about this, I think I've fixed in in hosting_docker, but that's not yet stable or included in aegir.

If you check the permissions of the DB user that is created, you'll notice a host of 172.***

Change this to % and start the containers again, and the sites will come back up.

Any help is appreciated, we are all volunteers here and don't have a lot of free time to spend!

[jonpugh]

Here's an old issue I created talking about the problem: https://www.drupal.org/node/2794915

Here's the code I'm working on to have docker Aegir Services for web, db, etc. https://www.drupal.org/project/hosting_docker

[apassi]

Restarting a container may alter the IP that is detected by Aegir, which is used for the database grants.

@jonpugh Hi, i have just this problem now. Sadly, i already did big conversion from legacy aegir to docker hostmaster, and i dont want to make convert back to legacy..

I think in docker we can manage static ip's, as a fast fix for this, but it is not docker way to work. I notice you hosting_docker module, and inspected its source code, and you have used docker container id, but i think the module should use docker container name, because id can and will change, but the container name is user defined. I will get back to this, maby with some patches.

[simwijs]

Having this problem as well. How did you fix it?

Trying to use this to deploy Aegir but it can't find the hostmaster when trying to restart container. Resulting in it reinstalling everything without the platforms, which then have to be installed manually.

[PaulLebmann]

Hi @simwijs, did you mount a volume at /var/aegir?
If you culd provide your docker-compose.yml or the way you try to start it, it would be easier to troubleshoot.

[simwijs]

Yes, see below. Simply using docker compose up

version: '3'
services:

  hostmaster:
    image: aegir/hostmaster:php74 # Locally built from ~/dockerfiles and tagged manually using 'docker build . -f dockerfile-php74 -t php74
    ports:
      - 80:80
      - 443:443
    hostname: <redacted>
    links:
      - database
    depends_on:
      - database
    environment:
      MYSQL_ROOT_PASSWORD: <redacted>
      AEGIR_CLIENT_EMAIL: <redacted>
      AEGIR_DATABASE_SERVER: database
    volumes:
      - /var/aegir:/var/aegir

  database:
    #image: mariadb:10.8.3 # mysql:8.0.30
    image: mysql:8.0.30
    environment:
      MYSQL_ROOT_PASSWORD:  <redacted>
    volumes:
      - mysql-data:/var/lib/mysql
      #- mariadb-data:/var/lib/mysql

volumes:
  mariadb-data:
  mysql-data:

and php74 image (diff is that it uses 20.04 and updated drush version to 8.4.10 instead of 8.3.6)

FROM ubuntu:20.04
ENV DEBIAN_FRONTEND=noninteractive
RUN apt-get update -qq && apt-get install -y -qq\
  apache2 \
  cron \
  curl \
  git-core \
  libapache2-mod-php \
  libpcre3-dev \
  mysql-client \
  nano \
  php-apcu \
  php-cli \
  php-common \
  php-curl \
  php-dev \
  php-fpm \
  php-gd \
  php-imap \
  php-json \
  php-mbstring \
  php-mysql \
  php-opcache \
  php-sqlite3 \
  php-xml \
  postfix \
  rsync \
  sudo \
  unzip \
  vim \
  wget

# Use --build-arg option when running docker build to set these variables.
# If wish to "mount" a volume to your host, set AEGIR_UID and AEGIR_GIT to your local user's UID.
# There are both ARG and ENV lines to make sure the value persists.
# See https://docs.docker.com/engine/reference/builder/#/arg
ARG AEGIR_UID=1000
ENV AEGIR_UID ${AEGIR_UID:-1000}

RUN echo "Creating user aegir with UID $AEGIR_UID and GID $AEGIR_GID"

RUN addgroup --gid $AEGIR_UID aegir
RUN adduser --uid $AEGIR_UID --gid $AEGIR_UID --system --shell /bin/bash --home /var/aegir aegir
RUN adduser aegir www-data
RUN a2enmod rewrite
RUN a2enmod ssl
RUN ln -s /var/aegir/config/apache.conf /etc/apache2/conf-available/aegir.conf
RUN ln -s /etc/apache2/conf-available/aegir.conf /etc/apache2/conf-enabled/aegir.conf

COPY sudoers-aegir /etc/sudoers.d/aegir
RUN chmod 0440 /etc/sudoers.d/aegir

RUN wget https://raw.githubusercontent.com/composer/getcomposer.org/e831e1e4d6cabfb11fa9657103cf728e6eb9e295/web/installer -O - -q | php -- --quiet
RUN cp composer.phar /usr/local/bin/composer


ENV DRUSH_VERSION=8.4.10
RUN wget https://github.com/drush-ops/drush/releases/download/$DRUSH_VERSION/drush.phar -O - -q > /usr/local/bin/drush
RUN chmod +x /usr/local/bin/composer
RUN chmod +x /usr/local/bin/drush

# Install fix-permissions and fix-ownership scripts
RUN wget http://cgit.drupalcode.org/hosting_tasks_extra/plain/fix_permissions/scripts/standalone-install-fix-permissions-ownership.sh
RUN bash standalone-install-fix-permissions-ownership.sh

COPY docker-entrypoint.sh /usr/local/bin/
RUN chmod +x /usr/local/bin/docker-entrypoint.sh

COPY run-tests.sh /usr/local/bin/
RUN chmod +x /usr/local/bin/run-tests.sh

#COPY docker-entrypoint-tests.sh /usr/local/bin/
#RUN chmod +x /usr/local/bin/docker-entrypoint-tests.sh

COPY docker-entrypoint-queue.sh /usr/local/bin/
RUN chmod +x /usr/local/bin/docker-entrypoint-queue.sh

# Prepare Aegir Logs folder.
RUN mkdir /var/log/aegir
RUN chown aegir:aegir /var/log/aegir
RUN echo 'Hello, Aegir.' > /var/log/aegir/system.log

# Don't install provision. Downstream tags will do this with the right version.
## Install Provision for all.
#ENV PROVISION_VERSION 7.x-3.x
#RUN mkdir -p /usr/share/drush/commands
#RUN drush dl --destination=/usr/share/drush/commands provision-$PROVISION_VERSION -y
ENV REGISTRY_REBUILD_VERSION 7.x-2.5
RUN drush dl --destination=/usr/share/drush/commands registry_rebuild-$REGISTRY_REBUILD_VERSION -y

USER aegir

RUN mkdir /var/aegir/config
RUN mkdir /var/aegir/.drush

# You may change this environment at run time. User UID 1 is created with this email address.
ENV AEGIR_CLIENT_EMAIL [email protected]
ENV AEGIR_CLIENT_NAME admin
ENV AEGIR_PROFILE hostmaster
ENV AEGIR_VERSION 7.x-3.x
ENV PROVISION_VERSION 7.x-3.x
ENV AEGIR_WORKING_COPY 0
ENV AEGIR_HTTP_SERVICE_TYPE apache

# Must be fixed across versions so we can upgrade containers.
ENV AEGIR_HOSTMASTER_ROOT /var/aegir/hostmaster

WORKDIR /var/aegir

# The Hostname of the database server to use
ENV AEGIR_DATABASE_SERVER database

# For dev images (7.x-3.x branch)
ENV AEGIR_MAKEFILE http://cgit.drupalcode.org/provision/plain/aegir.make

# For Releases:
# ENV AEGIR_MAKEFILE http://cgit.drupalcode.org/provision/plain/aegir-release.make?h=$AEGIR_VERSION

VOLUME /var/aegir

# docker-entrypoint.sh waits for mysql and runs hostmaster install
ENTRYPOINT ["docker-entrypoint.sh"]
CMD ["drush", "@hostmaster", "hosting-queued"]

[PaulLebmann]

Does it work with php72 pulled from docker hub?

I stopped using Aegir when switching to Drupal 8 and newer versions of drush. So I mostly lost track of the project.
According to Support Ubuntu 20.04 LTS it seems that there are problems with version Hostmaster 3.190 and Ubuntu 20.04.
Since Ubuntu 20.04 comes with PHP 7.4 which is eol in November and looking at the latest release dates and the issue queues of the Aegir related modules I think you should make sure, that you really need all the features Aegir gives you or if there is an alternative route for your project.

[simwijs]

I believe I tested that too but it didn't work, but it was while in a rush of things.

I have a customer who uses these versions already... but I switched to using the ansible role and it works fine to install it on a new EC2 instance (t3.small) using AWS. The ansible role can be found here and I'm using the policy to deploy

I told them to stop using aegir. Hassle to work with in these circumstances when it doesn't work "out of the box". :)