Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MDBF-804 - BB NGINX configuration in GH CI #590

Merged
merged 8 commits into from
Oct 15, 2024

Conversation

RazvanLiviuVarzaru
Copy link
Collaborator

@RazvanLiviuVarzaru RazvanLiviuVarzaru commented Oct 10, 2024

About This Patch:

  • Using templates (present since NGINX 1.19) to populate conf.d.
  • Templates allow for the use of environment variables defined in the .env files, enabling us to distinguish between PROD and DEV environments, particularly for the server name and certificate paths.
  • A proxy_params file is required according to the PROD configuration.
  • Mounting /etc/letsencrypt/live for SSL certificates. The base path is the same in both environments.
  • NGINX_ARTIFACTS_SSL_PATH variable is necessary because, in DEV, the same certificate is used for both CI and BB.
  • Attaching net_back to the NGINX container to facilitate communication with master-web via DNS.
  • Removing net_front from master-web; communication will be handled through NGINX.
  • NGINX access/error logs are written to the Docker-Compose relative path logs/nginx, which is needed for Zabbix collection.

TODO Before Migration to PROD:

  • Address all FIXME comments.
    • Cross-reference proxy pass. Attempt here: 9f8f1bc
    • helper_files directory name on hz-bbm2.
    • location /cloud-init ?

TODO Before Deployment in DEV:

  • Disable the HAProxy service.
  • not sure about the NGINX stats (attempt here: d2f8f13) and ACME-challenge . I may need your help @fauust . for ACME maybe switch to standalone like in DEV and in the post_hook restart the container?

All the tests were performed locally (with just NGINX, MariaDB, Master-Web and CrossBar). (one can use a self-signed certificate)

About This Patch:
- Using templates (present since NGINX 1.19) to populate conf.d.
- Templates allow for the use of environment variables defined in the .env files, enabling us to distinguish between PROD and DEV environments, particularly for the server name and certificate paths.
- A proxy_params file is required according to the PROD configuration.
- Mounting /etc/letsencrypt/live for SSL certificates. The base path is the same in both environments.
- NGINX_ARTIFACTS_SSL_PATH variable is necessary because, in DEV, the same certificate is used for both CI and BB.
- Attaching net_back to the NGINX container to facilitate communication with master-web via DNS.
- Removing net_front from master-web; communication will be handled through NGINX.
- NGINX access/error logs are written to the Docker-Compose relative path logs/nginx, which is needed for Zabbix collection.

TODO Before Migration to PROD:
- Address all FIXME comments.
  - Cross-reference proxy pass
  - helper_files directory name on hz-bbm2.
  - location /cloud-init ?

TODO Before Deployment in DEV:
- Disable the HAProxy service.
This will require some changes in Zabbix configuration for collecting these NGINX metrics. (replacing HAProxy metrics)
We don't have CrossReference in DEV, yet.
For PROD I've configured the WG IP, for DEV it's just the loopback interface and will raise a BAD GATEWAY error.
Copy link
Collaborator

@fauust fauust left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall looks good but a bit more work is needed I think.

What does not look ok is the fact that environment variables are made available to all containers, appart from nginx and master-web, they are not used and not necessary.

docker-compose/.env Outdated Show resolved Hide resolved
- /srv/buildbot/packages:/srv/buildbot/packages:ro
- /srv/buildbot/galera_packages:/srv/buildbot/galera_packages:ro
- /srv/buildbot/helper_files:/srv/buildbot/helper_files:ro
- /etc/letsencrypt/live:/etc/nginx/ssl:ro
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's add a comment here so that we make sure that we move this in /srv/buildbot (or somewhere else) when we move the renew routine from the host to containers.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure!

location /galera {
alias /srv/buildbot/galera_packages;
}
#FIX ME - Still needed? Not present in DEV.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, but no, remove it and let's find another solution for cloud-init images, it's polluting and confusing.


root /srv/buildbot/packages/;
location /helper_files {
alias /srv/buildbot/helper_files; #FIXME - for consistency, on hz-bbm2 let's rename it to helper_files instead of mariadb-shared-packages (current PROD)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok!

docker-compose/nginx/conf.d/monitoring.conf Show resolved Hide resolved
- add certbot container to docker-compose
- bind mount www path for certbot/nginx to use it in webroot mode
- bind mount certboot cnf path to allow nginx find the certificates

Fixes:
- remove location /cloud-init as per review
- remove NGINX_ARTIFACTS_SSL_PATH variable . Will use multiple domain names in PROD also.
- adapt bb and ci NGINX conf files to handle the acme challenge on port 80
- fix generate-config to avoid exposing nginx env variables to other containers other than nginx
@fauust fauust merged commit c8c6111 into MariaDB:dev Oct 15, 2024
2 of 3 checks passed
fauust added a commit to fauust/buildbot that referenced this pull request Oct 15, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

2 participants