Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ansible: add cloudflare-deploy role #3501

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

MoLow
Copy link
Member

@MoLow MoLow commented Sep 26, 2023

@MoLow
Copy link
Member Author

MoLow commented Sep 27, 2023

deployed to release-digitalocean-centos7-x64-1 for testing. worked as expected

@targos
Copy link
Member

targos commented Sep 27, 2023

I thought the goal was to have a sort of rsync? I don't see how this implementation will work with the release workflow, which includes a manual promomotion step from the releaser who signs the checksums.

@MoLow
Copy link
Member Author

MoLow commented Sep 27, 2023

I thought the goal was to have a sort of rsync? I don't see how this implementation will work with the release workflow, which includes a manual promomotion step from the releaser who signs the checksums.

can you elaborate more about the signing step? I was under the impression the release is uploaded through https://github.com/nodejs/node/blob/fef7927cc3a7a3fb81c355301aada4a679c5661c/Makefile#L1143-L1147 (and other places in this file) and https://github.com/nodejs/node/blob/fef7927cc3a7a3fb81c355301aada4a679c5661c/vcbuild.bat#L520-L538

this is also what logs at https://ci-release.nodejs.org/job/iojs+release/nodes=rhel8-x64-release/9643/console show.

@targos
Copy link
Member

targos commented Sep 27, 2023

The release is uploaded to a staging area on the www server. It cannot be accessed by the public/downloaded before the releaser executes the promotion script, which moves it to the publicly accessible directory.

@MoLow
Copy link
Member Author

MoLow commented Sep 27, 2023

The release is uploaded to a staging area on the www server. It cannot be accessed by the public/downloaded before the releaser executes the promotion script, which moves it to the publicly accessible directory.

where is this promotion script?
the logs of the release jobs (macos for example) have:

15:17:25 ssh node-www "mkdir -p nodejs/release/v20.7.0"
15:17:26 chmod 664 node-v20.7.0.pkg
15:17:26 scp -p node-v20.7.0.pkg node-www:nodejs/release/v20.7.0/node-v20.7.0.pkg
15:18:12 ssh node-www "touch nodejs/release/v20.7.0/node-v20.7.0.pkg.done"

this has the exact same timestamp as https://nodejs.org/dist/v20.7.0/

[node-v20.7.0.pkg](https://nodejs.org/dist/v20.7.0/node-v20.7.0.pkg)                                   18-Sep-2023 15:17            73602169

@MoLow
Copy link
Member Author

MoLow commented Sep 27, 2023

@targos maybe you are referring to the disttype parameter in the release job that determines weather the release is copied to https://nodejs.org/dist or to https://nodejs.org/download/ ?

@targos
Copy link
Member

targos commented Sep 27, 2023

Promotion script is here: https://github.com/nodejs/node/blob/main/tools/release.sh

AFAIK, all builds (release, nightly, rc, etc) are uploaded to the staging area. The difference between releases and other builds is that the releases are promoted manually, and the other builds are promoted automatically by a cron job.

@richardlau may know more about this.

@targos
Copy link
Member

targos commented Sep 27, 2023

/dist/ is only a virtual folder that points to /downloads/release/: https://github.com/nodejs/build/blob/e3059d702f5155fbb59432ab1b5053b6458474d9/ansible/www-standalone/resources/config/nodejs.org?plain=1#L63C12-L72

@MoLow
Copy link
Member Author

MoLow commented Sep 27, 2023

the entire goal here is to end up with a more resilient architecture,
so I believe that uploading directly to the r2 bucket is a better approach - at least where possible, it will dramatically reduce the dependence on a single www server - and can eventually make it redundant.

in the case of all build types except release (nightly, rc, test, v8-canary)- it makes sense to me to upload directly from Jenkins to R2
in case of release builds we can adjust https://github.com/nodejs/build/blob/0a1b2e9ad35ba7aab215e799529781a3522e00ce/ansible/www-standalone/tools/promote/promote_release.sh to upload to R2 (and perhaps adjust https://github.com/nodejs/node/blob/main/tools/release.sh in the future to do so)

WDYT?

@targos
Copy link
Member

targos commented Sep 27, 2023

I think the overall strategy should be discussed and agreed on within the WG before we make too many infra changes, that's all. My personal opinion isn't totally made on this.

@MoLow MoLow changed the title ansible: store cloudflare credentials on release builders ansible: add cloudflare-deploy role Sep 27, 2023
@richardlau
Copy link
Member

Promotion script is here: https://github.com/nodejs/node/blob/main/tools/release.sh

AFAIK, all builds (release, nightly, rc, etc) are uploaded to the staging area. The difference between releases and other builds is that the releases are promoted manually, and the other builds are promoted automatically by a cron job.

@richardlau may know more about this.

I added a diagram outline of the release process to this repository, with clickable labels on most of the boxes/steps linking to the relevant scripts (either in this repository or nodejs/node): https://github.com/nodejs/build/blob/main/doc/release-overview.md
It's slightly outdated now as we've moved the website off of our infrastructure (although I think all we may have done there is turn off the webhook from nodejs/nodejs.org).
I've also just noticed the diagram is missing the cron around the promotion for the non-release builds

- "*/30 * * * * dist /home/staging/tools/promote/promote_nightly.sh nodejs"
.

@richardlau
Copy link
Member

the entire goal here is to end up with a more resilient architecture, so I believe that uploading directly to the r2 bucket is a better approach - at least where possible, it will dramatically reduce the dependence on a single www server - and can eventually make it redundant.

Having had a quick glance at https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html and this PR the immediate problem I see is clients only for Linux, macOS and Windows, so we're going to have an issue on AIX (which is Unix, not Linux).

@richardlau
Copy link
Member

in the case of all build types except release (nightly, rc, test, v8-canary)- it makes sense to me to upload directly from Jenkins to R2 in case of release builds we can adjust https://github.com/nodejs/build/blob/0a1b2e9ad35ba7aab215e799529781a3522e00ce/ansible/www-standalone/tools/promote/promote_release.sh to upload to R2 (and perhaps adjust https://github.com/nodejs/node/blob/main/tools/release.sh in the future to do so)

We'll need to be uploading the SHASUM256.txt files to R2 from the promotion scripts for all build types as that is where we're currently generating the shasums

(cd "${dstdir}/${version}" && shasum -a256 $(ls node* openssl* iojs* win-*/* x64/* 2> /dev/null) > SHASUMS256.txt) || exit 1

via
${__dirname}/_resha.sh $site $dstdir $subdir

@targos
Copy link
Member

targos commented Sep 28, 2023

Having had a quick glance at https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html and this PR the immediate problem I see is clients only for Linux, macOS and Windows, so we're going to have an issue on AIX (which is Unix, not Linux).

It's probably possible to upload the file with curl or similar.

@targos
Copy link
Member

targos commented Sep 28, 2023

Shasum files are going to be challenging indeed.

@MoLow
Copy link
Member Author

MoLow commented Sep 28, 2023

It's probably possible to upload the file with curl or similar.

yes. it can also be installed with pip I think

Shasum files are going to be challenging indeed.

we can start with deploying from the promotion script, and in the future make the promotion script run from inside a Cloudflare worker

@MoLow
Copy link
Member Author

MoLow commented Sep 29, 2023

I think this would be a long term solution, added a short term implementation #3505, and will discuss in the build WG meeting

@jbergstroem
Copy link
Member

jbergstroem commented Sep 29, 2023

rclone also speaks s3 and uses a rsync-like protocol for deciding changes. It seems to have (third party) builds for AIX: https://github.com/skytap/rclone

It's probably possible to upload the file with curl or similar.

Yep, probs the best option.

@mhdawson
Copy link
Member

Discussed in build WG meeting today, agreed to remove from agend in favor of the top level issue about the R2 migration.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants