Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(publish.sh): use dedicated source folder for httpd service including only .htaccess files #777

Conversation

lemeurherve
Copy link
Member

@lemeurherve lemeurherve commented Apr 25, 2024

Ref. jenkins-infra/helpdesk#2649

Blocked by jenkins-infra/pipeline-library#854

This PR allows using a dedicated volume for httpd service:

  • separated www-content & www-redirections sources created with a filtered rsync copy from www2
  • including only the .htaccess files to avoid serving any file from it
  • on its own updates-jenkins-io-httpd file share, in the same storage account as updates-jenkins-io file share used for mirrorbits service
  • cherry-picked commit from chore(publish.sh) set up httpd fallback redirection to mirrors [new UC] #776 to allow patching only content in www-redirections
    • we might need to patch all .htaccess and not only the one at the root, TBD
  • mirrorbits and R2 buckets use www-content which doesn't contain any .htaccess file as they don't need/can't use/shouldn't have access to them

It will requires some job config changes: new file share service principal credentials on trusted, and adapt some variable names of the existing one.

Will prepare something to try out in our dedicated test folder https://trusted.ci.jenkins.io:1443/job/update_center_test_lemeurherve_helpdesk2649/, I'll let you know before doing anything on trusted.ci.jenkins.io

WIP in draft to share what I'm working on, not for immediate review.

@lemeurherve lemeurherve changed the title chore(publish.sh): use dedicated www4 for httpd service including only .htacess files chore(publish.sh): use dedicated source folder for httpd service including only .htaccess files Apr 26, 2024
@lemeurherve lemeurherve changed the title chore(publish.sh): use dedicated source folder for httpd service including only .htaccess files feat(publish.sh): use dedicated source folder for httpd service including only .htaccess files May 3, 2024
site/publish.sh Outdated Show resolved Hide resolved
@dduportal dduportal self-assigned this May 3, 2024
site/publish.sh Outdated Show resolved Hide resolved
dduportal pushed a commit to jenkins-infra/azure that referenced this pull request May 10, 2024
…r env files to access file shares (#690)

This PR adds an output with a command to generate a zip containing two
env files:
- .env-content for updates-jenkins-io File Share
- .env-redirections for updates-jenkins-io-httpd File Share

This zip file has to be uploaded as zip credentials on
trusted.ci.jenkins.io for the update-center2/site/publish.sh script to
work.

Ref:
- jenkins-infra/update-center2#777
- jenkins-infra/helpdesk#2649
lemeurherve added a commit that referenced this pull request May 10, 2024
site/publish.sh Outdated Show resolved Hide resolved
site/publish.sh Outdated Show resolved Hide resolved
site/publish.sh Outdated Show resolved Hide resolved
@dduportal
Copy link
Contributor

Update: I'm working on this PR, with local runs in the permanent agent

@dduportal dduportal force-pushed the helpdesk2649-dedicated-www4-for-httpd branch from 18c7a91 to c17011d Compare May 24, 2024 12:46
@dduportal
Copy link
Contributor

Update:

  • Fixed the shell script and tested it in debug mode locally in the agent.trusted.ci.jenkins.io VM
    • Applied recommandations
    • Fixed calls to the get-fileshare-url (needed variables to be exported)
    • Fixed rscyn www-* dirs. generations
  • Next steps before review:
    • Fix file credential (missing values)
    • Test with a freestyle job as an E2E test

dduportal added a commit to jenkins-infra/azure that referenced this pull request May 30, 2024
Related to jenkins-infra/helpdesk#2649 and
blocking jenkins-infra/update-center2#784 and
jenkins-infra/charts-secrets#30

This PR simplifies and rename outputs to ensure the script used to
generate the Update Center credential ZIP file in
jenkins-infra/charts-secrets#30 can extract the
proper output with the naming convention from
jenkins-infra/update-center2#777 and
jenkins-infra/update-center2#784 (`www-content`
and `www-redirections`).

- It avoids multiple levels of escaping challenges (current output
requires 3 levels of escaping: in HCL syntax AND in shell syntax for the
initial call AND on shell syntax in the file template itself)
- The only "coupling" in Terraform HCL is the output name which is an
acceptable contract: it avoid having to change things in the terraform
state when only renaming a variable
- No more risk to have a local env file committed by error and exposing
the secrets

---------

Signed-off-by: Damien Duportal <[email protected]>
@dduportal
Copy link
Contributor

Blocked by #784 and #787

@dduportal dduportal force-pushed the helpdesk2649-dedicated-www4-for-httpd branch from aa68205 to a19a03f Compare July 11, 2024 13:55
@dduportal dduportal marked this pull request as ready for review July 11, 2024 13:55
@dduportal
Copy link
Contributor

Update:

Copy link
Contributor

@dduportal dduportal left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested with success in a team pair session in trusted.ci.jenkins.io (same method as #789)

Copy took 2s (it was an incremental one).

@dduportal dduportal merged commit be07a4c into jenkins-infra:master Jul 11, 2024
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants