Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enhance Build Process #428

Merged
merged 23 commits into from
Dec 30, 2024
Merged

Enhance Build Process #428

merged 23 commits into from
Dec 30, 2024

Conversation

laardee
Copy link
Contributor

@laardee laardee commented Dec 19, 2024

This PR introduces some improvements and optimizations:

  • Splits Gulp runtime configurations into router-specific files for better modularity.
  • Updates Dockerfiles, light-http configs, and scripts to use parameters, avoiding unnecessary changes to git-tracked files.
  • Improves the clarity of error messages in specific scenarios.
  • Adds support for environment variables to skip selected build steps, reducing development time.
  • Skips pushing images to the Docker registry if credentials are not defined, ensuring the build process in unconfigured environments, like development.

Partly related to the old issue #189

Copy link
Member

@vesameskanen vesameskanen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the contribution. Using variables instead of modifying local version managed files is a good improvement.

Some notes, though:

  • Unused task/PatchDeploymentFiles.js should be deleted, the reference to it in gulpfile removed and README.md updated to reflect the changes.

  • Build process works but creates a broken OTP image. When running the image, kubernetes deployment shows an error:

    Reason: CrashLoopBackOff
    Last State: Terminated
    Reason: StartError
    Message: failed to create containerd task: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: exec: "/var/otp/entrypoint.sh": stat /var/otp/entrypoint.sh: no such file or directory: unknown

This error is a bit strange as I can find /var/otp/entrypoint.sh from the docker image. Any ideas?

task/Common.js Outdated
Comment on lines 10 to 11
const mapSrc = (id, url, fit, rules, replacements, request) => ({ id, url, fit, rules, replacements, request })

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it be more natural to import this one liner from util.js ? task folder contains gulp tasks and the main update logic.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

True, moved that to util.js

@vesameskanen
Copy link
Member

Found out that our container deployment mounts data storage to path /var/otp . I assume that this mounting overwrites the original folder where the entrypoint script is.

@laardee
Copy link
Contributor Author

laardee commented Dec 27, 2024

Found out that our container deployment mounts data storage to path /var/otp . I assume that this mounting overwrites the original folder where the entrypoint script is.

The entrypoint.sh is now in the parent /var/ directory.

Are these mounts and other cluster-related variables documented somewhere? I briefly tried to find those without success.

@vesameskanen
Copy link
Member

Opentripplanner-data-container, opentripplanner and their deployment configurations are indeed a complex system. It is hard to figure out how the actual runtime environment and configurations found from it get assembled, because the logic is split into many repositories.

The mentioned conflicting mount path is defined here:
https://github.com/HSLdevcom/digitransit-kubernetes-deploy/blob/master/roles/aks-apply/files/dev/opentripplanner-hsl-v2-dev.yml#L58

Environment variables for the data loader can be found here:
https://github.com/HSLdevcom/digitransit-kubernetes-deploy/blob/master/roles/aks-apply/files/dev/otp-data-builder-hsl-v3-dev.yml

These deployments are for HSL routing in development environment.

Copy link
Member

@vesameskanen vesameskanen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems to work now. I will ask second opinion from my colleague Joel before merging.

@vesameskanen vesameskanen merged commit 0b9a8f2 into HSLdevcom:v3 Dec 30, 2024
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants