Refactor environment setup and caching

[safoinme]

Describe changes

I implemented/fixed _ to achieve _.

Pre-requisites

Please ensure you have done the following:

• I have read the CONTRIBUTING.md document.
• If my change requires a change to docs, I have updated the documentation accordingly.
• I have added tests to cover my changes.
• I have based my new branch on develop and the open PR is targeting develop. If your branch wasn't based on develop read Contribution guide on rebasing branch to develop.
• If my changes require changes to the dashboard, these changes are communicated/requested.

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to change)
• Other (add details above)

Summary by CodeRabbit

• New Features

  • Introduced new workflows for integration tests and linting, enhancing testing capabilities across different environments.
  • Added a new deployment configuration for a ZenML server to support GitHub Actions.
  • New environment configuration for github-actions-server-docker-orchestrator added.

• Bug Fixes

  • Updated version constraints for the mlflow package to prevent compatibility issues.

• Documentation

  • Enhanced input options and descriptions for improved user interaction in workflows.

• Chores

  • Updated workflow environments to standardize execution across jobs.

[github-actions]

LLM Finetuning template updates in examples/llm_finetuning have been pushed.

[github-actions]

Classification template updates in examples/mlops_starter have been pushed.

[github-actions]

E2E template updates in examples/e2e have been pushed.

[gitguardian]

⚠️ GitGuardian has uncovered 12 secrets following the scan of your pull request.

Please consider investigating the findings and remediating the incidents. Failure to do so may lead to compromising the associated services or software components.

🔎 Detected hardcoded secrets in your pull request

GitGuardian id	GitGuardian status	Secret	Commit	Filename
3584396	Triggered	Generic Password	0903f65	.github/workflows/integration-test-fast.yml	View secret
3584396	Triggered	Generic Password	a712dcf	.github/workflows/integration-test-fast.yml	View secret
3584396	Triggered	Generic Password	a712dcf	.github/workflows/integration-test-fast.yml	View secret
3584396	Triggered	Generic Password	449fe79	.github/workflows/integration-test-fast.yml	View secret
3584396	Triggered	Generic Password	e6e729d	.github/workflows/integration-test-fast.yml	View secret
3584396	Triggered	Generic Password	e6e729d	.github/workflows/integration-test-fast.yml	View secret
3584396	Triggered	Generic Password	21378cf	.github/workflows/integration-test-fast.yml	View secret
3584396	Triggered	Generic Password	f6b4d69	.github/workflows/integration-test-fast.yml	View secret
3584396	Triggered	Generic Password	f6b4d69	.github/workflows/integration-test-fast-services.yml	View secret
3584396	Triggered	Generic Password	6a55f93	.github/workflows/integration-test-fast-services.yml	View secret
3584396	Triggered	Generic Password	de88ae4	.github/workflows/integration-test-fast-services.yml	View secret
3584396	Triggered	Generic Password	1d0bc0b	.github/workflows/integration-test-slow-services.yml	View secret

🛠 Guidelines to remediate hardcoded secrets

1. Understand the implications of revoking this secret by investigating where it is used in your code.
2. Replace and store your secrets safely. Learn here the best practices.
3. Revoke and rotate these secrets.
4. If possible, rewrite git history. Rewriting git history is not a trivial act. You might completely break other contributing developers' workflow and you risk accidentally deleting legitimate data.

To avoid such incidents in the future consider

• following these best practices for managing and storing secrets including API keys and other credentials
• install secret detection on pre-commit to catch secret before it leaves your machine and ease remediation.

---

^{🦉 GitGuardian detects secrets in your source code to help developers and security teams secure the modern development process. You are seeing this because you or someone else with access to this repository has authorized GitGuardian to scan your pull request.}

[github-actions]

E2E template updates in examples/e2e have been pushed.

[github-actions]

NLP template updates in examples/e2e_nlp have been pushed.

[github-actions]

LLM Finetuning template updates in examples/llm_finetuning have been pushed.

[github-actions]

Classification template updates in examples/mlops_starter have been pushed.

[github-actions]

E2E template updates in examples/e2e have been pushed.

[github-actions]

NLP template updates in examples/e2e_nlp have been pushed.

[bcdurak]

Left a few more comments and there were a few open still.