Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Copy project folder each job run #4001

Closed
wants to merge 1 commit into from

Conversation

AlanCoding
Copy link
Member

SUMMARY

This changes the behavior so that we copy the project folder for each job run.

Part of #282

ISSUE TYPE
  • Feature Pull Request
COMPONENT NAME
  • API
AWX VERSION
4.0.0
ADDITIONAL INFORMATION

@@ -148,18 +149,12 @@
register: doesRequirementsExist

- name: fetch galaxy roles from requirements.yml
command: ansible-galaxy install -r requirements.yml -p {{project_path|quote}}/roles/
command: ansible-galaxy install -r requirements.yml -p {{roles_destination|quote}}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably should figure out how to incorporate #3887

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That proposal has a lot of unanswered questions. Having multiple folders under roles/ seems like it wouldn't work right. I get the playbook-adjacent proposal, some variation of that might work.

If we do some variation of that request, I wouldn't anticipate much more trouble integrating it with this than with what we have now. It would be messy, involve searching around a couple places.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I certainly wouldn't be in love with keeping the requirements files in the same directory subtree that roles are getting installed to.

@softwarefactory-project-zuul
Copy link
Contributor

Build failed.

@@ -1217,9 +1220,6 @@ def run(self, pk, **kwargs):
module_args = ansible_runner.utils.args2cmdline(
params.get('module_args'),
)
else:
# otherwise, it's a playbook, so copy the project dir
copy_tree(cwd, os.path.join(private_data_dir, 'project'))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎉

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Still too early for celebration?

This wasn't removed, just moved to the pre_run_hook so that it happens for every job run, not just isolated.

@AlanCoding
Copy link
Member Author

I just realized the largest shortcoming of this - we have whitelisted the source project folder with bubblewrap. That needs to be removed.

@softwarefactory-project-zuul
Copy link
Contributor

Build succeeded.

@AlanCoding
Copy link
Member Author

Slight modification on my prior comment - the project folder is not whitelisted, cwd is, which used to be the project folder, but is no longer. However we already whitelist the private data dir. That means we redundantly whitelist it (ansible-runner removes duplicates, but subdirectories are not duplicates). So I pushed a commit for that.

before commit

[
  "bwrap",
  "--unshare-pid",
  "--dev-bind",
  "/",
  "/",
  "--proc",
  "/proc",
  "--bind",
  "/tmp/ansible_runner_pi_g930ge1w/tmpqnjwzhla",
  "/awx_devel/awx/job_status",
  "--bind",
  "/tmp/ansible_runner_pi_g930ge1w/tmpww6rluvi",
  "/etc/ssh",
  "--bind",
  "/tmp/ansible_runner_pi_g930ge1w/tmpqjdc1efc",
  "/projects",
  "--bind",
  "/tmp/ansible_runner_pi_g930ge1w/tmphktgb2kc",
  "/tmp",
  "--bind",
  "/tmp/ansible_runner_pi_g930ge1w/tmpspqpm3oc",
  "/var/lib/awx",
  "--bind",
  "/tmp/ansible_runner_pi_g930ge1w/tmpgia3_jtw",
  "/var/log",
  "--ro-bind",
  "/venv/ansible",
  "/venv/ansible",
  "--ro-bind",
  "/venv/awx",
  "/venv/awx",
  "--bind",
  "/tmp/awx_7909_7oy9v6cu",
  "/tmp/awx_7909_7oy9v6cu",
  "--bind",
  "/tmp/awx_7909_7oy9v6cu/project",
  "/tmp/awx_7909_7oy9v6cu/project",
  "--chdir",
  "/tmp/awx_7909_7oy9v6cu/project",
  "ansible-playbook",
  "-u",
  "root",
  "-i",
  "/tmp/awx_7909_7oy9v6cu/tmprnr6lc_j",
  "-e",
  "@/tmp/awx_7909_7oy9v6cu/env/extravars",
  "serial.yml"
]

after commit

[
  "bwrap",
  "--unshare-pid",
  "--dev-bind",
  "/",
  "/",
  "--proc",
  "/proc",
  "--bind",
  "/tmp/ansible_runner_pi_2tlhk0w4/tmpywfqwc7i",
  "/awx_devel/awx/job_status",
  "--bind",
  "/tmp/ansible_runner_pi_2tlhk0w4/tmp372woqjd",
  "/etc/ssh",
  "--bind",
  "/tmp/ansible_runner_pi_2tlhk0w4/tmplii4xxtr",
  "/projects",
  "--bind",
  "/tmp/ansible_runner_pi_2tlhk0w4/tmp1g16yc1v",
  "/tmp",
  "--bind",
  "/tmp/ansible_runner_pi_2tlhk0w4/tmpn9yi3kjr",
  "/var/lib/awx",
  "--bind",
  "/tmp/ansible_runner_pi_2tlhk0w4/tmpz0olqjv1",
  "/var/log",
  "--ro-bind",
  "/venv/ansible",
  "/venv/ansible",
  "--ro-bind",
  "/venv/awx",
  "/venv/awx",
  "--bind",
  "/tmp/awx_7911_klxsrslr",
  "/tmp/awx_7911_klxsrslr",
  "--chdir",
  "/tmp/awx_7911_klxsrslr/project",
  "ansible-playbook",
  "-u",
  "root",
  "-i",
  "/tmp/awx_7911_klxsrslr/tmpm8nukmnf",
  "-e",
  "@/tmp/awx_7911_klxsrslr/env/extravars",
  "serial.yml"
]

the job_args are already very complex and confusing, and I don't want to make them any more-so, so this is my final answer for this issue.

@softwarefactory-project-zuul
Copy link
Contributor

Build succeeded.

@AlanCoding AlanCoding changed the title [WIP] Copy project folder each job run Copy project folder each job run Jun 5, 2019
@softwarefactory-project-zuul
Copy link
Contributor

Build succeeded.

@AlanCoding
Copy link
Member Author

Rebased the PR, and it's ready for review now.

@softwarefactory-project-zuul
Copy link
Contributor

Build failed.

@AlanCoding
Copy link
Member Author

recheck

@softwarefactory-project-zuul
Copy link
Contributor

Build succeeded.

@softwarefactory-project-zuul
Copy link
Contributor

Build succeeded.

@softwarefactory-project-zuul
Copy link
Contributor

Build succeeded.

also do not add cwd to show_paths if it is
a subdirectory of private_data_dir, which
is already shown

pass the job private_data_dir to the local
project sync, and also add that directory
to the project sync show paths
@softwarefactory-project-zuul
Copy link
Contributor

Build succeeded.

@AlanCoding
Copy link
Member Author

Closing because I'm building on this for the rest of the feature and wasn't able to merge / rebase.

@AlanCoding AlanCoding closed this Jun 26, 2019
beeankha pushed a commit to beeankha/awx that referenced this pull request Dec 5, 2019
[3.6.2] Backport WFJT inventory rbac fix
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants