Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

500 error occurs when Webhook is executed with SURVEY set in Template or Workflow #5062

Closed
sky-joker opened this issue Oct 22, 2019 · 9 comments
Closed

500 error occurs when Webhook is executed with SURVEY set in Template or Workflow #5062

sky-joker opened this issue Oct 22, 2019 · 9 comments
Assignees
@ryanpetrello @appuk
Labels
component:api type:bug

Comments

@sky-joker
Copy link

ISSUE TYPE
  • Bug Report
SUMMARY

If I set the URL of AWX Template or Workflow (SURVEY added) to GitHub Webhook and execute it, a 500 error will occur.

Error Response Body

{"detail":"A server error has occurred."}

If SURVEY is not added, no error will occur.

ENVIRONMENT
  • AWX version: 8.0.0
  • AWX install method: docker on linux
  • Ansible version: 2.8.6
  • Operating System: CentOS Linux release 7.7.1908 (Core)
  • Web Browser: Chrome
STEPS TO REPRODUCE
  1. Create of AWX Template(Workflow).
  2. Added SURVEY variable in Template(Workflow).
  3. Enable execution of Template(Workflow) via webhook.
  4. Set the template(Workflow) URL to the GitHub webhook.
  5. Push to GitHub repository.
EXPECTED RESULTS

I was expecting the Template to execute via webhook.

ACTUAL RESULTS

500 Internal Server Error

ADDITIONAL INFORMATION
@ryanpetrello
Copy link
Contributor

@sky-joker any chance you could share the contents of your awx logs (with a 500 error, you should see a verbose error, which should give us some clues about what's wrong).

@sky-joker
Copy link
Author

Hi, @ryanpetrello

The error log is as follows.


Oct 22 11:50:11 ip-192-168-0-37 journal:
Oct 22 11:50:11 ip-192-168-0-37 journal: 2019-10-22 11:50:11,989 DEBUG    awx.api.views.webhooks header signature: b'bf7e8590fc3e6dbf96d6fb1e4f04ca7c6468e5ba'
Oct 22 11:50:11 ip-192-168-0-37 journal: 2019-10-22 11:50:11,990 DEBUG    awx.api.views.webhooks calculated signature: b'bf7e8590fc3e6dbf96d6fb1e4f04ca7c6468e5ba'
Oct 22 11:50:12 ip-192-168-0-37 journal: 2019-10-22 11:50:11,998 ERROR    django.request Internal Server Error: /api/v2/workflow_job_templates/11/github/
Oct 22 11:50:12 ip-192-168-0-37 journal: Traceback (most recent call last):
Oct 22 11:50:12 ip-192-168-0-37 journal:  File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/django/core/handlers/exception.py", line 34, in inner
Oct 22 11:50:12 ip-192-168-0-37 journal:    response = get_response(request)
Oct 22 11:50:12 ip-192-168-0-37 journal:  File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/django/core/handlers/base.py", line 115, in _get_response
Oct 22 11:50:12 ip-192-168-0-37 journal:    response = self.process_exception_by_middleware(e, request)
Oct 22 11:50:12 ip-192-168-0-37 journal:  File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/django/core/handlers/base.py", line 113, in _get_response
Oct 22 11:50:12 ip-192-168-0-37 journal:    response = wrapped_callback(request, *callback_args, **callback_kwargs)
Oct 22 11:50:12 ip-192-168-0-37 journal:  File "/usr/lib64/python3.6/contextlib.py", line 52, in inner
Oct 22 11:50:12 ip-192-168-0-37 journal:    return func(*args, **kwds)
Oct 22 11:50:12 ip-192-168-0-37 journal:  File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/django/views/decorators/csrf.py", line 54, in wrapped_view
Oct 22 11:50:12 ip-192-168-0-37 journal:    return view_func(*args, **kwargs)
Oct 22 11:50:12 ip-192-168-0-37 journal:  File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/django/views/generic/base.py", line 71, in view
Oct 22 11:50:12 ip-192-168-0-37 journal:    return self.dispatch(request, *args, **kwargs)
Oct 22 11:50:12 ip-192-168-0-37 journal:  File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/awx/api/generics.py", line 297, in dispatch
Oct 22 11:50:12 ip-192-168-0-37 journal:    return super(APIView, self).dispatch(request, *args, **kwargs)
Oct 22 11:50:12 ip-192-168-0-37 journal:  File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/rest_framework/views.py", line 495, in dispatch
Oct 22 11:50:12 ip-192-168-0-37 journal:    response = self.handle_exception(exc)
Oct 22 11:50:12 ip-192-168-0-37 journal:  File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/rest_framework/views.py", line 455, in handle_exception
Oct 22 11:50:12 ip-192-168-0-37 journal:    self.raise_uncaught_exception(exc)
Oct 22 11:50:12 ip-192-168-0-37 journal:  File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/rest_framework/views.py", line 492, in dispatch
Oct 22 11:50:12 ip-192-168-0-37 journal:    response = handler(request, *args, **kwargs)
Oct 22 11:50:12 ip-192-168-0-37 journal:  File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/django/views/decorators/csrf.py", line 54, in wrapped_view
Oct 22 11:50:12 ip-192-168-0-37 journal:    return view_func(*args, **kwargs)
Oct 22 11:50:12 ip-192-168-0-37 journal:  File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/awx/api/views/webhooks.py", line 163, in post
Oct 22 11:50:12 ip-192-168-0-37 journal:    new_job = obj.create_unified_job(**kwargs)
Oct 22 11:50:12 ip-192-168-0-37 journal:  File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/awx/main/models/workflow.py", line 472, in create_unified_job
Oct 22 11:50:12 ip-192-168-0-37 journal:    workflow_job = super(WorkflowJobTemplate, self).create_unified_job(**kwargs)
Oct 22 11:50:12 ip-192-168-0-37 journal:  File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/awx/main/models/unified_jobs.py", line 341, in create_unified_job
Oct 22 11:50:12 ip-192-168-0-37 journal:    encrypt_dict(kwargs.get('extra_vars', {}), password_list)
Oct 22 11:50:12 ip-192-168-0-37 journal:  File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/awx/main/utils/encryption.py", line 140, in encrypt_dict
Oct 22 11:50:12 ip-192-168-0-37 journal:    encrypt_fields = set(data.keys()).intersection(fields)
Oct 22 11:50:12 ip-192-168-0-37 journal: AttributeError: 'str' object has no attribute 'keys'
Oct 22 11:50:12 ip-192-168-0-37 journal: 2019-10-22 11:50:11,998 ERROR    django.request Internal Server Error: /api/v2/workflow_job_templates/11/github/
Oct 22 11:50:12 ip-192-168-0-37 journal: Traceback (most recent call last):
Oct 22 11:50:12 ip-192-168-0-37 journal:  File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/django/core/handlers/exception.py", line 34, in inner
Oct 22 11:50:12 ip-192-168-0-37 journal:    response = get_response(request)
Oct 22 11:50:12 ip-192-168-0-37 journal:  File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/django/core/handlers/base.py", line 115, in _get_response
Oct 22 11:50:12 ip-192-168-0-37 journal:    response = self.process_exception_by_middleware(e, request)
Oct 22 11:50:12 ip-192-168-0-37 journal:  File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/django/core/handlers/base.py", line 113, in _get_response
Oct 22 11:50:12 ip-192-168-0-37 journal:    response = wrapped_callback(request, *callback_args, **callback_kwargs)
Oct 22 11:50:12 ip-192-168-0-37 journal:  File "/usr/lib64/python3.6/contextlib.py", line 52, in inner
Oct 22 11:50:12 ip-192-168-0-37 journal:    return func(*args, **kwds)
Oct 22 11:50:12 ip-192-168-0-37 journal:  File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/django/views/decorators/csrf.py", line 54, in wrapped_view
Oct 22 11:50:12 ip-192-168-0-37 journal:    return view_func(*args, **kwargs)
Oct 22 11:50:12 ip-192-168-0-37 journal:  File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/django/views/generic/base.py", line 71, in view
Oct 22 11:50:12 ip-192-168-0-37 journal:    return self.dispatch(request, *args, **kwargs)
Oct 22 11:50:12 ip-192-168-0-37 journal:  File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/awx/api/generics.py", line 297, in dispatch
Oct 22 11:50:12 ip-192-168-0-37 journal:    return super(APIView, self).dispatch(request, *args, **kwargs)
Oct 22 11:50:12 ip-192-168-0-37 journal:  File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/rest_framework/views.py", line 495, in dispatch
Oct 22 11:50:12 ip-192-168-0-37 journal:    response = self.handle_exception(exc)
Oct 22 11:50:12 ip-192-168-0-37 journal:  File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/rest_framework/views.py", line 455, in handle_exception
Oct 22 11:50:12 ip-192-168-0-37 journal:    self.raise_uncaught_exception(exc)
Oct 22 11:50:12 ip-192-168-0-37 journal:  File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/rest_framework/views.py", line 492, in dispatch
Oct 22 11:50:12 ip-192-168-0-37 journal:    response = handler(request, *args, **kwargs)
Oct 22 11:50:12 ip-192-168-0-37 journal:  File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/django/views/decorators/csrf.py", line 54, in wrapped_view
Oct 22 11:50:12 ip-192-168-0-37 journal:    return view_func(*args, **kwargs)
Oct 22 11:50:12 ip-192-168-0-37 journal:  File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/awx/api/views/webhooks.py", line 163, in post
Oct 22 11:50:12 ip-192-168-0-37 journal:    new_job = obj.create_unified_job(**kwargs)
Oct 22 11:50:12 ip-192-168-0-37 journal:  File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/awx/main/models/workflow.py", line 472, in create_unified_job
Oct 22 11:50:12 ip-192-168-0-37 journal:    workflow_job = super(WorkflowJobTemplate, self).create_unified_job(**kwargs)
Oct 22 11:50:12 ip-192-168-0-37 journal:  File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/awx/main/models/unified_jobs.py", line 341, in create_unified_job
Oct 22 11:50:12 ip-192-168-0-37 journal:    encrypt_dict(kwargs.get('extra_vars', {}), password_list)
Oct 22 11:50:12 ip-192-168-0-37 journal:  File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/awx/main/utils/encryption.py", line 140, in encrypt_dict
Oct 22 11:50:12 ip-192-168-0-37 journal:    encrypt_fields = set(data.keys()).intersection(fields)
Oct 22 11:50:12 ip-192-168-0-37 journal: AttributeError: 'str' object has no attribute 'keys'
Oct 22 11:50:12 ip-192-168-0-37 journal: [pid: 163|app: 0|req: 12/82] 192.30.252.99 () {44 vars in 693 bytes} [Tue Oct 22 11:50:11 2019] POST /api/v2/workflow_job_templates/11/github/ => generated 41 bytes in 51 msecs (HTTP/1.1 500) 5 headers in 185 bytes (1 switches on core 0)
Oct 22 11:50:12 ip-192-168-0-37 journal: 192.30.252.99 - - [22/Oct/2019:11:50:12 +0000] "POST /api/v2/workflow_job_templates/11/github/ HTTP/1.1" 500 41 "-" "GitHub-Hookshot/795f661" "-"

thanks

@ryanpetrello
Copy link
Contributor

@sky-joker any chance you could share the output of /api/v2/job_templates/N/survey_spec/

...where N is the ID of the job template where you've defined the survey?

@ryanpetrello ryanpetrello self-assigned this Oct 22, 2019
@ryanpetrello
Copy link
Contributor

Okay, I'm able to reproduce this by:

  1. Setting up a JobTemplate that has a survey with a "password" field (no default provided).
  2. Launching this JobTemplate using a GitHub webhook.

@ryanpetrello
Copy link
Contributor

@sky-joker,

Any chance this diff helps for you?

diff --git a/awx/api/views/webhooks.py b/awx/api/views/webhooks.py
index e3ed6e64c9..dbdd3916bc 100644
--- a/awx/api/views/webhooks.py
+++ b/awx/api/views/webhooks.py
@@ -151,13 +151,13 @@ class WebhookReceiverBase(APIView):
                 'webhook_credential': obj.webhook_credential,
                 'webhook_guid': event_guid,
             },
-            'extra_vars': json.dumps({
+            'extra_vars': {
                 'tower_webhook_event_type': event_type,
                 'tower_webhook_event_guid': event_guid,
                 'tower_webhook_event_ref': event_ref,
                 'tower_webhook_status_api': status_api,
                 'tower_webhook_payload': request.data,
-            })
+            }
         }

         new_job = obj.create_unified_job(**kwargs)

@sky-joker
Copy link
Author

Okay, I'm able to reproduce this by:

  1. Setting up a JobTemplate that has a survey with a "password" field (no default provided).
  2. Launching this JobTemplate using a GitHub webhook.

Yes :)
The SURVEY I set is as follows.

{
    "name": "",
    "description": "",
    "spec": [
        {
            "question_name": "GITHUB TOKEN",
            "question_description": "",
            "required": true,
            "type": "password",
            "variable": "GitHub_TOKEN",
            "min": 0,
            "max": 64,
            "default": "$encrypted$",
            "choices": "",
            "new_question": true
        }
    ]
}

@ryanpetrello
Copy link
Contributor

I think something like this should help:

#5062 (comment)

I can put together a PR shortly.

@sky-joker
Copy link
Author

Applying this patch(#5062 (comment)) resolved this issue :)
Thank you @ryanpetrello for quick and polite response!

@dsesami dsesami self-assigned this Oct 22, 2019
@appuk appuk assigned appuk and unassigned dsesami Oct 22, 2019
@appuk
Copy link
Member

appuk commented Oct 25, 2019

Verified this issue, the job is launched successfully now. Hence, closing the issue

@appuk appuk closed this as completed Oct 25, 2019
ryanpetrello added a commit to ryanpetrello/awx that referenced this issue Oct 29, 2019
@ryanpetrello
fix a bug that breaks webhook launches when a survey is in use
560396c 
see: ansible#5062
ryanpetrello added a commit to ryanpetrello/awx that referenced this issue Oct 29, 2019
@ryanpetrello
fix a bug that breaks webhook launches when a survey is in use
ab6322a 
see: ansible#5062
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@ryanpetrello ryanpetrello

@appuk appuk

Labels
component:api type:bug
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@ryanpetrello @appuk @sky-joker @dsesami @kdelee