Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix error invalid URL escape "% ^" #736

Open
ParthaI opened this issue Jan 23, 2024 · 0 comments
Open

Fix error invalid URL escape "% ^" #736

ParthaI opened this issue Jan 23, 2024 · 0 comments
Labels
bug Something isn't working

Comments

@ParthaI
Copy link

ParthaI commented Jan 23, 2024

Describe the bug
The UnmarshalYAML() leveraged yaml.Unmarshal() from the Go package github.com/ghodss/yaml and url.QueryUnescape from net/url to decode and parse YAML into JSON.
The function QueryUnescape returns an error if any % is not followed by two hexadecimal digits.
Additionally, the function UnmarshalYAML was also throwing an error for the code block in the template.

Steampipe version (steampipe -v)
Example: Steampipe v0.21.3

Plugin version (steampipe plugin list)
Example: Steampipe plugin AWS 0.129.0

To reproduce
Try to parse the following template content:

CopyZipsFunction:
    Type: AWS::Lambda::Function
    Properties:
      Description: Copies objects from a source S3 bucket to a destination
      Handler: index.handler
      Runtime: python3.8
      Role: !GetAtt "CopyZipsRole.Arn"
      Timeout: 240
      Code:
        ZipFile: |
          import json
          import logging
          import threading
          import boto3
          import cfnresponse
          def copy_objects(source_bucket, dest_bucket, prefix, objects):
              s3 = boto3.client('s3')
              for o in objects:
                  key = prefix + o
                  copy_source = {
                      'Bucket': source_bucket,
                      'Key': key
                  }
                  print('copy_source: %s' % copy_source)
                  print('dest_bucket = %s'%dest_bucket)
                  print('key = %s' %key)
                  s3.copy_object(CopySource=copy_source, Bucket=dest_bucket,
                        Key=key)
          def delete_objects(bucket, prefix, objects):
              s3 = boto3.client('s3')
              objects = {'Objects': [{'Key': prefix + o} for o in objects]}
              s3.delete_objects(Bucket=bucket, Delete=objects)
          def timeout(event, context):
              logging.error('Execution is about to time out, sending failure response to CloudFormation')
              cfnresponse.send(event, context, cfnresponse.FAILED, {}, None)
          def handler(event, context):
              # make sure we send a failure to CloudFormation if the function
              # is going to timeout
              timer = threading.Timer((context.get_remaining_time_in_millis()
                        / 1000.00) - 0.5, timeout, args=[event, context])
              timer.start()
              print('Received event: %s' % json.dumps(event))
              status = cfnresponse.SUCCESS
              try:
                  source_bucket = event['ResourceProperties']['SourceBucket']
                  dest_bucket = event['ResourceProperties']['DestBucket']
                  prefix = event['ResourceProperties']['Prefix']
                  objects = event['ResourceProperties']['Objects']
                  if event['RequestType'] == 'Delete':
                      delete_objects(dest_bucket, prefix, objects)
                  else:
                      copy_objects(source_bucket, dest_bucket, prefix, objects)
              except Exception as e:
                  logging.error('Exception: %s' % e, exc_info=True)
                  status = cfnresponse.FAILED
              finally:
                  timer.cancel()
                  cfnresponse.send(event, context, status, {}, None)

Expected behavior
We should be able to parse the YAML content to JSON without any error.

Additional context
N/A

@ParthaI ParthaI added the bug Something isn't working label Jan 23, 2024
@ParthaI ParthaI transferred this issue from turbot/steampipe-plugin-aws Jan 23, 2024
@ParthaI ParthaI linked a pull request Jan 23, 2024 that will close this issue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant