Allow header values to be selected from selector

[Sammo98]

It would be very useful to be able to select header values within chains.

E.g. (taking the example from the docs)

chains:
  username:
    source: !file
      path: ./username.txt
  password:
    source: !file
      path: ./password.txt
  auth_token:
    source: !request
      recipe: login
      # selector: $.token instead of JSONPath, there should be some method of obtaining header values.
      selector: headers.token

requests:
  # This returns a response like {"token": "abc123"}
  login: !request
    method: POST
    url: "https://myfishes.fish/login"
    body: |
      {
        "username": "{{chains.username}}",
        "password": "{{chains.password}}"
      }

  get_user: !request
    method: GET
    url: "https://myfishes.fish/current-user"
    authentication: !bearer "{{chains.auth_token}}"

Either enable selector to be able to access header values or something akin to header_selector which allows for this to happen.

[LucasPickering]

My preference for design on this would be to add an additional field to the !request chain source type. I'll using component as a placeholder here, I don't love that but it essentially means "which part of the response do you want to pull from". This should be an enum (i.e. there's a fixed set of values), so it's represented by a YAML tag rather than a plain string. It'll default to !body to keep current behavior, but you could override it to !headers (or potentially other values in the future, like !cookies). An important decision is whether headers is implicitly converted to a JSON object, or whether you have to specify a specific header that you want to pull from, then operate on that value. For these examples, let's assume the response headers look something like:

Content-Type: application/json
token: asdf

Treat header as an object

chains:
  auth_token:
    source: !request
      recipe: login
      component: !headers
    selector: $.token

Pros

• Allows you to operate on multiple headers
• Easy to access all or specific values of a repeated header (e.g. Set-Cookie)

Cons

• Feels untuitive - you're implicitly building a JSON object. It's very rare (in fact I don't think I've ever seen it) to have a JSON value in a header, so you're basically wrapping raw strings in a JSON object which is odd
• Implementation would be kinda grody, which I think is indicative of a clumsy design

Specify header and treat it as a string

chains:
  auth_token:
    source: !request
      recipe: login
      component: !headers token

Pros

• Simpler, more clear
• Doesn't require converting to JSON

Cons

• Can't access multiple headers in the same chain
  • I don't think this is a huge deal, typically headers don't need to interact with each other
• No means for accessing multiple values of a header (if the header is repeated)

Implementation

For either design, the work involved would be roughly:

• Add the new field to the Request variant of ChainSource: models.rs
  • I want to emphasize that the name component is a placeholder, I'm not sure of a good name for it
  • This would need a new enum to define the possible values, to begin with just Body and Headers
• Implement the filtering in render.rs
• Add tests in template.rs
• Add docs in chain_source.md

Other Thoughts

I'm not sure if it was you @Sammo98 or someone else who commented about this on Reddit, but I believe the original use case was to be able to operate on cookies. Using this design you could get at a returned cookie response, by nesting this chain inside a shell command that parses the cookie. That would be very clunky though so I 100% would like to have first class cookie support in Slumber, I just don't think it needs to be an explicit goal of this ticket.

[Sammo98]

@LucasPickering Thanks for the response, it was me on Reddit as well, probably should have given the context of wanting to grab the cookie.

Nonetheless I can see this as a useful extension to slumber for certain systems that may want to store headers. Take for instance grabbing a CSRF token (I may be wrong about this being in the headers, maybe first class cookie support would solve this).

Whatever the case, thank you for the pointers, I'm happy to take a crack at this myself if that works?

[LucasPickering]

@Sammo98 Yeah go for it. And CSRF is typically sent via the X-Csrf-Token header, so this would handle that

[Sammo98]

@LucasPickering

Linked an initial basic attempt at this, it is working but still need to add tests, will try and get to that later. Please let me know if any obvious issues.

(Note, this would be of my first OSS contributions so apologies if something looks awry!)