(fix): HTTP error handling in VC API #4606

protocolwhisper · 2023-08-10T20:45:41Z

Issue Addressed

Closes #4597

Proposed Changes

Updated Handler Return Types: Handlers now return Response directly
Refactored blocking_signed_json_task function

Additional Info

No

CLAassistant · 2023-08-10T20:45:46Z

All committers have signed the CLA.

protocolwhisper · 2023-08-14T18:09:09Z

@michaelsproul, since this PR got merged, shall I update to use the function convert_rejection<T> as a crate in this scope?

validator_client/src/http_api/mod.rs

michaelsproul · 2023-08-17T04:50:40Z

@michaelsproul, since #4595 got merged, shall I update to use the function convert_rejection as a crate in this scope?

I think it's OK to duplicate the definition for now. Or we could move convert_rejection into warp_utils.

jimmygchen · 2023-08-18T03:04:23Z

@protocolwhisper looks like there are a few CI failures, would you mind taking a look?

This reverts commit 24fa06b.

protocolwhisper · 2023-08-19T05:19:15Z

@michaelsproul @jimmygchen Hello! I'm currently facing an issue. I've made changes to the function as follows:
However, after making these changes, I'm encountering numerous errors, and I'm not sure where to start in order to resolve them. I'd truly appreciate some guidance on this matter.

pub async fn blocking_signed_json_task<S, F, T>(signer: S, func: F) -> Response<Body>
where
    S: Fn(&[u8]) -> String,
    F: FnOnce() -> Result<T, Rejection> + Send + 'static,
    T: Reply + Send + 'static,
{
    // Get the Result from the blocking task
    let result = warp_utils::task::blocking_task(func).await;

    // Convert the result using your convert_rejection function
    let mut response = convert_rejection(result).await;

    // Process the response for the signature
    let body: &Vec<u8> = response.body(); // NOTE: Ensure `body` method exists and returns a Vec<u8>
    let signature = signer(body);
    let header_value = HeaderValue::from_str(&signature).expect("hash can be encoded as header");
    response.headers_mut().append("Signature", header_value);

    response
}

jimmygchen · 2023-08-21T01:11:36Z

validator_client/src/http_api/mod.rs

+///
+/// This function should *always* be used to convert rejections into responses. This prevents warp
+/// from trying to backtrack in strange ways. See: https://github.com/sigp/lighthouse/issues/3404
+pub async fn convert_with_header<T: Reply>(res: Result<T, warp::Rejection>) -> resp {


It seems like this function isn't used anywhere and can be removed? (There are some compilation errors as well)

jimmygchen · 2023-08-21T01:20:27Z

validator_client/src/http_api/mod.rs

-            response
-        })
+    let result = warp_utils::task::blocking_task(func).await;
+    let conversion = convert_rejection(result).await; // It handles the rejection


I'm not following the logic here, convert_rejection isn't available in the scope, perhaps you're trying to call the convert_with_header you defined above?

Right, I’ve already modified it but haven’t uploaded the last commit. I’m using ‘convert_with_header’ instead of ‘convert_rejection

I've made significant progress. I'll check tomorrow to see if only the .then is causing the failure, and then I'll modify the handlers that call blocking_signed_task :)

jimmygchen · 2023-08-21T01:23:36Z

@protocolwhisper I think it's best to go through the compiler errors - the explanations are usually pretty clear, there are a few easy ones to fix, e.g. method not found, variable not found, I'd suggest to fix those first if you're not sure where to start. I've pointed out 2 of these in the above comments, let us know if there's any specific error that you're struggling to resolve.

protocolwhisper · 2023-08-22T04:49:10Z

@michaelsproul @jimmygchen I've updated the latest PR with the necessary changes and it now passes the tests for the module. Let me know your feedback :) .

protocolwhisper · 2023-08-28T00:42:45Z

Hello 👋🏽, can I get your comments on this PR? I’d like to make changes since the function’s signature has been modified. @michaelsproul @jimmygchen

jimmygchen · 2023-08-28T01:33:34Z

Hi @protocolwhisper, thanks for the update!

It looks like @michaelsproul has triggered a CI run, and there are some CI failures possibly related to the changes, would you mind taking a look please?

failing tests: https://github.com/sigp/lighthouse/actions/runs/5934547690/job/16091676680?pr=4606
clippy issues: https://github.com/sigp/lighthouse/actions/runs/5934547690/job/16091678636?pr=4606 (you can run make lint locally to check as well)

protocolwhisper · 2023-08-28T02:50:09Z

Hi @protocolwhisper, thanks for the update!

It looks like @michaelsproul has triggered a CI run, and there are some CI failures possibly related to the changes, would you mind taking a look please?

failing tests: https://github.com/sigp/lighthouse/actions/runs/5934547690/job/16091676680?pr=4606

clippy issues: https://github.com/sigp/lighthouse/actions/runs/5934547690/job/16091678636?pr=4606 (you can run make lint locally to check as well)

Gotcha, I corrected the lint but since we changed the signature from pub async fn blocking_signed_json_task<S, F, T>( signer: S, func: F, ) -> Result<impl warp::Reply, warp::Rejection> to pub async fn blocking_signed_json_task<S, F, T>(signer: S, func: F) -> resp is this why the test are failing? So I should focus on those changes or the tests? I'm a little lost. When trying to reproduce I'm getting this :

This reverts commit f05c3fb.

jimmygchen · 2023-10-20T06:37:06Z

Feel free to ignore the test failure, it seems to be related to an issue we discovered earlier today, which is now being addressed in #4868. I'll re-run the fail jobs now.

common/warp_utils/src/task.rs

validator_client/src/http_api/mod.rs

jimmygchen · 2023-12-07T05:24:05Z

validator_client/src/http_api/mod.rs

            response
-        })
+        }


This is bit isn't really doing what the function is advertised to do; I think it probably falls under the responsibility of the blocking_signed_json_task function instead as it describes here:

lighthouse/validator_client/src/http_api/mod.rs

Lines 1277 to 1280 in f77a5a0

/// Executes `func` in blocking tokio task (i.e., where long-running tasks are permitted).

/// JSON-encodes the return value of `func`, using the `signer` function to produce a signature of

/// those bytes.

pub async fn blocking_signed_json_task<S, F, T>(signer: S, func: F) -> Response<Vec<u8>>

Is it possible to move this block there to keep the convert_rejection only do what it's supposed to do?

Convert a warp Rejection into a Response.

If it's too much of an issue, we could consider renaming this function to something like convert_into_json_response?

validator_client/src/http_api/mod.rs

jimmygchen

Hi @protocolwhisper
Thanks for the updates, I've added a few more comments. Let me know if you have any questions.

protocolwhisper · 2024-01-26T05:13:01Z

Hi @jimmygchen
I just updated both functions, changed the output and updated the error pattern as the one from beacon node

protocolwhisper · 2024-04-10T16:20:06Z

Heyy , can I get any comments about this?

michaelsproul · 2024-04-12T03:45:33Z

Sorry @protocolwhisper but we've decided to delete the response signing altogether, so this change has been rolled into the PR that implements that: #5529

protocolwhisper · 2024-04-30T00:08:12Z

Got it, I'll close this issue then

jimmygchen · 2024-04-30T00:23:23Z

@protocolwhisper Sorry this one didn't work out, thanks for your help though! 🙏

(fix): HTTP error handling in VC API

5577b03

protocolwhisper mentioned this pull request Aug 10, 2023

(fix): HTTP error handling in VC API #4598

Closed

michaelsproul added val-client Relates to the validator client binary ready-for-review The code is ready for review labels Aug 14, 2023

michaelsproul requested changes Aug 17, 2023

View reviewed changes

validator_client/src/http_api/mod.rs Outdated Show resolved Hide resolved

michaelsproul added waiting-on-author The reviewer has suggested changes and awaits thier implementation. and removed ready-for-review The code is ready for review labels Aug 17, 2023

protocolwhisper and others added 2 commits August 17, 2023 16:14

Merge branch 'sigp:unstable' into unstable

db932da

(rebase): HTTP error handling in VC API

edecf5a

protocolwhisper added 2 commits August 18, 2023 23:43

(rebase) : Not working

24fa06b

Revert "(rebase) : Not working"

166f0f5

This reverts commit 24fa06b.

(rebase) : Http vc api

09a0eb6

jimmygchen reviewed Aug 21, 2023

View reviewed changes

protocolwhisper added 2 commits August 21, 2023 01:12

(rebase): block_signed_task , blocking_task

802e7ea

(Fix) : Rejection handling in VC API

a27b18e

protocolwhisper requested review from michaelsproul and jimmygchen August 22, 2023 21:12

protocolito added 2 commits August 28, 2023 20:51

(rebase) : Fix lint

f05c3fb

Revert "(rebase) : Fix lint"

b608d2e

This reverts commit f05c3fb.

protocolwhisper added 4 commits October 31, 2023 00:15

Merge branch 'sigp:unstable' into unstable

3271d4a

Merge branch 'sigp:unstable' into unstable

52bdc5b

Merge branch 'sigp:unstable' into unstable

f6671a1

Merge branch 'sigp:unstable' into unstable

f77a5a0