Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How to convert Multipart to Stream? #930

Open
oovm opened this issue Dec 17, 2024 · 0 comments
Open

How to convert Multipart to Stream? #930

oovm opened this issue Dec 17, 2024 · 0 comments
Labels
question Further information is requested

Comments

@oovm
Copy link

oovm commented Dec 17, 2024

I'm developing a file forwarding function that accepts a Multipart from poem and sends a new Multipart via reqwest.

This is the current implementation:

pub async fn upload_file(mut multipart: Multipart) -> Result<StatusCode, poem::Error> {
    let mut file = None;
    while let Some(field) = multipart.next_field().await? {
        let name = field.name().unwrap_or_default();
        match name {
            "file" => {
                file = Some(field.bytes().await?);
            }
            _ => {}
        }
    }

    if let Some(file) = file {
        let resp = reqwest::Client::new()
            .post("https://dashscope.aliyuncs.com/compatible-mode/v1/files")
            .bearer_auth(&"SECRET")
            .multipart(reqwest::multipart::Form::new().text("purpose", "file-extract").part("file", Part::bytes(file)))
            .send()
            .await
            .map_err(|e| poem::Error::new(e, StatusCode::BAD_REQUEST))?;
        Ok(resp.status())
    }
    else {
        Err(poem::Error::from_string("Missing file", StatusCode::BAD_REQUEST))
    }
}

It works fine, but when a lot of users upload at the same time, the server runs out of memory,

I want to change to streaming input and streaming output.

The problem I'm having is that poem returns an AsyncRead, but reqwest expects a Stream, and I don't know how to convert that.

pub async fn upload_file_stream(mut multipart: Multipart) -> Result<StatusCode, poem::Error> {
    let mut file = None;
    let mut file_name = String::new();
    while let Some(field) = multipart.next_field().await? {
        let name = field.name().unwrap_or_default();
        match name {
            "file" => {
                file = Some(field.into_async_read());   // <--------------- change input here
            }
            "name" => {
                file_name = field.text().await?;
            }
            _ => {}
        }
    }

    if let Some(file) = file {
        let body = reqwest::Body::wrap_stream(file); // <------------ not work here
        let reader = reqwest::multipart::Part::stream(body).file_name(file_name);
        let resp = reqwest::Client::new()
            .post("https://dashscope.aliyuncs.com/compatible-mode/v1/files")
            .bearer_auth(&"SECRET")
            .multipart(reqwest::multipart::Form::new().text("purpose", "file-extract").part("file", Part::bytes(file)))
            .send()
            .await
            .map_err(|e| poem::Error::new(e, StatusCode::BAD_REQUEST))?;
        Ok(resp.status())
    }
    else {
        Err(poem::Error::from_string("Missing file", StatusCode::BAD_REQUEST))
    }
}
@oovm oovm added the question Further information is requested label Dec 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
question Further information is requested
Projects
None yet
Development

No branches or pull requests

1 participant