Support of File Uploads

[dtrinh100]

Are there any plans to support file uploads via: https://github.com/jaydenseric/graphql-multipart-request-spec?

[vetcher]

Why you want to upload files via graphql?

[dtrinh100]

@vetcher I would like an easy way to do file uploads in GraphQL, using a spec that works with the Apollo Client.

[icco]

I too would like this, just to keep my server only having one endpoint, and to save on duplication of things like authentication logic.

[sneko]

I'm also interested 😄

Does anyone have seen a Go implementation of this specification?

Thanks,

cc @vektah @vvakame

[MtthwBrwng]

I'm interested as-well.

@sneko The closest thing I've seen would be this project graphql-upload. Though it seems to be made for graphql-go.

[dehypnosis]

I made a hacky workaround to use gqlgen graphql handler.... while handling multipart/form-data request. i am using gin-gonic web server framework and this workaround works with apollo-client well.
i am waiting for this feature..!

A middleware factory which parse multipart/form-data request and remember the file streams, and then transform the original HTTP request of multipart/form-data as application/json content type.

package handler

import (
    "bytes"
    "context"
    "encoding/json"
    "github.com/99designs/gqlgen/graphql"
    "github.com/gin-gonic/gin"
    "io/ioutil"
    "log"
    "strings"
   ...
)

/* ref: GraphQL multipart file upload spec: https://github.com/jaydenseric/graphql-multipart-request-spec
this function transform multipart/form-data post request into application/json request and extract file streams
example of multipart/form-data request:

// single operation
curl localhost:3001/graphql \
  -F operations='{ "query": "mutation ($file: Upload!) { singleUpload(file: $file) { id } }", "variables": { "file": null } }' \
  -F map='{ "0": ["variables.file"] }' \
  -F [email protected]

// batched operations
curl localhost:3001/graphql \
  -F operations='[{ "query": "mutation ($file: Upload!) { singleUpload(file: $file) { id } }", "variables": { "file": null } }, { "query": "mutation($files: [Upload!]!) { multipleUpload(files: $files) { id } }", "variables": { "files": [null, null] } }]' \
  -F map='{ "0": ["0.variables.file"], "1": ["1.variables.files.0"], "2": ["1.variables.files.1"] }' \
  -F [email protected] \
  -F [email protected] \
  -F [email protected]
*/

type operation struct {
    Query         string                 `json:"query"`
    OperationName string                 `json:"operationName"`
    Variables     map[string]interface{} `json:"variables"`
}

func parseMultipartRequest(c *gin.Context) (f graphql.RequestMiddleware) {
    f = graphql.DefaultRequestMiddleware

    if c.ContentType() != "multipart/form-data" {
        return
    }
    form, err := c.MultipartForm()
    if err != nil {
        log.Println(err.Error())
        return
    }

    operations := make([]operation, 0)
    for _, operationJSON := range form.Value["operations"] {
        op := operation{}
        err := json.Unmarshal([]byte(operationJSON), &op)
        if err != nil {
            log.Println(err.Error())
            continue
        }
        operations = append(operations, op)
    }

    fileMap := make(map[string][]string)
    err = json.Unmarshal([]byte(form.Value["map"][0]), &fileMap)
    if err != nil {
        log.Println(err.Error())
        return
    }

    files := make(map[string]*model.UploadingFile)
    for fileKey, _ := range fileMap {
        fileHeader, err := c.FormFile(fileKey)
        if err != nil {
            log.Println(fileKey, err.Error())
            continue
        }
        fileReader, err := fileHeader.Open()
        if err != nil {
            log.Println(err.Error())
            continue
        }
        files[fileKey] = &model.UploadingFile{
            Content: fileReader,
            Name: fileHeader.Filename,
            Size: fileHeader.Size,
        }
    }

    // now change the body and content-type
    // it is hacky-way until gqlgen supports multipart/form-data request by self
    var doc interface{} = operations
    if len(operations) == 1 {
        doc = operations[0]
    }
    fakeBody, err := json.Marshal(doc)
    if err != nil {
        log.Println(err.Error())
        return
    }
    c.Request.Header.Set("content-type", "application/json")
    c.Request.Body = ioutil.NopCloser(bytes.NewReader(fakeBody))

    f = func(ctx context.Context, next func(ctx context.Context) []byte) []byte {
        req := graphql.GetRequestContext(ctx)
        variables := req.Variables
        for fileKey, variableFields := range fileMap {
            for _, variableField := range variableFields {
                fields := strings.Split(variableField, ".")
                if len(fields) <= 1 || fields[0] != "variables" { // respect spec: https://github.com/jaydenseric/graphql-multipart-request-spec
                    log.Println("invalid variable field in map", variableField)
                    continue
                }
                var obj = variables
                lastIndex := len(fields) - 2
                for index, path := range fields[1:] {
                    if _, ok := obj[path]; ok {
                        if index == lastIndex {
                            // set file
                            if obj[path], ok = files[fileKey]; ok {
                                log.Println("set file", variableField, files[fileKey])
                            }
                            break
                        } else if objInObj, ok := obj[path].(map[string]interface{}); ok {
                            obj = objInObj
                            continue
                        }
                    }

                    log.Println("invalid variable field in map", variableField)
                    break
                }
            }
        }
        return next(ctx)
    }
    return
}

A scalar type which represent file uploading stream

package model

import (
    "io"
)

// scalar type
type UploadingFile struct {
    Content io.Reader
    Name string
    Size int64
}

// GraphQL JSON -> UploadingFile
func (f *UploadingFile) UnmarshalGQL(gql interface{}) (err error) {
    // this scalar type will be unmarshaled while parsing multipart/form-data body
    // ref: ../handler/multipart.go
    if v, ok := gql.(*UploadingFile); ok {
        if v != nil {
            *f = *v
        }
    }
    return
}

// UploadingFile -> GraphQL JSON (RFC3339)
func (f UploadingFile) MarshalGQL(w io.Writer) {
    w.Write([]byte("null"))
}

Now attach the middleware into graphql handler

package handler

import (
    "context"
    "github.com/99designs/gqlgen/graphql"
    graphqlHandler "github.com/99designs/gqlgen/handler"
    ...
)


...

        gqlHandler := graphqlHandler.GraphQL(
            // create root schema with context derived values
            schema.NewExecutableSchema(schema.Config{
                Resolvers:  schema.NewResolverRoot(viewer, locale),
                Directives: schema.NewDirectiveRoot(viewer),
            }),

            ....

            // multipart/form-data parsing middleware
            graphqlHandler.RequestMiddleware(parseMultipartRequest(c)),
           ...

           // other options
        )

       gqlHandler.ServeHTTP(c.Writer, c.Request)

...

[MShoaei]

@dehypnosis would you try to make a pull request for this?

[hantonelli]

Hi @MShoaei I hope to be submiting a pull request this week. I already have it working, but I need to add some test to it. I also added support to submit the upload with a request mutation payload, so you can submit some extra fields with the file.

[mathewbyrne]

@hantonelli if you're planning a PR can you please ensure it's against next? We aren't accepting new features against master currently.

[hantonelli]

@mathewbyrne Sure!

[mathewbyrne]

Also, for a feature of this size it would be really great if you could write up a proposal as a separate issue with your planned approach for the development team to look at. It's really difficult to accept large PRs without a bit of context beforehand.

Sorry we're working on getting some contribution guidelines in place soon that will codify some of this.

[hantonelli]

@mathewbyrne Sure, I will.

[robert-zaremba]

Really looking forward for this feature!

[hantonelli]

@robert-zaremba @mathewbyrne I'm still working on the PR and proposal, hope to finish it soon and, that they would be good enough or at least a good start to add this feature :)

[smithaitufe]

I have been using this for about a year now.
It is just a middleware.

You can try it too.

[ghost]

@smithaitufe thanks for that !

@mathewbyrne Is this planned to be integrated into the gqlgen lib ?

[mathewbyrne]

It's not on our roadmap currently, but I think it makes sense to include at some point yes.

[hantonelli]

@mathewbyrne I finally find the time to make the proposal and the PR :)