How to retrieve multiple files continuously

[HuanjunKong]

After checking #562 , I post 10 images (500KB) to gin, code :

    func extractImgs(c *gin.Context) {
        prefix := "prefix"
        for ix := 0; ix < 10; ix++ {
            file, _, err := c.Request.FormFile(prefix + strconv.Itoa(ix))
            if err != nil {
                // do sth.
                return
            }
            // call face recognition algorithm
            face_rec_async(file)
        }
    }

c.Request.FormFile cost a lot time. It seems like that FormFile returns file after retrieved all 10 files.

Face recognition is time-consuming algorithm, this workflow should be better:

retrieve first file -> recognize face -> retrieve second file -> recognition -> ... ,

just like a link list.

My question is how to retrieve files like traverse linked list.

[milne-dev]

I have found a way that works by turning the multipart form into a reader. Here is a code snippet without full error handling:

reader, _ := c.Request.MultipartReader()

for {
    if part, err := reader.NextPart(); err == io.EOF {
        break
    }

    // part implements io.Reader
}

[javierprovecho]

@HuanjunKong sorry for late reply, .FormFile() is a wrap from net/http. If the form hasn't been parsed yet, it process all files by creating for each one a File object. After looking through the code, here should not be any delay after calling for the first file.

@Michael77 is another solution but I see that you could not access directly a file by name.

[jeffkoncz]

I had to do something similar--transferring multiple file uploads to S3 in parallel. Adding to @Michael77 method if you need access to the original filename you can use the mime package.

disposition, params, err := mime.ParseMediaType(part.Header.Get("Content-Disposition"))
if err != nil {
  break
}
filename := params["filename"]

[svperfecta]

hey @jeffkoncz any chance you have a larger example of this? I'm a bit stuck, doing the exact same thing.

[appleboy]

@genexp You can see the example https://github.com/gin-gonic/gin/blob/develop/examples/upload-file/multiple/main.go

[jeffkoncz]

@genexp Here's some code that is specific to transferring multiple file uploads to S3. Hopefully it can get you unstuck :) :

func MultipartUploadHandler(c *gin.Context) {
	multipart, err := c.Request.MultipartReader()
	if err != nil {
		log.Fatalln("Failed to create MultipartReader", err)
	}

	for {
		mimePart, err := multipart.NextPart()
		if err == io.EOF {
			break
		}
		if err != nil {
			log.Printf("Error reading multipart section: %v", err)
			break
		}
		disposition, params, err := mime.ParseMediaType(mimePart.Header.Get("Content-Disposition"))
		if err != nil {
			log.Printf("Invalid Content-Disposition: %v", err)
			break
		}

		// S3 Upload Manager
		uploader := s3manager.NewUploader(session.New(&aws.Config{Region: aws.String("us-west-2")}))
		result, err := uploader.Upload(&s3manager.UploadInput{
			Body:   mimePart,
			Bucket: aws.String("yourbucket"),
			Key:    aws.String(params["filename"]),
			ContentType: aws.String(mimePart.Header.Get("Content-Type")),
			ACL: aws.String("public-read"),
		})
		if err != nil {
			log.Fatalln("Failed to upload to S3", err)
		}
		
		c.JSON(http.StatusOK, gin.H{
			"location": result.Location,
		})
	}
}

[svperfecta]

@jeffkoncz Yeah, this is awesome. Thanks so much!

[geoherna]

Awesome! Thanks!