[JAVA] Bug generating request body for array of files upload

[wshands]

Java code generated for request body for multi-part form data does not put the content of the files in request body, instead it places the file path and name in the request body.

Git clone and build Swagger-codegen version 3.0.5 via instructions on Github README

Swagger yaml openapi-upload-files.yaml

openapi: 3.0.0
info:
  title: OpenApi file upload request body
  version: "1.0.0"
  contact:
    name: Walt Shands
    email: [email protected]
tags:
        - name: upload_test
          description: test multipart upload
paths:
  /runs:
    post:
      summary: Upload an array of files.
      description: >-
        This endpoint uploads an array of files.

      operationId: UploadFiles
      responses:
        '200':
          description: ''
        '400':
          description: The request is malformed.
        '401':
          description: The request is unauthorized.
        '403':
          description: The requester is not authorized to perform this action.
        '500':
          description: An unexpected error occurred.

      tags:
        - UploadFilesService
      requestBody:
        content:
          multipart/form-data:
            schema:
              type: object
              properties:
                file_list:
                  type: array
                  items:
                    type: string
                    format: binary

Build client from OpenApi yaml with Swagger Codegen

java -jar <path>/swagger-codegen-cli.jar generate -l java -i <path>/openapi-upload-files.yaml

Java class used for generation of POST request

package io.arrayoffiles;

import java.io.File;
import java.util.ArrayList;
import java.util.List;
import io.swagger.client.ApiClient;
import io.swagger.client.ApiException;
import io.swagger.client.api.UploadFilesServiceApi;

public class PostArrayOfFiles {
    public static void main(String[] args) {
        UploadFilesServiceApi apiInstance = new UploadFilesServiceApi();
        try {
            ApiClient apiClient = apiInstance.getApiClient();
            //apiClient.setBasePath("http://0.0.0.0:8080");
            apiClient.setBasePath("http://swagger.io");

            List<File> uploadFileList = new ArrayList<File>();
            File testfile = new File("Users/waltershands/junk/test1.txt");
            uploadFileList.add(testfile);
            testfile = new File("Users/waltershands/junk/test2.txt");
            uploadFileList.add(testfile);

            apiInstance.uploadFiles(uploadFileList);
        } catch (ApiException var4) {
            System.err.println("Exception when calling uploadFileList");
            var4.printStackTrace();
        }
    }
}

Compile test program

javac -cp <path to generated swagger client>/target/swagger-java-client-1.0.0.jar <path>/src/main/java/io/arrayoffiles/PostArrayOfFiles.java 

Monitor POST requests in another terminal window

sudo tcpdump -A -s 0 'tcp port 80 and (((ip[2:2] - ((ip[0]&0xf)<<2)) - ((tcp[12]&0xf0)>>2)) != 0)'

Run test program

java -debug -cp <path to PostArrayOfFiles class>/target/classes:<path to generated swagger client>target/*:<path to other needed swagger jars>/target/lib/*  io.arrayoffiles.PostArrayOfFiles

Produces this POST request body (seen in tcpdump window):

....    wU5POST /runs HTTP/1.1
User-Agent: Swagger-Codegen/1.0.0/java
Content-Type: multipart/form-data; boundary=e2b790d8-0149-47ac-8740-755c53c982c3
Content-Length: 223
Host: swagger.io
Connection: Keep-Alive
Accept-Encoding: gzip

--e2b790d8-0149-47ac-8740-755c53c982c3
Content-Disposition: form-data; name="file_list"
Content-Length: 67

Users/waltershands/junk/test1.txt,Users/waltershands/junk/test2.txt
--e2b790d8-0149-47ac-8740-755c53c982c3--
...

There should have been two bodies with Content-Disposition that included a 'filename="test<#>.txt" and the contents of the file

ApiClient.java creates a correct body only if the instance of the formParams Object is a File, but in the test case it is a List of Files and so the correct body is not created.

...
1109     public RequestBody buildRequestBodyMultipart(Map<String, Object> formParams) {$
1110         MultipartBuilder mpBuilder = new MultipartBuilder().type(MultipartBuilder.FORM);$
1111         for (Entry<String, Object> param : formParams.entrySet()) {$
1112             if (param.getValue() instanceof File) {$
1113                 File file = (File) param.getValue();$
1114                 Headers partHeaders = Headers.of("Content-Disposition", "form-data; name=\"" + param.getKey() + "\"; filename=\"" + fi     le.getName() + "\"");$
1115                 MediaType mediaType = MediaType.parse(guessContentTypeFromFile(file));$
1116                 mpBuilder.addPart(partHeaders, RequestBody.create(mediaType, file));$

[sgreene]

Seeing this exact issue in 3.0.5 when attempting to upload multiple zip files. Server side receives a single multipart body when there should be one per file.

[ChrisR48]

Encountered this bug using swagger-codegen-cli-3.0.24 to generate a Client for a provided swagger-api.
The client should upload multiple (Image)-files via api call to the server. The api definition is similar to the provided one (only json instead of yaml).

When using the api call, the files are not processed by the server.

We found a workaround by using org.springframework.core.io.FileSystemResource.FileSystemResource for the call. For this we added a type mapping to the codegen call like: --type-mappings File=org.springframework.core.io.Resource

With this the typ mapping the method signiture changed from:

public String addImages(List<File> images, String uuid) throws RestClientException {

to:

public String addImages(List<org.springframework.core.io.Resource> images, String uuid) throws RestClientException {

Unfortunatly we ran into another error than we jackson was not able to serialize the call:

Exception in thread "AWT-EventQueue-0" org.springframework.http.converter.HttpMessageConversionException: Type definition error: [simple type, class sun.nio.ch.ChannelInputStream]; nested exception is com.fasterxml.jackson.databind.exc.InvalidDefinitionException: No serializer found for class sun.nio.ch.ChannelInputStream and no properties discovered to create BeanSerializer (to avoid exception, disable SerializationFeature.FAIL_ON_EMPTY_BEANS) (through reference chain: java.util.ArrayList[0]->org.springframework.core.io.FileSystemResource["inputStream"]) at org.springframework.http.converter.json.AbstractJackson2HttpMessageConverter.writeInternal(AbstractJackson2HttpMessageConverter.java:351) ...

This error is affected by the call to the formParams variable inside the generated client:

final MultiValueMap<String, Object> formParams = new LinkedMultiValueMap<String, Object>(); queryParams.putAll(apiClient.parameterToMultiValueMap(null, "uuid", uuid)); if (images != null) formParams.add("images", images);

When checked in detail (via debugger), not the Resourcefiles are added to the MultiValueMap, but the (Array)list itself is placed inside a List for the key "images".

When changing add to addAll, all files are beeing send to the server and can be processed there.

Unfortunately I am missing the experience with the swagger project to convert this into a PR, but maybe this will help others when facing the same problem.

[Cenz]

I generated code using swagger-codegen-cli-3.0.27 and found problems with mulipart. One of them is failing with file lists as described in this issue. The other is with serailizing object to JSON.

The JSON issue can be either resolved in the object's class toString() method -the generated code needs patching- or inside method ApiClient.parameterToString(Object) where a specific instanceof is required.

To fix the file list issue I patched generated class ApiClient as such. (I'm sure better and prettier code could be envisioned.)

    public RequestBody buildRequestBodyMultipart(Map<String, Object> formParams) {
      MultipartBuilder mpBuilder = new MultipartBuilder().type(MultipartBuilder.FORM);
      for (Entry<String, Object> param : formParams.entrySet()) {
        if (isFileList(param.getValue())) {
          @SuppressWarnings("unchecked")
          List<File> files = (List<File>) param.getValue();
          for (File file : files) {
            Headers partHeaders = Headers.of("Content-Disposition",
                "form-data; name=\"" + param.getKey() + "\"; filename=\"" + file.getName() + "\"");
            MediaType mediaType = MediaType.parse(guessContentTypeFromFile(file));
            mpBuilder.addPart(partHeaders, RequestBody.create(mediaType, file));
          }
        } else if (param.getValue() instanceof File) {
          File file = (File) param.getValue();
          Headers partHeaders = Headers.of("Content-Disposition",
              "form-data; name=\"" + param.getKey() + "\"; filename=\"" + file.getName() + "\"");
          MediaType mediaType = MediaType.parse(guessContentTypeFromFile(file));
          mpBuilder.addPart(partHeaders, RequestBody.create(mediaType, file));
        } else {
          Headers partHeaders =
              Headers.of("Content-Disposition", "form-data; name=\"" + param.getKey() + "\"");
          mpBuilder.addPart(partHeaders,
              RequestBody.create(null, parameterToString(param.getValue())));
        }
      }
      return mpBuilder.build();
    }

    private boolean isFileList(Object value) {
      if (value == null) {
        return false;
      }
      if (!List.class.isAssignableFrom(value.getClass())) {
        return false;
      }
      List<?> cast = (List<?>) value;
      if (cast.isEmpty()) {
        return false;
      }
      if (cast.get(0) instanceof File) {
        return true;
      }
      return false;
    }