UnityWebRequest.Post() has multipart/form-data strangely formatted when sending request

A note before beginning, I know that WWW and WWWForm work, but since Unity is trying to get people off of that system, I would like to know why their new system breaks in my case. I’m using Unity 5.6

Goal: To take a screenshot and send it to a server

Problem: UnityWebRequest sends the data in a large data chunk rather than the correct multipart data even when it has the correct content-type header and boundary.

The server code was tested using tools such as postman to check it was working (server code isn’t mine but expects multipart form-data with a ‘file’ field and an image as it’s data) and the Unity and Postman requests were compared to identify the issue

This will be a long post, please bear with me as I explain what all I tried and how it broke:

//get screenshot and start coroutine to make request
//using dataPath so it is in assets folder for now.
Application.CaptureScreenshot(Application.dataPath + "/screenshot.png");
StartCoroutine(sendImage());

and the coroutine: (note: this code resulted in a 500 error from the server)

IEnumerator sendImage()
    {
    	/*validation to find file*/
        //read in file to bytes
    	byte[] img = File.ReadAllBytes(Application.dataPath + "/screenshot.png");

        //create multipart list
    	List<IMultipartFormSection> requestData = new List<IMultipartFormSection>();
        requestData.Add( new MultipartFormFileSection("file", img, 
        "screenshot.png", "image/png"));

        //url3 is a string url defined earlier
        UnityWebRequest request = UnityWebRequest.Post(url3, requestData);
        request.SetRequestHeader("Content-Type", "multipart/form-data");
        yield return request.Send();
        print("request completed with code: " + request.responseCode);
 
        if(request.isError) {
            print("Error: " + request.error);
        }
        else {
            print("Request Response: " + request.downloadHandler.text);
        }
    }

So that was the format giving us problems. Here is a screenshot from wireshark of what that request looked like.

For comparison, here is a good request made using Postman:

as you can see, the valid request breaks up the multipart form into it’s sections, where as the UnityWebRequest.Post() was not. Does anyone know why this is? The best guess a few others and myself could come up with was the fact that the .Post() forces the data to be URLEncoded via WWWTranscoder.URLEncode prior to transmission per the docs.

Can someone verify that this is indeed the case? I only saw 1 post that mentioned that after hours of searching and trying to fix this.

I will post an answer below with all of the methods we attempted/found to try and fix this so they are all in one place (seriously I had way too many tabs open with different posts trying to find solutions). We ended up using a hybrid solution using UnityWebRequest.Post() but using WWWForm data instead of the iMultipartFormSection class, but I’d love to hear if there is a better way.

So I found several methods in an attempt to fix this problem. Here are the top ones and why they didn’t work when I tried them.

Use .Put() then change the method to POST:
so this basically bypasses the encoding problem of .Post() which is especially useful if you are sending json strings as those get butchered through .Post() if using the constructor for string data. Here is how using a .Put() looks:

//.Put() takes in string url and either byte[] or string data
UnityWebRequest request = UnityWebRequest.Put(url3, requestData);
request.method = "POST";
request.Send()

So this works for JSON strings (we tested and confirmed) but when we tried sending a byte array of our fields we never actually could find/confirm the request coming out was a post coming out (no wireshark entry) and the server crashed with a 500. So for JSON this is a decent workaround but not for our problem.

Create request from scratch:
So there was another suggestion to craft the request from scratch. So we tried that. The code looks like this:

//generate a unique boundary
byte[] boundary = UnityWebRequest.GenerateBoundary();
//serialize form fields into byte[] => requires a bounday to put in between fields
byte[] formSections = UnityWebRequest.SerializeFormSections(requestData, boundary);
    
UnityWebRequest request = new UnityWebRequest(url3);
request.uploadHandler = new UploadHandlerRaw(formSections);

/*note: adding the boundary to the uploadHandler.contentType is essential! It won't have the boundary otherwise and won't know how to split up the fields; also it must be encoded to a string otherwise it just says prints the type, 'byte[]', which is not what you want*/

request.uploadHandler.contentType = "multipart/form-data; boundary=\"" + System.Text.Encoding.UTF8.GetString(boundary) + "\"";
request.downloadHandler = new DownloadHandlerBuffer();
request.method = "POST";
request.SetRequestHeader("Content-Type", "multipart/form-data");
yield return request.Send();

So this was actually pretty promising and almost worked for us. However, there were still problems. One was that the Content-Disposition of each field was empty. Not sure if this actually negatively affected anything, but since there is no field in the request object where I can manually set it, I could not change it. Second, the serialized fields added the boundary before each field but didn’t add a trailing boundary so there was no flag for where the field ended (in our use case of 1 field). I tried manually tacking one onto the end by manually creating a new byte and copying the boundary onto the end of that (like so):

byte[] boundary = UnityWebRequest.GenerateBoundary();
byte[] formSections = UnityWebRequest.SerializeFormSections(requestData, boundary);
byte[] reqData = new byte[formSections.Length + boundary.Length];
System.Buffer.BlockCopy(formSections, 0, reqData, 0, formSections.Length);
System.Buffer.BlockCopy(boundary, 0, reqData, formSections.Length, boundary.Length);
/*and use reqData and not formSections in the request*/

I actually got really close, but the end was not formatted like the valid response. The valid one had 2 ‘.’ icons before the ending boundary in wireshark while this attempt did not have those marks. I probably could get this to work with more finagling, but by this point I was irritated and tired so I went with our hybrid approach.

Hybrid:
This uses the WWWForm class but is a legacy method.

WWWForm form = new WWWForm();
form.AddBinaryData("file", img, "screenshot.png", "image/png");
UnityWebRequest request = UnityWebRequest.Post(url3, form);
yield return request.Send();
print("request completed with code: " + request.responseCode);
if(request.isError) {
    print("Error: " + request.error);
}
else {
    print("Request Response: " + request.downloadHandler.text);
 }

and that last one worked as intended.
So basically I wrote this question and this answer for 2 reasons:

  • Was I correct in my diagnosis of the problem and is there a better solution that tries to use the old methods as little as possible
  • If anyone esle experienced the same issues, here is all of the stuff I found and tried so it might save other people the trouble I had (also the 5 browser windows with 20 tabs open each of unity forum questions and other similat sites)

Thank you for posting this information; it got me most of the way to making multipart uploads work with Unity Web Request. You saw that the termination boundary wasn’t added but, in your example:

  1. I think you formatted the closing boundary incorrectly RFC1341(MIME) : 7 The Multipart content type.
  2. The quotes around the boundary are not in the spec.

I was able to get it working once I appended that boundary to my body byte.

// read a file and add it to the form
List<IMultipartFormSection> form= new List<IMultipartFormSection>
{
   new MultipartFormFileSection("file", IO.File.ReadAllBytes("somefile.gif"), "filename.gif", "image/gif")
};
// generate a boundary then convert the form to byte[]
byte[] boundary = UnityWebRequest.GenerateBoundary();
byte[] formSections = UnityWebRequest.SerializeFormSections(form, boundary);
// my termination string consisting of CRLF--{boundary}--
byte[] terminate = Encoding.UTF8.GetBytes(String.Concat("

–", Encoding.UTF8.GetString(boundary), “–”));
// Make my complete body from the two byte arrays
byte body = new byte[formSections.Length + terminate.Length];
Buffer.BlockCopy(formSections, 0, body, 0, formSections.Length);
Buffer.BlockCopy(terminate, 0, body, formSections.Length, terminate.Length);
// Set the content type - NO QUOTES around the boundary
string contentType = String.Concat(“multipart/form-data; boundary=”, Encoding.UTF8.GetString(boundary));
// Make my request object and add the raw body. Set anything else you need here
UnityWebRequest wr = new UnityWebRequest();
UploadHandler uploader = new UploadHandlerRaw(body);
uploader.contentType = contentType;
wr.uploadHandler = uploader;

I found unity adds quotes around the boundary in the content-type header which stumps some non Apache setups.

In addition unity adds extra newlines before the first boundary which again can cause some problems in some rare setups.

That’s why I did my own asset EasyWeb :slight_smile: to support streaming file to server with low memory footprint

It’s worth noting that even during errors, the response from the server can be read from request.downloadHandler.text and may contain useful data about the issue.


If the error is related to Content-Length, try setting:

request.chunkedTransfer=false;