Using a lightweight web server to debug requests

I’ve been working a lot with the Canvas API lately. One task was to add communication channels (e-mail addresses) to user accounts. I was able to add them successfully following the documentation’s cURL example using the skip_confirmation flag:

curl 'https://mycanvas.instructure.com/api/v1/users/[user id]/communication_channels'
-H "Authorization: Bearer [...]"
-d 'communication_channel[address]=chris@osric.com'
-d 'communication_channel[type]=email'
-d 'skip_confirmation=1'

However, when I ran the ColdFusion script I wrote, the user received notification of the addition even though the skip_confirmation flag was set:

<cfhttp url="https://mycanvas.instructure.com/api/v1/users/[user id]/communication_channels" method="post">
<cfhttpparam type="header" name="Authorization" value="Bearer [...]" />
<cfhttpparam type="formfield" name="communication_channel[address]" value="chris@osric.com" />
<cfhttpparam type="formfield" name="communication_channel[type]" value="email" />
<cfhttpparam type="formfield" name="skip_confirmation" value="1" />
</cfhttp>

Why didn’t the latter work as expected?

I needed to be able to tell what was different about the 2 requests, but it would be difficult to capture that outbound requests. Tools like Fiddler and WireShark could help, but I was sending one request from my local machine and another from a remote web server.

My idea for capturing the request data was to send the request to a server under my control. I grabbed a Python echo server and modified it slightly to print the data received. Then, instead of sending the requests to the Canvas API I sent them to the echo server. Here are the results of the 2 requests:

cURL
POST / HTTP/1.1
User-Agent: curl/7.24.0 (i686-pc-cygwin) libcurl/7.24.0 OpenSSL/0.9.8t zlib/1.2.5 libidn/1.22 libssh2/1.3.0
Host: osric.com:50000
Accept: */*
Authorization: Bearer [...]
Content-Length: 100
Content-Type: application/x-www-form-urlencoded

communication_channel[address]=chris@osric.com&communications_channel[type]=email&skip_confirmation=1

ColdFusion
POST / HTTP/1.1
User-Agent: ColdFusion
Content-Type: application/x-www-form-urlencoded
Connection: close
Authorization: Bearer [...]
Content-Length: 118
Host: osric.com:50000

communication%5Fchannel%5Baddress%5D=chris%40osric%2Ecom&communications%5Fchannel%5Btype%5D=email&skip%5Fconfirmation=1

I noted the different content lengths for the same data, and upon closer inspection, ColdFusion was URL-encoding characters. I added the encoded attribute to the cfhttpparam tags with a value of “no”, and then the request bodies were identical:

<cfhttp url="https://mycanvas.instructure.com/api/v1/users/[user id]/communication_channels" method="post">
<cfhttpparam type="header" name="Authorization" value="Bearer [...]" />
<cfhttpparam type="formfield" encoded="no" name="communication_channel[address]" value="chris@osric.com" />
<cfhttpparam type="formfield" encoded="no" name="communication_channel[type]" value="email" />
<cfhttpparam type="formfield" encoded="no" name="skip_confirmation" value="1" />
</cfhttp>

Not surprisingly that solved the problem.