The trials and tribulations of sending POST data to an Arduino
This tutorial / extended blog post was going to be part of a series on how to use an Arduino to remotely turn on a computer, which I may finish at a later date.
There are many reasons someone might want to POST to a webserver rather then GET. However, there is lack of POST accepting webserver implementations for Arduino. Today, I build one.
Building a GET webserver
Firstly, I looked into the Arduino examples, to see if there was a sensible way to build this server. This is how the Arduino foundation think you should do it, and we will modify this to accept a user input via a GET request, and if that works, try to get it to accept a POST request.
The first thing we need to do is get an input from the user. A common way to do usually would be to use a GET request. To do this on the Arduino, I send all of the HTTP request into the StringHTTP_req, letter by letter, as it comes in from client.read().
This can be done with the 2 simple lines.
char c = client.read(); // read the one incoming character from client
HTTP_req += c; // save the character to a big long string
Next we need to know when the request ends.
A standard HTTP request (with GET request) looks like this.
GET /?LED9=On HTTP/1.1\r\n
Host: 192.168.0.16:40000\r\n
Connection: keep-alive\r\n
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8\r\n
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/43.0.2357.124 Safari/537.36\r\n
Accept-Encoding: gzip, deflate, sdch\r\n
Accept-Language: en-US,en;q=0.8,en-GB;q=0.6\r\n
\r\n
The obvious ending is when there is a line with only \r\n on it (aka, a blank line).
So let's write some code to find when that happens.
We already have this code...
if (c == '\n') {
// you're starting a new line
currentLineIsBlank = true;
}
else if (c != '\r') {
// you've gotten a character on the current line
currentLineIsBlank = false;
}
}
...from the original example, which tells us when a line starts, and when there are characters on the current line.
So this means that everything contained within if (c == '\n' && currentLineIsBlank) {} is our response.
If we are sending our response, we can guarantee we already have our complete request.
This means we can parse it to find out anything we need to formulate our reply. For example, I use the following code to detect if the user has sent a get request, and what the contents of the request is, and what to do in each case.
if (HTTP_req.indexOf("GET /?LED9=On HTTP/1.1") > -1) { // see if URL ends in our get request
client.println("<a href=\"?LED9=Off\">Toggle the Pin?</a> LED is on!"); //add a link which will send a GET request if clicked to turn the pin off and display a message saying the LED is on.
digitalWrite(9, HIGH); //turn the LED on.
}
if (HTTP_req.indexOf("GET /?LED9=Off HTTP/1.1") > -1) { // see if URL ends in our get request
client.println("<a href=\"?LED9=On\">Toggle the Pin?</a> the LED is off!"); //add a link which will send a GET request if clicked to turn the pin on and display a message saying the LED is off.
digitalWrite(9, LOW); //turn the LED off.
}
This code is in the middle of the response sent back to the user.
Something to note: the string to search for must be GET /?LED9=Off HTTP/1.1 as opposed to LED9=Off and the same for the on state. This is because you must search for the GET request itself, or indexOf can find the similar strings in the HTTP Referer, if the browser has clicked through one of the links before, which will cause the wrong state to be shown the the LED to go to the wrong state.
Now we have the main programming bits sorted out, we just need to finish it up. Inside if (client.available()) {}, I added some more HTML to make the web page "look good" (if such a thing is possible without any CSS). I also sent the full HTTP request to the serial port for any debugging, emptied the request string with a HTTP_req = "";.
if (client.available()) statement should look like this...
if (client.available()) { // client data available to read
char c = client.read(); // read the one incoming character from client
HTTP_req += c; // save the character to a big long string
// last line of client request is blank and ends with \n
// respond to client only after last line received
if (c == '\n' && currentLineIsBlank) {
// send http response header
client.println("HTTP/1.1 200 OK");
client.println("Content-Type: text/html");
client.println("Connection: close");
client.println();
// send web page
client.println("<html>");
client.println("<body>");
client.println("<h1>LED</h1>");
client.println("<p>Click to switch LED on and off.</p>");
client.println("<form method=\"get\">");
if (HTTP_req.indexOf("GET /?LED9=On HTTP/1.1") > -1) { // see if URL ends in our get request
client.println("<a href=\"?LED9=Off\">Toggle the Pin?</a> LED is on!");
digitalWrite(9, HIGH);
Serial.println("statues: 1");
}
if (HTTP_req.indexOf("GET /?LED9=Off HTTP/1.1") > -1) { // switch LED off
client.println("<a href=\"?LED9=On\">Toggle the Pin?</a> the LED is off!");
digitalWrite(9, LOW);
Serial.println("statues: 0");
}
client.println("</form>");
client.println("</body>");
client.println("</html>");
Serial.print(HTTP_req);
HTTP_req = ""; // finished with request, empty string
break;
}
// every line of text received from the client ends with \r\n
if (c == '\n') {
// last character on line of received text
// starting new line with next character read
currentLineIsBlank = true;
}
else if (c != '\r') {
// a text character was received from client
currentLineIsBlank = false;
}
} // end if (client.available())
Code Review
So, end of article right? Wellllllllllllll, no. There are quite a few problems with this method. Firstly, if we are going to be making this control the power to a computer, we don't want people just running a port scan, finding this mysterious http server, then turning our computer off. So how can we secure it?
A password!
Obviously, we could just add another field, and another piece of code to look for the get request. However, GET requests aren't secure. Someone could easily go through your history and find the URL, and just read it to find the password.
We need to use a POST request.
Rebuilding the webserver for POST
Reading the POST request should be easy, right? All we need to do is read the HTTP request as we have done before. Well that's what I thought originally, so I set about looking for what the POST data in the request would look like. First we need a html form to submit, which will cause the browser to generate a post request.
<html>
<body>
<h1>LED</h1>
<p>Click to switch LED on and off.</p>
<form method="post">
<button name="LED9" value="On" type="submit">On</button> the LED is off!
</form>
</body>
</html>
When the button is clicked, it generates this HTTP request (which I captured with Wireshark) to our Arduino server.
POST / HTTP/1.1\r\n
Host: 192.168.0.16\r\n
Connection: keep-alive\r\n
Content-Length: 8\r\n
Cache-Control: max-age=0\r\n
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8\r\n
Origin: http://192.168.0.16\r\n
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/43.0.2357.124 Safari/537.36\r\n
Content-Type: application/x-www-form-urlencoded\r\n
Referer: http://192.168.0.16/\r\n
Accept-Encoding: gzip, deflate\r\n
Accept-Language: en-US,en;q=0.8,en-GB;q=0.6\r\n
\r\n
LED9=On
What's the difference? The data "LED9=On" is now stored on the last line, rather then the first. Surely this should work with our current code then? Nope. Because our current code checks for a \r\n followed by another \r\n to determine whether the request has ended. This can be seen by sending a POST request with data in the request body to the server. If you use the Arduino serial interface to see the requests as they come in, you will see your POST body is cut off. Crap.
So how do we find out POST body?
The easiest thing to do is sitting right under our noses... client.available()! client.available() returns the amount of bytes left from the client to read. When we reach 0, we have read everything from the client and therefore can interpret the request and respond to it (and I honestly don't know why it wasn't coded this way before, it seems considerably more sensible then scanning character by character for the double sets of newline characters). So I replaced c == '\n' && currentLineIsBlank with !client.available() and chucked out the rest of the currentLineIsBlank code from the bottom of the while bracket and... It works! Sort of...
It seems to be in both a state of "on" and "off". How could this be? When a browser navigates to a new page, it looks for a favicon.ico. Because the request sent to our webserver didn't have "LED9=On" when it came, it made the server think it was both on and off at the same time. However, there is an easy work around. Slap if (HTTP_req.indexOf("POST / HTTP/1.1") > -1){ around any part of the response. It prevents the web server responding if the request isn't a POST request. Simples!
Finished code
This will run the webserver on your network at http://192.168.0.16/. Have fun!
#include <SPI.h>
#include <Ethernet.h>
// MAC address from Ethernet shield sticker under board
byte mac[] = { 0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0xED };
IPAddress ip(192, 168, 0, 16); // IP address, may need to change depending on network
EthernetServer server(80); // create a server at port 80
String HTTP_req; // stores the HTTP request
boolean LED_status = 0; // state of LED, off by default
void setup()
{
Ethernet.begin(mac, ip); // initialize Ethernet device
server.begin(); // start to listen for clients
Serial.begin(9600); // for diagnostics
pinMode(9, OUTPUT); // LED on pin 13
}
void loop()
{
EthernetClient client = server.available(); // try to get client
if (client) { // got client?
while (client.connected()) {
if (client.available()) { // client data available to read
//Serial.println(client.available()); //this really slows your connection speed, uncomment at will.
char c = client.read();
//Serial.print(c); //this really slows your connection speed, uncomment at will.
HTTP_req += c; // save the HTTP request to a string
if (!client.available()) { //when this becomes false, they have sent their last byte.
// send http response header
client.println("HTTP/1.1 200 OK");
client.println("Content-Type: text/html");
client.println("Connection: close");
client.println();
// send web page
client.println("<html>");
client.println("<body>");
client.println("<h1>LED</h1>");
client.println("<p>Click to switch LED on and off.</p>");
client.println("<form method=\"post\">");
if (HTTP_req.indexOf("POST / HTTP/1.1") > -1){ //don't add anything if its not a post request.
if (HTTP_req.indexOf("LED9=On") > -1) { // see if URL ends in our get request
client.println("<button name='LED9' value='Off' type='submit'>Off</button> the LED is on!");
digitalWrite(9, HIGH);
}
else {
client.println("<button name='LED9' value='On' type='submit'>On</button> the LED is off!");
digitalWrite(9, LOW);
}
}
else{
client.println("<button name='LED9' value='On' type='submit'>On</button> the LED is off!"); //if its a GET request we know the LED is off
}
client.println("</form>");
client.println("</body>");
client.println("</html>");
//Serial.print(HTTP_req); //this really slows your connection speed, uncomment at will.
HTTP_req = "";
break; //we have got to the end, its time to stop this bit
}
}
}
delay(1); // give the user time to get it all
client.stop(); // get rid of this guy
}
}