Firing saved HTTP requests

Working with (and on) API's, testing them and figuring out what they send back exactly (beyond what the documentation tells you) involves sending a lot of (specific) requests. This usually comes down to repeating a lot of curl commands in small scripts or backtracking through my shell history for the curl command with those specific parameters.

Since HTTP is just plain text it should be trivial to save and load it from a file. While working on a node.js project figuring out the HTTP module internals and spending time on API's in my spare time I thought I could as well combine the two. So I wrote a (faily) simple script in node that can load an HTTP request from a file and show the response. This is the first script I wrote to flesh out the idea:

var util = require('util');
var fs   = require('fs');
var http = require('http');

// Check for an input file
if (2 >= process.argv.length) {
    console.error('An input file must be specified');
    process.exit(1);
}

// Pop the file of the script arguments (assume its the last argument)
var file = process.argv.pop();

// Try to read the input file
try {
    var content = fs.readFileSync(file, 'utf-8');
} catch (error) {
    console.error('Could not read file "' + file + '". Error: ' + error);
    process.exit(1);
}

// Split header and body
var parts = content.split(/\r?\n\r?\n/, 2);
var headerPart = parts.shift().trim();
var bodyPart = parts.length > 0 ? parts.shift().trim() : '';

// Process the headers
var headerLines = headerPart.split(/\r?\n/);
var requestLine = headerLines.shift();
var hostLine    = '';
var headers = {};
for (var i = 0; i < headerLines.length; i++) {
    var line = headerLines[i];
    var headerParts = line.split(':', 2);
    var headerKey = headerParts.shift().trim();
    var headerValue = headerParts.shift().trim();

    // Special case for Host header, we need this seperately for node's
    // request method. This could be circumvent by using the ClientRequest object directly
    if ('host' === headerKey.toLowerCase()) {
        hostLine = headerValue;
        continue;
    }
    // Save the header
    headers[headerKey] = headerValue;
}

// Build the request options
var options = {
    headers: headers
};

// Process the request line to retreive the method and path
var requestLineParts = requestLine.split(' ');
options.method = requestLineParts.shift().trim().toUpperCase();
options.path = requestLineParts.shift().trim();

// Process the host and port from the hostLine
var hostParts = hostLine.split(':', 2);
options.host = hostParts.shift().trim();
options.port = hostParts.length > 0 ? hostParts.shift().trim() : 80;

// Make the actual request
var request = http.request(options, function(response) {
    var body = '';

    // Write the status line
    util.puts('HTTP/' + response.httpVersion + ' ' + response.statusCode + ' ' + http.STATUS_CODES[response.statusCode]);

    // Make sure we can handle chunked transfer
    response.on('data', function(chunk) {
        body += chunk;
    });

    // Once done present the result
    response.on('end', function() {
        // Print the response headers
        for (header in response.headers) {
            var value = response.headers[header];
            util.puts(header + ': ' + value);
        }
        util.puts('');
        // Print the body
        util.puts(body);
    });
});

// Handle errors
request.on('error', function(error) {
    console.error('Error making request: ' + error);
    process.exit(2);
});


// Send the request with body
request.write(bodyPart);
request.end();
view raw nerf.js This Gist brought to you by GitHub

The script takes one argument and that is the path to the file containing an HTTP request. The request must follow the HTTP specification according to rfc2616 and must not contain any other text (just like any HTTP request send by a browser). A simple example of a request to google looks something like this:

GET / HTTP/1.1
Host: www.google.com:80
User-Agent: Mozilla/5.0

The script makes it easy for me to write a new request without having to search for the right (curl) commands. Since I can read HTTP pretty well it saves me time decyphering commands from my shell history. By giving sensible names to the request files I can easyli load the request I need. And since the requests are plain HTTP they can be saved from and loaded in other tools like Firebug or the Chrome Web Inspector and parsed by other languages.