Mmm Chuhnky

When not to use shell scripts

This is a straight rip from chapter 1 of advanced bash-scripting guide on the linux document project. Now I have to tell you, I have attempted to do some of the following, and it makes me laugh when they mention some of the things you should not do. Granted they make alot of sense but still isn’t it fun just to try it anyway.

If any of the above applies, consider a more powerful scripting language – perhaps Perl, Tcl, Python, Ruby –or possibly a compiled language such as C, C++, or Java. Even then, prototyping the application as a shell script might still be a useful development step.

Posted on 10 September 2010.

Evented Http

What? Evented Http? Thats right. Lets keep it brief. You know what node.js is, you know what eventmachine is. I’ve been working with eventmachine for a little while now and love it. It makes everything in ruby super fast aslong as you remember to avoid blocking calls. My only concern was, is it the fastest thing out there? Hmm probably not, but I wanted to see if it was upto the challenge. Node.js, the new kid in town, written in javascript running ontop of google’s blindingly fast v8 engine. There are alot of working examples of node out there and the one I was interested in is simple static file serving. Nothing special. Lets have a look.

var sys =  require("sys"),
    http = require("http"),
    url = require("url"),
    path = require("path"),
    fs = require("fs");

http.createServer(function(request, response) {
        var uri = url.parse(request.url).pathname;
        var filename = path.join(process.cwd(), uri);
        path.exists(filename, function(exists) {
                if(!exists) {
                    response.sendHeader(404, {"Content-Type": "text/plain"});
                    response.write("404\n");
                    response.close();
                    return;
                }
                
                fs.readFile(filename, "binary", function(err, file) {
                        if(err) {
                            response.sendHeader(500, {"Content-Type": "text/plain"});
                            response.write(err + "\n");
                            response.close();
                            return;
                        }
                        response.sendHeader(200);
                        response.write(file, "binary");
                        response.close();
                        
                    });
            });
    }).listen(8080);

Its pretty straight forward and I’m not going to explain it because honestly I couldnt do it justice. However I will bench it.

For anyone who has not used siege, it is a brilliant benchmarking tool which I highly recommend.

siege -b -r 500 -c 20 http://localhost:8080/foo.txt (20 concurrent requests, 500 runs)

The results:

transactions:                   10000 hits
Availability:                 100.00 %
Elapsed time:                  60.48 secs
Data transferred:            2042.44 MB
Response time:                  0.12 secs
Transaction rate:             165.34 trans/sec
Throughput:                    33.77 MB/sec
Concurrency:                   19.98
Successful transactions:       10000
Failed transactions:               0
Longest transaction:            1.52
Shortest transaction:           0.02
And now the eventmachine version:

require 'rubygems'
require 'eventmachine'
require 'evma_httpserver'
require 'uri'


class Static < EM::Connection
  include EM::HttpServer

  def post_init
    super
    no_environment_strings
  end

  def process_http_request
    @header_processing = false
    filename = Dir.pwd + URI.parse(@http_request_uri).path
    response = EM::DelegatedHttpResponse.new(self)

    if File.exists?(filename)
      response.status = 200
      response.content = File.readlines(filename,'b')
      response.send_response
    else
      response.status = 404
      response.content = "404 Error\n"
      response.send_response
    end
  end
end

EM.run {
  EM.epoll
  EM.start_server '0.0.0.0', 8081, Static
}
transactions:                   10000 hits
Availability:                 100.00 %
Elapsed time:                  19.88 secs
Data transferred:            2042.44 MB
Response time:                  0.04 secs
Transaction rate:             503.02 trans/sec
Throughput:                   102.74 MB/sec
Concurrency:                   19.98
Successful transactions:       10000
Failed transactions:               0
Longest transaction:            0.06
Shortest transaction:           0.02

What?! Eventmachine just walked all over node.js? I must be doing something wrong. Admittedly my code is alot shorter and has no real error handling, but thats just insane, a 40 sec difference, 500 trans/sec vs 165/sec for node. I’m using a 210k static file so its nothing massive. I’m definitely going to have to work on this and just strip the node code but there you have it.

Posted on 25 August 2010.