Benchmarking Mongrel, Apache, Rails, etc.

I'm doing some benchmarking of a Rails based web app (technically, a web service) to try to establish some baselines and to use in assessing build outs/hardware deployment, etc. To start off, I wanted to establish a baseline by using pure static content. The performance I got didn't seem quite right to me. Below is a message I sent to the mongrel-users mailing list. I figured this was a good blog item, and would like to solicit help on this...

I'm trying to do some initial benchmarking of our setup, mainly just to establish baselines. I'm essentially using the process Zed outlines in a previous message:
http://rubyforge.org/pipermail/mongrel-users/2006-May/000200.html

What I'm running into is that Mongrel appears only half as fast as Apache when serving a small static HTML file. If I then add in Apache with mod_proxy_balancer, going to a single Mongrel, it drops down to nearly about a third of what pure static Apache will do. This seems bogus to me, and I suspect I have either some configuration problem, or something. My understanding from what I've read is that Mongrel should be fairly close to Apache when serving static content (at least not only 50% as fast). Is that right as a generalization?

Here's some info to back this up.
- Server: HP DL360 dual 3.0GHz Xeons, 4GB RAM, 10k RPM SCSI disks
- OS: Fedora Core 6, up to date as of a few days ago
- Apache 2.2.3-5, with mod_proxy_balancer
- Mongrel 1.0.1, mongrel_cluster 0.2.1
- serving a static HTML file from the Rails app's public directory
- Testing using an identical server, that sits above this one in the rack, connected to a switch (so it's one hop)
- Using httperf for testing, with the following command:
httperf --server lab05 --port 80 --uri /mongrel_alive.html --num-conns 10000
The number of connections I vary to be near the 10 second mark...

Here's the results:

- Just Apache, num-conns=15000, ~1400 req/sec
- Direct to Mongrel on port 5000, num-conns=8000, ~740 req/sec
- Apache mod_proxy_balancer to a single Mongrel, num-conns=5000, ~475 req/sec

So, I'm incurring a massive penalty for the balancer/cluster setup. In production we will be using hardware load balancers, but even still, my understanding was that instead of a drop from 1400 to 740, it should be somewhat closer (say at least over 1000).

What would folks suggest, or what comments do you have?

Shortly after that, I decided to throw Pen and Nginx in the mix just as a random check on this. I haven't used them before so relied on what I found on the net, etc. for config, thus I could potentially get better results, but they yielded:

• Nginx with one Mongrel: ~612 req/sec
• Pen with one Mongrel: ~670 req/sec

I can see Apache being slower than these, as it's a much bigger and more complex app. So the question remains, should straight Mongrel be half as fast as Apache? And, should the load balancers affect the performance that much (although Pen is having only a 10% impact here).

It's Comcastic!

Well, I hope it will be Comcastic! Ordered up new phone, cable TV, and Internet service all in one with Comcast today, for our new house in Eugene. Yep, that's phone over cable. They had the fastest net service in the area (8MBit down, 784k up). We have a cable modem now, and have had really good luck with it, so I hope that continues.

The cable will include HD, and we have two of the new TiVo Series 3 HD units on the way from Amazon. Comcast will give us four cable cards (two per box) to put in them. Pretty cool, although more for my wife than me, as I barely watch TV these days.

The phone is pretty good too. It includes unlimited local and long distance calling, and 12 features (stuff like voice mail, call forwarding, etc.). All of this, setup for two rooms, etc, etc., is $130/month ($15 of that is for the 3 extra cable cards - they give you one for free). Seems fairly good, although if you did a lot of long distance calling this would probably start to pay off pretty well. Our cable modem now costs me $68/month (this is for the "residential router" package, which means I can "legally" (i.e. I don't violate the ToS) put a router and as many computers as I want behind it), my phone typically costs $45, and our DirecTV (with Tivo) costs about $70/month. So, that's $183, and thus I figure we're saving about $58/month (minus some, as the $130 through Comcast doesn't include the Tivo service).

ApolloCamp

For those of you interested in Apollo, check out the free ApolloCamp, on March 16th at the Adobe San Francisco office. See you there!

Using multiple Perforce depots and TextMate's P4 Bundle

I'm a big TextMate user. We're also a Perforce shop. TextMate has a P4 bundle, and it works great, but it's main problem is that it has minimal knowledge of your P4 settings, and certainly doesn't handle multiple P4 clients or depots. For example, I have a few different clientspecs I work with, and the server, port, clientspec, etc. all differ. So, it's not feasible to use a single .p4config file, or set the values in TextMate or globally in the environment. This makes using the P4 bundle essentially impossible for all but whichever one you pick as your main clientspec.

To alleviate this, and based on how I organize my code on my machine, I wrote a simple little script that is what I set the "TM_P4" textmate environment variable to. This script knows that all my code is kept in subdirectories off my ~/Code directory. So, I then stash specific .p4config files at the root of each of these as needed, and the script looks for those based on the currently open file, and sets that for the P4 settings. This seems to work well. Here's the script in case it's useful to you:

#!/bin/sh
#
# Script to run p4 commands in TextMate, and set the P4 variables
# depending on the location of the file being operated on.

# Where all my code lives
CODE_PATH="${HOME}/Code/"

# figure out the Code subdirectory for the current file in TextMate
P4_CONFIG_PATH=${TM_FILEPATH##$CODE_PATH}
P4_CONFIG_PATH=${P4_CONFIG_PATH%%/*}

# Reconstitute as path to potential config file
P4_CONFIG_FILE="${CODE_PATH}${P4_CONFIG_PATH}/.p4config"

# if dir contains a .p4config, set P4CONFIG to that
if [ -f $P4_CONFIG_FILE ]; then
 export P4CONFIG="$P4_CONFIG_FILE"
else
 export P4CONFIG="~/.p4config"
fi

# now do p4
/usr/local/bin/p4 $*

Loving the Blackjack, Wishing For Ubiquitous 3G

I've really been loving my Blackjack. The one bummer is how spoiled I am getting with 3G. When we move to Eugene there won't be 3G and I already find EDGE sucks. Hey Cingular - please put 3G everywhere!

Twitter: cool or just silly?

Check out Twitter, a relatively new web app. It's very simple, it shows the status of people (here's ). Status is any short bit of text the person wants to state. When I first saw this, I thought it was just plain silly. Gee, do you really need up to the minute (or whatever) updates on what people are doing? Could people really keep it up to date enough to be http://www2.blogger.com/img/gl.link.gifinteresting/useful? Would people actually use it?

Tonight I was checking it out, and noticed that they have an ActionScript 3 library so as to make using their public API trivial from Flash/Flex apps (I'm using Flex a fair bit these days). I then noticed that you can update your status via IM as well as via SMS from your mobile phone. Ok, now it's getting a bit more interesting. Maybe you can keep it fairly up to date with relative ease.

Also the AS3 lib and API, as well as the ability to shield your status from the public (only your Twitter "friends" can see it), made me think about tieing this in to the Agile tasking tool we're working on for RailsConf. It could potentially be interesting to have a "work" Twitter account, and use the API to show people's status. You could then tie it in to the tasking app, and anytime anyone changed a task's status (for example, they set a task to now be "in progress", the app would shoot out a Twitter message (aka a "tweet") to indicate that. Then using the public API you could show your team's "timeline" on a web page. Useful? Questionable. Geeky fun? You bet.

moblog from my new Blackjack

I got a Blackjack mobile phone the other day and just set it up for moblogging. This is my first test. As for the phone, I am truly loving it so far!

start

Essential COM autographed by Don Box, and other books

I've put up more books on eBay, including a copy of "Essential COM", signed by Don Box himself (eBay listing). I've also got books on Zope and Plone (listing, which has links to the others), and various other Windows books. I didn't sell a single one of the Java books I had up there, which kind of surprised me (nearly all were listed at $4).

Moving to Eugene, Oregon

We (my wife, kids, and I) will be moving to Eugene, Oregon in a couple months (probably in April). We signed the counter-offer on a house today, so hopefully escrow goes well, and we'll own it in March. We're pretty excited, but nervous of course as well. We don't know anyone there. My wife's parents and brother and his family will move later this year (no commune jokes please!). Looking forward to it though. It was a move completely by choice, and I will continue to work for Adobe from there (from the home office).

We spent last Saturday driving around with our real estate agent, and wound up still loving a house we'd seen back in October, which had recently had the price dropped, so we went for it. I have a map showing our drive and various waypoints up on Discover Machine. I recently got a GPS unit, so used it on the drive, which was really quite handy. I put a waypoint in for every house we went in to, as well as lunch spot, and so on. It was cool to look at the overall route afterwards and see where all we went from a 10,000 foot type of view. I uploaded my data from my GPS into Google Earth Plus, and then saved a KML file out, which can be uploaded to Discover Machine (a site done in Rails :).

This will of course be a very convenient location for RailsConf later this year, as well as cool conferences like OSCON.

I'm Presenting at RailsConf 2007

I just found out yesterday that the proposal my co-worker and I submitted for RailsConf 2007 was accepted! Quite exciting. I hope to say more later, but for the moment, the talk will be about using Rails and Adobe's new Apollo technology.