Benchmarking Mongrel, Apache, Rails, etc.

I'm doing some benchmarking of a Rails based web app (technically, a web service) to try to establish some baselines and to use in assessing build outs/hardware deployment, etc. To start off, I wanted to establish a baseline by using pure static content. The performance I got didn't seem quite right to me. Below is a message I sent to the mongrel-users mailing list. I figured this was a good blog item, and would like to solicit help on this...

I'm trying to do some initial benchmarking of our setup, mainly just to establish baselines. I'm essentially using the process Zed outlines in a previous message:
http://rubyforge.org/pipermail/mongrel-users/2006-May/000200.html

What I'm running into is that Mongrel appears only half as fast as Apache when serving a small static HTML file. If I then add in Apache with mod_proxy_balancer, going to a single Mongrel, it drops down to nearly about a third of what pure static Apache will do. This seems bogus to me, and I suspect I have either some configuration problem, or something. My understanding from what I've read is that Mongrel should be fairly close to Apache when serving static content (at least not only 50% as fast). Is that right as a generalization?

Here's some info to back this up.
- Server: HP DL360 dual 3.0GHz Xeons, 4GB RAM, 10k RPM SCSI disks
- OS: Fedora Core 6, up to date as of a few days ago
- Apache 2.2.3-5, with mod_proxy_balancer
- Mongrel 1.0.1, mongrel_cluster 0.2.1
- serving a static HTML file from the Rails app's public directory
- Testing using an identical server, that sits above this one in the rack, connected to a switch (so it's one hop)
- Using httperf for testing, with the following command:
httperf --server lab05 --port 80 --uri /mongrel_alive.html --num-conns 10000
The number of connections I vary to be near the 10 second mark...

Here's the results:

- Just Apache, num-conns=15000, ~1400 req/sec
- Direct to Mongrel on port 5000, num-conns=8000, ~740 req/sec
- Apache mod_proxy_balancer to a single Mongrel, num-conns=5000, ~475 req/sec

So, I'm incurring a massive penalty for the balancer/cluster setup. In production we will be using hardware load balancers, but even still, my understanding was that instead of a drop from 1400 to 740, it should be somewhat closer (say at least over 1000).

What would folks suggest, or what comments do you have?

Shortly after that, I decided to throw Pen and Nginx in the mix just as a random check on this. I haven't used them before so relied on what I found on the net, etc. for config, thus I could potentially get better results, but they yielded:

• Nginx with one Mongrel: ~612 req/sec
• Pen with one Mongrel: ~670 req/sec

I can see Apache being slower than these, as it's a much bigger and more complex app. So the question remains, should straight Mongrel be half as fast as Apache? And, should the load balancers affect the performance that much (although Pen is having only a 10% impact here).

It's Comcastic!

Well, I hope it will be Comcastic! Ordered up new phone, cable TV, and Internet service all in one with Comcast today, for our new house in Eugene. Yep, that's phone over cable. They had the fastest net service in the area (8MBit down, 784k up). We have a cable modem now, and have had really good luck with it, so I hope that continues.

The cable will include HD, and we have two of the new TiVo Series 3 HD units on the way from Amazon. Comcast will give us four cable cards (two per box) to put in them. Pretty cool, although more for my wife than me, as I barely watch TV these days.

The phone is pretty good too. It includes unlimited local and long distance calling, and 12 features (stuff like voice mail, call forwarding, etc.). All of this, setup for two rooms, etc, etc., is $130/month ($15 of that is for the 3 extra cable cards - they give you one for free). Seems fairly good, although if you did a lot of long distance calling this would probably start to pay off pretty well. Our cable modem now costs me $68/month (this is for the "residential router" package, which means I can "legally" (i.e. I don't violate the ToS) put a router and as many computers as I want behind it), my phone typically costs $45, and our DirecTV (with Tivo) costs about $70/month. So, that's $183, and thus I figure we're saving about $58/month (minus some, as the $130 through Comcast doesn't include the Tivo service).

ApolloCamp

For those of you interested in Apollo, check out the free ApolloCamp, on March 16th at the Adobe San Francisco office. See you there!

Using multiple Perforce depots and TextMate's P4 Bundle

I'm a big TextMate user. We're also a Perforce shop. TextMate has a P4 bundle, and it works great, but it's main problem is that it has minimal knowledge of your P4 settings, and certainly doesn't handle multiple P4 clients or depots. For example, I have a few different clientspecs I work with, and the server, port, clientspec, etc. all differ. So, it's not feasible to use a single .p4config file, or set the values in TextMate or globally in the environment. This makes using the P4 bundle essentially impossible for all but whichever one you pick as your main clientspec.

To alleviate this, and based on how I organize my code on my machine, I wrote a simple little script that is what I set the "TM_P4" textmate environment variable to. This script knows that all my code is kept in subdirectories off my ~/Code directory. So, I then stash specific .p4config files at the root of each of these as needed, and the script looks for those based on the currently open file, and sets that for the P4 settings. This seems to work well. Here's the script in case it's useful to you:

#!/bin/sh
#
# Script to run p4 commands in TextMate, and set the P4 variables
# depending on the location of the file being operated on.

# Where all my code lives
CODE_PATH="${HOME}/Code/"

# figure out the Code subdirectory for the current file in TextMate
P4_CONFIG_PATH=${TM_FILEPATH##$CODE_PATH}
P4_CONFIG_PATH=${P4_CONFIG_PATH%%/*}

# Reconstitute as path to potential config file
P4_CONFIG_FILE="${CODE_PATH}${P4_CONFIG_PATH}/.p4config"

# if dir contains a .p4config, set P4CONFIG to that
if [ -f $P4_CONFIG_FILE ]; then
 export P4CONFIG="$P4_CONFIG_FILE"
else
 export P4CONFIG="~/.p4config"
fi

# now do p4
/usr/local/bin/p4 $*

Loving the Blackjack, Wishing For Ubiquitous 3G

I've really been loving my Blackjack. The one bummer is how spoiled I am getting with 3G. When we move to Eugene there won't be 3G and I already find EDGE sucks. Hey Cingular - please put 3G everywhere!

Twitter: cool or just silly?

Check out Twitter, a relatively new web app. It's very simple, it shows the status of people (here's ). Status is any short bit of text the person wants to state. When I first saw this, I thought it was just plain silly. Gee, do you really need up to the minute (or whatever) updates on what people are doing? Could people really keep it up to date enough to be http://www2.blogger.com/img/gl.link.gifinteresting/useful? Would people actually use it?

Tonight I was checking it out, and noticed that they have an ActionScript 3 library so as to make using their public API trivial from Flash/Flex apps (I'm using Flex a fair bit these days). I then noticed that you can update your status via IM as well as via SMS from your mobile phone. Ok, now it's getting a bit more interesting. Maybe you can keep it fairly up to date with relative ease.

Also the AS3 lib and API, as well as the ability to shield your status from the public (only your Twitter "friends" can see it), made me think about tieing this in to the Agile tasking tool we're working on for RailsConf. It could potentially be interesting to have a "work" Twitter account, and use the API to show people's status. You could then tie it in to the tasking app, and anytime anyone changed a task's status (for example, they set a task to now be "in progress", the app would shoot out a Twitter message (aka a "tweet") to indicate that. Then using the public API you could show your team's "timeline" on a web page. Useful? Questionable. Geeky fun? You bet.

moblog from my new Blackjack

I got a Blackjack mobile phone the other day and just set it up for moblogging. This is my first test. As for the phone, I am truly loving it so far!

start

Essential COM autographed by Don Box, and other books

I've put up more books on eBay, including a copy of "Essential COM", signed by Don Box himself (eBay listing). I've also got books on Zope and Plone (listing, which has links to the others), and various other Windows books. I didn't sell a single one of the Java books I had up there, which kind of surprised me (nearly all were listed at $4).

Moving to Eugene, Oregon

We (my wife, kids, and I) will be moving to Eugene, Oregon in a couple months (probably in April). We signed the counter-offer on a house today, so hopefully escrow goes well, and we'll own it in March. We're pretty excited, but nervous of course as well. We don't know anyone there. My wife's parents and brother and his family will move later this year (no commune jokes please!). Looking forward to it though. It was a move completely by choice, and I will continue to work for Adobe from there (from the home office).

We spent last Saturday driving around with our real estate agent, and wound up still loving a house we'd seen back in October, which had recently had the price dropped, so we went for it. I have a map showing our drive and various waypoints up on Discover Machine. I recently got a GPS unit, so used it on the drive, which was really quite handy. I put a waypoint in for every house we went in to, as well as lunch spot, and so on. It was cool to look at the overall route afterwards and see where all we went from a 10,000 foot type of view. I uploaded my data from my GPS into Google Earth Plus, and then saved a KML file out, which can be uploaded to Discover Machine (a site done in Rails :).

This will of course be a very convenient location for RailsConf later this year, as well as cool conferences like OSCON.

I'm Presenting at RailsConf 2007

I just found out yesterday that the proposal my co-worker and I submitted for RailsConf 2007 was accepted! Quite exciting. I hope to say more later, but for the moment, the talk will be about using Rails and Adobe's new Apollo technology.

Adobe to Release PDF for Industry Standardization

Check out the press release! Adobe is releasing the FULL PDF 1.7 spec to get published as an ISO standard. Pretty cool.

I'm Now A GPS Geek

A good friend who is quite into GPS, and happens to work on the Google Earth team finally got me hooked on GPS. I picked up a Garmin GPSmap 60CSx unit at REI after attending their intro class the other day. I then proceeded to figure out how to get it all going on the Mac. Many thanks to Ben Sinclair, as he pretty much covers every angle in his article, "How to use a Garmin GPS with your Mac."

I'm using the combination of an Intel Mac, with Google Earth Plus, and then Parallels for the Garmin MapSource City Navigator software. I haven't gone for any TOPO stuff yet, and Google Earth seems to cover most of what I want trail/bike/ski wise so far (if I get more into backcountry skiing, then I will likely want the TOPO stuff).

Anyway, this is really cool. I can't wait to use it on my mountain bike. The first heavy use will be this weekend when we go to Eugene to scout houses and so on. I will track the entire day on Saturday and then import into Google Earth, etc. Also, because I have the City Navigator stuff on there, I've got intricate detail of city streets, businesses, turn-by-turn directions, and so on. Slick.

Note, the "x" series units from Garmin (I think all of them anyway) have microSD cards. You can store a ton of map data on these things. For example, the City Navigator stuff I put on mine, for which I did most of Oregon, and a good chunk of Northern CA, was about 50MB. The 60CSx comes with a 64MB microSD, but I have a 512MB on the way (I'd have gone bigger, but I heard that with the 1GB cards it creates a noticeable slowdown on the unit while it sifts through all that data, and 512MB seemed like plenty given what you can put in about 50-100MB).

Java Books for Sale

I just posted a slew of Java books for sale on eBay. They are all priced between $3-6. Some are current, some aren't. Here's a listing (and the link goes to the eBay listing):

• Graphic Java 2, Volume 2: Swing (3rd Edition)


• Graphic Java 1.2, Volume 1: AWT, Third Edition


• The Java Class Libraries, Volume 1: java.io, java.la...


• Patterns in Java, Volume 1, A Catalog of Reusable De...


• Patterns in Java, Volume 2


• Programming Web Services with XML-RPC (O'Reilly Inte...


• Java Web Services


• JSTL: Practical Guide for JSP Programmers (The Pract...


• Web Development with Java Server Pages


• Designing Enterprise Applications with the Java(TM) ...


• JavaServer Pages, 3rd Edition


• Java Threads, Second Edition Java 2


• Java(TM) Native Interface: Programmer's Guide and Sp...


• Art of Java Web Development: Struts, Tapestry, Commo...


• Building Imaging Applications with Java(TM) Technolo...

I will have more, on other subjects to come, but I've started to prune my library, in preparation for our move to Eugene.

For the curious, I use iSale on the Mac to post on eBay, it's superb. It easily filled in most of the auction on each book simply by me letting it search Amazon for the ISBN and importing the info (all automatically). The new version of iSale also has iSight support for taking quick pics, etc. But, the listing above was a simple hack by exporting my auction items from iSale (as CSV), plopping that in Excel and dropping out all but the eBay ID column and title column, then dropping that in TextMate, and simply wrapping those with the above HTML, trivially done by using TextMate's awesome column editing abilities (a regex search and replace would've done fine too, but this seemed even faster).

Skiing Silverton in Early January

Just firmed up a quick trip with two friends to ski Silverton (Colorado) in early January. One of the guys, Matt, just skied there today and said it was stellar. It's an interesting place, as it only has one lift, and during the main part of the season you must ski guided. This will be my first real backcountry excursion, although it should be relatively mellow in that regard (no skinning, hopefully minimal hiking as my fitness level is quite poor right now).

I also am hoping to take a lot of pictures. And, given my weak fitness, I expect to not last a whole day of this kind of skiing, so hopefully can also shoot some action shots of Matt and Barry. Matt will easily rip it up, as he's been skiing this kind of stuff all over CO and UT for the last few weeks (the lucky bum).

Dtrace: the anal probe

So I'm listening to the ps pipe grep podcast the other day, and with his always interesting sense of humor, Jason provides this superb quote,

Dtrace is an anal probe for your application.

Classic.

John Nack Slashdotted My Flickr Photos of Adobe

The good John Nack mentioned my photos of Adobe that are on Flickr. And, of course this was about as close as they will get to getting Slashdotted, er, I guess that'd bhttp://beta.blogger.com/img/gl.link.gife Dugg these days.

I should of course make one correction. John mentions I'm a Photoshop engineer. This is only partly true. I used to be full time on Photoshop, but my features are complete (wait for the Photoshop CS3 Beta tomorrow, and I'll blog about them). I'm now actually back to doing web services and web applications (hopefully more on that early next year).

The Photoshop beta is very exciting for those of us who've worked on it though. And of course it's great for folks who've been clamoring for a Mactel version. This will be on tomorrow.

There are some other interesting things on Labs recently as well. For example, check out Kuler and Soundbooth. Really fun and great stuff that's been happening at Adobe. And of course there's all the great Flash and Flex related bits, which I have personally been using a lot.

Amazon's UnSpun done with Ruby on Rails

Pretty cool... Amazon's latest service/site, UnSpun, was implemented with Ruby on Rails. I knew Amazon was looking at Rails, but this is great to see.

Winter Evening in Old Town Auburn

Winter Evening in Old Town Auburn
Originally uploaded by Christopher Bailey.

Gary and I went out for a quick photo shoot in the drizzle this evening. We started near the courthouse in Auburn, and then walked around Old Town. I will have to go back, as it took a while to start finding shots I was interested in. But, I got a few. Gary also took some nice ones.

Adobe Sixth Floor At Night and Other Pics

AdobeSixFloorAtNight.jpg
Originally uploaded by Christopher Bailey.

I took some night shots the other night, while working on servers in the data center (had some time to kill while waiting for Linux to install). This one is shot from the six floor walkway and basketball court area. There are some other shots that I did time-lapse that show some of the planes that fly directly over our buildings.

Hello BingoDisk, Goodbye iDisk

I've had a Bingo! disk for a little while now, and it's just making the iDisk of .Mac look pathetic. I don't use iDisk for much, except keeping some documents I need to keep available anywhere, and off-site, as well as sync'ing those across machines. But, iDisk performance is horrid, and Bingo disk is nice. So, I'm no longer bothering with iDisk, and hopefully I will not renew my .Mac account (not likely yet, because my main use for it is actually keeping Address Book, iCal, etc. in sync across various machines without having to think about it).

Camera and Camelbak Backup Solution #1

Following up on my previous post about needing a backpack that was both for camera gear and held a Camelbak, for use mountain biking, I've identified my first solution. After further discussion with Dakine, and having talked to various others, I'm going with the Ridge pack (in black/olive) and their Camera Block. Dakine suggested this, and believes it should work. The only question is that it may be a bit of a tight fit with a full Camelbak bladder and Camera Block, but we'll see.

Their packs, with the diagonal ski carry are still the most appealing to me for carrying a tripod. I also like the back access and full/instant access to all camera gear that it and the Camera Block brings. I should get these within a week or so, and will report again once I've had a chance to ride with it and try it all out.

Looking for combo camera and Camelbak pack

I've been searching for a backpack for mountain biking while carrying camera gear (digital SLR, extra lenses, etc.). There are many camera backpacks, but none seem to have a Camelbak/hydration sleeve/compartment. If you know of one, please let me know (add a comment). I've started compiling a list of some that may work, or that I might be able to customize or so on, on a backpack page. My current setup is just to individually carry camera+lens in a padding neoprene Zing pouch, and then carry lenses in separate lens bags, and stuff these in my Camelbak Transalp, which is a fairly large Camelbak. This works pretty well, but I'd also like to carry my tripod, and have a more organized setup, preferably with a bit easier camera access. If only the Dakine Sequence had Camelbak sleeve.

I haven't tried putting my tripod on the Transalp yet. I may be able to lash/secure it to it, but not sure. The tripod is a magfiber 4-section Manfrotto, so it's relatively light and a bit shorter than average. Regardless, it's not just a little thing to lash on.

We need a large scale web app developer/architect

For our team at Adobe, we're looking for an experienced developer who knows larger scale web apps: how to design them, build them, integrate with hardware stacks, and so on. We already hired the more junior spot of the two we have open, but check out my previous post for the description of the more senior position. This promises to be an exciting gig, as we're using cool technology, building some great apps, and the team is a fun one.

Lake Putt and the Wide Angle

LakePuttFloatingRock.jpg
Originally uploaded by Christopher Bailey.

Yesterday I drove up the hill to shoot, and try out my new EF-S 10-22 wide angle. This lens is a lot of fun! Not a lot of great photos from the shoot, but a couple nice ones. Here's the set of four on Flickr.

Photography Beginnings

I've started to get into photography, and recently finished a short class. The class was given by my father-in-law, who's an avid photography, and while he does some work for pay, it's primarily a hobby. The class was very good, as I am a serious n00b when it comes to real photography (i.e. using more than a point and shoot, and having proper aesthetics and composition). But, it's been really fun, and I'm starting to get some pictures. I put up a of some I took for the class.http://beta.blogger.com/img/gl.link.gif

Espresso vs. American Fear of Death

As seen on Labnotes:

“Serving size is about the American fear of death. Instead of a great, short experience, people want to prolong a mediocre experience.”

James Freeman of Blue Bottle Coffee

Upcoming Adobe Apollo Seminar

Adobe is doing another Understanding Apollo seminar. The last one was very good, so I'd encourage folks to check it out. I've been doing some work with Apollo and it's really cool. The ability to create web apps that work online and offline has been a huge interest of mine, and really ups the usefulness of many apps.

Web Developer Jobs at Adobe - Rails, Flex, RIA's, etc.

We're hiring at Adobe (we always are, but my team specifically)! We have two positions, with one being essentially more senior. See the job descriptions below. If you're interested, and meet the requirements, email me your resume (in PDF) at chris.bailey at adobe dot com.

Job JV110630

Position Summary:

The Digital Imaging Services team at Adobe Systems is looking for a superior web services developer to help us make our hosted applications and web services architectures best of class in an entrepreneurial and fast moving environment.

You need to have serious Web Development chops targeting LAMP-like platforms using the latest tools. This means you've implemented web services and hosted applications that utilize the latest dynamic techniques and languages, can simultaneously develop for multiple operating systems, know and use n-tier architectural patterns, can sling around SQL, a couple of scripting languages, and automate unit tests with ease, and may have even scaled up a hardware or network infrastructure or two. You should have demonstrated experience working iteratively and incrementally in an agile fashion with a high performance team. Be prepared to explain some of the architectures you've developed, answer coding questions, and tell us about your successes working with a dynamic team!

Knowledge & Skills:

• Expertise with developing multi-tier, distributed web application architectures and deploying in live production environments.
• Experience with Java, Ruby on Rails, or other current technology stacks required, experience with native code development in C/C++ a big plus. Four or more years of hands on LAMP (Linux, Apache, MySQL, Perl/Ruby) development.
• Experience using quality focused development practices such as heavy unit test and automation tool usage or Test Drive Development strongly desired.
• Experience of successful deployment of multiple iterations of a commercial/publicly accessible high traffic web service or application required.
• BS/MS degree in Computer Science (or equivalent).
• Must be able to work both independently and in a focused and efficient Agile engineering team that is geographically dispersed.

Job JV110631

Position Summary:

The Digital Imaging Services team at Adobe Systems is looking for a superior web services and infrastructure developer to help us make our hosted applications and web services architectures best of class in an entrepreneurial and fast moving environment.

You need to have serious Web Development chops targeting LAMP-like platforms using the latest tools. This means you've implemented web services and hosted applications that utilize the latest dynamic techniques and languages, can simultaneously develop for multiple operating systems, know and use n-tier architectural patterns, can sling around SQL, a couple of scripting languages, and automate unit tests with ease, and may have even scaled up a hardware or network infrastructure or two. You must have experience building infrastructure that is highly available and has had to grow quickly due to rapid uptake. You should have demonstrated experience working iteratively and incrementally in an agile fashion with a high performance team. Be prepared to explain some of the architectures you've developed, detail growing pains and solutions while scaling SW/HW/Net infrastructure, answer coding questions, and tell us about your successes working with a dynamic team!

Desired Talents:

• Analytic.
• Learner.
• Communication (written and oral).

Knowledge & Skills:

• Expertise with developing multi-tier, distributed web application architectures and deploying in live, high availability production environments.
• Expertise with current commodity based hardware, networking, and software infrastructure and related Operations exposure.
• Experience with two or more of Java, Ruby on Rails, Perl or other current technology stacks is required, experience with native code development in C/C++ a big plus. Eight or more years of hands on LAMP (Linux, Apache, MySQL, Perl/Ruby) or similar development.
• Experience using quality focused development practices such as heavy unit test and automation tool usage or Test Drive Development strongly desired.
• Experience of successful deployment of multiple iterations of a commercial/publicly accessible high traffic web service or application required.
• BS/MS degree in Computer Science (or equivalent).
• Must be able to work both independently and in a focused and efficient Agile engineering team that is geographically dispersed.

DualingMacLaptops

DualingMacLaptops
Originally uploaded by Chris Bailey.

This is the frequent configuration of my desk in my home office lately. That's a MacBook Pro sitting on top of a PowerBook. I've been doing some things lately where the MBP is a big advantage, so have been bringing it home, even though I have the PowerBook to use at home. The PB's days may be numbered.

Note, I don't have them both on (i.e. fully awake) at the same time. The MBP is currently on and in use, but the PB is sleeping. I also use the handy smcFanControl to ensure the MPB's fans are cranked up and keeping it nice and cool.

FirewoodStackWinter2006

FirewoodStackWinter2006
Originally uploaded by Chris Bailey.

Here's my morning workout - stacking a half cord of nice oak firewood. About another quarter of it is on the side of the house, as it doesn't all fit in the rack (and this is claimed to be a half cord firewood rack :)

Adobe Open Source ActionScript code - attend the chat today

Today, there is a Mozilla developer chat with Brenden Eich and Kevin Lynch (Adobe), discussing Adobe is contributing source code from the ActionScript Virtual Machine to Mozilla! Learn more here.

Encryption choice dramatically effects WiFi range

I've had range problems with the WiFi in our house for a while. And more than that, it seemed that the older iBook we have in the kitchen simply wouldn't deal with our encrypted WiFi. Last night, after setting up WDS (Wireless Distribution System: basically a way to link many WiFi base stations up to provide a bigger network range) I learned the bigger reason.

The kind of encryption seems to make a BIG difference. I had been using the strong WPA2 Personal encryption. This worked just fine within close range of the base station. But, terrible in other spaces that had worked before with an older base station. Switching to WEP 128-bit dramatically changed things, and now I have excellent coverage again.

The WDS setup also helped in one room in particular, and is relatively easy to set up. In case you're interested in that, here's two resources for doing it when using a Linksys WRT54G base station and then either Airport Express or Airport Extreme as the extension:

Linksys WRT54G, Airport Express, and WDS
Extend a Linksys WRT54G network via AirPort Express

Upshot: our kitchen laptop, which is about the furthest point from our upstairs base station went from sometimes seeing the network, but never being able to connect, to now having a solid 4 bar connection. Similar improvements in other places.

HalloweenPepperBBQ

HalloweenPepperBBQ
Originally uploaded by Chris Bailey.

I'm taking today off. I'll be doing two primary things: cooking and coding. I'm going to do some coding work on my side project, as well as cooking for the little family Halloween get together we're having. As you can see from the pic, I'm grilling various peppers, some of which will be eaten as is, others are diced and put into a salsa I'm making. I've already got the chili simmering on the stove, and the immense cheese platter is up next. But, right now, it's time to take a break and code for a little bit. Happy Halloween!

Fall Leaves of Beauty

FallLeafOct2006
Originally uploaded by Chris Bailey.

Here's a beautiful leaf from one of our backyard trees. Nearly all the leaves on this tree look like this right now, and it's just stunning. Taken this past weekend - the leaves have already changed quite a bit since.

Moving to Eugene, Oregon?

My wife, kids, and I have been thinking about moving for a while now. We can't stand the heat here in Rocklin, CA, and have been wanting a different kind of town - less strip malls, more outdoor and healthy living oriented culture, better coffee ;-) and so on. We're now planning to move to Eugene next year, which is relatively shocking to me. I had thought we'd pick Bend or Portland, and was fairly biased against Eugene, but then being thorough, went to visit last week. We wound up liking it quite a lot. Also, we went into 10 houses with a realator and liked a lot of them, and it's reasonably priced (by CA standards).

The main downside from my perspective so far, is that the startup and tech culture is relatively minimal, Portland being quite superior. But, I have a ton of experience working remotely (did so for 6 years for multiple employers), so know it can work. Mainly I'd just miss the energy of being colocated with your other startup cohorts. For the time being I'd simply plan to work remotely in my existing job.

All in all, quite exciting. We've even found a house we love, so the question now becomes whether we go for that, and own two houses until we can move and sell ours, or wait a bit. Might be a crazy next few months.

New Camera Gear

I'm quite excited, as I just ordered a new camera. I'm getting a Canon Digital Rebel XTi. I chose to get the "throwaway" 18-55 lens with it, which we did not do when we bought our Rebel XT, as instead we got the super sweet EF-S 17-85 lens, which is awesome. But, since we had the 17-85, and I also ordered a 70-300 IS lens, and knew I couldn't take the 17-85 away from my wife much, I got the cheap 18-55 to mostly fill the lower/wider angle end. We also recently got the EF-S 60mm macro lens which is pretty damn cool if I do say so.

Photography is a new thing for me, but I've been really enjoying it, in large part to using a DSLR. I used to use point-and-shoot digital cameras, and they were fun, are good for carrying on a mountain bike ride, etc., but the DSLR is just so amazing in comparison. I take much better pictures with it, in part I think because it makes me take a bit more time and simply the heft of it makes me more serious about it I guess.

I chose the XTi this time almost completely on price. Dell had this superb deal going on them, combined with a 20% coupon, which got me the body and 18-55 for $635 which is a screamin deal. I'd been debating between it and the 30D, which my father-in-law just got, as well as a friend. The 30D is super nice, but the Rebels are quite fine for me, and the nearly $400 I saved payed for most of the 70-300 lens.

GMail Package Tracking Rocks

One of the things that's been added to Gmail somewhat recently (AFAIK), is that it senses package tracking numbers in emails, and adds a little link to track it in the upper right of the mail window. This simply rocks. It's a little thing, but darn if it isn't just a big time saver and great convenience. Small things like this can really set services apart.

Bookmarklet for Pandora mini player

Matt recently turned me on to the Pandora music service. Very cool - check it out if you haven't. I always like to use their mini-player, but I wanted to be able to just bookmark that and have the mini player open. So, I created a bookmarklet for it. Drag the link below to your bookmark toolbar (in Flock or Firefox; other browsers use the appropriate mechanism). Now, whenever you click that link, it'll open a new window with the Pandora mini player.

Pandora mini player

Resources for Restful Authentication in Rails

I've been doing a fair bit of REST web service work in Rails apps lately. One of the interesting parts is doing the authentication pieces, and coordinating this with your regular interactive authentication. I've also been ramping up on the whole Simply Restful thing. I'm not totally sold on it, but the one thing that is definitely appealing about it is the further constraint it places on your code for which that primarily comes down to enforcing more consistency.

Anyway, in terms of REST WS API's, and authentication, there's a new Restful Authentication plugin that uses acts_as_authenticated. The San Diego ruby user's group had a video podcast and slides that touch on it briefly. These point to NAME's blog entry on using this for REST WS and authenticated feeds as well. Good stuff.

Why SSH stopped working on my Mac - with solution

I use SSH a lot, and recently it stopped working on one of my Macs here at home. I couldn't figure it out for a long time. The only thing I'd changed was that I got a new Linksys router (a WRTG54, which has WiFi and a switch). Previously I was using a Linksys router that was single port, and no WiFi.

Long story short, it came down to the DNS setting on my Mac. Even since I used SonicWall's, I had set my DNS entry to that of the router, not those of my ISP. This was nice, because then if the ISP changed DNS entries (which I pick up via DHCP), I'd be insulated. Well, apparently with the new Linksys, this somehow messes up SSH. It doesn't seem to bother anything else.

The one other nice change though, is that SSH connections now happen noticeably faster than they used to. Thought I'd pass this along in case it ever happens to anyone else.

Installing Rails, Apache 2.2, and Mongrel on Ubuntu Dapper 64-bit

I am setting up a system at SliceHost, on one of their Ubuntu Dapper slices. First, I'm not as familiar with Ubuntu as I am Fedora, but this is one reason I chose Ubuntu (to get more experience). Anyway, I found several other blog posts and such on the web about this, but none worked perfectly. I glommed mine together by using them though. Here they are:

• Simplistic Complexity
  
• Free Opinion
  
• Urban Puddle
  

And, here's what I did. First, uncomment the universe lines in /etc/apt/sources.list. Then...

# apt-get update# apt-get dist-upgrade
# apt-get install ruby ruby1.8-dev ri rdoc
# ln -s /usr/bin/irb1.8 /usr/local/bin/irb# wget http://rubyforge.org/frs/download.php/11289/rubygems-0.9.0.tgz
# tar xvzf rubygems-0.9.0.tgz
# cd rubygems-0.9.0
# ruby setup.rb
# cd ..
# rm -rf rubygems*
# gem install rails --include-dependencies
# dpkg --purge apache apache2  (this removes Apache if it exists, which it didn't)
# apt-get install build-essential zlib1g-dev
# wget http://apache.rmplc.co.uk/httpd/httpd-2.2.2.tar.gz
# tar xzvf httpd-2.2.2.tar.gz
# cd httpd-2.2.2
# ./configure --prefix=/usr/local/apache2 --enable-mods-shared=all --enable-deflate --enable-proxy --enable-proxy-balancer --enable-proxy-http
# make
# make install
# cd ..
# rm -rf httpd*
# apt-get install postfix  (Then, use the "Internet Site" default setting, and picked "WeighPals.com" as the name)
# apt-get install mysql-server mysql-common mysql-client libmysqlclient15-dev libmysqlclient15off
# apt-get install libmysql-ruby1.8
# gem install daemons gem_plugin mongrel mongrel_cluster --include-dependencies

technorati tags:RubyOnRails, Apache, Ubuntu, Ruby

Friday Night Coding

Some might consider it sad, but I'm really enjoying spending my solo Friday night coding.  My wife and kids headed to the Bay area earlier this evening to spend time with the grandparents and some of my wife's friends.  I have a project I've been wanting to work on for a while, so I'm taking the opportunity to hack for the evening.

Right now I'm enjoying:

• listening to Pandora, a very cool service
• sipping a decaf espresso (I don't need to stay up that late, and I had a regular espresso earlier this afternoon :)
• Hacking Rails code in TextMate, with ZenTest's autotest keeping me on track.  The project involves a few interesting things:
  
• Amazon's S3
• Music files
• Some "Web 2.0" features (natch)
• Client side code (just for Mac so far, but hopefully Linux and Windows as well)
• Figuring out where I'm going to host it.  I believe I will have one of TextDrive's new containers soon, but we'll see.
  

Hope you're enjoying your Friday night as well.

technorati tags:rubyonrails, TextMate, ZenTest, TextDrive, S3

RocklinSunset30Sep2006.jpg

RocklinSunset30Sep2006.jpg
Originally uploaded by Chris Bailey.

We had a beautiful sunset this evening. Made it home from a walk with the kids, ran in to grab the camera, and got what I could.

Gmail sucking lately?

Anyone else find that GMail is incredibly slow lately? Half the time it seems to time out or fail to do various things. This has been the case for several days now. I have a GMail domain account as well, and performance there seems fine, so that's interesting. It's a real bummer, makes using it rather painful.

Problems with RubyGems and Platform Specific Gems

Something that at least a few of us are annoyed with is the case where there is a ruby version, as well as a platform specific version of a gem. Good examples are mysql and mongrel, amongst others. I see a couple of issues here, but the crux of the issue is that this kills the ability to automate (non-interactively) installation.

First, why should the system even offer the win32 version of the gem to me if I'm not using a Windows machine? Fixing that would fix the problem for many people.

But, let's say you are doing multi-platform development, and in particular include in that Windows. I think RubyGems ought to be updated so that you can specify a platform variant when you install a gem, much as you can specify a version. This way, your automation script could simply specify the one it wanted, given the platform it was running on.

The other thing I'm wondering about is what solutions people have in terms of being able to freeze your gems, but where you have gems that have native code, and where you work on multiple platforms. This must be extremely common in the Rails world, where it seems the standard developers machine is MacOS X, and deployment servers are typically some UNIX flavor. Also, I've yet to try it, but I presume things don't behave well if you freeze gems on one platform and then go to another - I assume it freezes gems that include native code?

Anyone have solutions to these issues?

technorati tags:Ruby RubyOnRails

WebORB for Rails: connectivity between Flex and Flash Remoting clients and Ruby on Rails

WebORB for Ruby on Rails | Flex RPC and Flash Remoting for Ruby on Rails

This is just really cool. Take a look at the page. It makes putting a Flex front end on your RoR apps really easy. What's nice is that it's an easy install, and then it exposes your Rails objects as Flex Data SErvices or Flash Remoting gateway, which means that it's the natural way for Flex or Flash to work with it. In the past I've connected Flex apps up to Rails apps via web services and JSON as the data transport. This works just fine, but it appears this will be a more direct and natural integration for Flex. Here's the diagram from their site:

technorati tags:RubyOnRails, Flex, Flash, FlexDataSErvices, FlashRemoting, WebORB

Resizing hard drives in Parallels on Mac

Parallels is a very cool virtual machine system for Mac (and Windows and Linux). Particularly cool is the ability to resize the hard drive image you have for a given VM. But, the Parallels manual is not accurate on how to do this, or rather is missing some steps (maybe they don't intend for you to have to do this, but haven't finished this functionality?). A thread on their forum leads to the solution, but the thread is a bit jumbled and you need to put together the full solution. For ease, here's the step by step I used to make it work:

1. Shutdown Parallels


2. Run the "Image Tool" that comes with Parallels.


3. Resize the disk you want (it needs to be an "expandable" disk, but Parallels makes them this way by default. If yours is not for some reason, read their manual on how to convert it to one).


4. I did not bother making a backup of my image prior to this, but of course, standard cautions apply.


5. Now, what Image Tool actually does, is just to make the disk bigger, it doesn't actually affect the change on your virtual disk volume.


6. Go get the GPartEd LiveCD. You don't need to burn it to a CD.


7. Now, start up Parallels, and do two things for your virtual machine:
   
   


1. Change the boot order, so CDROM is first, and
   
   


2. In the CD options, change it to use a disk image, and then go browse to the GPartEd Live CD ISO you just downloaded.




8. Run the VM, which will boot into GPartEd. Accept all the default options.


9. Once GPartEd is running, select the initial partition, and click the Resize toolbar button. In the resulting dialog, drag it to fill the additional space, hit OK; then Apply the changes. Shutdown GPartEd.


10. Boot up your Parallels VM. If doing this for Windows, you'll see Windows do a chkdsk. Once booted, you'll see it say you have new devices and need to restart. This is fine, do so. Once it restarts and is back, you're good to go.

Headed to Scale With Rails Workshop; and Interesting Rails Deployment Options

Next week I'm headed to the Scale with Rails workshop. I'm hoping this will be quite useful and interesting. It will also be cool to test out TextDrive's new container hosting setup. I've been doing a lot of investigation on VPS and similar systems lately. Also, another very intriguing setup will be the Engine Yard hosted setup. I've been talking to Tom there, and I like what I hear so far. Even cooler is that they are in Sacramento, so I plan to get together with them in the near future and discuss even more. Another non-traditional, much more supported VPS setup is Rails Machine. These are all quite appealing to anyone who wants to concentrate on development of their app, and leave the OS maintenance and related bits to someone else that may have more expertise in that area. For my personal setups, this is quite appealing.

technorati tags:rubyonrails, TextDrive, RailsMachine, EngineYard, VPS

Using Non-CRUD Actions With Simply Restful Controllers

I've been making use of the simply restful integration in my Rails apps. However, one of the things that took a bit of digging to figure out, was how to map additional (non-CRUD) actions. If you try to simply use a URL of /controller/action that won't work, because it'll treat the action as an ID.

I came across this blog entry on Ryan's Scraps, which explained how to add more actions to the mix. That got me most of the way there. However, what happens if you want to support both GET and POST to an action? A good example of this is a login form. You want GET to show the form page, and POST to do a submit to the login action. As it turns out, you can do this easily with the following mapping:

map.resources :users, :new => { :login => :any }

For more good info, and more of a primer, on simply_restful, check out David Goodlad's post.

technorati tags:RubyOnRails, simply_restful

Flash Player 9 for Intel Macs Now Available!

The release version of Flash Player 9 is now available for Intel Macs.  Run and get it.

technorati tags:Flash

Confluence vs. Basecamp

I've recently been using Basecamp for some project management.  It was working fairly well.  I can definitely see the value if you are a consulting firm or similar where you are manageing multiple clients/projects.  The main drawback relates to their claimed strength of simplicity.  Sometimes it's too simple.  I'd like to see multiple todo pages for one.  Also, my real complaint is speed.  Basecamp seems very slow to me, painfully slow at times. 

On a new project, we evaluated Basecamp vs. using a wiki and sort of doing it ourselves.  Due to various constraints our wiki choices were MediaWiki (which is what powers Wikipedia, and many more), or Confluence.  MediaWiki definitely came up short.  We have a range of people using this, from engineers to marketing folks.  Wiki's sometimes present stumbling blocks to those who prefer to write in more of a word processor environment.  Also, we really wanted a good task list system.  We tried the two task extensions for MediaWiki, but neither seemed to actually work properly.

I then setup Confluence.  It should be noted that it is not open source and has a significant price tag that may be a prohibitive factor for many (note to open source folks though that they have arrangements for that).  Setup was simple drop-in and run a startup script (as well as create a database in your DB engine of choice). 

I am impressed!  Confluence is a superb wiki!  I've used several other wikis over the last year or two (we use twiki heavily as well, and I've tried a half dozen others).  Confluence takes it up a notch.  It feels polished and professional, and it's been a joy to use.  Also, the less-technical folks are finding it a lot nicer to use as well.  And, their task extension (we're using the Enhanced Task List macro) is quite nice, perfect for what we're after (fitting our Scrum use).

Another thing I like is how it manages hierarchies of pages, and provides automatic lists of child pages at the bottom of a page.  This allows you to forgo linking when you don't need to.  Also, the mail features, and RSS abilities are well done.  So, I'm sold, and we're proceeding full steam ahead with Confluence.

technorati tags:Confluence, Basecamp, MediaWiki, wiki

My contribution to PicoContainer

Ok, contribution is a bit reaching, but Paul Hammant was nice enough to list/credit me on his latest checkin to PicoContainer.  This is fun, because I'd done a very brief bit of pair programming with him, whilst at the Urban Grind cafe following the last day of OSCON.  We were working on the PicoContainer and JRuby stuff.

technorati tags:OSCON, OSCON06, PairProgramming, PicoContainer, JRuby, PaulHammant

OSCON: LiveJournal's talk

I attended the LiveJournal session on the last day of OSCON.  I've read previous slide decks from them on various scaling issues and their tools, etc.  Also, since I've been thinking about memcached (like everyone else I guess), and MogileFS, it made sense.

The presentation was mostly an overview of the main open source tools they provide: Perlbal, MogileFS, and memcached (see code.sixapart.com page for info).  Perlbal is their very configurable load balancer; MogileFS is a file store, not a file system; and memcached is a caching system.  They did not get into setup details on any of them.  They did cover some general aspects of LiveJournal's setup (and it sounds like VOX is similar), and a few points of interest, some relatively common ideas, some more detailed and useful ones (more on this below). 

MogileFS is quite interesting to me.  It is very similar to Amazon's S3 in general use, in that you store things essentially via keys, not via normal file system directories and files.  So, what's cool about this is that you can have a somewhat infinitely sized system where you don't worry about limitations on directory contents, file names, disk space, and so on.  Further, MogileFS allows for a "class" notion, which you configure to mean how many redundant copies you want.  The typical example is that for photo storage, you'd have an "originals" class where you might have 3-4 copies, and then a "thumbnails" class where you had maybe one or two (since you can regenerate these).  Also, there are namespaces, so you can partition data that way.  You must use a client library with MogileFS, but they are available in many languages.

As a general recommendation with their tools, they noted that you would likely start out with your normal system of some web servers and DB server(s).  Then, you'd add in memcached, next Perlbal in front, and finally add MogileFS (which essentially requires at least two machines, if not three or more depending on how you configure the three pieces involved for MogileFS).

They also noted that they use Gearman for job queues where the jobs don't matter, and either they're done in 10 seconds or they're gone.  They then use dschwartz for queues of durable tasks, that you care about.  You can send in multiple of the same task, and they get properly coalesced into only being done once.

Some of the interesting points I noted were:

• LiveJournal started on a shared hosting setup, then grew to dedicated, and so on until where they are today.  They are still in a single data center, although looking to either go to multiple, or move (their data center is in SF, and they would like to be in a less environmentally disasterous area).  They used to have a data center in Japan, but it wasn't yielding advantages.  This was info I gathered when I talked to them after the session.
• They use lots of cheap SATA drives, and they found that, at least today, the 250GB disks are better than the 500GB disks, because they are less susceptable to heat, and can be used to capacity, whereas the 500GB's could only be used up to about 350GB. 
  
• Further, they do not use RAID (5 anyway).  They do use RAID 0 or RAID 10 say for database systems, but not for the file systems.  With MogileFS, they don't need it, and it doesn't handle all problems, like power failures and so on.  Also, fsck doesn't run in parallel, so it can take a long time.
• Continuing this, use lots of small machines, so you spray the disk writes across them, as opposed to big honkin' systems.
  
• Make sure MySQL is not a SPOF (single point of failure)!  Single big MySQL boxes are not good, better to have smaller ones that are faster and cheaper and where MySQL can perform better (better/faster IO).
• They think they'll get rid of Apache within a year or so (Perlbal is enough).  They have BigIP boxes in front of Perlbal's for simple load balancing, and these are nice boxes, but they don't know how truly busy the Apaches are.
  
• MySQL is the only thing in their system that blocks.
• They're moving to dual, dual-core machines in 1U boxes to cut down on heat.  4GB of RAM per box.
• I didn't get the exact config of MySQL, but they said they have about 25 boxes running it, with varying configs for different things.
• They don't explicitly talk to the Odeo guys, but do see them regularly.  I had asked about this because Odeo is doing various Ruby versions of some of their tools (memcached and MogileFS I believe).
  

Note, for an older, seemingly slightly out of date slide deck, check out their presentation from last year's OSCON.  This year's presentation I have yet to find online.

technorati tags:OSCON, OSCON06, LiveJournal, Memcached, Perlbal, MogileFS, Gearman

Leaving OSCON

I'm at Portland Airport right now (hurray for free WiFi!), about to head home.  OSCON 2006 was great.  Far and away the best part was meeting people, and the impromptu gatherings.  Yesterday afternoon I hung out at Urban Grind with Paul Hammant, Greg Luck, James Duncan Davidson, and several others, and had a great time, then continued on to dinner with a subset.  We talked Java, Ruby, JRuby, Python, did a tiny bit of pair programming, and tried to find Perl folks (we finally found Matt, a young guy who's working on Parrot, and just finished and internship at Amazon).  Dinner was great, and I'm signing off now, as I have 3 hours of sleep and need to get on the plane.  I'll backfill various notes, session info, and stories over the coming week.

technorati tags:oscon, OSCON06, coffee, JamesDuncanDavidson, PaulHammant, GregLuck, Java, Ruby, JRuby

OSCON Day 1

Had a decent day at OSCON today. Started off with Amy Hoy's JavaScript Bootcamp. Unfortunately, like some folks I talked to at lunch, some of the sessions today were too high level. This one, for me, was just way too basic. It was a very brief intro to JS. From the description of the session, I knew there would be some tutorial, but it also mentioned getting into more AJAX and JS libraries, so I'd figured I'd sit through the initial part. But, alas, AJAX and things like prototype were covered for about 5 minutes at the end. So, while it was not a bad session as a JS intro (there were folks there that were clearly learning JS for the first time, etc.), it was basically a total waste for me. Amy overall was a fine presenter (especially given she was sick). The only dig I'd make was that she didn't realize do-while constructs were different than a plain while's. This is somewhat stunning in that this is the same across nearly all languages, and if they were the same, why would they have them both (oh wait, maybe Perl people would want those multiple ways ;-)

Lunch, or rather lunchtime (no lunch provided at conference) I talked to some folks about OSS in general and their interests (and mine). Two of them were into Asterisk, with one having set up a home Asterisk server so he could tie into his company's phone system. We also briefly talked about sshfs, which is something I'll have to look into, sounds interesting.

After lunch I headed to the Powell's mini-bookstore and lucked out in that Lucas Carlson was there signing his newly released Ruby Cookbook book. So, I picked that up, and one other book.

The afternoon session was a little better. I attended the Rails Guidebook session with Mike Clark and Dave Thomas. These guys are great speakers. Since I've been working with Rails for a bit, I knew roughly the first 75%. But, I hadn't yet gotten into RJS, so that was cool, even if brief. There were a few other tidbits that weren't in the slides, but were good, such as on deployment. Specifically, Dave mentioned that, paraphrasing, most of the people he knew that were running heavy traffic sites were using file based sessions on NFS volumes, in order to do distributed session management, as opposed to memcached. He mentioned that memcache[d] was a bit of a pain to set up. Also, the current "best" (my take, based on their comments, and various bloggers) deployment setup is Apache 2.2 with mod_proxy_balancer and Mongrel. I've mentioned this, with a few good links, in a recent post.

At the end of the day, I took the MAX train over to downtown PDX to meet coworker Mike Potter for dinner. We had great conversation and tasty Italian food at Pazzo Ristorante. So, the day improved as it went along, and that's a good trend. I took the picture above from my hotel room this evening. See you tomorrow...

technorati tags:oscon, OSCON06, AmyHoy, DaveThomas, MikeClark, RubyOnRails, Asterisk, sshfs, MikePotter, OpenSource

121 Degrees!

Ok, in case you didn't think it was hot, here's my neighbor's car's thermometer as he drove back into town. Ya, 121 degrees! Ok, so the car thermometer is not the most accurate, but close enough!

technorati tags:Temperature, Heat

OSCON Day 0

I've arrived!  w00t!  (cant I say that?)  Anyway, flew in, flight was only an hour late--listened to some podcasts and relaxed.  As hot as it may be here in Portland, it's not as hot as at home (113 today).  Took the MAX train from the airport to the hotel, super easy.  Checked in, then went and did the registration thing at OSCON.  It's about a 10 minute walk from the DoubleTree hotel here. 

After that, I came back to the room and wanted to do a video chat with my kids (4 year old daughter and 18mos old son).  But iChat was screwing up for some reason; and it didn't give me any useful errors!  Normally at hotels I may run into their network being too slow, but iChat will tell you that.  Nope, here it was just something obtuse so I sent it to Apple a half dozen times.  A real bummer though.  I'll have to try from OSCON tomorrow, or maybe Urban Grind (there's one a few blocks away).  Apparently this is where all the kewl kids hang out, although probably at the one in the Pearl.  So far in my short view radius I've only seen Starbucks :(  I'm now hitting the Delocator though.

Just ordered room service with a local brew.  Hopefully will go out tomorrow night.  Now time for a little code before dinner...

technorati tags:OSCON, OSCON06, Portland

FOSCON II: The Ruby Rodeo

Are you going to OSCON? Interested in Ruby? Then make sure to check out FOSCON! There will be speakers, pizza, etc. See you there.

technorati tags:OSCON, FOSCON, OSCAMP, Ruby

Virtual Servers and Web App Deployment

I'm getting very interested in server virtualization system, such as Xen.  And, more to the point, using virtual server hosting options, as opposed to either a regular shared host setup, or a fully dedicated machine/colo.  One reason is cost as related to the configuration of your setup (and this assumes you do not need to own the entire machine due to your application's needs).

For example, maybe I have a budget of around $500/month to get a web app going.  Well, that'd buy me a couple dedicated servers (at a say mid-priced place) with probably minimal to "no" support.  Or, it'd buy me somewhere between 8-10 virtual servers on a virtual server system.  The virtual servers are far more RAM constrained (this seems to be the big dividing line).  For example, those 8-10 server instances would have around 192MB of RAM.  This appears to be decent for running a basic Rails stack for example.

But getting back to it, the reason I find this appealing, at least for early deployment, is that it would allow me to mimic a more fully scaled solution simply due to having more "servers."  Instead of having say one web app server and one DB server with the full colo setup, you could have say a half dozen web app servers, a cluster of DB servers, and then maybe another box that you dump backups to, or dump logs to, or whatever.  What it really lets you do though is setup your fully scaled architecture, something like this, this, or this.

It all amounts to being able to test and setup your system in a more real-world scaled design from the get go, instead of having to redo that in the midst of your site being hammered with traffic.

technorati tags:scaling, deployment, RubyOnRails, Xen, VirtualServer

Time Breakdown of Modern Web Design

camembert.png (PNG Image, 550x410 pixels)

technorati tags:web, design

Store/Sync Your Data Online?

We're still working on the problem of synchronizing data online/offline, but the storage issue looms large. Will users be comfortable having all of their information stored online? I think so, but RIA developers need to take pains to ensure that the data is secure and trust is not misplaced.

Digital Backcountry - Ryan Stewart on Building the Web with Flex and the Flash Platform: RIAs in "World War 2.0"

The above topic in general, i.e. whether or not to keep a copy of your data, or synchronize your data online is becomming a bigger and bigger question and issue.  This is something I'm very interested in, and doing some work on myself. 

I don't believe it's a simple issue of all your data online or not.  I do however want ubiquitous access to my data, but most solutions so far come up very short.  Mainly this has to do with security and privacy.  I already use del.icio.us, Flickr, Backpack, Basecamp, Gmail, and various other tools that store some of my data online, and or hold a copy of some of that data.  But, I most surely do not keep my Quicken files online, or various other documents and information that I consider particularly sensitive or needing explicit security. 

But, I really want this.  I want all my data available on any computer I use.  I'm starting to look at things like S3 to create my own solutions.  One notion is to store everything on the server encrypted.  This works, but becomes a significant overhead when you start storing large amounts of data, or large files like media.  Maybe it is an option per file or per resource location.

I think it will be very interesting to see what Google and folks do in regard to this aspect.  Sure, GDisk would be great and all, but only for some things, at least until there's some level of security.  And believe me, just having some code of conduct, terms of service, and employee guidelines for those working at the data centers, doesn't mean there can't be abuse - you need to make it essentially impossible for folks to see your data (when you care about security/privacy of said data).

technorati tags:DataSynchronization, web2.0, RIA, Google, Amazon, S3, GDisk, privacy, security

Al Gore's Excellent Speech

I recently listened to a superb speech by Al Gore, which he presented to the Stanford Business School. Mr. Gore is a great speaker (he was far more engaging than I think past criticism has claimed). The speech was dead on, interesting, and damnit, people should just think and listen to this stuff!

The industrialized world is on a collision course with nature, says environmental hero and leading expert Al Gore, who passionately urges a Stanford Business School audience to take action to save the environment. In his presentation on global warming Gore presents with alarming clarity, conclusiveness, and humor, that the fact of global warming is not in question and that its consequences for the world we live in will be disastrous if left unchecked.

Odeo: Al Gore - The Earth in the Balance

technorati tags:AlGore, Weather, Environment, Green

This Morning's Capuccino

17Jul2006Espresso.jpg

Illy beans, non-fat milk, Illy cup, made on Jura Capresso Impressa S9.

technorati tags:Coffee, Cauccino, Illy, Jura, Capresso

Freezing Rails and Gems - how to leverage for version control?

Something I want to do for our development environment is to put Rails, and all our Gems, as well as other tools we use, all under version control.  The way we do this now is to freeze Rails, and then put all the other stuff in a directory in our code repository and only install from there.  I was wanting to freeze gems as well, but the way I understand this, it wouldn't work because it freezes what you have installed, which means it is platform dependent (assuming you have some gems that use native code).  I need to support development, currently primarily on MacOS X, but also on Windows, and then various UNIX flavors such as Linux, BSD, and Solaris.

I'm curious what other folks are doing to ensure their versions of everything are correct for any point in their source code control?  I also want to do this with non-Ruby technologies as well, so it's generic problem.  Things like MySQL, DarwinPorts, and so on.  The script I mentioned I want to write is aimed at handling much of this, but it'd be nice to have the actual "installed" variants in source control so you simply had to sync/update to latest and have everything except a few major bits (MySQL for example) guaranteed to be correct.

technorati tags:Ruby, rubyonrails, rubygems, sourcecodecontrol

Senator Ted Stevens: Master of English, and Internet Authority

Ted Stevens, the Senator who's head of the commerce committee, is well, an Internet idiot, to put it plainly (and in language hopefully he can understand, since he even seems to have a difficult job of putting English sentences together). As always, Jon Stewart has fun with it.

technorati tags:TedStevens, Politics, NetNeutrality

Explorer Destroyer: tweaking it to work on Blogger

Explorer Destroyer - Switch to Firefox, Make money from Adsense Referral

I've just set up Explorer Destroyer on my blog. Their instructions are good, but if you use Blogger to host your blog, and you have the little Blogger bar across the top of your page, you'll need to make one tweak to get the Explorer Destoryer box to show up completely. I've only done this for (and it probably only applies to) the Level 1 version. In the Level 1 script code, look for the following line (currently it's line 150 in the HTML they supply):

<div style="padding: 20px; background-color: #ffffbb; font-family: arial; font-size: 15px; font-weight: normal; color: #111111; line-height: 17px;"&gt;

Add a top margin to account for the Blogger bar, thus changing the line to:

<div style="margin-top: 30px; padding: 20px; background-color: #ffffbb; font-family: arial; font-size: 15px; font-weight: normal; color: #111111; line-height: 17px;"&gt;




technorati tags:Firefox, AdSense, InternetExplorer, ExplorerDestroyer, Blogger

New Job!

I'm very excited.  Beginning in August, after returning from OSCON, I will be changing jobs.  I'll still be at Adobe, but I'm moving from the Photoshop team, to a new team (that is just me (development wise) to start), and back to doing web apps and web services. I'm further psyched because I'll be doing Ruby on Rails (taking a project I was working on in my spare time and making it a "real" project), Flex, and other cool stuff.  I will do some minor continued work on Photoshop, but really be concentrated on this new gig.

I've been working on Rails stuff in my spare time for a while, but now I'll be joining the ranks of those getting paid to do it.  Plus while I have utmost respect for the Photoshop team, and really like the folks there (there is some truly amazing talent, although that's true throughout Adobe), I'll be back to my real strength (and love) which is web and network related apps (and maybe next year when CS3 ships, I can reveal what new stuff I worked on in Photoshop - some rather interesting bits indeed!). 

technorati tags:Adobe, Photoshop, RubyOnRails, Flex, Jobs

SmackBook Pro

SmackBook Pro is rather nifty.  For those of you/us using virtual desktop software (Virtue Desktops or Desktop Manager are the ones it works with), why bother using a hot key, a menu, or a panel, when you could just use your bare hand to smack the side of your laptop's monitor in order to change desktops?!  Check out the link, and watch the video:

Medallia Blog: SmackBook Pro Archives

technorati tags:Apple, virtual desktops, motion sensor

Blogged with Flock

Day 2 with Flock

So far I'm digging Flock.  I'm using it as my primary browser on all my machines now (4 Macs and a Windows XP machine).  I've run into two issues, one minor, the other, I'm not sure:

1. The favorites/bookmark bar does not appear to support folders, like it does in Firefox, etc.  This is a fairly major bummer to me.  I don't use a Favorites menu or anything beyond the bookmark bar.  I use bookmarks solely for the things I use on a regular basis, and thus this holds what I need--assuming I have folders. 
   
2. On Windows, Flock doesn't seem to be able to import all your Firefox settings.  It only seems to see/allow/support IE import, which is pointless for me (I do use IE occasionally, when I have to, and actually in that case I use Maxthon instead, but it's rare).
   

Also, I need to look at bookmark synchronization more.  I was under the impression that Flock did this, but it it's unclear if you are using full del.icio.us with it, if it does anything beyond that.

technorati tags:Flock, Maxthon, Firefox

Blogged with Flock

Trying out Flock

Flock — The web browser for you and your friends

I'm trying out the Flock browser, which is Firefox based. So far it's pretty darn cool. I'm not using every feature (I prefer NNW to Flock's RSS, but that isn't their intent to replace anyway). But, the Flickr integration is cool, and this post is testing out the Blogger integration. The del.icio.us integration rocks. Also, the UI is very nice, and I dig how they've already integrated some of the nicer extension features. I've of course already installed other extensions like Firebug, Web Developer, Gmail, Session Manager, and so on. Anyway, check it out, pretty interesting.

Also, their search box is great - it does multiple searches while you are typing, and shows them in a drop down. So, you can simultaneously search Google, Yahoo, Amazon, Wink, or whatever. It's very nice, and highly effective. I should note that some of these things may be existing Firefox extensions, but Flock simply integrates them really well, and avoids me having to manage that many more extensions.

About the only problem I've found so far is that it has the old focus problems on Mac, when running under CodeTek Virtual Desktop. This plagued Firefox way back, so I'm not sure why Flock still suffers from it, as Firefox fixed it a long time ago. But, I am currently switching all my Macs off CTVD anyway, because it doesn't work well at all on Intel Macs. I'm using Virtue Desktops on my MacBook Pro, and tonight switching the rest of my Macs to it.

technorati tags:Flock, Firefox, Flickr, del.icio.us, Blogger

Blogged with Flock

Rails environment setup and update script

I'm about to embark on creating a script/system for keeping a project's Rails and related environment up to date, as well as doing an initial setup. Gems and various Rails pieces already can do some auto-update and setup things, but I'm wanting something more professional, and something more explicit and exact in terms of versions used.

The idea is that this system will take a bare bones machine from essentially just having an OS, to full development or production ready. And then also, once set up, it will update it. So, it will install Ruby, Rails, Gems, database of choice, libraries needed (e.g. ImageMagick), other components used (e.g. DarwinPorts), etc. In combination with the source code control system (SCCM) (such as Perforce or Subversion), it will keep each piece up to date.

In addition, it will do things like install gems and other pieces from what you store in your SCCM - not just from the net or by doing say a generic "gem update". The reason to do this is that you can track the exact versions used at any given point in your application's development or deployment. You won't have to somehow separately document that you were using XYZ versions when you deployed 2.1.3 of your app, etc. I've used, and have heard others who take this further in the past, by also putting even the OS, development tools, and so on withint he source control system (I've never done the OS part, but I believe at Siemens they used to do that back in MacOS <= 9 days). This script also helps ensure all developers and deployment systems are in sync. At the moment I'm planning to do this with a bootstrap bash shell script, but then once Ruby is available, it'd kick over to a Rake script. Partly I'm choosing this route as a way to learn more about Rake. I'd be interested in hearing if anyone else has such a system in place, or if there's already something out there for this (that works well with Ruby and Rails).

Update: I forgot one other thing that plays in here. Our firewall rules at work do not allow rsync outside the firewall. This prevents things like DarwinPorts from working. The solution, silly as it is, is to take a laptop home, connect to a non-VPN'ed network, and do your DarwinPorts work. Then, you come back, and the ports themselves are cached, so you can then add them to your SCCM and install from there for other machines (who wants to lug a super heavy G5 home, ya, not me).

Tags: Ruby, RubyOnRails, Rake, Automation, Versioning

Driving a car: an excellent Scrum analogy

I've been reading, Agile Project Management with Scrum by Ken Schwaber. One of the things that struck me right off was in Mary Poppendieck's forward, where she provides an excellent analogy: that of driving a car. She describes how driving is made of a set of simple rules, and is a process by which we make small course corrections, frequent decisions, and so on, along the path we take. Yes, we have an end goal in mind, but how we get there is a process which the team (driver) determines along the way. This is unlike say a train route, where it's completely programmed and extremely well known before the trip even starts. A simple analogy, but probably the best fitting, strongest, and easiest to grasp that I've seen yet, in terms of Agile processes.

Tags: Agile, Scrum, Poppendieck

Adobe's San Jose building is world's first for energy conservation

I think this is very cool, as I'm into energy conservation and such. Adobe's West Tower building (the first of the three that are connected together at the San Jose HQ), just received the highest rating for energy and environmental design:

The Adobe tower is the world's first commercial office building to earn this highest recognition possible for energy and environmental design excellence under the USGBC's permanent LEED Existing Building (LEED-EB) standard.

The other two buildings that it connects to should get this rating shortly as well. It's nice knowing the company you work for takes these issues seriously. It's somewhat amazing to me that they do this, given the beautiful facility. Everyone has their own office, there are many "labs", good cafeteria (Google's main benefit over ours is that theirs is free - food isn't that different IMHO), top notch networking (Gigabit ethernet, WiFi all over), and so on.

Of course, I work in the Auburn, CA office, with four other people, so our facility isn't quite like this. But then we're only an hour drive from Tahoe :)

Articles:

• Adobe Wins Platinum Certification Awarded by U.S. Green Building Council
• Adobe Wins Top California Flex Your Power! Award for Energy Efficiency

Tags: Adobe, Green Building, Energy Efficiency

Going to Scale with Rails: Yes

The title of this post has more than one meaning, but for the purpose of this post... I've just signed up to attend the Scale with Rails seminar in Laguna Beach in August. I'm hoping that it provides a lot of good content and learning, as I start ramping up Rails apps.

I'll also be taking my family, and staying at my parents in Corona del Mar, which is just north of Laguna. We'll hit LEGOLAND the day after the seminar.

Tags: RubyOnRails, ScaleWithRails, Legoland

Going to Scale with Rails

I signed up for Scale with Rails this morning.  Hopefully it turns out to be quite useful and a good amount of learning and wisdom.  I'll write up some notes after I go.  I'm bringing my family with me, and staying at my parents house, just north of Laguna in Corona del Mar.  On Friday, I believe we'll be going to LEGOLAND :)