17 June 2008

RSpec View Testing Problems

I've been using RSpec exclusively on my latest project. I'd say I'm still fairly new to it, but it has won me over for the time being. However, the view testing has been a real problem. You can test views quite easily and nicely with RSpec, however, when you do something subtly wrong, it can cause RSpec to fail without any visible error! In fact, Autotest doesn't report a failure at all, and the only way I know it is failing is either I pay close attention to the number of examples it said it ran, or more likely, my cruisecontrol.rb CI server will actually fail (because it detects the commands exit status).

For example, while Autotest says everything is fine, the CI server will show something like this (note for easier reading I've trimmed this a bit):


/widgets/index.html.erb
- should render list of widgets

/bobbles/index.html.erb

Finished in 4.977586 seconds

175 examples, 0 failures
rake aborted!
Command /usr/bin/ruby1.8 -I"/var/cruisecontrolrb/projects/myproject/work/vendor/plugins/rspec/lib" "/var/cruisecontrolrb/projects/myproject/work/vendor/plugins/rspec/bin/spec" "spec/controllers/bobbles_controller_spec.rb" ... --options /var/cruisecontrolrb/projects/myproject/work/spec/cruisecontrol_rcov.opts failed

(See full trace by running task with --trace)


As you can see, RSpec is reporting 0 failures, yet rake fails. This is because in reality RSpec is returning an exit code of 1 instead of 0. But, looking at the output, it's certainly not revealing. Running with --trace is of no help either.

What I now know to pay attention to in situations like this is the fact that that last spec it ran, the /bobbles/index.html.erb one has no examples listed under it. That is thus the culprit (you can also argue that it's the last thing to "run" and then rake fails, so it's likely in this, etc.).

The real pain comes when you try to figure out what the heck is causing this. You have zero feedback, and no way that I know of to somehow debug or inspect the test to see what's failing. In my experience to date with these, it boils down to some problem in your mocks and stubs, but this can be difficult to figure out. I admit, the one that prompted me to write this blog entry is one I've still yet to figure out, and finally just punted on.

I've been hearing a lot about not testing views, or testing very little of the views. I agree in general on this, and am now looking into using Webrat to do integration testing to really test "view" functionality, and leave the rest of my view testing mostly to testing my helpers, controllers, and models. Here are a couple of blog entries related to all this:

Have Autotest speak to you

Update: the first .autotest I had in here was bogus, sorry about that. The .autotest file contents below work properly with Rspec tests at least.

I make extensive use of ZenTest's Autotest to constantly watch my test suite and ensure my app's tests are passing on my dev box/during development. Historically I've used Growl/growlnotify to get little popup notices indicating if my tests passed or failed. That's nice, and I've done the enhancements that add graphics and style it nicely, etc. But, in reality, I'm not always looking and sometimes don't see the messages. Plus, they can be somewhat distracting.

So, I've switched to using the handy say tool on the Mac (on Linux I think you could use "espeak", no clue on Windows, but then, uh, well, why are you doing dev work on Windows?! ;-)

The speaking is nice - I hear it, but don't get visually distracted. I use different voices for tests passed vs. failed too. This may not work great if you work in a cube farm, or even a cafe, but here at home, or in your own office, I think it's great. Here's my .autotest file as an example:


require 'autotest/redgreen'

module Autotest::Growl
def self.growl(title, msg, img, pri=0, stick="")
system "/usr/local/bin/growlnotify -n autotest --image #{img} -p #{pri} -m #{msg.inspect} #{title} #{stick}"
end

Autotest.add_hook :ran_command do |at|
results = at.results.last

unless results.nil?
output = results[/(\d+)\s+examples?,\s*(\d+)\s+failures?(,\s*(\d+)\s+pending)?/]
if output
failures = $~[2].to_i
pending = $~[4].to_i
end

if failures > 0
`/usr/bin/say -v Zarvox "you broke the code"`
elsif pending > 0
`/usr/bin/say -v Alex "Tests passed, with some pending"`
else
unless at.tainted
`/usr/bin/say -v Victoria "all tests passed"`
else
`/usr/bin/say -v Victoria "tests passed"`
end
end
end
end
end


To see what voices your system has available, open the Speech system preference pane, pull down "System Voice" and select "Show More Voices" to see the full list.

11 June 2008

Changelogs and Deployment Notification for Capistrano and Git

Early warning: this is a hack, which doesn't mean it's bad, just that it's not polished. However, I am documenting my solution for myself thus far, as well as figured others might find it useful...

Update: Added my shell command for doing deploys (see end of this post).

I wanted a way to automate a few things around deployments, and integrate this a bit with my continuous integration server. I use CruiseControl for the CI server, and previously blogged about setting up CC.rb with Git. The goals for this next task, and subject of this blog post are:


  • Tag the code on successful deploys. My CI server already tags the code anytime it does a successful build, but since I didn't cover that previously, I'll mention it here as well.

  • Notify a list of people via email whenever a new deploy happens.

  • Generate a changelog, based on Git commit messages (better make sure they're suitable reading for whoever gets your deploy notices!), and include this changelog in the deploy emails.

  • Have the CI tag I want to deploy as the only required piece of info/parameter when issuing a deploy command.



Tagging


First, I tag the code on any successful CI run. This tag is what I can then use as the Git tag to deploy. Capistrano supports this via the branch variable (set its value to the tag name). As you can guess, you can use pretty much any Git ID/tag/branch name for this. To do this, add a task to your cruise.rake file (or similar - wherever you define your custom CruiseControl command), and then ensure you run that task during a CruiseControl session. Here's my task:

desc "Tag the code on successful CI build"
task :ci_tag do
timestamp = Time.now.strftime("%Y%m%d%H%M%S")
tag_name = "CI_#{timestamp}"
# Create an empty file with our tag name, so we can easily go grab the tagname
# from the CI output page and do deploys, etc.
system("touch #{File.join(ENV['CC_BUILD_ARTIFACTS'], tag_name)}")
system("git tag -a -m 'Successful continuous integration build on #{timestamp}' #{tag_name}")
system("git push --tags")
end


From the above, you can see that I'll get tags of the form: CI_timestamp. Next up, I want to tag a successful deploy to indicate which commit/tag actually got deployed and when. This is handled via an after task in my Capistrano deploy.rb:

after "deploy:restart", "tag_last_deploy"
task :tag_last_deploy do
set :timestamp, Time.now
set :tag_name, "deployed_to_#{rails_env}_#{timestamp.to_i}"
`git tag -a -m "Tagging deploy to #{rails_env} at #{timestamp}" #{tag_name} #{branch}`
`git push --tags`
puts "Tagged release with #{tag_name}."
end


This will create tags like, deployed_to_staging_1213223458, and works for both staging and production (or any environment you're targeting - note the use of the rails_env variable - you may need to use something else). One thing to pay particular attention to, is that this tag is actually tagging another tag, as defined by the branch variable (mentioned above). In order for this to work though, you need to ensure that your tags are up to date locally. Thus, somewhere in your workflow you'll need to do a git pull --tags, if like me, your CI server is elsewhere and is generating those tags.

Ok, we're all tagged up, let's move on...

Notification



It turns out there's a nifty new plugin called Cap Gun that will take care of emailing a list of folks on deploy. Setup is covered in their README, but the one bit they don't mention, is that you can include a comment in the email message that goes out. I wanted to include a changelog in these emails, so I tapped into this comment attribute, setting it to the text of my changelog. To use the comment, you can either set it via -s comment="my lovely comment" on your Capistrano deploy command, or you can set the comment variable in your Capistrano deploy.rb or included script. More on that in a minute.

Changelogs


My changelog, so far, is very simple, it just pulls the comments for the Git commits that occurred since the last deploy (for the appropriate target), up to the tag specified (which in this case will be the CI tag you are about to deploy). To handle this, I use a small Ruby script, combined with the great Grit gem that lets one manipulate Git via a nice Ruby API. The script simply spits out a simple chunk of text that will be what gets put into the comment Capistrano variable for our deployment notifications. This is in particular where the "hack" comes into play. This script is not robust, does essentially no error checking, etc, etc. Use at your own risk! And with that, here it is:

#!/usr/bin/env ruby

require 'rubygems'
require 'mojombo-grit'
include Grit

unless ARGV.length == 2
puts "Usage: changelog.rb staging|production <commit-or-tag>"
puts " where commit-or-tag is the commit ID or tag you are planning to deploy"
exit -1
end

repo_location = File.expand_path(File.dirname(__FILE__) + '/..')
target = ARGV[0]
about_to_deploy_commit = ARGV[1]
repo = Repo.new(repo_location)

# Find the tag for the last deployed
tags = repo.tags.collect {|tag| tag.name }
tags.delete_if {|tag| !(tag =~ /^deployed_to_#{target}_/)}
tags.sort!
last_deployed_tag = tags[-1]

commits_for_changelog = repo.commits_between(last_deployed_tag, about_to_deploy_commit)
commits_for_changelog.reverse!

puts "Changes since last release:"
commits_for_changelog.each do |commit|
puts " "
puts " #{commit.message}"
end


To run through it briefly, it takes two parameters (and clearly, you can change this for your own deployment targets, etc.): a deployment target, and a tag (which can actually be a tag, a commit ID, branch, etc.). It sets up a repo variable for your Git repository using Grit, and then proceeds to find the last deployed tag for that deployment target. After that, it gets all the commits between that last deployed tag and the tag you specified as the second script argument, and prints out the commit messages.

To integrate this, I added this line to my Capistrano deploy.rb:

set :comment, `script/changelog.rb staging #{branch}`

As you can see, that one is specific to my staging environment, and lives inside my "staging" task in deploy.rb. Same, appropriately edited version goes for production.

Deployment Command


Lastly, I define a simple shell function to do my deploys, which ensures I have done a git pull so I have all the tags, and makes the command easier to remember and get right, etc:

stagemyproject () {
git pull
cap -s branch=$1 staging deploy:migrations
}


You would thus have a command line to do a deploy like this:

stagemyproject CI_20080612052417

That's it, and if you've managed to read this far, congrats, and if you've not only managed to read this far, but payed attention and got value out of it, well, cool.

For anyone who uses/adapts this, please do let me know improvements you make, or suggestions, or tweaks/changes, and so on. I've been using this for all of about a half dozen deploys so far. If (more like when) I make improvements, I'll update.

06 June 2008

Perforce for CruiseControl.rb now on GitHub

After getting another request for my Perforce implementation for CruiseControl.rb, I've put it up on GitHub (cruisecontrolrb_perforce), and also updated my previous blog entry on the subject.

Just note, I haven't used this since August 2007, so use at your own risk :)

04 June 2008

Rails, jQuery, auto-complete, and a New Plugin

Update: I've switched which jQuery autocomplete plugin I use for this, see my newer blog entry.

The other day, I made a whole switch from Prototype & Scriptaculous to jQuery. I've had the bug to do this for a while, and this is a new project, so I went for it. I don't have anything against Prototype, so my main impetus for this was a move towards Unobtrusive JavaScript, and also the speed aspect (the site I'm currently working on, if things go accordingly to plan, will do some pretty serious traffic). But, the unobtrusive JavaScript was the key, and really, my switch is more of a philosophy of approach rather than say a dislike for Prototype, etc. And, of course, it's something new to play with :)

Before I go any further, I'll state right now, I am not a JavaScript expert, and I've been using jQuery now for all of a couple hours.

One of the results of my switch however, was that I hacked DHH's auto_complete Rails plugin, to work for jQuery. Simple change. I tweaked the controller macro, and then gutted the JS helpers, as you just don't need those when using jQuery in this way. It does require the jquery-autocomplete plugin for jQuery. I've published my Rails plugin for this on GitHub as auto_complete_jquery.

Circling back around, here's what I did to get all this going. I did run into one issue (see step 10 below) that I'm still tracking down (easy solution in the interim, but I'd like to understand what's happening, so if you have comments, please let me know):


  1. Removed the Prototype and Scriptaculous JS files from the public/javascripts dir of my Rails app. You don't have to do this, but I am no longer using them, so saw no need to keep them there, and it helps ensure I don't mistakenly use something from them or include them in the view. This includes: prototype.js, controls.js, dragdrop.js, and effects.js.

  2. Removed the prototype-based Rails auto_complete plugin from vendor/plugins.

  3. Installed the latest minified jQuery file in public/javascripts/jquery.

  4. Installed the JS files for the jquery-autocomplete plugin, and its dependencies: jquery.templating.js, and jquery.ui.autocomplete.js. (see the jquery-autocomplete plugin for these files).

  5. Added the jquery.ui.autocomplete.css file to public/stylesheets.

  6. Installed my auto_complete_jquery plugin.

  7. Put the proper includes for the CSS file and the JS files in my application layout file:


    <%= stylesheet_link_tag 'jquery.ui.autocomplete' %>
    <%= javascript_include_tag 'jquery/jquery.min', 'jquery/jquery.templating', 'jquery/jquery.ui.autocomplete.ext', 'jquery/jquery.ui.autocomplete', :cache => 'jquery' %>

    Note that I keep my jQuery JS files in a subdir for organizational purposes, but you can modify as needed.

  8. Changed my existing auto-complete text fields that used the Rails Prototype based auto_complete plugin's helpers to just be plain old text fields, such as:
    <%= coffee.text_field :drink, :autocomplete =>"off" %>
    This is doing an auto-complete for the "drink" attribute of the Coffee model.

  9. I can simply leave any auto_complete_for calls that existed in my controller, as that works the same. If you had custom versions that were based on the code from the Prototype-based Rails plugin, just go look at the code in my plugin to see the differences, it's a simple change.

  10. Add the JavaScript that sets up the auto-complete for the given text field. This will typically look like:

    $(document).ready(function() {
    $("input#coffee_drink").autocomplete({ ajax: "auto_complete_for_coffee_drink" })
    });

    Where does this go? It depends. What I've been liking is using the JavaScript auto-include plugin, which creates a Rails-style convention for JavaScript files that pertain to individual actions, or are controller-wide. So in my case, this code would get placed in public/javascripts/views/coffees/new.js, or likely one directory up, as simply coffees.js (so that I can use it in any CoffeesController action that needs to auto-complete on coffee.drink. Without that plugin, you just put it in whatever JS file is appropriately included for the view you're using it in, etc. You can of course put it directly into the view in a script block, but then you aren't doing the whole Unobtrusive JavaScript thing as rigidly.
  11. Finally, what I found is that I had to add a route for this. This is the issue I mentioned above. It sort of makes sense, but what I'm unclear on is, why the prior standard/Prototype-based Rails auto_complete plugin didn't require a route. They both seem to use a GET, define the action the same way, and so on. I'm hoping I'm just missing something obvious. So, the route I added is:
    map.connect ':controller/auto_complete_for_coffee_drink', :action => 'auto_complete_for_coffee_drink', :format => 'json'



A bunch of steps, but pretty simple work. The app I'm doing this on is all of a few days old, so I hadn't gotten into use of much else in Prototype and so on, thus making the wholesale switch easy.

If you'd like to learn more about any of these things, and as a comprehensive set of links:


Enjoy!

Update: I removed the jQuery Dimensions JS file and include for it in my layout, as this is now included in the latest jQuery JS file itself.

Update 2: I don't know how the standard auto_complete plugin manages to do without routes, but here is a generic route for all auto-complete actions across controllers:

map.auto_complete ':controller/:action',
:requirements => { :action => /auto_complete_for_\S+/ },
:conditions => { :method => :get }

I hesitate to put this into the plugin, as routes can be quite tricky in more complex apps, and I wouldn't want to auto-hose someone :)

02 June 2008

Fixing Capistrano 2.3.0 and Git Deploy Problem

If you upgrade to Capistrano 2.3.0, and are doing deploys from a Git repository, you may find that all of a sudden you can no longer deploy. This is the case if you have no tags in your Git repo. Cap 2.3.0 changed one of the Git commands it uses and that apparently doesn't work right if you don't have tags. So, to solve the problem, you can simply create a single tag in your Git repository. The tag does not have to relate to your build at all, you only need one tag in the repo (not one per build or anything like that), etc. Once you create the tag, you can now deploy again.

To create a tag in Git, or, I think the "cooler" kind of tag, an annotated tag, you can do:

git tag -a tag_name

Replace "tag_name" with your tag name of course. The "-a" option says to make it an annotated tag, which lets you enter a comment about the tag. You can put whatever you want in there. I'm liking this potential use with my continuous integration server when it makes tags on successful builds. Lots of possibilities.

Finally, if you deploy from a remote repo, or if you have a remote repo (say on GitHub), you will need to push your tag. This does not automatically occur on a push, you need to add "--tags" option to git-push to include your tags:

git push --tags

Now you'll have your tags on your remote repo, and listed under the "all tags" tab on GitHub.