Planet Merb

Getting the Source

Setting up a clj file

Wrap-up

A few links

A little while ago, Carl and I starting digging into Rails’ initializer. We already made a number of improvements, such as adding the ability to add a new initializer at any step in the process, and to make it possible to have multiple initializers in a single process. The second improvement is the first step toward running multiple Rails apps in a single process, which requires moving all global Rails state into instances of objects, so each application can have its own contained configuration in its own object. More on this in the next few weeks.

As I detailed on the Engine Yard blog this week, when moving into a new area to refactor, it’s important to make sure that there are very good tests. Although the Rails initializer tests covered a fair amount of area, successfully getting the tests to pass did not guarantee that Rails booted. Thankfully, Sam Ruby’s tests were comprehensive enough to get us through the initial hump.

After making the initial change, we went back to see what we could do to improve the test suite. The biggest problem was a problem we’d already encountered in Merb: you can’t uninitialize Rails. Once you’ve run through the initialization process, many of the things that happen are permanent.

Our solution, which we committed to master today, is to create a new test mixin that runs each test case in its own process. Getting it working on OSX wasn’t trivial, but it was pretty elegant once we got down to it. All we did was override the run method on TestCase to fork before actually running the test. The child then runs the test (and makes whatever invasive changes it needs to), and communicates any methods that were called on the Test::Unit result object back to the parent.

The parent then replays those methods, which means that as far as the parent is concerned, all of the cases are part of a single suite, even though they are being run in a separate process. Figuring out what parts of Test::Unit to hook into took all of yesterday afternoon, but once we were done, it was only about 40 lines of code.

Today, we tackled getting the same module to work in environments that don’t support forking, like JRuby and Windows. Unfortunately, these environments are going to run these tests significantly more slowly, because they have to boot up a full process for each test case, where the forking version can simply use the setup already done in the parent process (which makes it almost as fast as running all the tests in a single process).

The solution was to emulate forking by shelling out to a new process that was identical to the one that was just launched, but with an extra constraint on the test name (basically, booting up the test suite multiple times, but each run only runs a single test). The subprocess then communicates back to the launching process using the same protocol as in the forking model, which means that we only had to change the code that ran the tests in isolation; everything else remains the same.

There was one final caveat, however. It turns out that in Test::Unit, using a combination of -t to specify the test case class and -n to specify the test case name doesn’t work. Test::Unit’s semantics are to include any test for which ANY of the appropriate filters match. I’m not proud of this, but what we did was a small monkey-patch of the Test::Unit collector in the subprocess only which does the right thing:

# Only in subprocess for windows / jruby.
if ENV['ISOLATION_TEST']
  require "test/unit/collector/objectspace"
  class Test::Unit::Collector::ObjectSpace
    def include?(test)
      super && test.method_name == ENV['ISOLATION_TEST']
    end
  end
end

Not great, but all in all, not all that much code (the entire module, including both forking and subprocess methods is just 98 lines of code).

A crazy couple of days yielding a pretty epic hack, but it works!

Over the weekend, it was discovered that the json_pure gem is subject to a DoS attack when parsing specially formed JSON objects. This vulnerability does not affect the json gem, which uses a C extension, or ActiveSupport::JSON, which is used in Rails.

By default, Merb uses the json gem, which is not vulnerable, but falls back to json_pure if json is not available. As a result, if you have json_pure but not json on your system, you may be vulnerable. Additionally, Ruby 1.9.1 (but not Ruby 1.9 trunk) ships with json_pure, which remains vulnerable.

The easiest way to immunize yourself from this problem, no matter what Ruby version you are on, is to upgrade to the latest version of json_pure, which resolves the vulnerability. Additionally, Merb 1.0.12 has been released, which monkey-patches json_pure to remove the vulnerability, but prints a warning encouraging you to upgrade to the latest. Merb 1.0.12 only adds this patch on top of 1.0.11, so it should be perfectly safe to upgrade.

I’ll occasionally be posting on the Engine Yard blog in addition to my posts here. Union Station has really stepped up lately in terms of technical content. We’ve had posts on pairing, TDD with Cucumber, Rack and a slew of other things, and I know there’s more in the pipe.

I posted today on how we’ve been refactoring Rails, and how you can apply what we’ve learned to your own projects: let me know what you think!