Planet Merb

MerbAdmin Installation and Overview

2010-03-13T07:34:32+00:00

Erik Michaels-Ober demonstrates the installation process and some of the functionality of MerbAdmin.

Watch it...

Learn About Merb Dependencies and the new Bundler!

2010-03-12T22:01:14+00:00

Carl Lerch and Yehuda Katz released this helpful screencast with Merb 1.0.6. It deals with how to utilize all the new features the new gem dependency bundler brings.

Watch it...

Ruby’s Implementation Does Not Define its Semantics

2010-02-25T06:22:34+00:00

When I was first getting started with Ruby, I heard a lot of talk about blocks, and how you could “cast” them to Procs by using the & operator when calling methods. Last week, in comments about my last post (Ruby is NOT a Callable Oriented Language (It’s Object Oriented)), I heard that claim again.

To be honest, I never really thought that much about it, and the idea of “casting” a block to a Proc never took hold in my mental model, but when discussing my post with a number of people, I realized that a lot of people have this concept in their mental model of Ruby.

It is not part of Ruby’s semantics.

In some cases, Ruby’s internal implementation performs optimizations by eliminating the creation of objects until you specifically ask for them. Those optimizations are completely invisible, and again, not part of Ruby’s semantics.

Let’s look at a few examples.

Blocks vs. Procs

Consider the following scenarios:

def foo
  yield
end
 
def bar(&block)
  puts block.object_id
  baz(&block)
end
 
def baz(&block)
  puts block.object_id
  yield
end
 
foo { puts "HELLO" } #=> "HELLO"
bar { puts "HELLO" } #=> "2148083200\n2148083200\nHELLO"

Here, I have three methods using blocks in different ways. In the first method (foo), I don’t specify the block at all, yielding to the implicit block. In the second case, I specify a block parameter, print its object_id, and then send it on to the baz method. In the third case, I specify the block, print out its object_id and yield to the block.

In Ruby’s semantics, these three uses are identical. In the first case, yield calls an implicit Proc object. In the second case, it takes an explicit Proc, then sends it on to the next method. In the last case, it takes an explicit Proc, but yields to the implicit copy of the same object.

So what’s the `&` thing for?

In Ruby, in addition to normal arguments, methods can receive a Proc in a special slot. All methods can receive such an argument, and Procs passed in that slot are silently ignored if not yielded to:

def foo
  puts "HELLO"
end
 
foo { something_crazy } #=> "HELLO"

On the other hand, if you want a method to receive a Proc in that slot, and thus be able to yield to it, you specify that by prefixing it with an &:

def foo
  yield
end
 
my_proc = Proc.new { puts "HELLO" }
foo(&my_proc)

Here, you’re telling the foo method that my_proc is not a normal argument; it should be placed into the proc slot and made available to yield.

Additionally, if you want access to the Proc object, you can give it a name:

def foo(&block)
  puts block.object_id
  yield
end
 
foo { puts "HELLO" } #=> "2148084320\nHELLO"

This simply means that you want access to the implicit Proc in a variable named block.

Because, in most cases, you’re passing in a block (using do/end or {}), and calling it using yield, Ruby provides some syntax sugar to make that simple case more pleasing. That does not, however, mean that there is a special block construct in Ruby’s semantics, nor does it mean that the & is casting the a block to a Proc.

You can tell that blocks are not being semantically wrapped and unwrapped because blocks passed along via & share the same object_id across methods.

Mental Models

For the following code there are two possible mental models.

def foo(&block)
  puts block.object_id
  yield
end
 
b = Proc.new { puts "OMG" }
puts b.object_id
foo(&b) #=> 2148084040\n2148084040\nOMG

In the first, the &b unwraps the Proc object, and the &block recasts it into a Proc. However, it somehow also wraps it back into the same wrapper that it came from into the first place.

In the second, the &b puts the b Proc into the block slot in foo’s argument list, and the &block gives the implicit Proc a name. There is no need to explain why the Proc has the same object_id; it is the same Object!

These two mental models are perfectly valid (the first actually reflects Ruby’s internal implementation). I claim that those who want to use the first mental model have the heavy burden of introducing the new concept to the Ruby language of a non-object block, and that as a result, it should be generally rejected.

Metaclasses

Similarly, in Ruby’s internal implementation, an Object does not get a metaclass until you ask for one.

obj = Object.new
# internally, obj does not have a metaclass here
 
obj.to_s
# internally, Ruby skips right up to Object when searching for #to_s, since
# it knows that no metaclass exists
 
def obj.hello
  puts "HELLO"
end
# now, Ruby internally rewrites obj's class pointer to point to a new internal
# metaclass which has the hello method on it
 
obj.to_s
# Now, Ruby searches obj's metaclass before jumping up to Object
 
obj.to_s
# Now, Ruby skips the search because it's already cached the method
# lookup

All of the comments in the above snippet are correct, but semantically, none of them are important. In all cases, Ruby is semantically looking for methods in obj’s metaclass, and when it doesn’t find any, it searches higher up. In order to improve performance, Ruby skips creating a metaclass if no methods or modules are added to an object’s metaclass, but that doesn’t change Ruby’s semantics. It’s just an optimization.

By thinking in terms of Ruby’s implementation, instead of Ruby’s semantics, you are forced to think about a mutable class pointer and consider the possibility that an object has no metaclass.

Mental Models

Again, there are two possible mental models. In the first, Ruby objects have a class pointer, which they manipulate to point to new metaclass objects which are created only when methods or modules are added to an object. Additionally, Ruby objects have a method cache, which they use to store method lookups. When a method is looked up twice, in this mental model, some classes are skipped because Ruby already knows that they don’t have the method.

In the second mental model, all Ruby objects have a metaclass, and method lookup always goes through the metaclass and up the superclass chain until a method is found.

As before, I claim that those who want to impose the first mental model on Ruby programmers have the heavy burden of introducing the new concepts of “class pointer” and “method cache”, which are not Ruby objects and have no visible implications on Ruby semantics.

Regular Expression Matches

In Ruby, certain regular expression operations create implicit local variables that reflect parts of the match:

def foo(str)
  str =~ /e(.)l/
  p $~
  p $`
  p $'
  p $1
  p $2
end
 
foo("hello") #=> #<MatchData "ell" 1:"l">\n"h"\n"o"\n"l"\nnil

This behavior is mostly inherited from Perl, and Matz has said a few times that he would not support Perl’s backrefs if he had it to do over again. However, the provide another opportune example of implicit objects in Ruby.

Mental Models

In this case, there are three possible mental models.

In the first mental model, if you don’t use any $ local variables, they don’t exist. When you use a specific one, it springs into existence. For instance, when using $1, Ruby looks at some internal representation of the last match and retrieves the last capture. If you use $~, Ruby creates a MatchData object out of it.

In the second mental model, when you call a method that uses regular expressions, Ruby walks back up the stack frames, and inserts the $ local variables on it when it finds the original caller. If you later use the variables, they are already there. Ruby must be a little bit clever, because the most recent frame on the stack (which might include C frames, Java frames, or internal Ruby frames in Rubinius) is not always the calling frame.

In the last mental model, when you call a method that uses regular expressions, there is an implicit match object available (similar to the implicit Proc object that is available in methods). The $~ variable is mapped to that implicit object, while the $1 variable is the equivalent of $~[1].

Again, this last mental model introduces the least burdensome ideas into Ruby. The first mental model introduces the idea of an internal representation of the last match, while the second mental model (which again, has the upside of being how most implementations actually do it) introduces the concept of stack frames, which are not Ruby objects.

The last mental model uses an actual Ruby object, and does not introduce new concepts. Again, I prefer it.

Conclusion

In a number of places, it is possible to imbue Ruby semantics with mental models that reflect the actual Ruby implementation, or the fact that it’s possible to imagine that a Ruby object only springs into existence when it is asked for.

However, these mental models require that Ruby programmers add non-objects to the semantics of Ruby, and requiring contortions to explain away Ruby’s own efforts to hide these internals from the higher-level constructs of the language. For instance, while Ruby internally wraps and unwraps Procs when passing them to methods, it makes sure that the Proc object attached to a block is always the same, in an effort to hide the internal details from programmers.

As a result, explaining Ruby’s semantics in terms of these internals requires contortions and new constructs that are not natively part of Ruby’s object model, and those explanations should be avoided.

To be clear, I am not arguing that it’s not useful to understand Ruby’s implementation. It can, in fact, be quite useful, just as it’s useful to understand how C’s calling convention works under the covers. However, just as day-to-day programmers in C don’t need to think about the emitted Assembler, day-to-day Ruby programmers don’t need to think about the implementation. And finally, just as C implementations are free to use different calling conventions without breaking existing processor-agnostic C (or the mental model that C programmers use), Ruby implementations are free to change the internal implementation of these constructs without breaking pure-Ruby code or the mental model Ruby programmers use.

Ruby is NOT a Callable Oriented Language (It’s Object Oriented)

2010-02-21T23:19:49+00:00

I recently ran across a presentation entitled Python vs. Ruby: A Battle to the Death. I didn’t consider it to be a particularly fair battle, and may well reply in more detail in a later post.

However, what struck me as most worthy of explanation was the presenter’s concern about the fact that Procs are not callable via parens.

x = Proc.new { puts "HELLO" }
x() #=> undefined method `x' for #<Object:0x1001bd298><
x.call #=> "HELLO"
x[]    #=> "HELLO"

For those coming from a callable-oriented language, like Python, this seems horribly inconsistent. Why are methods called with (), while Procs are called with [].

But what’s going on here is that Ruby doesn’t have a notion of “callable”, like Python. Instead, it has a pervasive notion of Object. Here, x is an instance of the Proc class. Both call and [] are methods on the Proc class.

Designing for the Common Case: Calling Methods

Coming from a callable-oriented language, this might seem jarring. But Ruby is designed around Objects and the common cases of working with objects.

Calling methods is far more common than wanting to get an instance of Method, so Ruby optimizes the case of calling methods, with a slightly less elegant form to access an instance:

class Greeter
  def say
    puts "Hello world!"
  end
end
 
Greeter.new.say #=> "Hello world!"
 
Greeter.method(:new) #=> #<Method: Class#new>
Greeter.new.method(:say) #=> #<Method: Greeter#say>
 
# This is so that you don't have to say:
 
Greeter.new().say()

Ruby considers the common case of calling methods, and optimizes that case while still making the less common case possible.

Designing for the Common Case: How Blocks Are Really Used

One of the reasons that the Proc.new case throws off Rubyists in debates is that Rubyists literally never call Proc objects using [].

In Ruby, Procs are the object passed to methods when using block syntax. Here is how Procs are actually used:

def hello
  puts "Hello world!"
  yield
  puts "Goodbye cruel world!"
end
 
hello { puts "I am in the world!" }

When examining languages that support passing anonymous functions to functions (like JavaScript), it turns out that the vast majority of such cases involve a single anonymous function. As a result, Matz (inspired by Smalltalk) built in the idea of a block as a core construct in Ruby. In the vast majority of cases, blocks are created using lightweight syntax ({} or do/end) and called using yield.

In some cases, blocks are passed from one method to the next, before they are finally called using yield:

def step1(&block)
  puts "Step 1"
  step2(&block)
end
 
def step2
  puts "Step 2"
  yield
end
 
step1 { puts "Do the action!" } #=> "Step 1\nStep 2\nDo the action!"

As you can see, Ruby builds in the idea of calling a block into the core language. I searched through Rails (a fairly large codebase) for instances of using [] to call a Proc and while we use blocks extremely commonly, we don’t use [] to call them.

I suspect that the reason this comes up is that people who are used to having to define standalone functions, pass them around, and then call them are looking for the analogous constructs in Ruby, but are missing the different paradigm used by Ruby.

Consistent Method Execution

Fundamentally, the issue here comes down to this:

def foo
  proc {}
end
 
foo()

In Ruby, methods are invoked with our without parentheses. All methods return values, which are always Objects. All Objects have methods. So foo() is a method call that returns a Proc object. It’s extremely consistent, with very few axioms. The fact that the axioms aren’t the same as those in a callable-oriented language doesn’t make them “weird”.

AbstractQueryFactoryFactories and alias_method_chain: The Ruby Way

2010-02-15T12:58:41+00:00

In the past week, I read a couple of posts that made me really want to respond with a coherent explanation of how I build modular Ruby code.

The first post, by Nick Kallen of Twitter, gushed about the benefits of PerQueryTimingOutQueryFactory and called out Ruby (and a slew of other “hipster” languages) for using language features (like my “favorite” alias_method_chain) and leveraging dynamicism to solve problems that he argues are more appropriately solved with laugh-inducing pattern names:

In a very dynamic language like Ruby, open classes and method aliasing (e.g., alias_method_chain) mitigate this problem, but they don’t solve it. If you manipulate a class to add logging, all instances of that class will have logging; you can’t take a surgical approach and say “just objects instantiated in this context”.

If you haven’t read it yet, you should probably read it now (at least skim it).

As if on cue, a post by Pivot Rob Olson demonstrated the lengths some Rubyists will go to torture alias_method_chain to solve essentially the same problem that Nick addressed.

In short, while I agree in principle with Nick, his examples and the jargon he used demonstrated exactly why so few Rubyists take his point seriously. It is possible to write modular code in Ruby with the same level of flexibility but with far less code and fewer concept hoops to jump through.

Let’s take a look at the problem Rob was trying to solve:

module Teacher
  def initialize
    puts "initializing teacher"
  end
end
 
class Person
  include Teacher
 
  def initialize
    puts "initializing person"
  end
end
 
# Desired output:
# > Person.new
# initializing teacher
# initializing person

This is a classic problem involving modularity. In essence, Rob wants to be able to “decorate” the Person class to include teacher traits.

Nick’s response would have been to create a factory that creates a Person proxy decorated with Teacher properties. And he would have been technically correct, but that description obscures the Ruby implementation, and makes it sound like we need new “Factory” and “Decorator” objects, as we do, in fact, need when programming in Java.

In Ruby, you’d solve this problem thusly:

# The base person implementation. Never instantiate this.
# Instead, create a subclass that mixes in appropriate modules.
class AbstractPerson
  def initialize
    puts "Initializing person"
  end
end
 
# Provide additional "teacher" functionality as a module. This can be
# mixed into subclasses of AbstractPerson, giving super access to
# methods on AbstractPerson
module Teacher
  def initialize
    puts "Initializing teacher"
    super
  end
end
 
# Our actual Person class. Mix in whatever modules you want to
# add new functionality.
class Person < AbstractPerson
  include Teacher
end
 
# > Person.new
# Initializing teacher
# Initializing person

Including modules essentially decorates existing classes with additional functionality. You can include multiple modules to layer on existing functionality, but you don’t need to create special factory or decorator objects to make this work.

For those following along, the classes used here are “factories”, and the modules are “decorators”. But just as it’s not useful to constantly think about classes as “structs with function pointers” because that’s historically how they were implemented, I’d argue it’s not useful to constantly think about classes and modules as factories and decorators, simply because they’re analogous to those concepts in languages like Java.

The Case of the PerQueryTimingFactoryFactory

Nick’s example is actually a great example of a case where modularity is important. In this case, he has a base Query class that he wants to extend to add support for timeouts. He wrote his solution in Scala; I’ll transcode it into Ruby.

Feel free to skim the examples that follow. I’m transcoding the Scala into Ruby to demonstrate something which you will be able to understand without fully understanding the examples.

class QueryProxy
  def initialize(query)
    @query = query
  end
 
  def select
    delegate { @query.select { yield } }
  end
 
  def execute
    delegate { @query.execute }
  end
 
  def cancel
    @query.cancel
  end  
 
  def delegate
    yield
  end
end

Then, in order to add support for Timeouts, he creates a new subclass of QueryProxy:

class TimingOutQuery < QueryProxy
  def initialize(query, timeout)
    @timeout = timeout
    @query   = query
  end
 
  def delegate
    begin
      Timeout.timeout(@timeout) do
        yield
      end
    rescue Timeout::Error
      cancel
      raise SqlTimeoutException
    end
  end
end

Next, in order to instantiate a TimingOutQuery, he creates a TimingOutQueryFactory:

class TimingOutQueryFactory
  def initialize(query_factory, timeout)
    @query_factory = query_factory
    @timeout = timeout
  end
 
  def self.call(connection, query, *args)
    TimingOutQuery.new(@query_factory.call(connection, query, *args), timeout)
  end
end

As his coup de grâce, he shows how, now that everything is so modular, it is trivial to extend this system to support timeouts that were per-query.

class PerQueryTimingOutQueryFactory
  def initialize(query_factory, timeouts)
    @query_factory = query_factory
    @timeouts = timeouts
  end
 
  def self.call(connection, query, *args)
    TimingOutQuery.new(@query_factory.call(connection, query, *args), @timeouts[query])
  end
end

This is all true. By using factories and proxies, as you would in Java, this Ruby code is modular. It is possible to create a new kind of QueryFactory trivially.

However, this code, by tacking close to vocabulary created to describe Java patterns, rebuilds functionality that exists natively in Ruby. It would be equivalent to creating a Hash of Procs in Ruby when a Class would do.

The Case: Solved

Ruby natively provides factories, proxies and decorators via language features. In fact, that vocabulary obscures the obvious solution to Nick’s problem.

# No need for a proxy at all, so we skip it
 
module Timeout
  # super allows us to delegate to the Query this
  # module is included into, even inside a block
  def select
    timeout { super }
  end
 
  def execute
    timeout { super }
  end
 
  # We get the cancel delegation natively, because
  # we can use subclasses, rather than separate
  # proxy object, to implement the proxy
 
private
  # Since we're not using a proxy, we'll just implement
  # the timeout method directly, and skip "delegate"
  def timeout
    # The Timeout module expects a duration method
    # which classes that include Timeout should provide
    Timeout.timeout(duration) do
      yield
    end
  rescue Timeout::Error
    cancel
    raise SqlTimeoutException
  end
end
 
# Classes in Ruby serve double duty as "proxies" and
# "factories". This behavior is part of Ruby semantics.
class TimingOutQuery < Query
  include Timeout
 
private
  # implement duration to hardcode the value of 1
  def duration
    1
  end
end
 
# Creating a second subclass of Query, this time with
# per-query timeout semantics.
class PerQueryTimingOutQuery < Query
  TIMEOUTS = Hash.new(0.5).merge("query1" => 1, "query2" => 3)
 
  include Timeout
 
private
  def duration
    TIMEOUTS[query]
  end
end

As Nick would point out, what we’re doing here, from a very abstract perspective, isn’t all that different from his example. Our subclasses are proxies, our modules are decorators, and our classes are serving as factories. However, forcing that verbiage on built-in Ruby language features, in my opinion, only serves to complicate matters. More importantly, by starting to think about the problem in terms of the Java-inspired patterns, it’s easy to end up building code that looks more like Nick’s example than my solution above.

For the record, I think that designing modularly is very important, and while Ruby provides built-in support for these modular patterns, we don’t see enough usage of them. However, we should not assume that the reason for the overuse of poor modularity patterns (like alias_method_chain) result from a lack of discussion around proxies, decorators, and factories.

By the way, ActionController in Rails 3 provides an abstract superclass called ActionController::Metal, a series of modules that users can mix in to subclasses however they like, and a pre-built ActionController::Base with all the modules mixed in (to provide the convenient “default” experience). Additionally, users or extensions can easily provide additional modules to mix in to ActionController::Metal subclasses. This is precisely the pattern I am describing here, and I strongly recommend that Rubyists use it more when writing code they wish to be modular.

Postscript: Scala

When researching for this article, I wondered why Nick hadn’t used Ruby’s equivalent to modules (traits) in his examples. It would be possible to write Scala code that was extremely similar to my preferred solution to the problem. I asked both Nick and the guys in #scala. Both said that while traits could solve this problem in Scala, they could not be used flexibly enough at runtime.

In particular, Nick wanted to be able to read the list of “decorators” to use at runtime, and compose something that could create queries with the appropriate elements. According to the guys in #scala, it’s a well-understood issue, and Kevin Wright has a compiler plugin to solve this exact problem.

Finally, the guys there seemed to generally agree with my central thesis: that thinking about problems in terms of patterns originally devised for Java can leave a better, more implementation-appropriate solution sitting on the table, even when the better solution can be thought of in terms of the older pattern (with some contortions).

The Blind Men and the Elephant: A Story of Noobs

2010-02-10T02:19:39+00:00

If you will indulge me, I’d like to paraphrase a familiar tale:

Once upon a time, deep in the forest, there was a tribe of elephant curators. The elders of this tribe kept sophisticated, detailed notes about the proper care and feeding of elephants, and the villagers tended to follow along.

Eventually, they dedicated a large section of the local library to books and articles on the care and feeding of elephants.

One day, a group of blind nomads appeared in the village. Each of the blind men went to greet the villagers, and were met with welcomes. Wanting to be helpful, they walked over to one of the elephants and tried to learn about it.

The first man, who stood next to the elephant’s tail said, “I feel a snake”.

The second, who stood next to the elephant’s leg said, “I feel a tree trunk”.

And so on.

One of the group of nomads, who thought he felt a snake, went to the elders of the village and asked, “I would like to help. How can I feed this creature?”

The elder replied: “Sir, if you can’t be bothered to search the library for information on the care and feeding of elephants, surely you are wasting our time”.

Years passed and after a grueling series of trials, the blind men became integrated into the culture, becoming some of the most successful at caring for the elephants.

Eventually, another group of blind nomads appeared. The entire village, including the original group, proceeded to berate the new travelers. “We’ve spent quite a bit of time putting together a section of the library about how to care for these animals. You are wasting our time”.

Of course, the travelers did not know the creatures were elephants, and so they entered the library, searching for books on feeding snakes and caring for trees.

Eventually, an elderly blind man, of the original group stood up and said: “Have we forgotten than we, too, started in this confused state. We should help these travelers and perhaps they will become as wise and helpful as we became”.

To many participants in open source communities, this is a familiar tale. When a developer first comes across an open source project, either to use it in a project or to help, he is like a blind man feeling an elephant.

It’s easy to spit out “lmgtfy.com” or RTFM, but in truth, these beginners barely know where to look. All too often, we (open source leaders) assume that if someone couldn’t figure out the right search term on Google, they can never become a viable community member.

When I first started working on Rails, I distinctly remember not knowing what the request method in Rails controllers was. To some degree, this could be attributed to its exclusion in api.rubyonrails.org, but some judicious Googling turned up the ActionController::Request class. Writing this post years later, the request method still does not reside in the API docs, but I found the Request documentation in seconds.

The problem is that a new developer simply has no conceptual model for the problem at all. In most cases, the “noob” can stare at “the f***ing manual” all day and simply fail to find something staring him in the face. Importantly, this does not reflect a failing on the part of the new developer. Virtually everyone I know who worked their way from noob to senior Rails developer starting feeling around the elephant.

As open source leaders, if we are interested in growing our communities, we should treat new developers as confused people with real potential. That’s not to say that sinking dozens of hours down a black hole is a good use of time. On the other hand, the mismatch between how we think about problems once we become experienced and the way we feel around like a blind man when getting started makes the experience of getting started with an open source project far more painful than it needs to be.

Using Bundler in Real Life

2010-02-09T11:12:00+00:00

A lot of people have asked me what the recommended workflows for bundler are. Turns out, they’re quite simple.

Let’s step through a few use-cases.

You Get a Repo for the First Time

You’ve just checked out a git (or other) repository for an application that uses bundler. Regardless of any other features of bundler in use, just run:

bundle install

This will resolve all dependencies and install the ones that aren’t already installed in system gems or in your system’s bundler cache.

You Update a Repo Using Bundler

If you update a repository using bundler, and it has updated its dependencies in the Gemfile, regardless of any other features in use, just run:

bundle install

As above, this will resolve all dependencies and install any gems that are not already installed.

You have created a new Rails application

If you’ve created a new Rails application, go inside it and run:

bundle install

This will make sure that Rails’ dependencies (such as SQLite3) are available in system gems or your system’s bundler cache. In most cases, this will not need to install anything.

To check whether your system already satisfied the application’s dependencies, run:

bundle check

As you work in the application, you may wish to add dependencies. To do so, update the Gemfile, and run:

bundle install

You are ready to share your new Rails application

Now that you have a working Rails application, and wish to share, you might wish to ensure that your collaborators get the same versions of the gems as you have. For instance, if webrat specified “nokogiri >= 1.4″ as a dependency, you might want to ensure that an update to nokogiri does not change the actual gems that bundle install will install.

To achieve this, run:

bundle lock

This will create a new file called Gemfile.lock in your root directory that contains the dependencies that you specified, as well as the fully resolved dependency graph. Check that file into source control.

When your collaborators receive the repository and run bundle install, bundler will use the resolved dependencies in Gemfile.lock.

You have a locked application, and wish to add a new dependency

If you add a new dependency to your Gemfile in a locked application, Bundler will give you an error if you try to perform any operations.

You will want to run bundle unlock to remove the lock, then bundle install to ensure that the new dependencies are installed on your system, and finally bundle lock again to relock your application to the new dependencies.

We will add a command in a near-future version to perform all these steps for you (something like bundle install –relock).

You want a self-contained application

In many cases, it is desirable to be able to have a self-contained application that you can share with others which contains all of the required gems.

In addition to a general desire to remove a dependency on Gemcutter, you might have dependencies on gems that are not on a publicly accessible gem repository.

To collect up all gems and place them into your app, run:

bundle pack

When running bundle install in the future, Bundler will use packed gems, if available, in preference to gems available in other sources.

Conclusion

I hope these workflows have clarified the intent of Bundler 0.9 (and 1.0). During our work on earlier versions, the lack of these workflows came up again and again as a source of frustration. This was the primary reason for the big changes in 0.9, so I hope you find them useful.

The Building Blocks of Ruby

2010-02-07T14:53:40+00:00

When showing off cool features of Ruby to the uninitiated (or to a language sparring partner), the excited Rubyist often shows off Ruby’s “powerful block syntax”. Unfortunately, the Rubyist uses “powerful block syntax” as shorthand for a number of features that the Pythonista or Javaist simply has no context for.

To start, we usually point at Rake, Rspec or Sinatra as examples of awesome usage of block syntax:

get "/hello" do
  "Hello World"
end

In response, Pythonistas usually point to these syntaxes as roughly equivalent:

@get('/hi')
def hello():
  return "Hello World"
 
def hello() -> "/hi":
  return "Hello World"

While the Python version may not be quite as pretty, nothing about them screams “Ruby has much stronger capabilities here”. Instead, by using examples like Sinatra, Rubyists trade in an argument about great semantic power for one about superficial beauty.

Rubyists, Pythonistas and others working on web development share a common language in JavaScript. When describing blocks to “outsiders” who share a common knowledge of JavaScript, we tend to point at JavaScript functions as a close analogue. Unfortunately, this only furthers the confusion.

On the Ruby side, when PHP or Java announces that they’re “adding closures”, many of us don’t stop to ask “what kind of closures?”

Cut to the Chase

Let’s cut to the chase and use a better example of the utility of Ruby blocks.

def append(location, data)
  path = Pathname.new(location)
  raise "Location does not exist" unless path.exist?
 
  File.open(path, "a") do |file|
    file.puts YAML.dump(data)
  end
 
  return data
end

Here, the File.open method takes a block. It then opens a new file (in “append” mode), and yields the open file into the block. When the block completes, Ruby closes the file. Except that Ruby doesn’t just close the file when the block completes; it guarantees that the File will be closed, even if executing the block results in a raise. Let’s take a look at the implementation of File in Rubinius:

def self.open(*args)
  io = new *args
 
  return io unless block_given?
 
  begin
    yield io
  ensure
    begin
      io.close unless io.closed?
    rescue StandardError
      # nothing, just swallow them.
    end
  end
end

This means that you can wrap up idioms like pervasive try/catch/finally in methods.

# Without blocks
def append(location, data)
  path = Pathname.new(location)
  raise "Location does not exist" unless path.exist?
 
  begin
    file = File.open(path, "a")
    file.puts YAML.dump(data)
  ensure
    file.close
  end
 
  return data
end

Because Ruby runs ensure clauses even when the exception happened in a block, programmers can reliably ensure that Ruby executes teardown logic hidden away in abstractions.

This example only demonstrates the power of well-designed lambdas. With the addition of one small additional feature, Ruby’s blocks become something altogether different.

def write(location, data)
  path = Pathname.new(location)
  raise "Location does not exist" unless path.exist?
 
  File.open(path, "w") do |file|
    return false if Digest::MD5.hexdigest(file.read) == data.hash
    file.puts YAML.dump(data)
  end
 
  return true
end

In the above case, imagine that writing the data to disk is quite expensive, and we can skip writing if the MD5 hash of the file’s contents match a hash method on the data. Here, we’ll return false if the method did not write to disk, and true if the method did.

Ruby’s blocks support non-local-return (some references), which means that a return from the block behaves identically to returning from the block’s original context. In this case, returning from inside the block returns from the write method, but Ruby will still run the ensure block closing the file.

You can think of non-local-return as behaving something like:

def write(location, data)
  path = Pathname.new(location)
  raise "Location does not exist" unless path.exist?
 
  File.open(path, "w") do |file|
    raise Return.new(false) if Digest::MD5.hexdigest(file.read) == data.hash
    file.puts YAML.dump(data)
  end
 
  return true
rescue Return => e
  return e.object
end

where Return is Return = Struct.new(:object).

Of course, any reasonable lambda implementation will support this, but Ruby’s version has the benefit of feeling just like a normal return, and requiring much less chrome to achieve it. It also behaves well in scenarios that already use rescue or ensure, avoiding mind-warping combinations.

Further, Ruby also supports super inside of blocks. Imagine the write method was defined on a subclass of a simpler class whose write method took the raw data from the file and printed it to a log.

def write(location, data)
  path = Pathname.new(location)
  raise "Location does not exist" unless path.exist?
 
  File.open(path, "w") do |file|
    file_data = file.read
    super(location, file_data)
    return false if Digest::MD5.hexdigest(file_data) == data.hash
    file.puts YAML.dump(data)
  end
 
  return true
end

In a purer lambda scenario, we would need to store off a reference to the self, then use that reference inside the lambda:

def write(location, data)
  path = Pathname.new(location)
  raise "Location does not exist" unless path.exist?
 
  this = self
  File.open(path, "w") do |file|
    file_data = file.read
 
    # imaginary Ruby construct that would be needed without
    # non-local-super
    this.super.write(location, file_data)
    raise Return.new(false) if Digest::MD5.hexdigest(file_data) == data.hash
    file.puts YAML.dump(data)
  end
 
  return true
rescue Return => e
  return e.object
end

You can also yield to a method’s block inside a block. Imagine that the write method is called with a block that chooses the correct data to use based on whether the file is executable:

def write(location)
  path = Pathname.new(location)
  raise "Location does not exist" unless path.exist?
 
  File.open(path, "w") do |file|
    file_data = file.read
    super(location)
    data = yield file
    return false if Digest::MD5.hexdigest(file_data) == data.hash
    file.puts YAML.dump(data)
  end
 
  return true
end

This would be called via:

write("/path/to/file") do |file|
  if file.executable?
    "#!/usr/bin/env ruby\nputs 'Hello World!'"
  else
    "Hello World!"
  end
end

In a pure-lambda language, we would take the block in as a normal argument to the function, then call it inside the closure:

def write(location, block)
  path = Pathname.new(location)
  raise "Location does not exist" unless path.exist?
 
  this = self
  File.open(path, "w") do |file|
    file_data = file.read
 
    # imaginary Ruby construct that would be needed without
    # non-local-super
    this.super.write(location, file_data)
    data = block.call(file)
    raise Return.new(false) if Digest::MD5.hexdigest(file_data) == data.hash
    file.puts YAML.dump(data)
  end
 
  return true
rescue Return => e
  return e.object
end

The real benefit of Ruby’s approach comes from the fact that the code inside the block would be identical if the method did not take a block. Consider the identical method, except taking a File instead of a location:

def write(file)
  file_data = file.read
  super(file)
  data = yield file
  return false if Digest::MD5.hexdigest(file_data) == data.hash
  file.puts YAML.dump(data)
  return true
end

Without the block, the Ruby code looks exactly the same. This means that Ruby programmers can more easily abstract out repeated patterns into methods that take blocks without having to rewrite a bunch of code. It also means that using a block does not interrupt the normal flow of code, and it’s possible to create new “control flow” constructs that behave almost identically to built-in control flow constructs like if and while.

Rails uses this to good effect with respond_to, which provides convenient syntax for declaring content negotiation:

def index
  @people = Person.find(:all)
 
  respond_to do |format|
    format.html # default action is render
    format.xml { render :xml => @people.xml }
  end
end

Because of the way Ruby blocks work, you can also return from any of the format blocks:

def index
  @people = Person.find(:all)
 
  respond_to do |format|
    format.html { redirect_to(person_path(@people.first)) and return }
    format.xml  { render :xml => @people.xml }
    format.json { render :json => @people.json }
  end
 
  session[:web_service] = true
end

Here, we returned from the HTML format after redirecting, allowing us to take additional action (setting a :web_service key on the session) for other cases (XML and JSON mime types).

Keep in mind that the code above is a demonstration of a number of features of Ruby’s blocks. It’s very rare to see return, yield and super all used in a single block. That said, Ruby programmers commonly use one or more of these constructs inside blocks, because their usage is seamless.

So Why Are Ruby’s Blocks Better?

If you made it this far, let’s take a look at another use of blocks in Ruby: mutex synchronization.

Java supports synchronization via a special synchronized keyword:

class Example {
  final Lock lock = new Lock();
 
  void example() {
    synchronized(lock) {
      // do dangerous stuff here
    }
  }
}

Essentially, Java provides a special construct for expressing the idea that it should run a block of code once at a time for a given instance of the synchronization object. Because Java provides a special construct, you can return from inside the synchronization block, and the Java runtime does the appropriate things.

Similarly, Python required the use of try/finally until Python 2.5, when they added a special language feature to handle the try/finally idiom:

class Example:
  # old
  def example(self):
    lock.acquire()
    try:
      ... access shared resource
    finally:
      lock.release() # release lock, no matter what
 
  # new
  def example(self):
    with lock:
      ... access shared resource

In Python’s 2.5’s case, the object passed to with must implement a special protocol (including __enter__ and __exit__ methods), so the with statement cannot be used like Ruby’s general-purpose, lightweight blocks.

Ruby represents the same concept using a method that takes a block:

class Example
  @@lock = Mutex.new
 
  def example
    @@lock.synchronize do
      # do dangerous stuff here
    end
  end
end

Importantly, synchronize is a normal Ruby method. The original version, written in pure Ruby, looks like this:

def synchronize
  lock
  begin
    yield
  ensure
    unlock
  end
end

It has all the hallmarks of what we’ve discussed so far. It locks, yields to the block, and ensures that the lock will be released. This means that if a Ruby programmer returns from inside the block, synchronize will behave correctly.

This example demonstrates the key power of Ruby’s blocks: they can easily replace language constructs. In this case, a Ruby programmer can take unsafe code, plop it inside a synchronization block, and it will continue to work.

Postscript

I’ve historically written my posts without very many links, mostly out of a fear of links going out of date. I’ve received increasing requests for more annotations in my posts, so I’ll start doing that. Let me know if you think my annotations in this post were useful, and feel free to give me any suggestions on that front that you find useful.

notes from yet another rails 3 upgrade

2010-02-04T12:20:29+00:00

These tips and tricks are valid as of today, February 3rd, 2010. Who knows what next week will bring, but for now, I thought compiling some of the techniques and hacks I've seen might help others.

Rails Related

If you see a "no such file to require: rails/all" you probably don't have Rails installed correctly. I was basing my upgrade on the great screencast produced by Topfunky over at peepcode.com, and his method didn't work for me. To get things running I added this to my Gemfile:


path "/Users/jack/git/rails", :glob => "{*/,}*.gemspec"
gem "rails", "3.0.pre"
gem "rails", "3.0.0.beta" # Just changed today it seems

Obviously, change the path to wherever you have rails checked out locally.

HAML

If you see something like this: Missing template articles/index with {:formats=>[:html]} in view path

you probably need to install haml.


$ haml --rails .

and then you'll likely see an error about html_safe!

Geoff has a nice little idea going with a config/initializers/rails3_hacks.rb file which I've also done. You can drop this in and get by the errors for now:


projects/app master > cat config/initializers/rails3_hacks.rb 
class String
  def html_safe!() self end
  def html_safe?() true end
  def safe_concat(value) value+self end
end

WillPaginate

Looks like Mislav and other's have been doing a lot of work in upgrading the plugin. If you see a "no paginate method for Class...." error, try cloning the source down locally, check out the rails 3 branch,build and install that gem and then update your Gemfile:


...
gem 'will_paginate', '3.0.pre'
...

It also seemed that WillPaginate wasn't being required into my app. For now I've created a config/initializers/requires.rb file and dropped in random requires for attention later. Putting a "require 'will_paginate'" in there fixed that up nicely.

Update:
Seems something's broken now. I'm getting this error:


uninitialized constant ActiveRecord::Calculations::CALCULATIONS_OPTIONS

Seems WP relies on that constant without checking for its existence, and it looks like its gone now. Easy enough to quick patch it if you need to get by it, but I imagine it'll be fixed in the rails3 branch soon enough.
Assorted Issues

HTMLEntities doesn't seem to be working, so I commented that out for now. For a great beginning list of what gems should be working, take a look here: http://wiki.rubyonrails.org/rails/version3/plugins_and_gems

Devise: not ready for rails 3 yet. I think they're trying to see what routing changes will settle and how that will affect their code. I'm going to see if I can pop over into that and help out a bit as I'd love to keep using it in this app.

new_rails_defaults.rb: I had an error from this file and realized it's old anyway, so git rm and it's gone.

Passenger

I also overhauled a bunch of other ruby related items on my boxes recently, and one of those items was passenger. After updating to 2.2.9, reinstalling the module, changing RailsEnv to RackEnv, things are mostly good. Remember to point to the right ruby if you're using rvm. I also deleted the config.ru file per their recommendation: "Smart spawning (the mechanism with which REE’s 33% memory reduction is implemented) is *not* supported for Rack apps. This means that if you want to utilize smart spawning with Rails 3, then you should remove your config.ru file."

I think that's it for now. I imagine I'll update this page as I deal with new issues. Pretty excited about the new changes and starting to actually make use of them.

Update

Previous notes were from my macbook pro. Now that I'm back on the iMac I'm finding other issues. Latest is:


/Users/jack/git/rails/activesupport/lib/active_support/dependencies.rb:167:in `require': no such file to load -- rails/commands/rails (LoadError)

I'm actually not sure if that was something I did as it looks like its definitely just "require 'rails/commands'" in the latest source, but either way, if you're seeing that, just take off the second rails in that line. Correct line should be:


require 'rails/commands'

at the end of your script/rails.

Bundler

Somehow just found myself facing this:


projects/app master > bundle install
/Users/jack/.rvm/gems/ree-1.8.7-2010.01/bin/bundle: line 1: require: command not found

A quick cat and I see something's changed, no #!...ruby line to see. Realized I had bundler 0.9.0 and 0.9.0.pre4, so I uninstalled them both, and reinstalled with:


$ gem install bundler --pre

and now have bundler-0.9.0.pre5 installed.

Still things are borked, but it seems its a result of github being down: "Page did not respond in a timely fashion."

Will try again in a few to make sure this gets me back on track.

Update

Ok, github's back, and just saw a tweet from Wycats. Looks like 0.9.0 final is out now, but it's recommended to wait til tomorrow and 0.9.1 + some docs. YMMV.

Application != ApplicationController

So, the app I'm converting was originally written in Merb, was partially converted to Rails 2, and is now on its way towards Rails 3. In this transition I've generally remembered what pieces to change from Merb to Rails, but I had a nice stumble earlier in forgetting to change a controller. So, if you find yourself in a place where you have a route that you know works, but is not recognized, make sure your controller is named whatever_controller.rb, and contains the code:


class WhateverController  ApplicationController
...
end

not " Application". You'll an error about a route missing and could easily spend a couple hours missing the obvious.

Routes

I keep tripping myself up with this. Lets say you have a route defined as:


  match 'foobar/:foobar' => 'foobar#foobar', :as => :foobar

These calls will all fail:


ree-1.8.7-2009.10 > app.foobar_path
ActionController::RoutingError: No route matches {:action=>"foobar", :controller=>"foobar"}
... from (irb):3
ree-1.8.7-2009.10 > app.foobar_path('')
ActionController::RoutingError: No route matches {:use_defaults=>false, :action=>"foobar", :foobar=>"", :controller=>"foobar"}
... from (irb):4
ree-1.8.7-2009.10 > app.foobar_path('test')
 => "/foobar/test"

until that last one which passes in a value. This makes sense, as the route defines foobar as a requirement, and not passing in a value is an 'error'. It's a little less obvious that an empty string doesn't satisfy it to me, and definitely not obvious when you have this in a loop and an object just happens to be missing its value for foobar.

One way around this is to make the :foobar piece optional, and handle that reality in your controller elsewhere. You could also protect against this by checking that object.foobar exists in your loop, and not linking unless it does.

I've also just realized that for a certain route of mine, one that passes in domain names, sending in 'euraeka.com' will result in an error. I'm not sure if this is the recommended solution, but its letting me progress for now:


  match 'domain/:domain' => 'search#domain', :as => :domain, :domain => /.+/

I'm using domain here instead of host as I think host might be reserved somewhere. Couldn't seem to get it to work before, so if you're having that issue, let me know.

How and why I joined the “suit people”

2010-02-02T19:19:26+00:00

It is now official: I have traded my freedom & home office for a job title, an Aeron chair in a cubicle and a 401K.

I received my new employee package and, in less than a week, I will officially become a full-time employee at SCEA (Sony Computer Entertainment America).

I’m going to work in the PlayStation department, working on PS3, PS2 and PSP game titles developed by various game studios.

Why ‘o why?

Why leave behind a happy life of indie contracting to join corporate America?

For many reasons actually:

A Team

Being a consultant I have been working with other independent consultants and existing teams. Nonetheless, I really miss being part of a stable team which grows together and learns from each other as we go through new projects and maintain old ones.

Long term plan

As a consultant, I usually start projects or “rescue” existing projects. I work on them for a little while and then move on. It’s exciting and rewarding but you don’t really pay the consequences of your mistakes. You usually don’t have to maintain the code you wrote and you rarely deal with the mistakes you made.

It sounds good, nobody likes to maintain the crappy awesome code they wrote 2 years ago and most developers love working on new stuff. But at the same time, to become a better engineer you need to learn from you mistakes and assuming responsibility for your bad decisions is part of the process.

It might sound weird, but I’m actually excited to work on long term projects and feel some sort of ownership over the projects. Having to support games for many years means that I’d better not mess up the implementation. And if I do, I hope I’ll quickly learn from my mistakes.

Avoiding burn out

There is no secret: when you are passionate about what you do, you have a hard time stopping and taking a break. I’m a recovering workaholic and it’s really hard for me to say no when I’m presented the possibility to work on interesting projects. I love what I do and I keep writing code even after I’m done with client work.

The problem is that this can start me on the slippery slope to isolating myself from friends, family and people who don’t share the same passion. I’m really lucky that my wife is a geek and loves hanging out at conferences, looking at code and playing with my buggy prototypes. But still, I spend too much time “playing” with my computer and I just can’t manage my free time wisely.

Having a full time position will hopefully help me put boundaries and will hopefully teach me to disconnect from work.

Exciting projects

Hey, let’s be honest, how many geeks do you know don’t want to work in the video game industry? By the way, if you don’t have a PS3, they are now at $299 and on top of getting an awesome console you get a blue ray player! (And no, I do not receive any bonuses or commissions for mentioning the console or promoting it in my blog. I had to pay for my own like everyone else.)

Corporate America? Are you going to write Java now?

No, I’m mainly going to stick to the language I love: Ruby.
From time to time I will probably use other languages here and there, but that usually makes me love Ruby even more. The reality is that Ruby’s power and flexibility seem to be appreciated by SCEA, which makes sense when you have tight deadlines and a lot of new technologies to deal with. Ruby is a perfect match!

As you can guess, I can’t go into any detail about how and why Sony uses Ruby, but let me just say that while games are still usually written in C++, they are becoming more and more interactive and need to communicate with game servers where some logic operates. Game players also need to interact with other gamers as well as check their gaming progress online, as well as the progress of the players around them etc… Basically, outside of the game engine and the console SDK, there is a lot of potential for Ruby.

Coming back to Corporate America, I have to say that I’ve known my future manager for a few years now. He’s always been a fervent Ruby advocate and has introduced lots of teams to the happiness of Ruby & Rails development. He’s also a great developer who’s contributing patches to major projects and has a bunch of cool stuff on github. To give you an idea, my job description mentions Rails, Merb, Sinatra, CouchdB, MongoDB, Redis, AWS. All these Ruby technologies are actually already used in production or are being seriously evaluated.

I’m also really looking forward to join the existing team. I know I’m going to love working with a bunch of awesome developers coming from various backgrounds.

Those who know me, know that I’m not a morning person. And while your typical office job is categorized as ‘9-5′, don’t feel too bad for me. I will be joining the video game product department, and morning people are rather rare in these kinds of groups

Conclusion

I’m really excited about this opportunity. For me, it is proof again that the Ruby revolution took place and that the Enterprise is evolving. Of course, time will tell if I am right, but I am quite confident.

Also, Sony is always looking for new, talented people who want to push the entertainment world to the next level. Feel free to keep in touch with me if you are interested in joining the fun.

Undo/Redo in MacRuby

2010-02-02T10:19:44+00:00

As I’m working on my upcoming O’Reilly MacRuby book, I’m writing quite a lot of example code. I have spent the last few weeks digging through most of the Foundation framework classes to hopefully make Cocoa more accessible to Ruby developers.

In some instances things might look quite weird to someone new to Cocoa in some cases, things seem almost too easy. Here is an example implementing a undo/redo functionality using Foundations’ NSUndoManager.

framework 'Foundation'
class Player
  attr_accessor :x, :y
 
  def initialize
    @x = @y = 0
  end
 
  def undo_manager
    @manager ||= NSUndoManager.alloc.init
  end
 
  def left
    undo_manager.prepareWithInvocationTarget(self).right
    @x -= 1
  end
 
  def right
    undo_manager.prepareWithInvocationTarget(self).left
    @x += 1
  end
end

Which you can use as such:

 
>> lara = Player.new
=> <Player:0x200267c80 @y=0 @x=0>
>> lara.undo_manager.canUndo
=> false # normal since we did not do anything yet
>> lara.left
=> -1
>> lara.x # -1
=> -1
>> lara.undo_manager.canUndo
=> true # now we can undo, so let's try
>> lara.undo_manager.undo # undo back to initial position
=> #<NSUndoManager:0x200257560>
>> lara.x
=> 0
>> lara.undo_manager.canUndo
=> false # we can't anymore which makes sense 
>> lara.undo_manager.canRedo
=> true # however we can redo what we just undone
>> lara.undo_manager.redo # redo to before we called undo
=> #<NSUndoManager:0x200257560>
>> lara.x
=> -1

The above example was tested in macirb but as you can see, actions can be undone and redone very very easily. This is just a quick preview of what you can do using Ruby + Cocoa and hopefully it will give you some cool ideas to implement.

Bundler 0.9: Heading Toward 1.0

2010-02-01T08:50:57+00:00

Over the past two years, Carl and I have been working on-again off-again on the problem of dependency resolution. Fabien Franzen implemented the first partial solution for Merb, with thor merb:gem:install.

When we started working on Rails, we knew we wanted to finally crack the nut, making it possible for Rails itself to have some of its own dependencies, and solving some persistent, long-term problems with gem plugins.

Earlier this year, we released bundler, our first real attempt to solve this problem. Among other things, it shipped with the first true dependency resolver for Rubygems. It also modeled the entire dependency process on top of Rubygems, allowing us to support git repositories in the same dependency resolution process as Rubygems themselves.

Over the next few months, we refined the bundler quite a bit. We’re proud of the fact that individuals, web shops, and deployment companies have adopted bundler, and Gemfile has become a standard way of expressing gem dependencies for an app.

During this time, we’ve also received quite a bit of feedback. As we approach the release of Bundler 1.0, which we hope to ship along with Rails 3.0 final, we took the opportunity to take a look at all the feedback people have sent so far.

Having done so, we’re proud to announce Bundler 0.9 with radically improved workflows that fit our needs and the needs of those who have contributed feedback and patches.

The Cache

Before Bundler 0.9, Bundler installed gems and git repositories to a local application cache. When we looked at the deployment workflow, we found that while people liked the ability to cache their .gem files in their application, they had a lot of trouble at deployment time.

Essentially, people (and we!) wanted the ability to expand the gems to a central cache that they could reuse across deployments. People tried various symlinking approaches, but it became clear to us Bundler 1.0 needed to support this workflow natively.

On a related note, people who developed a number of applications on a single machine wanted to be able to reuse a single system cache across their application, and not need to connect to remotes so often when they already had all the gems they needed on their systems.

Finally, Rails itself grated against the default Bundler workflow. Most people installing a new Rails app run gem install rails, then rails my_app, and then want to go into their application and start working. But with Bundler 0.5 to 0.8, they needed to hit the remotes again even though, by definition, the system gem cache already had all the gems they needed.

Runtime

The fully-packaged application works fantastically for deployment, enabling a repeatable, reliable development to staging to production workflow. However, the need to explicitly bundle the application after added a new dependency in the very early stages of an application’s life cycle feels more like a compile step than a lean mean agile machine.

In Bundler 0.9, we’re adding the ability to run your application against gems already on your system, with the same reliability that you’ve grown to love from our dependency resolver.

Once you’ve gotten your application working, you can lock down the dependencies, so coworkers or production will use exactly the same environment that worked for you.

This allows you to use your common system gem repository across multiple apps, with a fully resolved virtual environment for each application based on the app’s dependencies specified in the Gemfile.

Locking

A lot of people (including me) love the idea of storing the .gem files in the application to create a single deployable unit with no network dependencies.

Storing the .gem files in the repository also provides a record of the fully resolved dependencies for your application, so that a new released gem between testing and deployment cannot change the environment. In Bundler 0.9, we provide a mechanism for you to save that record without having to also store the .gem files.

This won’t give you a single, dependency-less deployment package, but it will save you from unexpected environment changes.

New Commands

When we started Bundler, we had just one command: gem bundle. Over the following months, the command took on a life of its own, with a slew of flags representing both options and verbs. Bundler 1.0 will have a small, streamlined list of commands, each with its own set of flags:

bundle pack

Take all .gem files needed for the application and place them into the local application. By default, bundler places them in vendor/gems.

bundle lock

Resolve all dependencies in the Gemfile, and store a record of that process. In the future, use that record, preventing any changes in the environment or in the remote sources from changing the gems used by the application.

bundle install

Install the bundle specified by the Gemfile to the cache. By default, Bundler installs the gems (and git repositories) into system gems. You can override this to install into another location, or into your local application, just like in Bundler 0.8 and before.

After cloning a new repository for the first time, you will probably want to run this command.

If you packed gems, Bundler will use the gems from your application. If you packed and locked gems, Bundler will install the gems immediately without updating the remote sources.

bundle check

Check to see whether your cache has all the required dependencies. If it does not, Bundler will print out the list of missing gems.

bundle exec …

Run the specified command using the local environment. For instance, if you have Rails 2.3 and 3.0 installed, if you run bundle exec rails . in a Rails 2.3 application, Bundler will run the rails command using Rails 2.3 (and other gems in the current environment).

Environments

In Bundler 0.9, as in Bundler 0.8 and before, the Gemfile specifies an app-specific environment. New in Bundler 0.9, you may store the .gem files and unpacked gems in a central location, as well as in the application itself.

Because Bundler ships with a dependency resolver, you do not need to specify named environments and switch between them. Instead, the environments are virtual, based on the gems you specify as application dependencies in the Gemfile, and you still get all the benefits of a shared, system-wide cache.

SafeBuffers and Rails 3.0

2010-02-01T07:41:19+00:00

As you may have read, Rails adds XSS protection by default in Rails 3. This means that you no longer have to manually escape user input with the h helper, because Rails will automatically escape it for you.

However, it’s not as simple as all that. Consider the following:

Hello <strong>friends</strong>!
 
<%= tag(:p, some_text) %>
<%= some_text %>

In the above example, we have a few different scenarios involving HTML tags. First off, Rails should not escape the strong tag surrounding “friends”, because it is unambiguously not user input. Second, Rails should escape some_text in the <p> tag, but not the <p> tag itself. Finally, the some_text in the final tag should be escaped.

If some_text is <script>evil_js</script>, the above should output:

Hello <strong>friends</strong>!
 
<p>&lt;script&gt;evil_js&lt;/script&gt;</p>
&lt;script&gt;evil_js&lt;/script&gt;

In order to make this happen, we have introduced a new pervasive concept called html_safe into Rails applications. If a String is html_safe (which Rails determines by calling html_safe? on the String), ERB may insert it unaltered into the output. If it is not safe, ERB must first escape it before inserting it into the output.

def tag(name, options = nil, open = false, escape = true)
  "<#{name}#{tag_options(options, escape) if options}#{open ? ">" : " />"}".html_safe
end

Here, Rails creates the tag, telling tag_options to escape the contents, and then marks the entire body as safe. As a result, the <p> and </p> will emerge unaltered, while Rails will escape the user-supplied content.

The first implementation of this, in Koz’s rails-xss plugin, accomplished the above requirements by adding a new flag to all Strings. Rails, or Rails applications, could mark any String as safe, and Rails overrode + and << to mark the resulting String appropriately based on the input Strings.

However, during my last performance pass of Rails, I noticed that overriding every String concatenation resulted in quite a bit of performance overhead. Worse, the performance overhead was linear with the number of <%= %> in a template, so larger templates didn’t absorb the cost (as they would if the problem was once-per-template).

Thinking about the problem more, I realized (and confirmed with Koz, Jeremy, and Evan Phoenix of Rubinius), that we could implement roughly the same feature-set in a more performant way with a smaller API impact on Ruby. Because the problem itself is reasonably complex, I won’t go into a lot of detail about the old implementation, but will explain how you should use the XSS protection with the new implementation. If you already used Koz’s plugin or are working with the prereleases of Rails, you’ll notice that today’s commit changes very little.

SafeBuffer

In Rails 3, the ERB buffer is an instance of ActiveSupport::SafeBuffer. SafeBuffer inherits from String, overriding +, concat and << so that:

If the other String is safe (another SafeBuffer), the buffer concatenates it directly
If the other String is unsafe (a plain String), the buffer escapes it first, then concatenates it

Calling html_safe on a plain String returns a SafeBuffer wrapper. Because SafeBuffer inherits from String, Ruby creates this wrapper extremely efficiently (just sharing the internal char * storage).

As a result of this implementation, I was starting to see a lot of the following idiom in the codebase:

buffer << other_string.html_safe

Here, Rails is creating a new SafeBuffer for the other_string, then passing it to the << method of the original SafeBuffer, which then checks to see if it is safe. For cases like this, I created a new safe_concat method on the buffer which uses the original, native concat method, skipping both the need to create a new SafeBuffer and the need to check it.

Similarly, concat and safe_concat in ActionView proxy to the concat and safe_concat on the buffer itself, so you can use safe_concat in a helper if you have some HTML you want to concatenate to the buffer with no checks and without escaping.

ERB uses safe_concat internally on the parts of the template outside of <% %> tags, which means that with the changes I pushed today, the XSS protection code adds no performance impact to those cases (basically, all of the plain text in your templates).

Finally, ERB can now detect the raw helper at compile time, so if you do something like <%= raw some_stuff %>, ERB will use safe_concat internally, skipping the runtime creation of a SafeBuffer and checks for html_safety.

Summary

In summary, the XSS protection has the following characteristics:

If a plain String is passed into a <%= %>, Rails always escapes it
If a SafeBuffer is passed into a <%= %>, Rails does not escape it. To get a SafeBuffer from a String, call html_safe on it. The XSS system has a very small performance impact on this case, limited to a guard calling the html_safe? method
If you use the raw helper in a <%= %>, Rails detects it at compile-time of the template, resulting in zero performance impact from the XSS system on that concatenation
Rails does not escape any part of a template that is not in an ERB tag. Because Rails handles this at template compile-time, this results in zero performance impact from the XSS system on these concatenations

In comparison, the initial implementation of XSS impacted each concatenation or + of String, had impact even if the app used the raw helper, and even on plain Strings in templates.

That said, I want to extend personal thanks to Koz for getting the first draft out the door. It worked, demonstrated the concept, and let the community test it out. All in all, an excellent first pass.

Corporations Cannot Have Natural Rights… Duh

2010-02-01T06:47:19+00:00

When I first read through the Citizens United decision that essentially made it illegal to “discriminate” against corporate political speech, I found myself very torn. On the one hand, I’m an extremely avid supporter of free speech rights, and Kennedy wrote the majority decision to tug at my heartstrings:

Premised on mistrust of governmental power, the First Amendment stands against attempts to disfavor certain subjects or viewpoints or to distinguish among different speakers, which may be a means to control content.

On the other hand, something felt wrong.

In the days that followed, I realized the the court had pulled a bait-and-switch, finding a way to split defenders of free speech, preventing a consistent response to the ruling, and resulting in a fair bit of liberal support for the ruling.

If you read the above quote closely, you can see the lie hiding in plain sight. The court, without drawing much notice, calls corporations “speakers” in the context of the First Amendment.

The problem is simple. The Bill of Rights exists to protect, for the long term, a set of natural rights. If you don’t believe me, read the writings of the anti-federalists, the guys who argued against the constitution and successfully agitated for the addition of the Bill of Rights in the first place:

That insatiable thirst for unconditional controul over our fellow-creatures, and the facility of sounds to convey essentially different ideas, produced the first Bill of Rights ever prefixed to a Frame of Government. The people, although fully sensible that they reserved every tittle of power they did not expressly grant away, yet afraid that the words made use of, to express those rights so granted might convey more than they originally intended, they chose at the same moment to express in different language those rights which the agreement did not include, and which they never designed to part with, endeavoring thereby to prevent any cause for future altercation and the intrusion into society of that doctrine of tacit implication which has been the favorite theme of every tyrant from the origin of all governments to the present day.

The language is a big crufty, but the meaning is clear. Without government, people have certain rights. They choose to surrender some of them to the government. The Constitution outlines the powers they have surrendered, and the Bill of Rights outlines specific cases where those powers may not tread.

However, corporations do not exist outside of government. They are not “creatures”. Instead, corporations are creations of the government, and therefore cannot reserve any rights via a bill of rights.

Looking at the quote from the ruling again:

Premised on mistrust of governmental power, the First Amendment stands against attempts to disfavor certain subjects or viewpoints or to distinguish among different speakers, which may be a means to control content.

The ruling fundamentally subverts that actual purpose of the Bill of Rights, using language that appeals to defenders of free speech to apply to entities that, by their very definition, the First Amendment cannot cover.

In short, the Bill of Rights protects the natural rights of people who have already surrendered a fair bit of their autonomy to the federal government. Corporations, by definition, cannot have natural rights, and since they don’t exist outside of government, cannot have surrendered any autonomy to the government. Therefore, the Bill of Rights, and the First Amendment, cannot apply to them. QED.

MacRuby 0.5 final is out

2010-02-01T03:13:36+00:00

After going through two betas, MacRuby 0.5 final is now released and can be downloaded by clicking on the icon below:

MacRuby 0.5

Don’t worry about having MacRuby and Ruby 1.8.x or 1.9 installed, MacRuby is namespaced and won’t affect your current Ruby installations, just download and launch the installer. (Note: The build was compiled for SnowLeopard only)

You can read all the details of the release on the MacRuby website.

So what changed since 0.4? Too many things for me to list them here but basically 0.5 uses LLVM to compile code and make MacRuby faster and integrate better with the Obj-c runtime. However since the last beta, here is what changed:

HotCocoa is now a separate gem
improved AOT compilation
Grand Central Dispatch support – use all your cores without the pain of threads. Read this post for more info.

0.5 is a solid release which I consider production ready, I personally wrote a few of small Cocoa apps in MacRuby and everything has been working very well. Of course, I’m also excited about the new stuff in 0.6 trunk like the debugger previewed a few weeks ago: http://merbist.com/2010/01/18/how-to-detect-cylons-with-macruby/ but also some drastic changes in the primitive classes that I might cover later on.

Finally, people are asking if the iPad will be able to run apps running in MacRuby. Unfortunately, the current answer is: no. The two issues with the IPhone/iP*d OS are the lack of Garbage Collector and support for BridgeSupport (needed to define CocoaTouch constants available from MacRuby). However, this matter is being discussed on the mailing list and progress is made by contributors (the core team primarily focusing on the desktop).

That is going to be an exciting Ruby week as MacRuby 0.5 is now out and Rails 3 beta/RC0 is expected really soon.

How I use Foursquare

2010-01-31T06:00:00+00:00

There’s been a little discussion around my Twitter circle about how or why Foursquare is useful. (I’m glad I’m not the only one who sees the irony of using Twitter to ask that question, since the exact same thing was - and still is - asked about Twitter.)

I think Foursquare is fun, and has a decent chance of being a long-term player in the location game. Of course I thought that about Dodgeball, too. For the benefit of the Foursquare doubters, here’s my tips on how to use it “right”. YMMV.

Be somewhere with other players
Foursquare is only fun with other people. If you’re the only one checking in near you, you’ll get bored. So unless you’re a real early adopter, wait until Foursquare has traction in your city. I live in a town that has very little activity, but I travel to Atlanta a lot; Foursquare is much more fun when I’m there.
Have some friends
Not only do you need people near you playing, you need to be friends with some of them. Real friends. Friends you’d share your cell number with. Friends you would enjoy having drop by your table when you’re at the same restaurant. Friends you’d like to fight for control of the neighborhood coffee shop.
Don’t have too many friends
Don’t add everyone from Twitter and Facebook. Be selective. I have maybe 20 Foursquare friends, and all but one are people I know in person.
Don’t automatically post every check-in to Twitter
The whole point of Foursquare having a separate set of friends is that you can limit who you share your location with. If you post every update to Twitter, you’re not only over-sharing to your followers, but you lose that intimacy.
Post selective updates to Twitter
You can always choose to post a check-in to Twitter on an individual basis, like if you’re at a cool restaurant you’ve never been to before, or at a conference. The advantage over just tweeting about it is a 4sq link that includes tips from other people, Yelp links, a map, phone numbers…much more info than just a geocoded location.
Play the game
Foursquare’s game mechanics are fun. Enjoy them. Battle it out for mayoral control of the taco stand. Stake your claim at every doughnut shop within 25 miles. Pick some badges and try to win them.
Discover new places, find your friends
On the rare occasion that I find myself in a city where I have Foursqaure friends, no plans, and the time and energy for some, I’ll pull up Foursquare and see if anyone is already out and about. If they are, I may join them, or at least click a button to send a SMS or a tweet and see what’s going on.

That’s how I use Foursquare. I get a reasonable amount of pleasure from it. It’s amusing, and sometimes helps me find something to do. The best way to “get” Foursquare is to try it for a while.

The Irony of the iPad: A GREAT Day for Open Technologies

2010-01-28T00:03:49+00:00

With the announcement of the iPad, the usual suspects have come out decrying a closed, proprietary, fully locked down system.

For instance, a story on the top of Hacker News today says:

This is what I asked in January 2007 on this site, shortly after the original iPhone was launched:

“1. Will Apple lock down the iPhone, blocking Flash, Java, custom widgets, and open development from its new platform?

2. Could Apple’s multi-touch patents actually stifle growth of new, interactive displays?”

Unfortunately, that turned out to prescient

And the FSF is out there calling this an unprecedented march of DRM:

With new tablet device, Apple’s Steve Jobs pushes unprecedented extension of DRM to a new class of general purpose computers

It’s a fair initial reaction. Apple didn’t build a general-purpose computer as its next entry into the market. Instead, they built a heavily proprietary, locked down device. In order to install an application onto the device, Apple must approve the application.

I don’t need to address the merits of the argument against how Apple handles native applications, because it’s irrelevant. A much, much larger force is at work here.

With the iPad, Apple has created two platforms. First, they have produced a heavily proprietary, native platform that requires Apple approval and has significant Apple restrictions. But ironically, with their heavy focus on improving the quality of Safari and the HTML standard, they have shipped the iPad with a platform based on open, unencumbered technologies.

If you haven’t been paying attention, over the past couple of years, the web platform has gotten offline APIs, improved caching support, local storage (on Safari, that includes an on-device SQLite database accessible through JavaScript), CSS-based animations, and custom, downloadable fonts. Mobile Safari has support for gestures, Geolocation, and hardware-accelerated graphics.

Additionally, Apple has remained at the forefront of these technologies, literally building some of them for mobile devices (hardware-accelerated animations were built for the iPhone, and by extension, the iPad). The Open Source Webkit project has remained extremely active, and in fact, has only accelerated progress since Apple first released its Native SDK, so Apple’s “locked down” strategy has a very carefully carved out intentional exception.

Apple even makes it easy to take a web app and put it on the home screen amongst normal apps. When you do this, the iPhone downloads all the assets in the HTML5 cache manifest to make the work better as an offline app. This is how I use Gmail on my iPhone (because Google knows what’s going on in this space, they leverage new tech in Safari quite well). When Apple rejected Google Voice, Google immediately built a Safari version of the app. The way they tell it:

Already, Google says it is readying a replacement for the Google Voice app that will offer exactly the same features as the rejected app—except that it will take the form of a specialized, iPhone-shaped Web page

Ironically, despite claims that not allowing Flash or Java represent a victory for proprietary technologies and a loss for open technologies, they represent quite the opposite. By restricting the web platform on the iPhone and iPad to open, patent-free, technologies, Apple has created a highly desirable market for pure-HTML5 apps. This is, frankly, a win for supporters of open technologies.

How to detect Cylons with MacRuby

2010-01-19T00:13:37+00:00

Over the weekend, MacRuby’s trunk became version 0.6 and the bug fixing is currently done in both the 0.5 branch and trunk. Based on MacRuby’s usual release cycle I would expect a 0.5 beta3 or 0.5 final to be released soon so most of the work can be focused on trunk.

I’ll let you check on the TODO list to see what was done in 0.5 and what is in the plan for 0.6.

However, there is one feature in 0.6 that I know lots of you will just love! The good news is that Laurent already committed a very early version of his work so I figured, I should share the good news with you:

Introducing MacRuby’s debugger!

If you were expecting to read: “a Cylon detector!”, keep on reading.

Again, this feature is in really early development since it’s scheduled for 0.6 and 0.5 final is not out yet. But if you install MacRuby using the nightly builds or build from trunk, you can already play with the debugger.

Let me give you a really quick tour of the debugger.

Let’s imagine that we were given the task to debug the cylon detector written by Gaius Baltar which looks like that:

characters = %w{Adama Apollo Baltar Roslin StarBuck Six}
 
def cylon?(character)
  false
end
 
characters.each do |character|
  if cylon?(character)
    puts "#{character} is a Cylon!"
  else
    puts "#{character} is not a cylon."
  end
end

Here is what happens when I execute the script:

$ macruby cylon_detector.rb 
Adama is not a cylon.
Apollo is not a cylon.
Baltar is not a cylon.
Roslin is not a cylon.
StarBuck is not a cylon.
Six is not a cylon.

The only problem is that we all know that Six is a Cylon, the detector isn’t working right so let’s debug it:

$ macrubyd cylon_detector.rb
Starting program.
cylon_detector.rb:1> b cylon_detector.rb:8 if character == 'Six'
Added breakpoint 1.
cylon_detector.rb:1> c
Adama is not a cylon.
Apollo is not a cylon.
Baltar is not a cylon.
Roslin is not a cylon.
StarBuck is not a cylon.
cylon_detector.rb:8> p cylon?(character)
=> false
cylon_detector.rb:8> p "This detector is broken!"
=> "This detector is broken!"
cylon_detector.rb:8> p def cylon?(character); character == 'Six'; end
=> nil
cylon_detector.rb:8> p cylon?(character)
=> true
cylon_detector.rb:8> p cylon?('Matt')
=> false
cylon_detector.rb:8> c
Six is a Cylon!
Program exited.

The first thing we do is to add a conditional breakpoint:

b cylon_detector.rb:8 if character == 'Six'

Basically, the debugger will add a breakpoint at line 8 which will only be active when the value of ‘character’ is equal to ‘Six’.
Now that the breakpoint added, we can continue the program execution and just wait until we reach the defined condition.

cylon_detector.rb:1> c

Once we reach the breakpoint, we evaluate the result of “cylon?(character)” by using the p command. We see that the result is “false” when we know for sure that it should be true since the value of the character variable is ‘Six’ and she is a cylon. At this point, you might have guessed that somewhat acted as a cylon agent and I pretended to fix the problem by overwriting the “cylon?” method:

cylon_detector.rb:8> p def cylon?(character); character == 'Six'; end

Now that the method is overwritten, I can check that Six is recognized as being a cylon:

cylon_detector.rb:8> p cylon?(character)
=> true

and also check that I am not detected a cylon:

cylon_detector.rb:8> p cylon?('Matt')
=> false

I can now continue the execution of the program and see that Six is detected as a Cylon!

Of course this is just a very early version of the debugger and we will see lots of improvement in the next few weeks. Who knows someone might even create a GUI for the debugger and/or a Xcode integration.

Anyway, the point being that MacRuby developers should expect a lot of awesome stuff coming up their way soon. (also be careful about the skin jobs around you, cylon detectors can’t be trusted!)

Controlling iTunes with MacRuby

2010-01-18T03:54:19+00:00

Since Mac OS X v10.5, Apple added a technology called Scripting Bridge which allows to control and communicate with scriptable applications such as Mail, iChat or iTunes.

A few weeks back, I showed how to control iChat with MacRuby. This time I’m going to show you how to control iTunes.

Here is a small script that I wrote to wake me up in music every morning.

#!/usr/local/bin/macruby
framework 'Foundation'
framework 'ScriptingBridge'
 
itunes = SBApplication.applicationWithBundleIdentifier("com.apple.itunes")
load_bridge_support_file 'iTunes.bridgesupport'
itunes.run
 
class SBElementArray
  def [](value)
    self.objectWithName(value)
  end
end
 
itunes.stop
playlist = itunes.sources["Library"].userPlaylists["morning"]
playlist.playOnce(false) if playlist

The idea is that I have a Mac Mini streaming music through speakers connected to an AirportExpress in my bedroom.

Let’s go through the script quickly.

We start by loading two frameworks, Foundation and ScriptingBridge.
Now that we have ScriptingBridge loaded, we can control iTunes. To do that, we use:
SBApplication.applicationWithBundleIdentifier(“com.apple.itunes”)
We then load a bridgesupport file that contains the enumerated constants from the iTunes scriptable dictionary.

We make sure iTunes is running by calling #run on the application object.

Before using iTune scriptable interface, we are making the API a bit nicer, it’s totally unnecessary but it makes our code look better.

itunes.sources returns an instance of SBElementArray which is not really an array nor a hash.

The rest of the code is pretty simple, we find the library, find the playlist called ‘morning’ and play it if found.

So you might wonder two things:

What is the iTunes.bridgesupport file?
How do you know what methods are available to control iTunes?

bridgesupport file

The bridgesupport file is important since it defines the required constants.
Apple provides a metadata generator called gen_bridge_metadata which generates a bridgesupport file.

Here is what the documentation says:

$ man gen_bridge_metadata

NAME
gen_bridge_metadata -- Objective-C Bridges Metadata Generator
SYNOPSIS
gen_bridge_metadata [options...] headers...
DESCRIPTION
gen_bridge_metadata is a tool that generates bridging metadata information for a given framework or set of head-
ers. The Objective-C bridges supported in Mac OS X, such as RubyCocoa (Ruby) and PyObjC (Python), read this
information at runtime.
Metadata files describe the parts of an Objective-C framework that the bridges cannot automatically handle. These
are primarily the ANSI C elements of the framework -- functions, constants, enumerations, and so on -- but also
include special cases such as functions or methods that accept pointer-like arguments. These special cases must
be manually specified in separate files called exceptions. The gen_bridge_metadata tool can then read in the
exceptions file when it generates the framework metadata.
The file extension used for metadata files should be .bridgesupport.
Certain elements, such as inline functions, cannot be described in the metadata files. It is therefore required
to generate a dynamic library in order to make the bridges use them. The gen_bridge_metadata tool can take care
of that for you.
The file extension for the dynamic libraries should be .dylib.
You should install metadata files in one of three filesystem locations. For example, for a framework named
MyFramework that is installed as /Library/Frameworks/MyFramework.framework, you can install the
MyFramework.bridgesupport and MyFramework.dylib files in one of the following possible locations, in order of
priority:
o /Library/Frameworks/MyFramework/Resources/BridgeSupport
o /Library/BridgeSupport
o ~/Library/BridgeSupport

The problem is that we don’t have a framework or header file to generate a bridgesupport file for.
So, what we need a header file for iTunes, turns out we have a tool to do that:

$ sdef /Applications/iTunes.app | sdp -fh --basename iTunes

I won’t go in the details of what sdef and sdp do, just check their manual page.
Running the command above will create a iTunes.h which we can use to create a bridgesupport file.
Here is the generated header file: http://gist.github.com/279657

Now, let’s create a bidgesupport file:

$ gen_bridge_metadata -c '-I.' iTunes.h &gt; iTunes.bridgesupport

An that’s how we get the bridgesupport file. (see file: http://gist.github.com/279698)

iTunes Documentation

The easiest way to understand what’s available to you is to open iTunes’ dictionary in the AppleScript Editor.

Otherwise you can study the iTunes.h file.

I wrote a very dumb parser to give you an idea of the methods and properties available when controlling iTunes via ScriptingBridge, here is the output:

Class: iTunesPrintSettings
Properties:
copies (the number of copies of a document to be printed)
collating (Should printed copies be collated?)
startingPage (the first page of the document to be printed)
endingPage (the last page of the document to be printed)
pagesAcross (number of logical pages laid across a physical page)
pagesDown (number of logical pages laid out down a physical page)
errorHandling (how errors are handled)
requestedPrintTime (the time at which the desktop printer should print the document)
printerFeatures (printer specific options)
faxNumber (for fax number)
targetPrinter (for target printer)
 
Method: printPrintDialog:(BOOL)printDialog withProperties:(iTunesPrintSettings *)withProperties kind:(iTunesEKnd)kind theme:(NSString *)theme
Returned: void
Print the specified object(s)
----
Method: close
Returned: void
Close an object
----
Method: delete
Returned: void
Delete an element from an object
----
Method: duplicateTo:(SBObject *)to
Returned: SBObject
Duplicate one or more object(s)
----
Method: exists
Returned: BOOL
Verify if an object exists
----
Method: open
Returned: void
open the specified object(s)
----
Method: playOnce:(BOOL)once
Returned: void
play the current track or the specified track or file.
----
 
Class: iTunesApplication
Properties:
currentEncoder (the currently selected encoder (MP3, AIFF, WAV, etc.))
currentEQPreset (the currently selected equalizer preset)
currentPlaylist (the playlist containing the currently targeted track)
currentStreamTitle (the name of the current song in the playing stream (provided by streaming server))
currentStreamURL (the URL of the playing stream or streaming web site (provided by streaming server))
currentTrack (the current targeted track)
currentVisual (the currently selected visual plug-in)
EQEnabled (is the equalizer enabled?)
fixedIndexing (true if all AppleScript track indices should be independent of the play order of the owning playlist.)
frontmost (is iTunes the frontmost application?)
fullScreen (are visuals displayed using the entire screen?)
name (the name of the application)
mute (has the sound output been muted?)
playerPosition (the player’s position within the currently playing track in seconds.)
playerState (is iTunes stopped, paused, or playing?)
selection (the selection visible to the user)
soundVolume (the sound output volume (0 = minimum, 100 = maximum))
version (the version of iTunes)
visualsEnabled (are visuals currently being displayed?)
visualSize (the size of the displayed visual)
 
Method: browserWindows
Returned: SBElementArray
----
Method: encoders
Returned: SBElementArray
----
Method: EQPresets
Returned: SBElementArray
----
Method: EQWindows
Returned: SBElementArray
----
Method: playlistWindows
Returned: SBElementArray
----
Method: sources
Returned: SBElementArray
----
Method: visuals
Returned: SBElementArray
----
Method: windows
Returned: SBElementArray
----
Method: printPrintDialog:(BOOL)printDialog withProperties:(iTunesPrintSettings *)withProperties kind:(iTunesEKnd)kind theme:(NSString *)theme
Returned: void
Print the specified object(s)
----
Method: run
Returned: void
run iTunes
----
Method: quit
Returned: void
quit iTunes
----
Method: add:(NSArray *)x to:(SBObject *)to
Returned: iTunesTrack
add one or more files to a playlist
----
Method: backTrack
Returned: void
reposition to beginning of current track or go to previous track if already at start of current track
----
Method: convert:(NSArray *)x
Returned: iTunesTrack
convert one or more files or tracks
----
Method: fastForward
Returned: void
skip forward in a playing track
----
Method: nextTrack
Returned: void
advance to the next track in the current playlist
----
Method: pause
Returned: void
pause playback
----
Method: playOnce:(BOOL)once
Returned: void
play the current track or the specified track or file.
----
Method: playpause
Returned: void
toggle the playing/paused state of the current track
----
Method: previousTrack
Returned: void
return to the previous track in the current playlist
----
Method: resume
Returned: void
disable fast forward/rewind and resume playback, if playing.
----
Method: rewind
Returned: void
skip backwards in a playing track
----
Method: stop
Returned: void
stop playback
----
Method: update
Returned: void
update the specified iPod
----
Method: eject
Returned: void
eject the specified iPod
----
Method: subscribe:(NSString *)x
Returned: void
subscribe to a podcast feed
----
Method: updateAllPodcasts
Returned: void
update all subscribed podcast feeds
----
Method: updatePodcast
Returned: void
update podcast feed
----
Method: openLocation:(NSString *)x
Returned: void
Opens a Music Store or audio stream URL
----
 
Class: iTunesItem
Properties:
container (the container of the item)
index (The index of the item in internal application order.)
name (the name of the item)
persistentID (the id of the item as a hexidecimal string. This id does not change over time.)
 
Method: id
Returned: NSInteger
the id of the item
----
Method: printPrintDialog:(BOOL)printDialog withProperties:(iTunesPrintSettings *)withProperties kind:(iTunesEKnd)kind theme:(NSString *)theme
Returned: void
Print the specified object(s)
----
Method: close
Returned: void
Close an object
----
Method: delete
Returned: void
Delete an element from an object
----
Method: duplicateTo:(SBObject *)to
Returned: SBObject
Duplicate one or more object(s)
----
Method: exists
Returned: BOOL
Verify if an object exists
----
Method: open
Returned: void
open the specified object(s)
----
Method: playOnce:(BOOL)once
Returned: void
play the current track or the specified track or file.
----
Method: reveal
Returned: void
reveal and select a track or playlist
----
 
Class: iTunesPlaylist
Properties:
duration (the total length of all songs (in seconds))
name (the name of the playlist)
parent (folder which contains this playlist (if any))
shuffle (play the songs in this playlist in random order?)
size (the total size of all songs (in bytes))
songRepeat (playback repeat mode)
specialKind (special playlist kind)
time (the length of all songs in MM:SS format)
visible (is this playlist visible in the Source list?)
 
Method: tracks
Returned: SBElementArray
----
Method: moveTo:(SBObject *)to
Returned: void
Move playlist(s) to a new location
----
Method: searchFor:(NSString *)for_ only:(iTunesESrA)only
Returned: iTunesTrack
search a playlist for tracks matching the search string. Identical to entering search text in the Search field in iTunes.
----
 
Class: iTunesAudioCDPlaylist
Properties:
artist (the artist of the CD)
compilation (is this CD a compilation album?)
composer (the composer of the CD)
discCount (the total number of discs in this CD’s album)
discNumber (the index of this CD disc in the source album)
genre (the genre of the CD)
year (the year the album was recorded/released)
 
Method: audioCDTracks
Returned: SBElementArray
----
 
Class: iTunesDevicePlaylist
Method: deviceTracks
Returned: SBElementArray
----
 
Class: iTunesLibraryPlaylist
Method: fileTracks
Returned: SBElementArray
----
Method: URLTracks
Returned: SBElementArray
----
Method: sharedTracks
Returned: SBElementArray
----
 
Class: iTunesRadioTunerPlaylist
Method: URLTracks
Returned: SBElementArray
----
 
Class: iTunesSource
Properties:
capacity (the total size of the source if it has a fixed size)
freeSpace (the free space on the source if it has a fixed size)
kind ()
 
Method: audioCDPlaylists
Returned: SBElementArray
----
Method: devicePlaylists
Returned: SBElementArray
----
Method: libraryPlaylists
Returned: SBElementArray
----
Method: playlists
Returned: SBElementArray
----
Method: radioTunerPlaylists
Returned: SBElementArray
----
Method: userPlaylists
Returned: SBElementArray
----
Method: update
Returned: void
update the specified iPod
----
Method: eject
Returned: void
eject the specified iPod
----
 
Class: iTunesTrack
Properties:
album (the album name of the track)
albumArtist (the album artist of the track)
albumRating (the rating of the album for this track (0 to 100))
albumRatingKind (the rating kind of the album rating for this track)
artist (the artist/source of the track)
bitRate (the bit rate of the track (in kbps))
bookmark (the bookmark time of the track in seconds)
bookmarkable (is the playback position for this track remembered?)
bpm (the tempo of this track in beats per minute)
category (the category of the track)
comment (freeform notes about the track)
compilation (is this track from a compilation album?)
composer (the composer of the track)
databaseID (the common, unique ID for this track. If two tracks in different playlists have the same database ID, they are sharing the same data.)
dateAdded (the date the track was added to the playlist)
objectDescription (the description of the track)
discCount (the total number of discs in the source album)
discNumber (the index of the disc containing this track on the source album)
duration (the length of the track in seconds)
enabled (is this track checked for playback?)
episodeID (the episode ID of the track)
episodeNumber (the episode number of the track)
EQ (the name of the EQ preset of the track)
finish (the stop time of the track in seconds)
gapless (is this track from a gapless album?)
genre (the music/audio genre (category) of the track)
grouping (the grouping (piece) of the track. Generally used to denote movements within a classical work.)
kind (a text description of the track)
longDescription ()
lyrics (the lyrics of the track)
modificationDate (the modification date of the content of this track)
playedCount (number of times this track has been played)
playedDate (the date and time this track was last played)
podcast (is this track a podcast episode?)
rating (the rating of this track (0 to 100))
ratingKind (the rating kind of this track)
releaseDate (the release date of this track)
sampleRate (the sample rate of the track (in Hz))
seasonNumber (the season number of the track)
shufflable (is this track included when shuffling?)
skippedCount (number of times this track has been skipped)
skippedDate (the date and time this track was last skipped)
show (the show name of the track)
sortAlbum (override string to use for the track when sorting by album)
sortArtist (override string to use for the track when sorting by artist)
sortAlbumArtist (override string to use for the track when sorting by album artist)
sortName (override string to use for the track when sorting by name)
sortComposer (override string to use for the track when sorting by composer)
sortShow (override string to use for the track when sorting by show name)
size (the size of the track (in bytes))
start (the start time of the track in seconds)
time (the length of the track in MM:SS format)
trackCount (the total number of tracks on the source album)
trackNumber (the index of the track on the source album)
unplayed (is this track unplayed?)
videoKind (kind of video track)
volumeAdjustment (relative volume adjustment of the track (-100% to 100%))
year (the year the track was recorded/released)
 
Method: artworks
Returned: SBElementArray
----
 
Class: iTunesFileTrack
Properties:
location (the location of the file represented by this track)
 
Method: refresh
Returned: void
update file track information from the current information in the track’s file
----
 
Class: iTunesURLTrack
Properties:
address (the URL for this track)
 
Method: download
Returned: void
download podcast episode
----
 
Class: iTunesUserPlaylist
Properties:
shared (is this playlist shared?)
smart (is this a Smart Playlist?)
 
Method: fileTracks
Returned: SBElementArray
----
Method: URLTracks
Returned: SBElementArray
----
Method: sharedTracks
Returned: SBElementArray
----

jQuery 1.4 and Malformed JSON

2010-01-15T09:40:17+00:00

Today, we released jQuery 1.4, a mostly backward compatible release with a few minor quirks. One of these quirks is that jQuery now uses the native JSON parser where available, rejecting malformed JSON.

This probably sounds fine, except that by malformed JSON, we mostly mean:

{foo: "bar"}

Because this is valid JavaScript, people have long assumed it was valid JSON, and a lot of hand-coded JSON is written like this. Thankfully, JSON automatically generated by Rails does not have this problem, so most of you guys should not have an issue here.

If you have server-side JSON that can’t be quickly fixed in the transition to jQuery 1.4, you can use the compatibility plugin, which tries to patch a number of small incompatibilities, or you can use this workaround:

$.ajax({url: "/url", 
  dataType: "json",
  success: function(json) {
    // do something with json
  }
});
 
// becomes
 
$.ajax({url: "/url",
  dataType: "text",
  success: function(text) {
    json = eval("(" + text + ")");
    // do something with JSON
  }
});

Essentially, this is what jQuery used to do to evaluate JSON, so with these small changes you can get the old behavior.

Again, though, you should probably be modifying your busted server-side code to stop sending malformed JSON!

ActiveModel: Make Any Ruby Object Feel Like ActiveRecord

2010-01-11T05:56:58+00:00

Rails 2.3 has a ton of really nice functionality locked up in monolithic components. I’ve posted quite a bit about how we’ve opened up a lot of that functionality in ActionPack, making it easier to reuse the router, dispatcher, and individual parts of ActionController. ActiveModel is another way we’ve exposed useful functionality to you in Rails 3.

Before I Begin, The ActiveModel API

Before I begin, there are two major elements to ActiveModel. The first is the ActiveModel API, the interface that models must adhere to in order to gain compatibility with ActionPack’s helpers. I’ll be talking more about that soon, but for now, the important thing about the ActiveModel API is that your models can become ActiveModel compliant without using a single line of Rails code.

In order to help you ensure that your models are compliant, ActiveModel comes with a module called ActiveModel::Lint that you can include into your test cases to test compliance with the API:

class LintTest < ActiveModel::TestCase
  include ActiveModel::Lint::Tests
 
  class CompliantModel
    extend ActiveModel::Naming
 
    def to_model
      self
    end
 
    def valid?()      true end
    def new_record?() true end
    def destroyed?()  true end
 
    def errors
      obj = Object.new
      def obj.[](key)         [] end
      def obj.full_messages() [] end
      obj
    end
  end
 
  def setup
    @model = CompliantModel.new
  end
end

The ActiveModel::Lint::Tests provide a series of tests that are run against the @model, testing for compliance.

ActiveModel Modules

The second interesting part of ActiveModel is a series of modules provided by ActiveModel that you can use to implement common model functionality on your own Ruby objects. These modules were extracted from ActiveRecord, and are now included in ActiveRecord.

Because we’re dogfooding these modules, you can be assured that APIs you bring in to your models will remain consistent with ActiveRecord, and that they’ll continue to be maintained in future releases of Rails.

The ActiveModel comes with internationalization baked in, providing an avenue for much better community sharing around translating error messages and the like.

The Validations System

This was perhaps the most frustrating coupling in ActiveRecord, because it meant that people writing libraries for, say, CouchDB had to choose between painstakingly copying the API over, allowing inconsistencies to creep in, or just inventing a whole new API.

Validations have a few different elements.

First, declaring the validations themselves. You’ve seen the usage before in ActiveRecord:

class Person < ActiveRecord::Base
  validates_presence_of :first_name, :last_name
end

To do the same thing for a plain old Ruby object, simply do the following:

class Person
  include ActiveModel::Validations
 
  validates_presence_of :first_name, :last_name
 
  attr_accessor :first_name, :last_name
  def initialize(first_name, last_name)
    @first_name, @last_name = first_name, last_name
  end
end

The validations system calls read_attribute_for_validation to get the attribute, but by default, it aliases that method to send, which supports the standard Ruby attribute system of attr_accessor.

To use a more custom attribute lookup, you can do:

class Person
  include ActiveModel::Validations
 
  validates_presence_of :first_name, :last_name
 
  def initialize(attributes = {})
    @attributes = attributes
  end
 
  def read_attribute_for_validation(key)
    @attributes[key]
  end
end

Let’s look at what a validator actually is. First of all, the validates_presence_of

method:

def validates_presence_of(*attr_names)
  validates_with PresenceValidator, _merge_attributes(attr_names)
end

You can see that validates_presence_of is using the more primitive validates_with, passing it the validator class, merging in {:attributes => attribute_names} into the options passed to the validator. Next, the validator itself:

class PresenceValidator < EachValidator
  def validate(record)
    record.errors.add_on_blank(attributes, options[:message])
  end
end

The EachValidator that it inherits from validates each attribute with the validate method. In this case, it adds the error message to the record, only if the attribute is blank.

The add_on_blank method does add(attribute, :blank, :default => custom_message) if value.blank? (among other things), which is adding the localized :blank message to the object. If you take a look at the built-in locale/en.yml looks like:

en:
  errors:
    # The default format use in full error messages.
    format: "{{attribute}} {{message}}"
 
    # The values :model, :attribute and :value are always available for interpolation
    # The value :count is available when applicable. Can be used for pluralization.
    messages:
      inclusion: "is not included in the list"
      exclusion: "is reserved"
      invalid: "is invalid"
      confirmation: "doesn't match confirmation"
      accepted: "must be accepted"
      empty: "can't be empty"
      blank: "can't be blank"
      too_long: "is too long (maximum is {{count}} characters)"
      too_short: "is too short (minimum is {{count}} characters)"
      wrong_length: "is the wrong length (should be {{count}} characters)"
      not_a_number: "is not a number"
      greater_than: "must be greater than {{count}}"
      greater_than_or_equal_to: "must be greater than or equal to {{count}}"
      equal_to: "must be equal to {{count}}"
      less_than: "must be less than {{count}}"
      less_than_or_equal_to: "must be less than or equal to {{count}}"
      odd: "must be odd"
      even: "must be even"

As a result, the error message will read first_name can't be blank.

The Error object is also a part of ActiveModel.

Serialization

ActiveRecord also comes with default serialization for JSON and XML, allowing you to do things like: @person.to_json(:except => :comment).

The main important part of the serialization support is adding general support for specifying the attributes to include across all serializers. That means that you can do @person.to_xml(:except => :comment) as well.

To add serialization support to your own model, you will need to include the serialization module and implement attributes. Check it out:

class Person
  include ActiveModel::Serialization
 
  attr_accessor :attributes
  def initialize(attributes)
    @attributes = attributes
  end
end
 
p = Person.new(:first_name => "Yukihiro", :last_name => "Matsumoto")
p.to_json #=> %|{"first_name": "Yukihiro", "last_name": "Matsumoto"}|
p.to_json(:only => :first_name) #=> %|{"first_name": "Yukihiro"}|

You can also pass in a :methods option to specify methods to call for certain attributes that are determined dynamically.

Here's the Person model with validations and serialization:

class Person
  include ActiveModel::Validations
  include ActiveModel::Serialization
 
  validates_presence_of :first_name, :last_name
 
  attr_accessor :attributes
  def initialize(attributes = {})
    @attributes = attributes
  end
 
  def read_attribute_for_validation(key)
    @attributes[key]
  end
end

Others

Those are just two of the modules available in ActiveModel. Some others include:

AttributeMethods: Makes it easy to add attributes that are set like table_name :foo
Callbacks: ActiveRecord-style lifecycle callbacks.
Dirty: Support for dirty tracking
Naming: Default implementations of model.model_name, which are used by ActionPack (for instance, when you do render :partial => model
Observing: ActiveRecord-style observers
StateMachine: A simple state-machine implementation for models
Translation: The core translation support

This mostly reflects the first step of ActiveRecord extractions done by Josh Peek for his Google Summer of Code project last summer. Over time, I expect to see more extractions from ActiveRecord and more abstractions built up around ActiveModel.

I also expect to see a community building up around things like adding new validators, translations, serializers and more, especially now that they can be reused not only in ActiveRecord, but in MongoMapper, Cassandra Object, and other ORMs that leverage ActiveModel's built-in modules.

Amazon Kindle 2

2010-01-06T01:36:12+00:00

For Noël, my wife got me a new toy: the Amazon Kindle 2.

Kindle vs Nook

Before my wife bought what became my new favorite gadget, we checked on the Barnes & Noble Nook.

The primary problem with the Nook was that it was out of stock. That pushed me to dig deeper and really compare both devices.

The Nook had some interesting features like wifi, small color screen and the book lending feature.

However, I don’t really care about a small, clunky extra color screen since I just want to read books. The wifi is nice… but what am I going to do with it? Lending books is a cool feature, but this feature isn’t available for all books and is apparently seriously limited.

So I decided to focus on what I really needed instead of what I could use.

The main purpose of an e-book is reading, page loading should be fast and the user experience should be at least as good as when reading a ‘real’ book.
I need to be able to have a great choice of books and if possible in multiple languages.
I would like to be able to read the safarionline books that I can access online.

From what I read almost everywhere, the nook is slow and a bit buggy (it’s a first generation after all) and doesn’t come with a web browser. I also checked on the ebooks available on Amazon and Barnes & Noble. Amazon has way more ebooks and that’s really what motivated my decision. The other thing is that Amazon is present world wide and I expect them to start selling more ebooks in French and Spanish.

The other thing that pushed me to buy a Kindle was the fact that I know so many people having one and being so pleased. During a recent trip to San Diego, Yehuda Katz told me about how much he loves his Kindle and he recommended a book to read. I started reading the book on my iPhone (using kindle for iPhone) and was really wanted a bigger screen.

Kindle vs iTablet

A lot of people asked me why I didn’t wait for the big announcement of the iTablet because it might be a Kindle killer.

First, the iTablet is currently just a rumor and if we would believe the rumor, the iTablet will sell around $1,000 vs $250 for a kindle. The tablet will probably do a lot of cool stuff my e-reader can’t do… but I don’t care I chose the kindle because I wanted a simple device to read book and only do that. Actually, the kindle + iphone combo works great for me. I used to read a lot but because I have been moving so many times I started reading less because I always get rid of my books. Also, when traveling a lot, carrying your books with you is a pain. (especially now that most airlines charge you per bag)

Kindle and tech books/documentation.

If you want a kindle to read tech books, don’t get a 6″, get the kindle DX instead. Even though the 6″ natively supports PDF files, the text is usually too small and the Kindle doesn’t support zooming (yet).

The DX is perfect to read PDFs because the ratio matches the print expectations. I was thinking about converting all the Cocoa docs to a kindle friendly format since the kindle supports search etc.. But the truth is that it’s not worth it. The display on a 6″ screen is clunky, it just doesn’t feel right. I didn’t try any of the pragprog ebooks but something tells me they will just look better on the DX.

Since I wanted a kindle to read books and not browse tech documentation, I’m still really happy with 6″.

If you also have kindle or are planning on buying one soon, here are two books I’ve been enjoying reading:

Flashforward by Robert J. Sawyer

The Talent Code by Daniel Coyle

The Maximal Usage Doctrine for Open Source

2010-01-05T23:36:36+00:00

I’ve worked on a number of open source projects over the past several years (the most prominent being Merb, Ruby on Rails and jQuery) and have begun to form some thoughts about the usage (aka adoption) of open source projects and the practical effects of license styles.

The Playing Field

There are essentially two kinds of licenses popularly used in open source[1].

The type I’ve worked with most extensively in the BSD or MIT-style license. This license allows unlimited usage, modification, distribution, and commercialization of the source code, with two caveats. First, the copyright notice must be distributed with the source code. Second, to the extent legally enforceable, if you use my MIT-licensed source code, you may not sue me for the effects of using the source.

The other major license type is called copyleft, and essentially leverages copyright law to insist that any modifications to the original source must themselves be made public. The practical effect of this license could not be effected without government-enforced copyright laws.

The most popular copyleft license is GPL, which is used by Linux, and which also has viral characteristics. In short, if you link code covered by the GPL into your own code, you are required to open your code as well (if you distribute it). The GPL is not very clear about what this means for more dynamic languages (is requiring a Ruby file “linking”?).

A less viral version of the GPL, the LGPL, requires that any modifications you make to the software itself be released, but drops the “linking” requirement.

Because of the uncertainty surrounding the viral characteristics of the GPL, legal departments in big corporations are very hostile to the GPL license. Software covered by the LGPL license is more likely to get approval by corporate legal, and MIT/BSD licenses are the most well-liked (probably because they don’t confer any obligations on the corporation).

What I Want When I Write Open Source

When I work on a serious open source project, like Ruby on Rails or jQuery, I have relatively simple desires.

One. I want as many people as possible to use the software I am working on. This is probably a selfish, egoist desire, but it may also be a desire to see useful software actually used.

Two. I want some subset of the many users to be willing to report bugs, and for the varied situations the code is used in to help improve the quality of the software as a result. This is essentially “given enough eyeballs, all bugs are shallow”.

Three. I want people who are interested in improving the software to submit patches. Specifically, I want to receive patches from people who have thought through the problem the bug is trying to fix, and want to receive fewer patches from people who have hacked together a solution just to get it to work. In essence, a high signal-to-noise ratio of patches is more important than a high volume of patches.

Four. I want people who are interested in improving the software long term to become long-term contributors and eventually committers.

Of these concerns, numbers two and three are the highest priorities. I want to expose the code to as much real-world pressure as possible, and I want as many high-quality patches to fix those bugs as possible. If I had to choose between those two, I would pick number two: exposing my code to real-world stress is the best way to rapidly ferret out bugs and incorrect design assumptions that I know of.

Meeting Those Requirements

Starting out with the easiest, my first desire, to have my software used as much as possible, is most easily satisfied by an extremely liberal usage policy. Adding restrictions on the use of software I write reduces its adoption almost by definition.

Much more importantly, the same can be said about exposing code to real world stresses. By far the most important way to achieve this goal is to make it as easy as possible for as many people as possible to use the code.

If only 1% of all proprietary users of the source ever report bugs, that’s 1% of potentially thousands of users, as opposed to 100% of the zero proprietary users who were able to use the software under a more restrictive usage scheme. In practice, this number is much more than 1%, as proprietary users of software experience and report bugs just like open source users do.

The only real counter-argument to this is that by forcing users to contribute, some number of proprietary users will be forced to become open source users, and their contributions will outweigh the smaller contributions of proprietary users. In practice, proprietary users choose proprietary solutions instead when they are forced to choose between restrictive open source usage schemes and other proprietary software.

There is also much to be said for exposing open source tools into proprietary environments.

There is also much to be said for introducing proprietary developers to the open source ecosystem. Proprietary developers have access to things like paid time, access to obscure usage scenarios, and access to markets with low open source penetration that the open source community lacks.

The next major desire I have while working on open source is a steady stream of high-quality patches. This should be advantage copyleft, because all users of the software are forced to contribute back. However, since copyleft licenses are not used in proprietary environments anyway, the patches to open source projects from those environments under more permissive licenses are much more numerous. Again, even if only a few percent of proprietary users contribute back to the project, that is significantly more contributions than the 100% of zero proprietary users.

Also importantly, the patches are contributed by much more dedicated users of the software, instead of being force-contributions. I have never heard a team member on an open source project say that inadequate patches are received by permissive-license software, and that this problem would be solved by going to a more restrictive model.

Finally, I can only speak for jQuery and Rails (and other smaller open source projects I work on), but a large number of new long-term contributors became involved while working on corporate projects, where a permissive license made the decision to use the software in the first place feasible.

Lower Barrier to Entry Helps Meet Open Source Goals

Regardless of how much of the above argument you agree with, it is clear that copyleft licenses intentionally impose a higher barrier to entry for usage than more permissive licenses.

For projects that use more permissive licenses, the fact that many proprietary users of their software do not contribute back changes is a feature, not a bug.

That’s because we don’t focus on all of the people who use our software without contributing back. Instead, we focus on all the users, bug reports, patches, and long term contributors we have gained by keeping the barrier to entry as low as possible.

In the end, the world of ubiquitous open source is here. And we made it without having to resort to coercion or the formation of an entirely self-contained set of software. We made it by building great software and convincing real users that it was better than the alternative.

Postscript: Linux

Linux is a peculiar example because its license has not impeded its usage much. In part, that is because most users of Linux do not make changes to it, and the linking characteristics of the GPL license rarely come into play.

In cases like this, I would argue that copyleft licenses are close enough to usage maximization to get most of the benefits of more permissive licenses. However, it’s not clear to me what benefits the copyleft licenses provide in those (relatively rare) cases.

[1] There are other kinds of licenses, like the Affero license, which is even more restrictive than the GPL license. I would classify Affero as copyleft.

The Craziest F***ing Bug I’ve Ever Seen

2010-01-03T01:58:33+00:00

This afternoon, I was telling a friend about one of my exploits tracking down a pretty crazy heisenbug, and he said he thought other people would be interested in hearing about it. So let me tell you about it.

Before you continue, if you’re not interested in relatively arcane technical details, feel free to skip this post. It’s here mainly because a friend said he thought people would be interested in it.

Our Story Begins

Our story begins with a small change in the way Ruby 1.9 handles implicit coercion. The most common case of implicit coercion is in Array#flatten. The basic logic is that Ruby checks to see if an element of the Array can be itself coerced into an Array, which is then does before flattening.

There are two steps to the process:

Check to see if the element can be coerced into an Array
If it can, call to_ary on the element, and repeat the process recursively

In Ruby 1.8, the process is essentially the following:

if obj.respond_to?(:to_ary)
  obj.__send__(:to_ary)
else
  obj
end

In Ruby 1.9, it was changed to:

begin
  obj.__send__(:to_ary)
rescue NoMethodError
  obj
end

Of course, the internal code was implemented in C, but you get the idea. This change subtly effects objects that implement method_missing:

class MyObject
  def method_missing(meth, *)
    if meth == :testing
      puts "TESTING"
    else
      puts "Calling Super"
      super
    end
  end
end

In Ruby 1.8, #flatten will first call #respond_to? which returns false and therefore doesn’t ever trigger the #method_missing. In Ruby 1.9, #method_missing will be triggered blindly, and because the call to super should raise a NoMethodError, we will essentially get the same result.

There are some subtle differences here, but for the vast majority of cases, the behavior is identical or close enough to not matter.

A Weird Quirk

When the above change landed in Ruby 1.9.2’s head, Rails started experiencing weird, intermittent behavior. As you might know, Rails overrides method_missing on NilClass to provide a guess about what object you were expecting instead of nil. This feature is called “whiny nils” and can be enabled or disabled in Rails applications.

def method_missing(method, *args, &block)
  if klass = METHOD_CLASS_MAP[method]
    raise_nil_warning_for klass, method, caller
  else
    super
  end
end

But sometimes, when nil was inside an Array, and the Array was flattened, the flatten failed with a NameError: undefined local variable or method `to_ary' for nil:NilClass, which resulted in a failure in the flatten entirely.

The bug appeared intermittently, apparently unrelated to the code that was calling it. We worked around it by catching the NameError and reraising a NoMethodError, but the existence of the bug was rather baffling.

The Bug Reappears

When working on bundler, Carl and I saw the bug again. This time, we didn’t want to let it go. In this case, we had four virtually identical tests with four identical stack traces leading to NoMethodError or NameError seemingly at random. Something didn’t add up.

After hunting bugs for a few hours (having dug deeply into Ruby’s source), I brought in Evan Phoenix (of Rubinius). At first, Evan was baffled as well, but he correctly pointed out that the key to understanding what was going on was the difference between NameError and NoMethodError in Ruby.

x = Object.new
x.no_method #=> NoMethodError
 
class Testing
  def vcall_no_method
    no_method #=> NameError
  end
 
  def call_no_method
    self.no_method #=> NoMethodError
  end
end

In essence, when Ruby sees a method call with implicit self (which might be a typo’ed local variable), it raises a NameError, rather than a NoMethodError.

However, it still didn’t add up, because the call to to_ary was internal, and shouldn’t have been randomly interpreted as a vcall (a call to a method with implicit self) or a normal call.

Tracking it Down

Evan dug deeply into the Ruby source, and found how Ruby was determining whether the call was a vcall or a regular call.

static inline VALUE
method_missing(VALUE obj, ID id, int argc, const VALUE *argv, int call_status)
{
    VALUE *nargv, result, argv_ary = 0;
    rb_thread_t *th = GET_THREAD();
 
    th->method_missing_reason = call_status;
    th->passed_block = 0;
    // ...
}

Before calling the method_missing method itself, Ruby sets a thread-local variable called call_status that reflects whether or not the original call was a vcall or a normal call.

Upon further examination, Evan discovered that the call to method_missing in the case of type coercion did not change the thread-local method_missing_reason.

As a result, Ruby raised a NoMethodError or NameError based upon whether the last call to method_missing was a vcall or regular call.

The fix was a simple one-line patch, which allowed us to remove the workaround in ActiveSupport, but it was by far the craziest f***ing bug I’ve ever seen

Spinning up a new Rails app

2010-01-01T03:06:52+00:00

So people have been attempting to get a Rails app up and running recently. I also have some apps in development on Rails 3, so I’ve been experiencing some of the same problems many others have.

The other night, I worked with sferik to start porting merb-admin over to Rails. Because this process involved being on edge Rails, we got the process honed to a very simple, small, repeatable process.

The Steps

Step 1: Check out Rails

$ git clone git://github.com/rails/rails.git

Step 2: Generate a new app

$ ruby rails/railties/bin/rails new_app
$ cd new_app

Step 3: Edit the app’s Gemfile

# Add to the top
directory "/path/to/rails", :glob => "{*/,}*.gemspec"
git "git://github.com/rails/arel.git"
git "git://github.com/rails/rack.git"

Step 4: Bundle

$ gem bundle

Done

Everything should now work: script/server, script/console, etc.

If you want to check your copy of Rails into your app, you can copy it into the app and then change your Gemfile to point to the relative location.

For instance, if you copy it into vendor/rails, you can make the first line of the Gemfile directory "vendor/rails", :glob => => "{*/,}*.gemspec". You’ll want to run gem bundle again after changing the Gemfile, of course.

IM new year countdown with MacRuby

2009-12-31T18:25:50+00:00

Here is the geekiest way I found to wish Happy New Year to my IM contacts:

framework 'ScriptingBridge'
app = SBApplication.applicationWithBundleIdentifier("com.apple.iChat")
original_status = app.statusMessage
new_year = Time.mktime(2010, 1, 1, 0, 0)
 
loop do
  now = Time.now
  time_left = (new_year - now).ceil
  if time_left > 0
    app.statusMessage = "#{time_left} seconds left until 2010 (EST)"
  else
    app.statusMessage = "Happy New Year 2010!"
    exit
  end
  sleep(1)
end

If you are alone at home playing WOW, you can also trigger iTunes to play a mp3 file with crowd noise and people shouting ‘Happy New Year 2010‘!

Fun with MacRuby

2009-12-30T03:09:45+00:00

To be ready for 2010, I’m taking some time off relaxing and spending time with my family in Florida.

During my free time, I’ve been reading, catching up on movies and TV shows and worked on the MacRuby book that I am writing for O’Reilly.

I wrote a bunch of small apps, played with various APIs and every single time I was amazed by all the goodies Apple makes available to developers. My most recent discovery is very simple but I wanted to share it with you.

I often type text in English, French and Spanish and I even mix the languages from time to time. SnowLeopard comes with a great spellchecker that auto detects the language I’m typing in and is most of the time correct. It’s a very impressive feature and I was wondering if, as a MacRuby developer, I could use one of Apple’s lib to detect what language is being used. I dug through the documentation but didn’t find anything. I started looking at some header files and found the API to use

framework 'Foundation'
class String
  def language
    CFStringTokenizerCopyBestStringLanguage(self, CFRangeMake(0, self.size))
  end
end
 
puts "Bonne année!".language
# =&gt; "fr"
puts "Happy new year!".language
# =&gt; "en"
puts "¡Feliz año nuevo!".language
# =&gt; "es"
puts "Felice anno nuovo!".language
# =&gt; "it"
puts "أعياد سعيدة".language
# =&gt; "ar"
puts "明けましておめでとうございます。".language
# =&gt; "ja"

The documentation says that the result is not guaranteed to be accurate and that typically 200-400 characters are required to reliably guess the language of a string. (CFStringTokenizer Doc)

Probably not the most useful piece of code, but really cool none the less

Happy new year!

The Rails 3 Router: Rack it Up

2009-12-26T06:51:15+00:00

In my previous post about generic actions in Rails 3, I made reference to significant improvements in the router. Some of those have been covered on other blogs, but the full scope of the improvements hasn’t yet been covered.

In this post, I’ll cover a number of the larger design decisions, as well as specific improvements that have been made. Most of these features were in the Merb router, but the Rails DSL is more fully developed, and the fuller emphasis on Rack is a strong improvement from the Merb approach.

Improved DSL

While the old map.connect DSL still works just fine, the new standard DSL is less verbose and more readable.

# old way
ActionController::Routing::Routes.draw do |map|
  map.connect "/main/:id", :controller => "main", :action => "home"
end
 
# new way
Basecamp::Application.routes do
  match "/main/:id", :to => "main#home"
end

First, the routes are attached to your application, which is now its own object and used throughout Railties. Second, we no longer need map, and the new DSL (match/to) is more expressive. Finally, we have a shortcut for controller/action pairs ("main#home" is {:controller => "main", :action => "home").

Another useful shortcut allows you to specify the method more simply than before:

Basecamp::Application.routes do
  post "/main/:id", :to => "main#home", :as => :homepage
end

The :as in the above example specifies a named route, and creates the homepage_url et al helpers as in Rails 2.

Rack It Up

When designing the new router, we all agreed that it should be built first as a standalone piece of functionality, with Rails sugar added on top. As a result, we used rack-mount, which was built by Josh Peek as a standalone Rack router.

Internally, the router simply matches requests to a rack endpoint, and knows nothing about controllers or controller semantics. Essentially, the router is designed to work like this:

Basecamp::Application.routes do
  match "/home", :to => HomeApp
end

This will match requests with the /home path, and dispatches them to a valid Rack application at HomeApp. This means that dispatching to a Sinatra app is trivial:

class HomeApp < Sinatra::Base
  get "/" do
    "Hello World!"
  end
end
 
Basecamp::Application.routes do
  match "/home", :to => HomeApp
end

The one small piece of the puzzle that might have you wondering at this point is that in the previous section, I showed the usage of :to => "main#home", and now I say that :to takes a Rack application.

Another improvement in Rails 3 bridges this gap. In Rails 3, PostsController.action(:index) returns a fully valid Rack application pointing at the index action of PostsController. So main#home is simply a shortcut for MainController.action(:home), and it otherwise is identical to providing a Sinatra application.

As I posted before, this is also the engine behind match "/foo", :to => redirect("/bar").

Expanded Constraints

Probably the most common desired improvement to the Rails 2 router has been support for routing based on subdomains. There is currently a plugin called subdomain_routes that implements this functionality as follows:

ActionController::Routing::Routes.draw do |map|
  map.subdomain :support do |support|
    support.resources :tickets
    ...
  end
end

This solves the most common case, but the reality is that this is just one common case. In truth, it should be possible to constrain routes based not just on path segments, method, and subdomain, but also based on any element of the request.

The Rails 3 router exposes this functionality. Here is how you would constrain requests based on subdomains in Rails 3:

Basecamp::Application.routes do
  match "/foo/bar", :to => "foo#bar", :constraints => {:subdomain => "support"}
end

These constraints can include path segments as well as any method on ActionDispatch::Request. You could use a String or a regular expression, so :constraints => {:subdomain => /support\d/} would be valid as well.

Arbitrary constraints can also be specified in block form, as follows:

Basecamp::Application.routes do
  constraints(:subdomain => "support") do
    match "/foo/bar", :to => "foo#bar"
  end
end

Finally, constraints can be specified as objects:

class SupportSubdomain
  def self.matches?(request)
    request.subdomain == "support"
  end
end
 
Basecamp::Application.routes do
  constraints(SupportSubdomain) do
    match "/foo/bar", :to => "foo#bar"
  end
end

Optional Segments

In Rails 2.3 and earlier, there were some optional segments. Unfortunately, they were hardcoded names and not controllable. Since we’re using a generic router, magical optional segment names and semantics would not do. And having exposed support for optional segments in Merb was pretty nice. So we added them.

# Rails 2.3
ActionController::Routing::Routes.draw do |map|
  # Note that :action and :id are optional, and
  # :format is implicit
  map.connect "/:controller/:action/:id"
end
 
# Rails 3
Basecamp::Application.routes do
  # equivalent
  match "/:controller(/:action(/:id))(.:format)"
end

In Rails 3, we can be explicit about the optional segments, and even nest optional segments. If we want the format to be a prefix path, we can do match "(/:format)/home" and the format is optional. We can use a similar technique to add an optional company ID prefix or a locale.

Pervasive Blocks

You may have noticed this already, but as a general rule, if you can specify something as an inline condition, you can also specify it as a block constraint.

Basecamp::Application.routes do
  controller :home do
    match "/:action"
  end
end

In the above example, we are not required to specify the controller inline, because we specified it via a block. You can use this for subdomains, controller restrictions, request method (get etc. take a block). There is also a scope method that can be used to scope a block of routes under a top-level path:

Basecamp::Application.routes do
  scope "/home" do
    match "/:action", :to => "homepage"
  end
end

The above route would match /home/hello/foo to homepage#foo.

Closing

There are additional (substantial) improvements around resources, which I will save for another time, assuming someone else doesn’t get to it first.

Generic Actions in Rails 3

2009-12-21T05:14:18+00:00

So Django has an interesting feature called “generic views”, which essentially allow you to to render a template with generic code. In Rails, the same feature would be called “generic actions” (just a terminology difference).

This was possible, but somewhat difficult in Rails 2.x, but it’s a breeze in Rails 3.

Let’s take a look at a simple generic view in Django, the “redirect_to” view:

urlpatterns = patterns('django.views.generic.simple',
    ('^foo/(?P<id>\d+)/$', 'redirect_to', {'url': '/bar/%(id)s/'}),
)

This essentially redirects "/foo/<id>" to "/bar/<id>s/". In Rails 2.3, a way to achieve equivalent behavior was to create a generic controller that handled this:

class GenericController < ApplicationController
  def redirect
    redirect_to(params[:url] % params, params[:options])
  end
end

And then you could use this in your router:

map.connect "/foo/:id", :controller => "generic", :action => "redirect", :url => "/bar/%{id}s"

This uses the new Ruby 1.9 interpolation syntax (“%{first} %{last}” % {:foo => “hello”, :bar => “sir”} == “hello sir”) that has been backported to Ruby 1.8 via ActiveSupport.

Better With Rails 3

However, this is a bit clumsy, and requires us to have a special controller to handle this (relatively simple) case. It also saddles us with the conceptual overhead of a controller in the router itself.

Here’s how you do the same thing in Rails 3:

match "/foo/:id", :to => redirect("/bar/%{id}s")

This is built-into Rails 3’s router, but the way it works is actually pretty cool. The Rails 3 router is conceptually decoupled from Rails itself, and the :to key points at a Rack endpoint. For instance, the following would be a valid route in Rails 3:

match "/foo", :to => proc {|env| [200, {}, ["Hello world"]] }

The redirect method simply returns a rack endpoint that knows how to handle the redirection:

def redirect(*args, &block)
  options = args.last.is_a?(Hash) ? args.pop : {}
 
  path = args.shift || block
  path_proc = path.is_a?(Proc) ? path : proc {|params| path % params }
  status = options[:status] || 301
 
  lambda do |env|
    req = Rack::Request.new(env)
    params = path_proc.call(env["action_dispatch.request.path_parameters"])
    url = req.scheme + '://' + req.host + params
    [status, {'Location' => url, 'Content-Type' => 'text/html'}, ['Moved Permanently']]
  end
end

There’s a few things going on here, but the important part is the last few lines, where the redirect method returns a valid Rack endpoint. If you look closely at the code, you can see that the following would be valid as well:

match "/api/v1/:api", :to => 
  redirect {|params| "/api/v2/#{params[:api].pluralize}" }
 
# and
 
match "/api/v1/:api", :to => 
  redirect(:status => 302) {|params| "/api/v2/#{params[:api].pluralize}" }

Another Generic Action

Another nice generic action that Django provides is allowing you to render a template directly without needing an explicit action. It looks like this:

urlpatterns = patterns('django.views.generic.simple',
    (r'^foo/$',             'direct_to_template', {'template': 'foo_index.html'}),
    (r'^foo/(?P<id>\d+)/$', 'direct_to_template', {'template': 'foo_detail.html'}),
)

This provides a special mechanism for rendering a template directly from the Django router. Again, this could be implemented by creating a special controller in Rails 2 and used as follows:

class GenericController < ApplicationController
  def direct_to_template
    render(params[:options])
  end
end
 
# Router
map.connect "/foo", :controller => "generic", :action => "direct_to_template", :options => {:template => "foo_detail"}

A Prettier API

A nicer way to do this would be something like this:

match "/foo", :to => render("foo")

For the sake of clarity, let’s say that directly rendered templates will come out of app/views/direct unless otherwise specified. Also, let’s say that the render method should work identically to the render method used in Rails controllers themselves, so that render :template => "foo", :status => 201, :content_type => Mime::JSON et al will work as expected.

In order to make this work, we’ll use ActionController::Metal, which exposes a Rack-compatible object with access to all of the powers of a full ActionController::Base object.

class RenderDirectly < ActionController::Metal
  include ActionController::Rendering
  include ActionController::Layouts
 
  append_view_path Rails.root.join("app", "views", "direct")
  append_view_path Rails.root.join("app", "views")
 
  layout "application"
 
  def index
    render *env["generic_views.render_args"]
  end
end
 
module GenericActions
  module Render
    def render(*args)
      app = RenderDirectly.action(:index)
      lambda do |env|
        env["generic_views.render_args"] = args
        app.call(env)
      end
    end
  end
end

The trick here is that we’re subclassing ActionController::Metal and pulling in just Rendering and Layouts, which gives you full access to the normal rendering API without any of the other overhead of normal controllers. We add both the direct directory and the normal view directory to the view path, which means that any templates you place inside app/views/direct will take be used first, but it’ll fall back to the normal view directory for layouts or partials. We also specify that the layout is application, which is not the default in Rails 3 in this case since our metal controller does not inherit from ApplicationController.

Note for the Curious

In all normal application cases, Rails will look up the inheritance chain for a named layout matching the controller name. This means that the Rails 2 behavior, which allows you to provide a layout named after the controller, still works exactly the same as before, and that ApplicationController is just another controller name, and application.html.erb is its default layout.

And then, the actual use in your application:

Rails.application.routes do
  extend GenericActions
 
  match "/foo", :to => render("foo_index")
  # match "/foo" => render("foo_index") is a valid shortcut for the simple case
  match "/foo/:id", :constraints => {:id => /\d+/}, :to => render("foo_detail")
end

Of course, because we’re using a real controller shell, you’ll be able to use any other options available on the render (like :status, :content_type, :location, :action, :layout, etc.).

My Proposed Solution to the "Can't Comment On RTs" Problem: Retweet-Replies

2009-12-13T06:00:00+00:00

I love retweets. More importantly, I love the new, native Retweets. I know there are plenty of people still up in arms about them, or refusing to use them because they don’t like change, or for a whole bunch of reasons that I thought were silly at the time and didn’t remember long enough to refute them.

But that’s not what I’m writing about tonight.

There is one real deficiency with native Retweets. Ev even mentioned it in his blog post, which is excellent, and required reading before I’ll debate why native Retweets are awesome with you.

@ev:

The other thing some people will not like is that, unlike organic RTs, there’s no way to annotate or leave your own comment when you retweet something with the new system…we have some ideas there, and it’s possible we’ll build that in at a later date. (This point should not be missed.)

This is important, and it’s a big reason some people don’t want to use the native Retweets. Even @jack isn’t using them.

I have a solution. It’s a really simple solution, but unfortunately it requires a little bit of work from Twitter. I’d rather this were a completely user-behavior or client-side solution, but it is what it is.

If I retweet @someone, and then reply to the retweet, push my reply to all of my followers, even if they don’t follow @someone.

If Twitter made this simple change (to replies, you’ll notice, not to retweets), it would mean I could use an entire 140 characters to annotate my retweet, and that annotation doesn’t need Yet Another special case in Twitter clients, or an API change, or something new for users to learn.

It would also mean innovative clients could add a “RT/Reply” button that would mark a tweet for retweeting, get the reply, and then do the retweet and the reply back to back so they show up next to each other.

I call this idea “Retweet-Replies”, since, you know…that’s what they are.

What do you think?

CC: #twitterapi

Introducing Moralist

2009-12-12T06:00:00+00:00

I love Twitter lists. Really. I think they’re one of the most important features Twitter has rolled out in the past year, even more than geotweets.

First off, I think they’ll slowly be the end of Follow Friday. Why plug people one day a week when you can have a list of people you suggest all the time? Even better, you can have multiple lists, organized by topic.

More importantly, though, lists mean I can have multiple timelines. I have my main timeline of people I follow all the time. When they say something, I want to see it. (I haven’t completely cleaned up my following list yet, but I dropped more than 100 people when I started using lists.) There are a lot of the people who are funny, but I don’t need to see every tweet from them. They’re on my @ivey/funny-people list, and I read them when I have time.

The problem is, not only do I have my list of funny people, but I also follow a few other people’s lists of funny people, like @Favstar/rising-stars and @Favstar/top-50-funny. There’s a lot of overlap between these lists, which means I see the same tweets 2 or 3 times. That bugs me.

So I fixed it. First, I made a little library called, creatively, twitter-lists. It uses the twitter gem to do some set-based operations on Twitter lists, mainly Union and Intersection. It’s not fully fleshed out yet, but it’s usable. Patches welcome.

Then, I made an app that uses it: Moralist. Moralist makes it easy to combine lists in interesting ways. I’m using it to merge @Favstar/rising-stars, @Favstar/top-50-funny, and @ivey/funny-people into a single list, @ivey/hah. This is the list I read when I have a few minutes and need a chuckle (although Favstar’s Favit iPhone app is replacing it, but that’s another story.)

Anyway, try it out. It refreshes once a day, it crashes sometimes, and it comes with no warranty. If people find it useful, I might bump it up to a paid Heroku plan, but for now it meets my needs so I’m leaving it alone.

PS: here’s a snippet from Moralist to make a twitter-auth app able to use the twitter gem using the same set of keys: http://gist.github.com/255214

Thanks for the Retweets

2009-12-11T06:00:00+00:00

Lately I’ve been seeing a handful of tweets per day that say something like, “Thanks to @soandso, @friend, @someoneelse for the retweets today!” and … it doesn’t feel right, to me.

I get it. Twitter’s about social, and connections, and all that jazz. If someone retweets something you said, they’re lending you a little bit of their credibility. That’s a nice thing to do, and it’s worth thanking them for it, especially if it’s something you’d really like to get out there.

If it was a DM, or a reply to that one person, I wouldn’t notice. (I might not even see it.) Which is why people use the more public list of thank-yous, of course.

Somehow, though, it seems like it’s less about a genuine thank-you, and more of the self-promotion feedback loop we see over and over again on Twitter. What do you think? Am I being cynical and reading it wrong?

Lots of rubies, now what?

2009-11-30T21:38:53+00:00

If you were at RubyConf 2009 or looked at the schedule, you saw that the big thing happening in the Ruby scene is the maturation of a many of the Ruby implementations:
BlueRuby, HotRuby, IronRuby, JRuby, MacRuby, Maglev, MRI, REE, Rubinius, SmallRuby…
Ruby developers really have plenty of choice when it comes to choosing an alternative Ruby implementation.

Is that a bad thing?

Turns out, Matz , Ruby author mentioned one ore time during RubyConf that it was actually a good thing for the Ruby ecosystem.

But maybe we should take a step back and look at the big picture. So here are some of my thoughts on the topic. (disclaimer: being part of the MacRuby team, I’m certainly biased)

Stop comparing alternative Ruby implementations against Ruby 1.8.x.

Even though a lot of people are still using Ruby 1.8.x, it’s totally unfair to compare against such an old version of Matz’s Ruby.
Ruby 1.9 is what people should compare against and believe me, Ruby 1.9.2 is going to change the game (much faster, cleaned up version).
Hopefully, all the Ruby alternative implementations will quickly become 1.9 compatible.

(On a site note, Rails 2.3.5 was released a few days ago which fixes some of the last few remaining Ruby 1.9 bugs)

Don’t trust the benchmarks

As we all know, there are lies, damn lies and statistics. Evan Phoenix explained that very clearly with lots of examples during his RubyConf talk and I totally agree with him.
Micro benchmarks only help implementers know where they are standing and find performance issue.
Let’s take a very simple example: JRuby.
What you might not know is that most JRuby benchmarks are usually ‘warmed up’. Basically, to get the results you see in most reported benchmarks, you need to run the benchmark in a loop X amount of times and then run the benchmark. The reason for that is that JRuby uses HotSpot which needs to process the code to optimize the JITting and finally get your code to run faster.
The problem is that it is not a fair comparison. I was benchmarking Rails3 for instance and JRuby starts way slower than Ruby 1.8.6 and takes a little while to get as fast/faster than 1.9.1.
This is something you need to take in consideration since each deployment will slow down your app and you probably shouldn’t use JRuby if you are not using long running processes.
Another example would be MacRuby, even though the MacRuby project never released any benchmarks, it’s known to be a fast implementation.
Currently, the allocation of objects in MacRuby is actually still a bit costly and if you benchmark that, you can show that MacRuby is slow. Other operations are super optimized and both Rubinius and MacRuby will be way faster than anyone else. (check Phoenix’s talk at Ruby conf for more examples)
So, before choosing an implementation for performance reasons, benchmark against the code you are going to use.

There is only one real original Ruby

I think nobody will disagree with the fact that there is only one Ruby reference and that’s Matz.
Therefore is only one “real” Ruby and that is Matz’s. (Matz actually disagreed with me when I said that, since all the other implementations are also “real” but you get the point)
There is nothing wrong with Ruby’s alternatives but they stay alternatives to the original
MRI might not be the best option for every single case out there but it is and will stay the reference.

During RubyConf, I was joking with Matz and Koichi that maybe they should claim a Ruby tax to the Ruby implementations.
The reality is that Ruby is maintained by 5 to 10 contributors, most of them doing that on the side of their full time job.
Most of the other implementations have full time staff working hard on their implementations. That’s exciting but sad at the same time.

During the different talks given by the Ruby core team, it was clear that they really want to bridge the West/East gap and asked for help. It’s very honorable on their part and I hope we will see more contributions coming from outside Japan.

Finally, I think that even if people work hard to write the best Ruby implementation ever, let’s not forget to respect Matz’s project, team and achievements. People spent hours/weeks/months creating the Ruby language we love and even though there are some aspects that could be and are being improved, let’s not forget that when criticizing/comparing their work.

There is not only one way to do things

I don’t know where that’s coming from, but some people in our community seem to believe that there is only one way to do things and if you don’t do it their way, you do it wrong.
I personally have a hard time believing that and I even think it goes against what Ruby was designed for. (However, I must admit that sometimes I did/do think that some approaches were totally wrong :p)
It reminds me of people trying to convince you that God exists, other people trying to convince you that their religion is the only true one, or finally the other people trying to convince believers that they are wrong.
Maybe it is just that as humans we have a hard time dealing with people coming to a different conclusion than us and therefore we have to convince them that we are right and they are wrong.

The way I see it, for any new Ruby project, you should start be using with Ruby1.9. If it doesn’t fit your needs, then, consider an alternative. Here is how I look at the other alternatives (feel free to disagree):

BlueRuby – use if you are writing a SAP app
HotRuby – use if you want to use Ruby to write a Flash app
IronRuby – use it if you want to integrate with .NET and/or write a silverlight app
JRuby – use if you want to integrate with the Java world
MacRuby – use it if you want to integrate with Cocoa/want a native OSX application
Maglev – use if you want to use an object persistence layer and a distributed shared cache
REE – use if you have a ruby 1.8 web app that uses too much memory
Rubinius – use if you want to know what’s going on in your code
SmallRuby – don’t use it yet, but keep an eye on it

This is the way I see the various existing implementations and I think this is where they shine and why they were created in the first place. They are all complimentary and help bring the Ruby language further. The goal is always the same.

The future is exciting

Ruby is getting everywhere or almost. Matz’s Ruby is going to get multiVM support, various other improvements and get even faster and faster.
Alternative implementations are getting more and more mature and they grow the community by bringing people from different communities (.NET, java, obj-c/cocoa)

At the end of the day, we are all beneficiaries of the work done by all the implementers, thank you guys and keep up the good work!

The Ruby Revolution, take II

2009-11-16T16:05:55+00:00

Mohandas Karamchand Gandhi

My recent ‘Ruby revolution being over‘ blog post generated quite a lot of comments.
Let’s be honest, I did not expect less from the readers.

However, I noticed three types of reactions I would like to address:

It was not a Ruby revolution, it was a Rails revolution
The revolution has stalled due to no major enterprise backing
The revolution will only be over when we will reach a greater adoption

First of all, as Joe correctly mentioned, for me, the revolution is not about specifics or individuals. It’s really about the big picture.

The influence Ruby had and still has on the IT world seems to be undermined by some.
Ruby is a dynamic, truly Object Oriented programming scripting language designed for humans first.
The real paradigm shift is in the fact that Ruby was designed to make programming fast, enjoyable and easy instead of being optimized for the machines running it.
This is for me the essence of the revolution and it is meant to transcend the scope of the Ruby language.

The way I see it, Yukihiro Matsumoto (Matz) is more of an artist than a technician. He had a vision for software development. Programming languages cannot be optimized/designed for both machines and humans, the language designer has to choose which one he wants to privilege.

Most programming languages believe that it’s up to the programmer to make an extra effort since he is smarter and easier to optimize than a machine. Matz questioned this approached and decided to turn things around. The result is one of the reasons why developers seem to just fall in love with Ruby.

It was not a Ruby revolution, it was a Rails revolution.

I am not denying that there *also* was a Rails revolution.
But if you look at it, Rails and its revolution are a direct effect from Ruby’s revolution.
One might argue that it is actually an extension of Ruby’s philosophy. But what is Rails if not a web framework designed to make web development fast, easy and enjoyable?
Without Ruby there would not have been Rails and that was my point, the underlying revolution comes from the language itself.

The revolution has stalled due to no major enterprise backing.

That’s an interesting comment. It is true that .NET and Java are still dominating the enterprise world. But let’s be clear, Ruby was not designed to please “suit people”.
And to this day, there is still a strong feeling, from some individuals against the enterprise.
In the past, DHH openly said that he did not care nor wanted to hear about the enterprise, more recently, Obie Fernandez, during one of his talks said: “Fuck the enterprise” (49:39).
But the truth is that Ruby and the so called enterprise, both, are changing.
The smart people in the enterprise world saw potential in Ruby and decided to give it a chance. An easy way to include Ruby’s philosophy without breaking the fragile enterprise equilibrium was to inject Ruby in the midst of well known and respected technologies such as Java and .NET. The enterprise can now use “re-branded Ruby versions” with “new taste or ‘improved’ flavor” like JRuby, Scala, groovy, IronRuby.
I work for some enterprise clients and I can tell you that they ‘also’ use Ruby. Mainly because developers love the language.
Microsoft, Apple and SAP investing in their own implementation of the language is yet another example that the enterprise recognizes the value of Matz’s work.
Nobody can blame them to try to make Ruby fit more their requirements.
So, at the end of the day, Ruby is not the #1 enterprise language and Rails isn’t used by the large majority of enterprise web apps, but that is NOT the point. Ruby has influenced the enterprise and we will see its effects for many years.

The revolution will only be over when we will reach a greater adoption

Saying that is missing the point entirely. A revolution is a step towards a situation change. Things don’t change right away after a revolution. It takes a long time for mentalities to evolve and for people to change their habits.
The consequences of a revolution are to be studied over the decades following the event. Take smalltalk for instance. Smalltalk adoption was not that great, however it brought a paradigm shift that directly influenced languages such as Ruby, Python and Objective-C.
So, again, do not focus on the adoption but instead look at the influence of the Ruby revolution and the ripple effect around it.

Metaprogramming in Ruby: It’s All About the Self

2009-11-16T03:02:20+00:00

After writing my last post on Rails plugin idioms, I realized that Ruby metaprogramming, at its core, is actually quite simple.

It comes down to the fact that all Ruby code is executed code–there is no separate compile or runtime phase. In Ruby, every line of code is executed against a particular self. Consider the following five snippets:

class Person
  def self.species
    "Homo Sapien"
  end
end
 
class Person
  class << self
    def species
      "Homo Sapien"
    end
  end
end
 
class << Person
  def species
    "Homo Sapien"
  end
end
 
Person.instance_eval do
  def species
    "Homo Sapien"
  end
end
 
def Person.species
  "Homo Sapien"
end

All five of these snippets define a Person.species that returns Homo Sapien. Now consider another set of snippets:

class Person
  def name
    "Matz"
  end
end
 
Person.class_eval do
  def name
    "Matz"
  end
end

These snippets all define a method called name on the Person class. So Person.new.name will return “Matz”. For those familiar with Ruby, this isn’t news. When learning about metaprogramming, each of these snippets is presented in isolation: another mechanism for getting methods where they “belong”. In fact, however, there is a single unified reason that all of these snippets work the way they do.

First, it is important to understand how Ruby’s metaclass works. When you first learn Ruby, you learn about the concept of the class, and that each object in Ruby has one:

class Person
end
 
Person.class #=> Class
 
class Class
  def loud_name
    "#{name.upcase}!"
  end
end
 
Person.loud_name #=> "PERSON!"

Person is an instance of Class, so any methods added to Class are available on Person as well. What they don’t tell you, however, is that each object in Ruby also has its own metaclass, a Class that can have methods, but is only attached to the object itself.

matz = Object.new
def matz.speak
  "Place your burden to machine's shoulders"
end

What’s going on here is that we’re adding the speak method to matz’s metaclass, and the matz object inherits from its metaclass and then Object. The reason this is somewhat less clear than ideal is that the metaclass is invisible in Ruby:

matz = Object.new
def matz.speak
  "Place your burden to machine's shoulders"
end
 
matz.class #=> Object

In fact, matz’s “class” is its invisible metaclass. We can even get access to the metaclass:

metaclass = class << matz; self; end
metaclass.instance_methods.grep(/speak/) #=> ["speak"]

At this point in other articles on this topic, you’re probably struggling to keep all of the details in your head; it seems as though there are so many rules. And what’s this class << matz thing anyway?

It turns out that all of these weird rules collapse down into a single concept: control over the self in a given part of the code. Let’s go back and take a look at some of the snippets we looked at earlier:

class Person
  def name
    "Matz"
  end
 
  self.name #=> "Person"
end

Here, we are adding the name method to the Person class. Once we say class Person, the self until the end of the block is the Person class itself.

Person.class_eval do
  def name
    "Matz"
  end
 
  self.name #=> "Person"
end

Here, we’re doing exactly the same thing: adding the name method to instances of the Person class. In this case, class_eval is setting the self to Person until the end of the block. This is all perfectly straight forward when dealing with classes, but it’s equally straight forward when dealing with metaclasses:

def Person.species
  "Homo Sapien"
end
 
Person.name #=> "Person"

As in the matz example earlier, we are defining the species method on Person’s metaclass. We have not manipulated self, but you can see using def with an object attaches the method to the object’s metaclass.

class Person
  def self.species
    "Homo Sapien"
  end
 
  self.name #=> "Person"
end

Here, we have opened the Person class, setting the self to Person for the duration of the block, as in the example above. However, we are defining a method on Person’s metaclass here, since we’re defining the method on an object (self). Also, you can see that self.name while inside the person class is identical to Person.name while outside it.

class << Person
  def species
    "Homo Sapien"
  end
 
  self.name #=> ""
end

Ruby provides a syntax for accessing an object’s metaclass directly. By doing class << Person, we are setting self to Person’s metaclass for the duration of the block. As a result, the species method is added to Person’s metaclass, rather than the class itself.

class Person
  class << self
    def species
      "Homo Sapien"
    end
 
    self.name #=> ""
  end
end

Here, we combine several of the techniques. First, we open Person, making self equal to the Person class. Next, we do class << self, making self equal to Person’s metaclass. When we then define the species method, it is defined on Person’s metaclass.

Person.instance_eval do
  def species
    "Homo Sapien"
  end
 
  self.name #=> "Person"
end

The last case, instance_eval, actually does something interesting. It breaks apart the self into the self that is used to execute methods and the self that is used when new methods are defined. When instance_eval is used, new methods are defined on the metaclass, but the self is the object itself.

In some of these cases, the multiple ways to achieve the same thing arise naturally out of Ruby’s semantics. After this explanation, it should be clear that def Person.species, class << Person; def species, and class Person; class << self; def species aren’t three ways to achieve the same thing by design, but that they arise out of Ruby’s flexibility with regard to specifying what self is at any given point in your program.

On the other hand, class_eval is slightly different. Because it take a block, rather than act as a keyword, it captures the local variables surrounding it. This can provide powerful DSL capabilities, in addition to controlling the self used in a code block. But other than that, they are exactly identical to the other constructs used here.

Finally, instance_eval breaks apart the self into two parts, while also giving you access to local variables defined outside of it.

In the following table, defines a new scope means that code inside the block does not have access to local variables outside of the block.

mechanism	method resolution	method definition	new scope?
class Person	Person	same	yes
class Person	Person’s metaclass	same	yes
Person.class_eval	Person	same	no
Person.instance_eval	Person	Person’s metaclass	no

Also note that class_eval is only available to Modules (note that Class inherits from Module) and is an alias for module_eval. Additionally, instance_exec, which was added to Ruby in 1.8.7, works exactly like instance_eval, except that it also allows you to send variables into the block.

UPDATE: Thank you to Yugui of the Ruby core team for correcting the original post, which ignored the fact that self is broken into two in the case of instance_eval.

Better Ruby Idioms

2009-11-12T08:17:03+00:00

Carl and I have been working on the plugins system over the past few days. As part of that process, we read through the Rails Plugin Guide. While reading through the guide, we noticed a number of idioms presented in the guide that are serious overkill for the task at hand.

I don’t blame the author of the guide; the idioms presented are roughly the same that have been used since the early days of Rails. However, looking at them brought back memories of my early days using Rails, when the code made me feel as though Ruby was full of magic incantations and ceremony to accomplish relatively simple things.

Here’s an example:

module Yaffle
  def self.included(base)
    base.send :extend, ClassMethods
  end
 
  module ClassMethods
    # any method placed here will apply to classes, like Hickwall
    def acts_as_something
      send :include, InstanceMethods
    end
  end
 
  module InstanceMethods
    # any method placed here will apply to instaces, like @hickwall
  end
end

To begin with, the send is completely unneeded. The acts_as_something method will be run on the Class itself, giving the method access to include, a private method.

This code intended to be used as follows:

class ActiveRecord::Base
  include Yaffle
end
 
class Article < ActiveRecord::Base
  acts_as_yaffle
end

What the code does is:

Register a hook so that when the module is included, the ClassMethods are extended onto the class
In that module, define a method that includes the InstanceMethods
So that you can say acts_as_something in your code

The crazy thing about all of this is that it’s completely reinventing the module system that Ruby already has. This would be exactly identical:

module Yaffle
  # any method placed here will apply to classes, like Hickwall
  def acts_as_something
    send :include, InstanceMethods
  end
 
  module InstanceMethods
    # any method placed here will apply to instances, like @hickwall
  end
end

To be used via:

class ActiveRecord::Base
  extend Yaffle
end
 
class Article < ActiveRecord::Base
  acts_as_yaffle
end

In a nutshell, there’s no point in overriding include to behave like extend when Ruby provides both!

To take this a bit further, you could do:

module Yaffle
  # any method placed here will apply to instances, like @hickwall, 
  # because that's how modules work!
end

To be used via:

class Article < ActiveRecord::Base
  include Yaffle
end

In effect, the initial code (override included hook to extend a method on, which then includes a module) is two layers of abstraction around a simple Ruby include!

Let’s look at a few more examples:

module Yaffle
  def self.included(base)
    base.send :extend, ClassMethods
  end
 
  module ClassMethods
    def acts_as_yaffle(options = {})
      cattr_accessor :yaffle_text_field
      self.yaffle_text_field = (options[:yaffle_text_field] || :last_squawk).to_s
    end
  end
end
 
ActiveRecord::Base.send :include, Yaffle

Again, we have the idiom of overriding include to behave like extend (instead of just using extend!).

A better solution:

module Yaffle
  def acts_as_yaffle(options = {})
    cattr_accessor :yaffle_text_field
    self.yaffle_text_field = options[:yaffle_text_field].to_s || "last_squawk"
  end
end
 
ActiveRecord::Base.extend Yaffle

In this case, it’s appropriate to use an acts_as_yaffle, since you’re providing additional options which could not be encapsulated using the normal Ruby extend.

Another “more advanced” case:

module Yaffle
  def self.included(base)
    base.send :extend, ClassMethods
  end
 
  module ClassMethods
    def acts_as_yaffle(options = {})
      cattr_accessor :yaffle_text_field
      self.yaffle_text_field = (options[:yaffle_text_field] || :last_squawk).to_s
      send :include, InstanceMethods
    end
  end
 
  module InstanceMethods
    def squawk(string)
      write_attribute(self.class.yaffle_text_field, string.to_squawk)
    end
  end
end
 
ActiveRecord::Base.send :include, Yaffle

Again, we have the idiom of overriding include to pretend to be an extend, and a send where it is not needed. Identical functionality:

module Yaffle
  def acts_as_yaffle(options = {})
    cattr_accessor :yaffle_text_field
    self.yaffle_text_field = (options[:yaffle_text_field] || :last_squawk).to_s
    include InstanceMethods
  end
 
  module InstanceMethods
    def squawk(string)
      write_attribute(self.class.yaffle_text_field, string.to_squawk)
    end
  end
end
 
ActiveRecord::Base.extend Yaffle

Of course, it is also possible to do:

module Yaffle
  def squawk(string)
    write_attribute(self.class.yaffle_text_field, string.to_squawk)
  end
end
 
class ActiveRecord::Base
  def self.acts_as_yaffle(options = {})
    cattr_accessor :yaffle_text_field
    self.yaffle_text_field = (options[:yaffle_text_field] || :last_squawk).to_s
    include Yaffle
  end
end

Since the module is always included in ActiveRecord::Base, there is no reason that the earlier code, with its additional modules and use of extend, is superior to simply reopening the class and adding the acts_as_yaffle method directly. Now we can put the squawk method directly inside the Yaffle module, where it can be included cleanly.

It may not seem like a huge deal, but it significantly reduces the amount of apparent magic in the plugin pattern, making it more accessible for new users. Additionally, it exposes the new user to include and extend quickly, instead of making them feel as though they were magic incantations requiring the use of send and special modules named ClassMethods in order to get them to work.

To be clear, I’m not saying that these idioms aren’t sometimes needed in special, advanced cases. However, I am saying that in the most common cases, they’re huge overkill that obscures the real functionality and confuses users.

The Ruby revolution is over

2009-11-09T08:10:16+00:00

According to wikipedia, a revolution (from the latin revolutio, “a turn around”) is a fundamental change in power or organizational structures that takes place in a relatively short period of time.

Somehow, I believe this is exactly what Ruby has done in the programming world, especially with the help of Rails. Over the last few years, Ruby lead a mini revolution in the midst of software development. Thanks to Ruby, developers now look at software development differently. One thing for sure, it pushed DHH to write Rails and then convinced thousands of people to develop Ruby based applications on a daily basis.

How did it happen?

Let’s take a look at history of revolutions. Some people get frustrated their situation, they try to find workarounds until it’s just too much and the revolution kicks in.

Ruby came up with a new holistic perspective on things. Unlike most other programming languages, one of the main key value of Ruby is that writing code should feel right for the developer. You feel good about it because the language was written for humans and not machines. Basically, the language was designed to make you productive because it’s designed to please you.

As people were discovering web 2.0, Ruby also came with an opinionated framework, pushing for productivity, testing, simplicity and elegance. People started to see a new way of doing things and it quickly became the new, cool technology. Rails became a buzz word, developers were hired to work on cool projects, and books were selling by the thousands.

What did it change?

If you ask my mom, she would probably say: nothing, except that now my son works from his home office and he seems to really enjoy what he does for living.

Relatively speaking, Ruby did not change the way we work or live. However, I believe that it has influenced many software developers around the globe. Why else do you think that companies like Microsoft, Apple or SAP are working on their own implementation of the Ruby language?

When I first discovered Ruby, I was amazed at how “right” it felt, at how much fun it was to write code using its syntax and idioms. Now, if I don’t get that feeling when testing a programming language, I think there is something wrong.

The Ruby community also revived the Agile/XP world. Testing being a strong value of the community, we spent a lot of time discussing TDD, BDD, integration test as well as other practices such as pair programming, code review, sprints etc..

A few years ago, when people were asking me what programming language I would write their app in, I would reply Ruby and had to explain what it was, why it is great and would have to answer a lot of questions from potential clients. Nowadays, people don’t even argue, sites like hulu.com, twitter.com, yellowpages.com and many others are written in Ruby and it’s just part of the tools known to work very well.

The revolution is over!

Yes, Ruby made it’s revolution and the world “has changed”. But a real movement doesn’t die after its revolution, that’s actually when it has to be strong and defend its values.

This doesn’t mean that Ruby is dead or that Rails is “passé”. To the contrary, Ruby imposed itself as a new valued and respected player, a new standard if you will.

Ruby is certainly not the “new kid in the block”anymore nor the “popular kid”, however lots of older kids seem to want to have her on their team. (.NET, Java, Objective-C can all use Ruby)

The TDD + Ruby combo doesn’t surprise anyone anymore and the Enterprise is slowly but surely adopting Ruby. Ruby is now just getting better, tools and libraries are improving and the amount of users is growing.

Certainly the Ruby community is still small compared to other software developer communities, but the fundamental change was done and we are now working on improvement and keeping things running smoothly, growing and getting new ideas inspired by our experience and other communities.

Long live Ruby!

See my follow up.

Using the New Gem Bundler Today

2009-11-03T09:53:40+00:00

As you might have heard, Carl and I released a new project that allows you to bundle your gems (both pure-ruby and native) with your application. Before I get into the process for using the bundler today, I’d like to go into the design goals of the project.

The bundler should allow the specification of all dependencies in a separate place from the application itself. In other words, it should be possible to determine the dependencies for an application without needing to start up the application.
The bundler should have a built-in dependency resolving mechanism, so it can determine the required gems for an entire set of dependencies.
Once the dependencies are resolved, it should be possible to get the application up and running on a new system without needing to check Rubyforge (or gemcutter) again. This is especially important for compiled gems (it should be possible to get the list of required gems once and compile on remote systems as desired).
Above all else, the bundler should provide a reproducible installation of Ruby applications. New gem releases or down remote servers should not be able to impact the successful installation of an application. In most cases, git clone; gem bundle should be all that is needed to get an application on a new system and up and running.
Finally, the bundler should not assume anything about Rails applications. While it should work flawlessly in the context of a Rails application, this should be because a Rails application is a Ruby application.

Using the Bundler Today

To use the gem bundler today in a non-Rails application, follow the following steps:

gem install bundler
Create a Gemfile in the root of your application
Add dependencies to your Gemfile. See below for more details on the sorts of things you can do in the Gemfile. At the simplest level, gem "gem_name", "version" will add a dependency of the gem and version to your application
At the root, run gem bundle. The bundler should tell you that it is resolving dependencies, then downloading and installing the gems.
Add vendor/gems/gems, vendor/gems/specifications, vendor/gems/doc, and vendor/gems/environment.rb to your .gitignore file
Inside your application, require vendor/gems/environment.rb to add the bundled dependencies to your load paths.
Use Bundler.require_env :optional_environment to actually require the files.
After committing, run gem bundle in a fresh clone to re-expand the gems. Since you left the vendor/gems/cache in your source control, new machines will be guaranteed to use the same files as the original machine, requiring no remote dependencies

The bundler will also install binaries into the app’s bin directory. You can, therefore, run bin/rackup for instance, which will ensure that the local bundle, rather than the system, is used. You can also run gem exec rackup, which runs any command using the local bundle. This allows things like gem exec ruby -e "puts Nokogiri::VERSION" or the even more adventurous gem exec bash, which will open a new shell in the context of the bundle.

Gemfile

You can do any of the following in the Gemfile:

gem "name", "version": version may be a strict version or a version requirement like >= 1.0.6. The version is optional.
gem "name", "version", :require_as => "file": the require_as allows you to specify which file should be required when the require_env is called. By default, it is the gem’s name
gem "name", "version", :only => :testing: The environment name can be anything. It is used later in your require_env call. You may specify either :only, or :except constraints
gem "name", "version", :git => "git://github.com/wycats/thor": Specify a git repository to be used to satisfy the dependency. You must use a hard dependency (“1.0.6″) rather than a soft dependency (“>= 1.0.6″). If a .gemspec is found in the repository, it is used for further dependency lookup. If the repository has multiple .gemspecs, each directory will a .gemspec will be considered a gem.
gem "name", "version", :git => "git://github.com/wycats/thor", :branch => "experimental": Further specify a branch, tag, or ref to use. All of :branch, :tag, and :ref are valid options
gem "name", "version", :vendored_at => "vendor/nokogiri": In the next version of bundler, this option will be changing to :path. This specifies that the dependency can be found in the local file system, rather than remotely. It is resolved relative to the location of the Gemfile
clear_sources: Empties the list of gem sources to search inside of.
source "http://gems.github.com": Adds a gem source to the list of available gem sources.
bundle_path "vendor/my_gems": Changes the default location of bundled gems from vendor/gems
bin_path "my_executables": Changes the default location of the installed executables
disable_system_gems: Without this command, both bundled gems and system gems will be used. You can therefore have things like ruby-debug in your system and use it. However, it also means that you may be using something in development mode that is installed on your system but not available in production. For this reason, it is best to disable_system_gems
disable_rubygems: This completely disables rubygems, reducing startup times considerably. However, it often doesn’t work if libraries you are using depend on features of Rubygems. In this mode, the bundler shims the features of Rubygems that we know people are using, but it’s possible that someone is using a feature we’re unaware of. You are free to try disable_rubygems first, then remove it if it doesn’t work. Note that Rails 2.3 cannot be made to work in this mode
only :environment { gem "rails" }: You can use only or except in block mode to specify a number of gems at once

Bundler process

When you run gem bundle, a few things happen. First, the bundler attempts to resolve your list of dependencies against the gems you have already bundled. If they don’t resolve, the metadata for each specified source is fetched and the gems are downloaded. Next (either way), the bundler checks to see whether the downloaded gems are expanded. For any gem that is not yet expanded, the bundler expands it. Finally, the bundler creates the environment.rb file with the new settings. This means that running gem bundler over and over again will be extremely fast, because after the first time, all gems are downloaded and expanded. If you change settings, like disable_rubygems, running gem bundle again will simply regenerate the environment.rb.

Rails 2.3

To get this working with Rails 2.3, you need to create a preinitializer.rb and insert the following:

require "#{File.dirname(__FILE__)}/../vendor/bundler_gems/environment"
 
class Rails::Boot
  def run
    load_initializer
    extend_environment
    Rails::Initializer.run(:set_load_path)
  end
 
  def extend_environment
    Rails::Initializer.class_eval do
      old_load = instance_method(:load_environment)
      define_method(:load_environment) do
        Bundler.require_env RAILS_ENV
        old_load.bind(self).call
      end
    end
  end
end

It’s a bit ugly, but you can copy and paste that code and forget it. Astute readers will notice that we’re using vendor/bundler_gems/environment.rb. This is because Rails 2.3 attaches special, irrevocable meaning to vendor/gems. As a result, make sure to do the following in your Gemfile: bundle_path "vendor/bundler_gems".

Gemcutter uses this setup and it’s working great for them.

Bundler 0.7

We’re going to be releasing Bundler 0.7 tomorrow. It has some new features:

List outdated gems by passing --outdated-gems. Bundler conservatively does not update your gems simply because a new version came out that satisfies the requirement. This is so that you can be sure that the versions running on your local machine will make it safely to production. This will allow you to check for outdated gems so you can decide whether to update your gems with –update. Hat tip to manfred, who submitted this patch as part of his submission to the Rumble at Ruby en Rails
Specify the build requirements for gems in a YAML file that you specify with --build-options. The file looks something like this:
```
mysql:
  config: /path/to/mysql_config
```
This is equivalent to –with-mysql-config=/path/to/mysql_config
Specify :bundle => false to indicate that you want to use system gems for a particular dependency. This will ensure that it gets resolved correctly during dependency resolution but does not need to be included in the bundle
Support for multiple bundles containing multiple platforms. This is especially useful for people moving back and forth between Ruby 1.8 and 1.9 and don’t want to constantly have to nuke and rebundle
Deprecate :vendored_at and replace with :path

A new directory DSL method in the Gemfile:

directory "vendor/rails" do
  gem "activesupport", :path =&gt; "activesupport" # :path is optional if it's identical to the gem name
                                                # the version is optional if it can be determined from
                                                # a gemspec
end

You can do the same with the git DSL method

git "git://github.com/rails/rails.git" do # :branch, :tag, or :ref work here
  gem "activesupport", :path =&gt; "activesupport" # same rules as directory, except that the files are
                                                # first downloaded from git.
end

Fix some bugs in resolving prerelease dependencies

MacRuby tips: browse for folder or file dialog

2009-10-25T23:52:14+00:00

This is yet another pretty simple tip.
Use case: let say you want your applications users to choose one or multiple files or folder on their file system. A good example would be that you want the user to choose a file to process or a folder where to save some data.

In the example above, I added a browse button and a text field.

I would like my users to click on the browse button, locate a folder and display it in the text field.

In your MacRuby controller, use a simple action method as well as an accessor to the text field:

attr_accessor :destination_path
 
def browse(sender)
end

Now, in Interface builder bind the destination_path outlet to the text field you want to use to display the path and bind the button to the browse action.

Let’s go back to our action method and let’s create a dialog panel, set some options and handle the user selection:

def browse(sender)
  # Create the File Open Dialog class.
  dialog = NSOpenPanel.openPanel
  # Disable the selection of files in the dialog.
  dialog.canChooseFiles = false
  # Enable the selection of directories in the dialog.
  dialog.canChooseDirectories = true
  # Disable the selection of multiple items in the dialog.
  dialog.allowsMultipleSelection = false
 
  # Display the dialog and process the selected folder
  if dialog.runModalForDirectory(nil, file:nil) == NSOKButton
  # if we had a allowed for the selection of multiple items
  # we would have want to loop through the selection
    destination_path.stringValue = dialog.filenames.first
  end
end

That’s it, your user can now browse for a folder and the selection will be displayed in the text field. Look at the NSOpenPanel documentation for more details on the Cocoa API.

RailsSummit – an amazing adventure

2009-10-23T18:17:45+00:00

I have had the opportunity to go to and speak at many conferences but this year was the first time I had the chance to go to RailsSummit in São Paulo, Brazil.

I was really looking forward to this trip and I have to say it went beyond my expectations. I had really good feedback from people like the Phusion guys (Ninh & Hongli), Gregg Pollack and others.

For those who don’t know, RailsSummit is one of the biggest, if not the biggest, Ruby event in Latin America. It’s organized yearly by Locaweb, the #1 hosting company in Brazil (props to Akita & Cristiane for their work).

As part of my involvement in the Ruby community I have met a lot of Brazilians always willing to help and especially giving time to translate content. (A good example would be the Portuguese version of the Rails wiki or José Valim GSOC contribution) However, I did not realize how fast Rails was growing over here.

To come back to the conference, I have to say it was one of the best conference I have gone to. Chad Fowler, who gave the opening keynote, later told me that it reminded him of the first Ruby conferences in the US . For me, what made a huge difference was the fact that it was a very positive conference. People were happy to be here, eager to share and you could feel the passion. Unfortunately, I missed the first few Ruby conferences, but I can totally imagine how must have been. Passionate people, not trying to push their products but instead, share the cool stuff they’ve been working on. This is exactly the feeling I had during this conference.

Maybe it’s because I don’t understand Portuguese well enough or maybe it’s just a cultural thing, but the people at the conference were just super friendly and always willing to help. I was really glad to meet those who have been using some of my work, some new people to Ruby and people who don’t do Ruby but were just interested in knowing what was going on in the Ruby world.

The schedule was pretty packed and the discussions very interesting, you could certainly feel the excitement in the air. Ruby seems to catching up quickly over there. Brazilian Rubyists seem to be very pragmatic and a good illustration of that was certainly made by Vinicus Teles, Brazil Agile XPert, who shared tips on how to release a successful product.

I stayed a few days after the conference and went to visit the Rubyists in Rio. Rio is a great city. It has some of the best soccer players in the world and some seriously talented software developers. Ruby & Rails are not just the new trendy startup secret to success, companies like globo.com, currently the largest TV network in Latin America and the fourth largest in the world, also started using Ruby and Rails. I had the opportunity to visit their office and meet their teams. It’s very exciting to see how they use Agile/XP and Ruby and how they seem to be so successful. But I will keep that for another post.

Overall, even though the actual traveling to/from Brazil was a bit long, RailsSummit was a blast. I really hope to be able to come back next year, and by then, hopefully my Portuguese will have improved.

MacRuby tips: capturing keyboard events

2009-10-10T03:24:04+00:00

If you are writing any type of games you might want your users to interact with your application using their keyboards.

This is actually not that hard. The approach is simple and fast forward if you are used to Cocoa.

Everything starts in Interface Builder, add a custom view instance to your window.

Now switch to your project and a new file with a class called KeyboardControlView and make in inherit from NSView. We are creating a subview of NSView so we will be able to make our top view “layer” use this subclass.

class KeyboardControlView &lt; NSView
  attr_accessor :game_controller
 
  def acceptsFirstResponder
    true
  end
 
end

As you can see in the example above, I added an attribute accessor. attr_accessor class method creates getters and setters. It’s basically the same as writing:

 def game_controller=(value)
  @game_controller = value
end
 
def game_controller
  @game_controller
end

MacRuby is keeping an eye on these accessors and let bind outlets to them.
But let’s not get ahead of ourselves, we’ll keep that for another time.

Let’s go back to our newly created class. Notice, we also added a method called `
acceptsFirstResponder` and returns true. acceptsFirstResponder returns false by default.
But in this case we want it to return true so our new class instance can be first in the responder chain.

Now that our class is ready, let’s go back to Interface Builder, select our new custom view and click on the inspector button.

Click on the (i) icon and in the Class field choose our new KeyboardControlView.
Yep, our new class just shows up by magic, it’s also called the lrz effect, just don’t ask
So now when our application starts, a new instance of our NSView class is created and Cocoa will call different methods based on events triggered.

The two methods we are interested in reimplementing are keyDown and keyUp. They get called when a key gets pressed or released.

def keyDown(event)
  characters = event.characters
  if characters.length == 1 &amp;&amp; !event.isARepeat
    character = characters.characterAtIndex(0)
    if character == NSLeftArrowFunctionKey
      puts "LEFT pressed"
    elsif character == NSRightArrowFunctionKey
      puts "RIGHT pressed"
    elsif character == NSUpArrowFunctionKey
      puts "UP pressed"
    elsif character == NSDownArrowFunctionKey
      puts "DOWN pressed"
    end
  end
 super
end

I don’t think the code above needs much explanation. The only things that you might not understand are ‘event.isARepeat’. This method returns true if the user left his/her finger on the key. The other thing is the use of the ’super’ call at the end of the method. Basically, we reopened a method that was already defined and we don’t want to just overwrite it, we just want to inject out code within the existing method, so once we are done handling the event, we just pass it back to original method.

Final result:

class KeyboardControlView &lt; NSView
  attr_accessor :game_controller
 
  def acceptsFirstResponder
    true
  end
 
  def keyDown(event)
    characters = event.characters
    if characters.length == 1 &amp;&amp; !event.isARepeat
      character = characters.characterAtIndex(0)
      if character == NSLeftArrowFunctionKey
        puts "LEFT pressed"
      elsif character == NSRightArrowFunctionKey
        puts "RIGHT pressed"
      elsif character == NSUpArrowFunctionKey
        puts "UP pressed"
      elsif character == NSDownArrowFunctionKey
  	puts "DOWN pressed"
      end
    end
    super
  end
 
  # Deals with keyboard keys being released
  def keyUp(event)
    characters = event.characters
    if characters.length == 1
      character = characters.characterAtIndex(0)
      if character == NSLeftArrowFunctionKey
       puts "LEFT released"
      elsif character == NSRightArrowFunctionKey
        puts "RIGHT released"
      elsif character == NSUpArrowFunctionKey
       puts "UP released"
      elsif character == NSDownArrowFunctionKey
        puts "DOWN released"
      end
    end
    super
  end
 
end

Now it’s up to you to handle the other keystrokes and do whatever you want. That’s it for this tip, I hope it helps.

MacRuby 0.5 beta 1 and Textorize

2009-10-08T04:45:02+00:00

Good news everyone!

MacRuby beta 1 has been released! Official announcement here.

Download MacRuby 0.5 beta1

Note that the download is only for SnowLeopard, intel machines.

Lots of great stuff in this new release, the first one based on LLVM. Check the Laurent’s post to learn more about the work done on compilation, optimization, concurrency, compatibility and Cocoa interaction. And a big thank you to Laurent Sansonetti who is putting so much effort in this project!

However, don’t forget it’s still a beta release and you might encounter bugs. Feel free to report them in the bug tracker or ask on the mailing list.

On a different topic, the other day, John Gruber from Daring Fireball wrote a quick note about Thomas Fuchs’ textorize script which since got its own place on the internet http://textorize.org/.

Textorize is a Ruby-based font rasterizer command line utility for Mac OS X. It generates PNG files from an input string and options, using Mac OS X’s pristine typography facilities. As John said, it’s a case where a few lines of Ruby code beat Photoshop.

Thomas version is based on RubyCocoa which is great… but not MacRuby.

To celebrate MacRuby 0.5 beta1, I ported the gem over and pushed it to the excellent gemcutter.org facility.

After installing MacRuby beta, follow these directives:

$ macgem sources -a http://gemcutter.org
$ sudo macgem install textorize-mr
$ textorize -f"Didot" -s200 "MacRuby 0.5b1"
$ open output.png

And you will get a subpixel antialiased fancy bitmap like that:

Check http://textorize.org/ for more examples and http://github.com/mattetti/textorize for the source code.

MacRuby tips: how to play an audio file

2009-10-07T03:00:08+00:00

Let’s say you would like to play an audio file in your MacRuby app/script, how would you do?
It’s actually pretty simple, you just need to use NSSound. If we wanted to use a system sound we could do:

NSSound.soundNamed('Basso').play

But let’s look at a more advanced example with some Cocoa patterns. We will loop through all the audio files in a folder and we will play them one after the other.

#!/usr/bin/env macruby
framework 'Cocoa'
# Cocoa documentation reference:
# http://developer.apple.com/mac/library/documentation/Cocoa/Reference/ApplicationKit/Classes/NSSound_Class/Reference/Reference.html
 
def play_sound
  if @sounds.empty?
    NSApplication.sharedApplication.terminate(nil)
  else
    sound_file = @sounds.shift
    s = NSSound.alloc.initWithContentsOfFile(sound_file, byReference: false)
    puts "previewing #{sound_file}"
    s.delegate = self
    s.play
  end
end
 
# This is a delegate method called by the sound object
def sound(sound, didFinishPlaying: state)
  play_sound if state
end
 
# Delegate method called when the app finished loading
def applicationDidFinishLaunching(notification)
  @sounds = Dir.glob("/System/Library/Sounds/*.aiff")
  play_sound
end
 
# We are delegating the application to self so the script will know when
# it finished loading
NSApplication.sharedApplication.delegate = self
NSApplication.sharedApplication.run

On my machine, I get the following output:

$ macruby macrubysound.rb
previewing /System/Library/Sounds/Basso.aiff
previewing /System/Library/Sounds/Blow.aiff
previewing /System/Library/Sounds/Bottle.aiff
previewing /System/Library/Sounds/Frog.aiff
previewing /System/Library/Sounds/Funk.aiff
previewing /System/Library/Sounds/Glass.aiff
previewing /System/Library/Sounds/Hero.aiff
previewing /System/Library/Sounds/Morse.aiff
previewing /System/Library/Sounds/Ping.aiff
previewing /System/Library/Sounds/Pop.aiff
previewing /System/Library/Sounds/Purr.aiff
previewing /System/Library/Sounds/Sosumi.aiff
previewing /System/Library/Sounds/Submarine.aiff
previewing /System/Library/Sounds/Tink.aiff

Cocoa delegation might seem a bit strange at first when you come from Ruby. In Ruby we rarely do any type of async delegation so let’s quickly look at what’s going on in this script.

The first thing we do is loading the cocoa framework which makes total sense, then we define a bunch of methods and finally we get to:

NSApplication.sharedApplication.delegate = self
NSApplication.sharedApplication.run

We are setting the run loop to delegate to set, which means that when an event is triggered, the delegation method will be called on self (our script).
Once that done, we are starting out run loop.

Once the application is loaded the applicationDidFinishLaunching delegate is being triggered (line 24). At this point, we are looking for all the sound files in the sound system folder and storing them in an instance variable (line 25). Finally, we are calling the play_sound method (line 26).

The play_sound method checks that we have some audio files left to play (line 7), otherwise it quits the app. (line 8 ) If we still have some files in the queue, we get the first one (line 10) and use it to create an instance of NSSound (line 11).

Before playing the NSSound instance, we are setting its delegate to our script (self) (line 13). If you read NSSound Cocoa documentation for #play, you will notice that #play “initiates playback asynchronously and returns control to your application. Therefore, your application can continue doing work while the audio is playing.” In our case, we don’t want to do that, otherwise all the sounds will play at the same time.

The documentation also mentions a delegation you can use to avoid that. This is exactly what we did when we implemented: def sound(sound, didFinishPlaying: state) (line 19)

Rubyists might be surprised by the method signature. MacRuby extended Ruby to support Objective-C selectors.

When the sound is playing, this delegate gets called, we check on the state of the sound, if it finished playing then we call play_sound again.

That’s it! It’s a very elegant implementation giving us the benefits of both Ruby and Cocoa.

MacRuby tips: embed a custom font

2009-10-06T17:23:44+00:00

Let say you want to release your MacRuby app and use a custom embedded font?
You probably don’t want to force your users to install the font.
Well, don’t worry, just put the font file in your resources folder and use the following code:

font_location = NSBundle.mainBundle.pathForResource('MyCustomFont', ofType: 'ttf')
font_url = NSURL.fileURLWithPath(font_location)
# in MacRuby, always make sure that cocoa constants start by an uppercase
CTFontManagerRegisterFontsForURL(font_url, KCTFontManagerScopeProcess, nil)

That’s it, now your custom font is available and you can change your textfield instance’s font like that:

text_field.font = NSFont.fontWithName('MyCustomFont', size:24)

The only tricky things here were to know the Cocoa API call and to know that even if the Cocoa API references the constant to use as kCTFontManagerScopeProcess, because in Ruby, constants start by an uppercase, you need to convert it to: KCTFontManagerScopeProcess.

MacRuby soon to reach a new milestone

2009-10-06T07:58:53+00:00

Laurent just posted a MacRuby status update on the mailing list and the first official beta of MacRuby 0.5 should be released pretty soon.

Let’s quickly look at Laurent’s report:

Early backtracing support.
Much better AOT compilation. Parts of the standard library are now pre-compiled for testing.
Migrated to LLVM top of tree.
Dispatcher performance is now back to normal (we lost about 30% due to gcc not inlining code).
Many bug fixes.

In lay terms, backtracing is what you see when your app crashes or has a problem, it’s the list of methods called before the exception was raised and the line where the error happened. Currently the backtrace is similar to what you would have with Ruby 1.9, however objective-c exceptions are not supported and there is still some work to do.

AOT compilation or Ahead Of Time compilation is the process of compiling a script into machine code. I already covered that feature earlier. Progress has been made and now some Ruby standard libraries are now pre-compiled in MacRuby. The two main advantages of doing AOT compilation are startup speed and obfuscation. Two important features for desktop applications or for when you want to license your server app.

Updating LLVM doesn’t mean much for end users. With the support of the MacRuby team, Claudio setup a nightly build bot http://macruby.icoretech.org/ allowing you to download nightly builds for SnowLeopard. The app is a Sinatra app that could can checkout on github: http://github.com/masterkain/macruby-nightlies-web What’s interesting is that Apple is sponsoring both LLVM and MacRuby which will hopefully bring some synergy and help both projects.

Fixed dispatcher, this is a just a perf bug fix only affecting people on trunk.

I personally used MacRuby quite a lot recently. I have been writing a 2D video game for my RubyConf talk.

It’s a simple game only using Ruby and Cocoa. It’s not a fancy game and, no, it doesn’t run on the iPhone yet

I never wrote a game in Cocoa and I decided not to use an existing framework like Gosu or cocos2d, instead I decided to write everything from scratch. Using CoreAnimation instead of OpenGL, the task wasn’t that hard at all. in less than 1,000 LOC (before refactoring), I have a fully working game.

I will be previewing the game at RailsSummit and will show the code and explain how to get there at RubyConf.

When I started getting involved with MacRuby, I really did not think I would write a video game in Ruby and actually enjoy it. At the end of the day, I will more than likely use MacRuby for desktop/mobile/server apps more than games, but it’s awesome to be able to use your favorite language to do other things than what you are usually paid to do.

MacRuby is coming along very nicely, and I’m really excited about the multitude of new options offered to Ruby developers on OSX and can’t wait for 0.5 final and see what people will do with it.

Inherited Templates With Rails

2009-10-05T03:14:36+00:00

A nice feature of some template languages (for instance, Django’s) is the ability to create templates that can be “inherited” by other templates. In effect, the goal is to create a template that has some missing content, and let “inheritors” fill in that content downstream.

It’s not terribly difficult to add this feature to Rails; let’s first design the API. First, the parent template (_parent.html.erb):

Before in parent
<%= first_from_child %>
<%= second_from_child %>
After in parent

We just use normal partial locals for the parent API, because they’re a well-understood construct and pretty easy to work with. Next, the “subclass” (_child.html.erb):

Before in child
<% override_template("parent") do |template| -%>
  <% template.set_content(:first_from_child) do -%>first_from_child<% end -%>
  <% template.set_content(:second_from_child) do -%>second_from_child<% end -%>
<% end -%>
After in child

Here, we use Ruby’s block syntax to create a context for supplying the content for the template. We expect the following output:

Before in child
Before in parent
first_from_child
second_from_child
After in parent
After in child

There are two parts to this implementation. First, let’s create the implementation for the override_template method that will be exposed onto ActionView::Base:

module ActionView
  module InheritedTemplates
    class Collector
    end
 
    def override_template(name, &block)
      locals = Collector.collect(self, &block)
      concat(render :partial => name, :locals => locals)
    end
  end
 
  class Base
    include InheritedTemplates
  end
end

To start, we create the method, deferring the heavy lifting to the Collector.collect method. The API we’ve designed asks #collect to hand back a Hash of locals, which we can pass unmodified into the “super” template.

The collector is a pretty straight-forward implementation, but it deserves some explanation:

class Collector
  def self.collect(view)
    collector = new(view)
    yield collector
    collector.content
  end
 
  attr_reader :content
  def initialize(view)
    @view = view
    @content = {}
  end
 
  def set_content(name, &block)
    @content[name] = @view.capture(&block)
  end
end
 
def override_template(name, &block)
  locals = Collector.collect(self, &block)
  concat(render :partial => name, :locals => locals)
end

The first thing we do is define the collect method. It’s a basic DSL implementation, creating an instance of itself and evaluating the block in its context. In order to fully understand what’s going on here, you need to see what the block looks like, once it’s compiled from ERB into pure-Ruby:

override_template("parent") do |template|
  template.set_content(:first_from_child) do; @output_buffer << "first_from_child"; end
  template.set_content(:second_from_child) do; @output_buffer << "second_from_child"; end
end

We start by creating a new collector, yielding the collector to the block passed to override_template. Inside the block, template.set_content is called twice; each call sets a key and value in the @content Hash. When done, the Hash will look like {:first_from_child => "first_from_child", :second_from_child => "second_from_child"}. The collector then returns the content Hash.

When that’s done, override_template simply passes that locals Hash through to render :partial, and concats the output to the buffer.

Afterthought

Those following my blog might remember that I showed how we can improve block helpers by smarter compilation. If that fix was in place, here’s what _child.html.erb would look like:

Before in child
<%= override_template("parent") do |template| -%>
  <% template.set_content(:first_from_child) do -%>first_from_child<% end -%>
  <% template.set_content(:second_from_child) do -%>second_from_child<% end -%>
<% end -%>
After in child

More importantly, here’s what override_template would look like:

def override_template(name, &block)
  locals = Collector.collect(self, &block)
  render :partial => name, :locals => locals
end

We’re actually saved a bit of pain here in the unfixed case, because override_template can only be called from inside templates, so we don’t need to use block_called_from_erb? to disambiguate. Had this been a normal block helper, override_template would have looked like:

def override_template(name, &block)
  locals = Collector.collect(self, &block)
  response = render(:partial => name, :locals => locals)
 
  if block_called_from_erb?(&block)
    concat(response)
  else
    response
  end
end

Of course, realizing all of this is non-trivial, and the reason we’re going to be moving toward the simple block helper syntax above.

Emulating Smalltalk’s Conditionals in Ruby

2009-10-04T08:27:07+00:00

If you follow my blog, you know that I enjoy emulating language features from other languages (like Python) in pure-Ruby. On the flight back from London, I read through Smalltalk Best Practice Patterns, and was reminded that Smalltalk doesn’t have built-in conditionals.

Instead, they use method calls (aka message sends) to do the heavy lifting:

anObject isNil
  ifTrue:  [ Transcript show: 'true'. ]
  ifFalse: [ Transcript show: 'false'. ].

In contrast, the typical way to do that in Ruby is:

if an_object.nil?
  puts "true"
else
  puts "false"
end

However, the Smalltalk approach is fully compatible with Ruby’s pure-OO approach (which is, in fact, inherited from Smalltalk in the first place). So why not be able to do:

(an_object.nil?).
  if_true  { puts "true" }.
  if_false { puts "false" }

In Ruby, unlike Smalltalk, message sends use a “.“, so we need to use periods between the message sends to link them together. With that exception, the semantics of the two languages are roughly identical. And, as usual, the implementation in Ruby is pretty straight-forward:

class Object
  def if_true
    yield
    self
  end
 
  def if_false
    self
  end
end
 
class NilClass
  def if_true
    self
  end
 
  def if_false
    yield
    self
  end
end
 
class FalseClass
  def if_true
    self
  end
 
  def if_false
    yield
    self
  end
end

In Ruby, false and nil are “falsy”, while all other values are “truthy”. Since everything in Ruby inherits from Object, we can simply define if_true and if_false on Object, FalseClass, and NilClass. If the block should be called (if_true on truthy values, or if_false on falsy values), we yield. In all cases, we return “self” so that we can chain if_true to if_false. And that’s it.

Of course, this means extending core classes, which some people are shy about doing, but it does mirror the semantics of Smalltalk.

“The Bar for Success in Our Industry” — A Quibble

2009-09-03T02:04:32+00:00

I read The bar for success in our industry is too low with interest. I generally like and agree with the argument that as an entrepreneur, people should build businesses around sustainable practices that involve actually making money in the short-term. The fact that the advice comes from a successful business like 37 Signals, which has actually made sustainable, long-term money, makes it further credible.

I also agree that the analysis of Evernote as a “success” on the basis of projected income earned incrementally (a month at a time) over a year from now is a pretty bizarre way to define present success.

That said, I think their success blinds them to alternative, more risky ways of being successful with technology. In Jason’s post, he analogizes web businesses to stores:

If there was an airline that flew more passengers than anyone else, but lost money on each one, would we call it a success? If there was a restaurant that served more people than anyone else, but lost money on each meal served, would we call it a success? If there was a store that sold more product than anyone else, but took a loss on each one, would we call it a success? Would the business press hold these companies up as business model successes? Would anyone? Interesting, maybe. Promising, sure. But successful? Then what the hell is going on with the coverage of our industry?

There are, however, an entirely different mode of doing business that is well-represented in the traditional sphere. While it is certainly more risky, large companies often sink millions or billions of dollars into research, hoping it will pay off by providing them a sellable product. Microsoft, for instance, spent more than 9 billion dollars on research in FY 2008, almost double what it spent on income tax. Around one in six dollars that comes in to Microsoft in revenue are spent on research.

By Jason’s argument, Microsoft Research is not “successful”, and if someone says it is, they are using an unacceptably low bar for success. It would be more appropriate to think of Microsoft as aggregating significant amounts of risky propositions into a single department, on the gamble that enough of those propositions will be successful to make the expenditure worth it, even if it takes a while for those propositions to pay off.

In effect, venture capitalists are making the exact same gamble. The underlying business model is still the same: plunk down a bunch of cash on research projects that, if successful, will result in significant payoff. The difference is that Internet businesses are very amenable to a more distributed model, where individual folks with ideas do the research, shouldering some of the risk of failure and some of the reward for success.

In other words, these sorts of businesses are simply accumulating value–in followers, users, or even eyeballs. They will be successful when they can sell that value (usually, all at once), just as Microsoft will be successful when they can ship products based on years of research (usually, all at once). According to Jason’s analysis, Twitter is not a success. That is a fundamentally flawed analysis as they have built enough value at this point to be converted into a lump-sum payoff, even if they have not yet cashed in.

However, none of this is to say that as an entrepreneur, you should build your business around the hope of such a payoff. It’s a highly risky way to build a business, and is subsidizing venture capitalists, who don’t actually share in the risk (like Microsoft, they are aggregating risk into a larger research pool). But if taking big risks is your thing, taking on a long-term research project is a perfectly reasonable way to build a business.

Simplifying Rails Block Helpers (With a Side of Rubinius)

2009-08-31T06:09:49+00:00

We all know that <%= string %> emits a String in ERB. And <% string %> runs Ruby code, but does not emit a String. When starting working with Rails, you almost expect the syntax for block helpers to be:

<%= content_tag(:div) do %>
  The content
<% end %>

Why doesn’t it work that way?

It has to do with how the ERB parser works, looking at each line individually. When it sees <% %>, it evaluates the code as a line of Ruby. When it sees <%= %>, it evaluates the inside of the ERB tag, and calls to_s on it.

This:

<% form_for(@object) do %>
Stuff
<% end %>

gets effectively converted to:

form_for(@object) do
_buf << ("Stuff").to_s
end

On the other hand, this:

<%= form_for(@object) do %>
Stuff
<% end %>

gets converted to:

_buf << (form_for(@object) do).to_s
_buf << ("Stuff").to_s
end

which isn’t valid Ruby. So we use the first approach, and then let the helper itself, rather than ERB, be responsible for concatenating to the buffer. Sadly, it leads to significantly more complex helpers.

Let’s take a look at the implementation of content_tag.

def content_tag(name, content_or_options_with_block = nil, options = nil, escape = true, &block)
  if block_given?
    options = content_or_options_with_block if content_or_options_with_block.is_a?(Hash)
    content_tag = content_tag_string(name, capture(&block), options, escape)
 
    if block_called_from_erb?(block)
      concat(content_tag)
    else
      content_tag
    end
  else
    content_tag_string(name, content_or_options_with_block, options, escape)
  end
end

The important chunk here is the middle, inside of the if block_given? section. The first few lines just get the actual contents, using the capture helper to pull out the contents of the block. But then you get this:

if block_called_from_erb?(block)
  concat(content_tag)
else
  content_tag
end

This is actually a requirement for writing a block helper of any kind in Rails. First, Rails checks to see if the block is being called from ERB. If so, it takes care of concatenating to the buffer. Otherwise, the caller simply wants a String back, so it returns it.

Worse, here’s the implementation of block_called_from_erb?:

BLOCK_CALLED_FROM_ERB = 'defined? __in_erb_template'
 
# Check whether we're called from an erb template.
# We'd return a string in any other case, but erb <%= ... %>
# can't take an <% end %> later on, so we have to use <% ... %>
# and implicitly concat.
def block_called_from_erb?(block)
  block && eval(BLOCK_CALLED_FROM_ERB, block)
end

So every time you use a block helper in Rails, or use a helper which uses a block helper, Rails is forced to eval into the block to determine what the context is.

In Merb, we solved this problem by using this syntax:

<%= form_for(@object) do %>
Stuff
<% end =%>

And while everyone agrees that the opening <%= is a reasonable change, the closing =%> is a bit grating. However, it allows us to compile the above code into:

_buf << (form_for(@object) do
_buf << ("Stuff").to_s
end).to_s

That’s because we tag the end with a special ERB tag that allows us to attach a ).to_s to the end. We use Erubis, which lets us control the compilation process more finely, to hook into this process.

Rails 3 will use Erubis regardless of this problem to implement on-by-default XSS protection, but I needed a solution that didn’t require the closing =%> (ideally).

Evan (lead on Rubinius) hit upon a rather ingenious idea: use Ruby operator precedence to get around the need to know where the end was. Effectively, compile into the following:

_buf << capture_obj << form_for(@object) do
_buf << ("Stuff").to_s
end

where capture_obj is:

class CaptureObject
  def <<(obj)
    @object = obj
    self
  end
 
  def to_str
    @object.to_s
  end
 
  def to_s
    @object.to_s
  end
end

Unfortunately, with one hand Ruby operator precedence giveth, and with one hand it taketh away. In order to test this, I tried using a helper that returned an object, rather than a String (valid in ERB). In ERB, this would call to_s on the object. When I tried to run this code with the CaptureObject, I got:

template template:1:in `<<': can't convert Object into String (TypeError)
   from template template:1:in `template'
   from helper_spike.rb:48

Evan and I were both a bit baffled by this (although it retrospect we probably shouldn’t have been), and we hit on the idea to try running the code through Rubinius and look at its backtrace:

An exception occurred running helper_spike.rb
    Coercion error: #<Object:0x60a>.to_str => String failed:
(No method 'to_str' on an instance of Object.) (TypeError)
 
Backtrace:
                       Type.coerce_to at kernel/common/type.rb:22
           Kernel(String)#StringValue at kernel/common/kernel.rb:82
                            String#<< at kernel/common/string.rb:93
                   MyContext#template at template template:1
                      main.__script__ at helper_spike.rb:48

By looking at Rubinius’ backtrace, we quickly realized that the order of operations was wrong, and to_str was getting called on the return value from the helper, rather than the CaptureObject. As I tweeted immediately thereafter, the information available in Rubinius’ backtrace is just phenomenal, exposing enough information to really see what’s going on. Because the internals of Rubinius are written in Ruby, the Ruby backtrace goes all the way through to the Type.coerce_to method.

After realizing that, we changed the implementation of CaptureObject to take the buffer in its initializer, and have it handle concatenating to the buffer. The compiled code now looks like:

capture_obj << form_for(@object) do
_buf << ("Stuff").to_s
end

and the CaptureObject looks like:

class CaptureObject
  def initialize(buf)
    @buf = buf
  end
 
  def <<(obj)
    @buf << obj.to_s
  end
end

Now, Ruby’s operator precedence will bind the do to the form_for, and the return value of form_for will be to_s‘ed and concatenated to the buffer.

And the best thing is the implementation of content_tag once that’s done:

def content_tag(name, content = nil, options = nil, escape = true, &block)
  if block_given?
    options = content if content.is_a?(Hash)
    content = capture(&block)
  end
  content_tag_string(name, content, options, escape)
end

We can simply return a String and ERB handles the concatenation work. That’s the important part: helper writers should be able to think of block helpers the same way they think about traditional helpers. Somewhat less importantly, we’ll be able to eliminate evaling into untold numbers of blocks at runtime.

This was only an experiment, and the specific details still need to be worked out (how do we do this without breaking untold numbers of existing applications), I’m very happy with this solution, which provides the simplicity and performance enhancement of the Merb solution without the ugly =%>.

Blog status

2009-08-31T00:35:01+00:00

I had people asking me how come I was not blogging as much lately. Well, on top of being really busy, I have been blogging on other blogs such as the official Rails blog.

There aren’t a lot of Merb news, we are waiting for Carl and Yehuda to sign out on the 1.1 release and will hopefully soon start the migration work to Rails3. If you want more news about Rails3, check Yehuda’s blog and the official Rails blog. Lots of exciting things are coming up.

Finally, if you are planning on upgrading your Mac to Snow Leopard, these instructions are also valid for all Ruby developers on an Intel mac.

How to Build Sinatra on Rails 3

2009-08-27T19:13:28+00:00

In Ruby, we have the great fortune to have one major framework (Rails) and a number of minor frameworks that drive innovation forward. One of the great minor frameworks which has been getting a lot of traction recently is Sinatra, primarily because it exposes a great DSL for writing small, single-purpose apps.

Here’s an example of a simple Sinatra application.

class MyApp < Sinatra::Base
  set :views, File.dirname(__FILE__)
  enable :sessions
 
  before do
    halt if session[:fail] == true
  end
 
  get "/hello" do
    "Hello world"
  end
 
  get "/world" do
    @name = "Carl"
    erb :awesomesauce
  end
 
  get "/fail" do
    session[:fail] = true
    "You failed"
  end
end

There’s a lot of functionality packed into this little package. You can declare some code to be run before all actions, declare actions and the URL they should be routed from, use rendering semantics, and even use sessions.

We’ve been saying that Rails 3 is flexible enough to use as a framework toolkit–let’s prove it by using Rails to build the subset of the Sinatra DSL described above.

Let’s start with a very tiny subset of the DSL:

class MyApp < Sinatra::Base
  get "/hello" do
    "HELLO World"
  end
 
  post "/world" do
    "Hello WORLD"
  end
end

The first step is to declare the Sinatra base class:

module Sinatra
  class Base < ActionController::Metal
    include ActionController::RackConvenience
  end
end

We start off by making Sinatra::Base a subclass of the bare metal ActionController implementation, which provides just enough infrastructure to get going. We also include the RackConvenience module, which provides request and response and handles some basic Rack tasks for us.

Next, let’s add support for the GET and POST method:

class Sinatra::Base
  def self.inherited(klass)
    klass.class_eval { @_routes = [] }
  end
 
  class << self
    def get(uri, options = {}, &block)  route(:get,  uri, options, &block) end
    def post(uri, options = {}, &block) route(:post, uri, options, &block) end
 
    def route(http_method, uri, options, &block)
      action_name = "[#{http_method}] #{uri}"
      @_routes << {:method => http_method.to_s.upcase, :uri => uri,
                   :action => action_name, :options => options}
      define_method(action_name, &block)
    end
  end
end

We’ve simply defined some class methods on the Sinatra::Base to store off routing details for the get and post methods, and creating a new method named [GET] /hello. This is a bit of an interesting Ruby trick; while the def keyword has strict semantics for method names, define_method allows any string.

Now we need to wire up the actual routing. There are a number of options, including the Rails router (rack-mount, rack-router, and usher are all new, working Rails-like routers). We’ll use Usher, a fast Rails-like router written by Josh Hull.

class << Sinatra::Base
  def to_app
    routes, controller = @_routes, self
 
    Usher::Interface.for(:rack) do
      routes.each do |route|
        add(route[:uri], :conditions => {:method => route[:method]}.merge(route[:options])).
          to(controller.action(route[:action]))
      end
    end
  end
end

Here, we define to_app, which is used by Rack to convert a parameter to run into a valid Rack application. We create a new Usher interface, and add a route for each route created by Sinatra. Because Usher::Interface.for uses instance_eval for its DSL, we store off the routes and controller in local variables that will still be available in the closure.

One little detail here: In Rails 3, each action in a controller is a valid rack endpoint. You get the endpoint by doing ControllerName.action(method_name). Here, we’re simply pulling out the action named “[GET] /hello” that we created in route.

The final piece of the puzzle is covering the action processing in the controller itself. For this, we will mostly reuse the default action processing, with a small change:

class Sinatra::Base
  def process_action(*)
    self.response_body = super
  end
end

What’s happening here is that Rails does not treat the return value of the action as significant, instead expecting it to be set using render, but Sinatra treats the returned string as significant. As a result, we set the response_body to the return value of the action.

Next, let’s add session support.

class << Sinatra::Base
  def set(name, value)
    send("_set_#{name}", value)
  end
 
  def enable(name)
    set(name, true)
  end
 
  def _set_sessions(value)
    @_sessions = value
    include ActionController::Session if value
  end
 
  def to_app
    routes, controller = @_routes, self
 
    app = Usher::Interface.for(:rack) do
      routes.each do |route|
        add(route[:uri], :conditions => {:method => route[:method]}.merge(route[:options])).
          to(controller.action(route[:action]))
      end
    end
 
    if @_sessions
      app = ActionDispatch::Session::CookieStore.new(app, {:key => "_secret_key",
        :secret => Digest::SHA2.hexdigest(Time.now.to_s + rand(100).to_s)})
    end
 
    app
  end
end

There’s a few things going on here. First, Sinatra provides an API for setting options: set :option, :value. In Sinatra, enable :option is equivalent to set :option, true. To simplify adding new options, we just delegate set :whatever, value to a call to _set_whatever(value).

We then implement _set_sessions(value) to include ActionController::Session, which provides the session helper. In to_app, we wrap the original application in an ActionDispatch::Session::CookieStore if sessions were set.

Next, we want to add in support for callbacks (before do). It’s only a few lines:

class Sinatra::Base
  include AbstractController::Callbacks
end
 
class << Sinatra::Base
  alias before before_filter
end

Basically, we pull in the normal Rails callback code, and then rename before_filter to before and we’re good to go.

Finally, let’s dig into rendering.

class Sinatra::Base
  include ActionController::RenderingController
 
  def sinatra_render_file(name)
    render :template => name.to_s
  end
 
  def sinatra_render_inline(string, type)
    render :inline => string, :type => type
  end
 
  %w(haml erb builder).each do |type|
    define_method(type) do |thing|
      return sinatra_render_inline(thing, type) if thing.is_a?(String)
      return sinatra_render_file(thing)
    end
  end
end
 
class << Sinatra::Base
  alias _set_views append_view_path
end

We include the RenderController module, which provides rendering support. Sinatra supports a few different syntaxes for rendering. It supports erb :template_name which renders the ERB template named template_name. It also supports erb "Some String", which renders the string uses the ERB engine.

Rails supports both of those via render :template and render :inline, so we simply defer to that functionality in each case. We also handle Sinatra’s set :views, view_path by delegating to append_view_path.

You can check out the full repository at https://github.com/wycats/railsnatra/

So there you have it, a large subset of the Sinatra DSL written in Rails in under 100 lines of code. And if you want to add in more advanced Rails features, like layouts, flash, respond_to, file streaming, or conditional get support, it’s just a simple module inclusion away.

Why I Won't Be Eating a Dozen Krispy Kremes Today

2009-08-27T05:00:00+00:00

Tuesday night, in a discussion with Don, Jennifer, Shalini and Brad, I made the statement that I did not believe it would be that challenging to eat a dozen hot Krispy Kremes in a single sitting.

I stand by that assertion.

However, I will not be making any attempt to prove that I can do this. One dozen Krispy Kreme doughnuts contains 2400 calories and 144g of fat. While I believe it would not be difficult to consume this much, that doesn’t mean it’s a good idea. There are many things that I could do, and might even enjoy doing, that I do not do, because they’re not healthy or in my best interests.

If, in the future, I feel a desire to eat a dozen Krispy Kremes, I’ll document the experience.

My 10 Favorite Things About the Ruby Language

2009-08-24T16:57:44+00:00

I work with Ruby every single day, and over time have come to really enjoy using it. Here’s a list of some specific things that I really like about Ruby. Some of them are obvious, and some are shared with other languages. The purpose is to share things I like about Ruby, not to compare and contrast with any specific language.

1. Dynamic Typing

There are very good things about statically typed languages, such as compile-time verifiability and IDE support. However, in my experience, dynamic typing really helps get projects bootstrapped and smooths along changes, especially in the early to middle stages of a project.

I’m very happy that I don’t need to create a formal interface for my new objects to implement simply to enable me to easily swap out that class for another later on.

2. Duck Typing

This is effectively just an extension of Dynamic Typing. In Ruby, methods that expect to be able to operate on String objects don’t do checks for is_a?(String). They check whether the object respond_to?(:to_str) and then calls to_str on the Object if it does. Similarly, objects that represent Paths in Ruby can implement a to_path method to provide the path representation.

In Rails, we use this technique for objects that have “model” characteristics by expecting them to respond_to?(:to_model). This allows us to support any object in relevant contexts, provided those objects can supply us with a “model” representation of themselves.

3. Awesome Modules

Ruby provides a language feature similar to “traits” in Scala, Squeak, and Perl. Effectively, Ruby modules allow the dynamic addition of new elements of the class hierarchy at runtime. The use of super is dynamically evaluated at runtime to take into consideration any modules that might have been added, making it easy to extend functionality on a superclass as many times as desired, without being forced to decide where super will land at class declaration time.

Additionally, Ruby modules provide the lifecycle hooks append_features and included, which make it possible to use modules robustly to isolate extensions from one another and to dynamically extend classes on the basis of feature inclusion.

4. Class Bodies Aren’t Special

In Ruby, class bodies aren’t a special context. They’re simply a context where self points at the class object. If you’ve used Rails, you’ve probably seen code like this:

class Comment < ActiveRecord::Base
  validates_presence_of :post_id
end

It may look like validates_presence_of is a language feature, but it’s actually a method being called on Comment that is provided by ActiveRecord::Base.

That method can execute arbitrary code, also in the context of the class, including creating new methods, executing other pieces of code, or updating a class instance variable. Unlike Java annotations, which must be run at compile-time, Ruby class bodies can take runtime information into consideration, such as dynamically supplied options or the results of evaluating other code.

5. String Eval

This is likely a heresy. I’m not referring to arbitrary runtime String eval here, but rather String eval that is used to create methods early in the boot process of a Ruby application.

This can make it possible to take Ruby-defined structures, like Rails routes or AOP-definitions, and compile them into Ruby methods. Of course, it is possible to implement these things as add-ons to other languages, but Ruby makes it possible to implement these sorts of things in pure Ruby. It is, to a large degree, a self-hosting language.

6. Blocks and Lambdas

I’ve said this a few times and I’ll repeat myself: I don’t consider languages without anonymous lambdas to be powerful enough for me to use day-to-day. These constructs are actually extremely common, and found in languages as diverse as Ruby, JavaScript, Scala, Clojure, and of course Lisp.

They make it possible to implement block-scoped constructs that look like language features. The most common example usage is for File operations. In languages without lambdas, users are forced to use an inline “ensure” block every in the same lexical scope that they originally opened the file in, to ensure that the resource is closed.

In Java:

static void run(String in) 
throws FileNotFoundException {
  File input = new File(in);
  String line; Scanner reader = null;
  try {
    reader = new Scanner(input);
    while(reader.hasNextLine()) {
      System.out.println(reader.nextLine());
    }
  } finally { reader.close(); }
}

Among other things, the Java version needs to wrap the creation of the Scanner in a try block so it can be guaranteed to be closed. In contrast, the Ruby version:

def run(input)
  File.open(input, "r") do |f|
    f.each_line {|line| puts line }
  end
end

Because of the existence of blocks, it is possible to abstract away the need to close the File in a single location, minimizing programmer error and reducing duplication.

7. Combo Attack: Self-Hosting Language

The combination of several of the above features produce real-life examples of ways that we can “extend” the Ruby language in Rails. Consider the following:

  respond_to do |format|
    if @user.save
      flash[:notice] = 'User was successfully created.'
      format.html { redirect_to(@user) }
      format.xml { render :xml => @user, :status =>ted, :location => @user }
    else
      format.html { render :action => "new" }
      format.xml { render :xml => @user.errors, :status => :unprocessable_entity }
    end
  end

In this case, we’re able to seamlessly mix methods (respond_to) with normal Ruby code (if and else) to produce a new block-scoped construct. Ruby’s semantics for blocks allow us to return or yield from inside the block, further blending the boundaries of the code-block and language constructs like if or while.

In Rails 3, we introduced:

class PeopleController < ApplicationController
  respond_to :html, :xml, :json
 
  def index
    @people = Person.find(:all)
    respond_with(@people)
  end
end

Here, respond_to is provided at the class-level. It tells Rails that respond_with (in index) should accept HTML, XML, or JSON as response formats. If the user asked for a different format, we automatically return a 406 error (Not Acceptable).

If you dig in a bit deeper, you can see that the respond_to method is defined as:

def respond_to(*mimes)
  options = mimes.extract_options!
 
  only_actions   = Array(options.delete(:only))
  except_actions = Array(options.delete(:except))
 
  mimes.each do |mime|
    mime = mime.to_sym
    mimes_for_respond_to[mime]          = {}
    mimes_for_respond_to[mime][:only]   = only_actions   unless only_actions.empty?
    mimes_for_respond_to[mime][:except] = except_actions unless except_actions.empty?
  end
end

This method is defined on the ActionController::MimeResponds::ClassMethods module, which is pulled into ActionController::Base. Additionally, mimes_for_respond_to is defined using class_inheritable_reader in the included lifecycle hook for the module. The class_inheritable_reader method (macro?) uses class_eval to add methods onto the class in question to emulate the built-in attr_accessor functionality.

Understanding all of the details isn’t important. What’s important is that using the Ruby features we described above, it’s possible to create layers of abstraction that can appear to add features to the Ruby language.

A developer looking at ActionController::MimeResponds need not understand how class_inheritable_reader works–he just needs to understand the basic functionality. And a developer looking at the API documentation need not understand how the class-level respond_to is implemented–she just needs to understand the provided functionality. With that said, peeling back each layer leads to simple abstractions that build on other abstractions. There’s no need to peel back the whole curtain at once.

8. Nice Literals

I often forget about this when programming in Ruby, only to crash back down to earth when using a language with fewer, less expressive literals.

Ruby has literals for just about everything:

Strings: single-line, double-line, interpolated
Numbers: binary, octal, decimal, hex
Null: nil
Boolean: true, false
Arrays: [1,2], %w(each word is element)
Hashes: {key => value} and in Ruby 1.9 {key: value}
Regular expressions: /hello/, %r{hello/path}, %r{hello#{interpolated}}
Symbols: :name and :”weird string”
Block: { block literal }

And I think I’m missing some. While it may seem academic, good, readable literals can increase the programmer’s ability to write short but extremely expressive code. It’s of course possible to achieve the same sorts of things as you can with literal Hashes by instantiating a new Hash object and pushing the keys and values on one at a time, but it reduces their utility as method parameters, for instance.

The terseness of the Hash literal has allowed Ruby programmers to effectively add a limited keyword argument feature to the language without having to get approval by the language designers. Yet another small example of self-hosting.

9. Everything is an Object, and All Code is Executed and Has a `self`

I showed this to some degree earlier, but a lot of the reason that Class bodies work the way they do is a consequence of the unfailing object orientation of the Ruby language. Inside of a class body, Ruby is simply executing code with a self pointing at the class. Additionally, nothing is special about the class context; it is possible to evaluate code in a class’ context from any location. Consider:

module Util
  def self.evaluate(klass)
    klass.class_eval do
      def hello
        puts "#{self} says Hello!" 
      end
    end
  end
end
 
class PersonName < String
  Util.evaluate(self)
end

This is exactly equivalent to:

class PersonName < String
  def hello
    puts "#{self} says Hello!" 
  end
end

By removing the artificial boundaries between code in different locations, Ruby reduces the conceptual overhead of creating abstractions. And this is the result of a strong, consistent object model.

One more example on this topic. This idiom is quite common in Ruby: possibly_nil && possibly_nil.method_name. Since nil is just an object in Ruby, sending it a message it does not understand will result in a NoMethodError. Some developers suggested the following syntax: possibly_nil.try(:method_name). This can be implemented in Ruby as follows:

class Object
  alias_method :try, :__send__
end
 
class NilClass
  def try
    nil
  end
end

Essentially, this adds the method try to every Object. When the Object is nil, try simply returns nil. When the object is not nil, try just calls the method in question.

Using targeted application of Ruby’s open classes, combined with the fact that everything in Ruby, including nil, is an object, we were able to create a new Ruby feature. Again, this isn’t such a big deal, but it’s another case where the right choices in the language can allow us to create useful abstractions.

10. Rack

I’m going to cheat a little bit since Rack isn’t part of the Ruby language, but it does demonstrate some useful things about it. First of all, the Rack library only hit 1.0 earlier this year, and already every single Ruby web framework is Rack compliant. If you use a Ruby framework, you can be guaranteed that it uses Rack, and any standard Rack middleware will work.

This was all done without any backward compatibility sacrifices, a tribute to the flexibility of the Ruby language.

Rack itself also leverages Ruby features to do its work. The Rack API looks like this:

Rack::Builder.new do
  use Some::Middleware, param
  use Some::Other::Middleware
  run Application
end

In this brief code snippet, a number of things are at work. First, a block is passed to Rack::Builder. Second, that block is evaluated in the context of a new instance of Rack::Builder, which gives it access to the use and run methods. Third, the parameter passed to use and run is a class literal, which in Ruby is a simple object. This allows Rack to call passed_in_middleware.new(app, param), where new is just a method call on the Class object Some::Middleware.

And in case you think the code to implement that would be heinous, here it is:

class Rack::Builder
  def initialize(&block)
    @ins = []
    instance_eval(&block) if block_given?
  end
 
  def use(middleware, *args, &block)
    @ins << lambda { |app| middleware.new(app, *args, &block) }
  end
 
  def run(app)
    @ins << app #lambda { |nothing| app }
  end
end

This is all that’s needed to implement the code I showed above the creates a new Rack application. Instantiating the middleware chain is a simple affair as well:

def to_app
  inner_app = @ins.last
  @ins[0...-1].reverse_each { |app| inner_app = app.call(inner_app) }
  inner_app
end
 
def call(env)
  to_app.call(env)
end

First, we take the last element from the chain, which is our endpoint. We then loop over the remaining elements in reverse, instantiating each middleware with the next element in the chain, and return the resulting object.

Finally, we define a call method on the Builder, which is required by the Rack specifically, that calls to_app and passes the environment through, kicking off the chain.

Through the use of a number of the techniques described in the post, we were able to create a Rack-compliant application that supports Rack middleware in under two dozen lines of code.

Ruby, Rack and CouchDB = lots of awesomeness

2009-07-27T21:49:20+00:00

Over the weekend, I spent some time working on a Ruby + Rack +CouchDB project. Three technologies that I know quite well but that I never put to work together at the same time, at least not directly. Let’s call this Part I.

Before we get started, let me introduce each component:

Ruby : if you are reading this blog, you more than likely know at least a little bit about, what I consider, one of the most enjoyable programming language out there. It’s also a very flexible language that lets us do some interesting things. I could have chosen Python to do the same project but that’s a whole different topic. For this project we will do something Ruby excels at: reopening existing classes and injecting more code.
Rack: a webserver interface written in Ruby and inspired by Python’s WSGI. Basically, it’s a defined API to interact between webservers and web frameworks. It’s used by most common Ruby web frameworks, from Sinatra to Rails (btw, Rails3 is going to be even more Rack-focused than it already is). So, very simply put, the webserver receives a request, passes it to Rack, that converts it, passes it to your web framework and the web framework sends a response in the expected format (more on Rack later).
CouchDB: Apache’s document-oriented database. RESTful API, schema-less, written in Erlang with built-in support for map/reduce. For this project, I’m using CouchRest, a Ruby wrapper for Couch.

Goal: Log Couch requests and analyze data

Let’s say we have a Rails, Sinatra or Merb application and we are using CouchRest (maybe we are using CouchRest and ActiveRecord, but let’s ignore that for now).

Everything works fine but we would like to profile our app a little and maybe optimize the DB usage. The default framework loggers don’t support Couch. The easy way would be to tail the Couch logs or look at the logs in CouchDBX. Now, while that works, we can’t really see what DB calls are made per action, so it makes any optimization work a bit tedious. (Note that Rails3 will have some better conventions for logging, making things even easier)

So, let’s see how to fix that. Let’s start by looking at Rack.

Rack Middleware

Instead of hacking a web framework specific solution, let’s use Rack. Rack is dead simple, you just need to write a class that has a call method.
In our case, we don’t care about modifying the response, we just want to instrument our app. We just want our middleware to be transparent and let our webserver deal with it normally.

Here we go … that wasn’t hard, was it? We keep the application reference in the @app variable when a new instance of the middleware is created. Then when the middleware is called, we just call the rest of the chain and pretend nothing happened.

As you can see, we just added some logging info around the request. Let’s do one better and save the logs in CouchDB:

Again, nothing complicated. In our rackup file we defined which Couch database to use and we passed it to our middleware (we change our initialize method signature to take the DB).
Finally, instead of printing out the logs, we are saving them to the database.

W00t! At this point all our requests have been saved in the DB with all the data there, ready to be manipulated by some map/reduce views we will write. For the record, you might want to use the bulk_save approach in CouchDB which will wait for X amount of records to save them in the DB all at once. Couch also let’s you send new documents, but only save it to the DB every X documents or X seconds.

As you can see, our document contains the timestamps and the full environment as a hash.

All of that is nice, but even though we get a lot of information, we could not actually see any of the DB calls made in each request. Let’s fix that and inject our logger in CouchRest (you could apply the same approach to any adapter).

Let’s reopen the HTTP Abstraction layer class used by CouchRest and inject some instrumentation:

Again, nothing fancy, we are just opening the module, reopening the methods and wrapping our code around the super call (for those who don’t know, super calls the original method).

This is all for Part I. In Part II, we’ll see how to process the logs and make all that data useful.

By the way, if you make it to RailsSummit, I will be giving a talk on Rails3 and the new exciting stuff you will be able to do including Rack based stuff, CouchDB, MongoDB, new DataMapper etc..

Rubygems Good Practice

2009-07-24T14:40:49+00:00

Rubygems provides two things for the Ruby community.

A remote repository/packaging format and installer
A runtime dependency manager

The key to good rubygems practice is to treat these two elements of Rubygems as separate from each other. Someone might use the Rubygems packaging format and the Rubygems distribution but not want to use the Rubygems runtime.

And why should they? The Rubygems runtime is mainly responsible for setting up the appropriate load-paths, and if you are able to get the load paths set up correctly, why should you care about Rubygems at all?

In other words, you should write your libraries so that their only requirement is being in the load path. Users might then use Rubygems to get your library in the load path, or they might check it out of git and add it themselves.

It sounds pretty straight-forward but there are a few common pitfalls:

Using `gem` inside your gems

It’s reasonably common to see code like this inside of a gem:

gem "extlib", ">= 1.0.8"
require "extlib"

This should be entirely unnecessary. While using Kernel.gem in an application makes perfect sense, gems themselves should use their gem specification to provide dependent versions. When used with Rubygems, Rubygems will automatically add the appropriate dependencies to the load path. When not using Rubygems, the users can add the dependencies themselves.

Keep in mind that whether or not you use Rubygems, you can use require and it will do the right thing. If the file is in the load path (because you put it there or because Rubygems put it there), it will just work. If it’s not in the loadpath, Rubygems will look for a matching gem to add to the load path (by overriding require).

Rescuing from Gem::LoadError

This idiom is also reasonably common:

begin
  gem "my_gem", ">= 1.0.6"
  require "my_gem"
rescue Gem::LoadError
  # handle the error somehow
end

The right solution here is to avoid the gem call, as I said above, and rescue from plain LoadError. The Rubygems runtime sometimes raises Gem::LoadError, but that inherits from regular LoadError, so you’re free to rescue from that and catch cases with and without the rubygems runtime.

Conclusion

Declare you gem version dependencies in your gem specification and use simple requires in your library. If you need to catch the case where the dependency could not be found, rescue from LoadError.

And that’s all there is to it. Your library will work fine with or without the Rubygems runtime :)

Rails 3: The Great Decoupling

2009-07-20T07:06:58+00:00

In working on Rails 3 over the past 6 months, I have focsed rather extensively on decoupling components from each other.

Why should ActionController care whether it’s talking to ActionView or just something that duck-types like ActionView? Of course, the key to making this work well is to keep the interfaces between components as small as possible, so that implementing an ActionView lookalike is a matter of implementing just a few methods, not dozens.

While I was preparing for my talk at RubyKaigi, I was trying to find the smallest possible examples that demonstrate some of this stuff. It went really well, but I noticed a few areas that could be improved even further, producing an even more compelling demonstration.

This weekend, I focused on cleaning up those interfaces, so we have small and clearly documented mechanisms for interfacing with Rails components. I want to focus on ActionView in this post, which I’ll demonstrate with an example.

$:.push "rails/activesupport/lib"
$:.push "rails/actionpack/lib"
 
require "action_controller"
 
class Kaigi < ActionController::Http
  include AbstractController::Callbacks
  include ActionController::RackConvenience
  include ActionController::Renderer
  include ActionController::Layouts
  include ActionView::Context
 
  before_filter :set_name
  append_view_path "views"
 
  def _action_view
    self
  end
 
  def controller
    self
  end
 
  DEFAULT_LAYOUT = Object.new.tap {|l| def l.render(*) yield end }
 
  def _render_template_from_controller(template, layout = DEFAULT_LAYOUT, options = {}, partial = false)
    ret = template.render(self, {})
    layout.render(self, {}) { ret }
  end
 
  def index
    render :template => "template"
  end
 
  def alt
    render :template => "template", :layout => "alt"
  end
 
  private
  def set_name
    @name = params[:name]
  end
end
 
app = Rack::Builder.new do
  map("/kaigi") {  run Kaigi.action(:index) }
  map("/kaigi/alt") { run Kaigi.action(:alt) }
end.to_app
 
Rack::Handler::Mongrel.run app, :Port => 3000

There’s a bunch going on here, but the important thing is that you can run this file with just ruby, and it’ll serve up /kaigi and /kaigi/alt. It will serve templates from the local “/views” directory, and correctly handle before filters just fine.

Let’s look at this a piece at a time:

$:.push "rails/activesupport/lib"
$:.push "rails/actionpack/lib"
 
require "action_controller"

This is just boilerplace. I symlinked rails to a directory under this file and required action_controller. Note that simply requiring ActionController is extremely cheap — no features have been used yet

class Kaigi < ActionController::Http
  include AbstractController::Callbacks
  include ActionController::RackConvenience
  include ActionController::Renderer
  include ActionController::Layouts
  include ActionView::Context
end

I inherited my class from ActionController::Http. I then included a number of features, include Rack convenience methods (request/response), the Renderer, and Layouts. I also made the controller itself the view context. I will discuss this more in just a moment.

  before_filter :set_name

This is the normal Rail before_filter. I didn’t need to do anything else to get this functionality other than include AbstractController::Callbacks

  append_view_path "views"

Because we’re not in a Rails app, our view paths haven’t been pre-populated. No problem: it’s just a one-liner to set them ourselves.

The next part is the interesting part. In Rails 3, while ActionView::Base remains the default view context, the interface between ActionController and ActionView is extremely well defined. Specifically:

A view context must include ActionView::Context. This just adds the compiled templates, so they can be called from the context
A view context must provide a _render_template_from_controller method, which takes a template object, a layout, and additional options
A view context may optionally also provide a _render_partial_from_controller, to handle render :partial => @some_object
In order to use ActionView::Helpers, a view context must have a pointer back to its original controller

That’s it! That’s the entire ActionController=>ActionView interface.

  def _action_view
    self
  end
 
  def controller
    self
  end

Here, we specify that the view context is just self, and define controller, required by view contexts. Effectively, we have merged the controller and view context (mainly just to see if it could be done ;) )

  DEFAULT_LAYOUT = Object.new.tap {|l| def l.render(*) yield end }

Next, we make a default layout. This is just a simple proc that provides a render method that yields to the block. It will simplify:

  def _render_template_from_controller(template, layout = DEFAULT_LAYOUT, options = {}, partial = false)
    ret = template.render(self, {})
    layout.render(self, {}) { ret }
  end

Here, we supply the required _render_template_from_controller. The template object that is passed in is a standard Rails Template which has a render method on it. That method takes the view context and any locals. For this example, we pass in self as the view context, and do not provide any locals. Next, we call render on the layout, passing in the return value of template.render. The reason we created a default is to make the case of a layout identical to the case without.

  def index
    render :template => "template"
  end
 
  def alt
    render :template => "template", :layout => "alt"
  end
 
  private
  def set_name
    @name = params[:name]
  end

This is a standard Rails controller.

app = Rack::Builder.new do
  map("/kaigi") {  run Kaigi.action(:index) }
  map("/kaigi/alt") { run Kaigi.action(:alt) }
end.to_app
 
Rack::Handler::Mongrel.run app, :Port => 3000

Finally, rather than use the Rails router, we just wire the controller up directly using Rack. In Rails 3, ControllerName.action(:action_name) returns a rack-compatible endpoint, so we can wire them up directly.

And that’s all there is to it!

Note: I’m not sure if I still need to say this, but stuff like this is purely a demonstration of the power of the internals, and does not reflect changes to the public API or the way people use Rails by default. Everyone on the Rails team is strongly committed to retaining the same excellent startup experience and set of good conventional defaults. That will not be changing in 3.0.

getting started with ruby 1.9

2009-07-18T14:19:19+00:00

Yehuda Katz has a nice post over at his blog talking about Ruby 1.9 and what we need to do to move our community towards it.

Lots of great points are brought up, but for me, the biggest issue is just not having the time to fiddle with things, setup nice little aliases, etc. Thankfully, one of the nice guys over at Relevance has done this for us: spicy-config

On a quick side note, Chad's configs also helped me get up and running with zsh, and that's been a whole other fun trip.

You'll want to clone his repo and take a look in at his .zshrc file along with the ruby_installer and ruby_switcher files inside .zsh. All you have to do is source the files to get the functions defined, and run what you're interested in. I did set things up a little different given that I don't use the Leopard install of ruby...but if you're reading this, I assume you can see what you'd want to change if need be.

That's about it. I can now say "use_ruby_191" and it quickly switches me over, and then 'use_ruby' to jump back to 1.8.6.

Great stuff Chad, thanks!

What do we need to get on Ruby 1.9?

2009-07-18T04:57:22+00:00

A year ago, I was very skeptical of Ruby 1.9. There were a lot of changes in it, and it seemed like it was going to be a mammoth job to get things running on it. The benefits did not seem to outweigh the costs of switching, especially since Ruby 1.9 was not yet adequately stable to justify the big switch.

At this point, however, it seems as though Ruby 1.9 has stabilized (with 1.9.2 on the horizon), and there are some benefits that seem to obviously justify a switch (such as fast, integrated I18n, better performance in general, blocks that can have default arguments and take blocks, etc.).

Perhaps more importantly though, Ruby’s language implementors have shifted their focus to Ruby 1.9. It has become increasingly difficult to get enhancements in Ruby 1.8, because it is no longer trunk Ruby. Getting community momentum behind Ruby 1.9 would enable us to make productive suggestions to Matz and the other language implementors. Instead, we seem to get a new monthly patch fixing Ruby 1.8.

So my question is: what do we as a community need to shift momentum to 1.9. I’m don’t want a generic answer, like “we need to feel good about it”. I’m asking you what is stopping you today from using Ruby 1.9 for your next project. Is there a library that doesn’t work? Is there a new language feature that causes so much disruption to your existing programming patterns to make a switch untenable?

I suspect that we are all just comfortable in Ruby 1.8, but would actually be mostly fine upgrading to Ruby 1.9. I also suspect that there are small issues I’m not personally aware of, but which have blocked some of you from upgrading. Rails 2.3 and 3.0 (edge) work fine on Ruby 1.9, and I’d like to see what we can do to make Ruby 1.9 a good recommended option for new projects.

Thoughts?

a couple quick git tips

2009-07-16T20:59:43+00:00

The other day someone asked in #git how to fix something in the middle of an unsuccessful rebase. So, I thought I'd rebase some of my own code, reorder things so there must be a conflict, and then try to duplicate their question.

Only problem was that there was no error, and I hadn't really paid attention to which commits I reordered. Thankfully, I got a bit of help from a gent named doener.

I'm including a rather long gist showing the process. Basically, git reflog is your friend, and you should play with this at some point in a test repo like I've done here.

Note: anytime you git reset --hard be very sure and confident of what you're doing.

Ruby authentication of CouchDB requests

2009-07-13T21:00:13+00:00

CouchDB is an awesome technology. I’m lucky enough to work on quite a big project where we decided to switch from MySQL to Couch for various reasons.

One of the many things I like with Couch is that it handles attachments and can replicate them as well as serve them for you using the Erlang based builtin webserver. (you can load balance your dbs and do some other really cool stuff)

Let’s take a use case. Let’s imagine that you have a web app with logged in users. Every user can have their own avatar.

No big deal, you get the user to upload his/her avatar to your app and add it to the user document in the database. To serve it from the database, you just need to create a proxy in nginx/apache and redirect the virtual avatar url to the protected DB making sure the request is a GET request.

Add to that a caching solution like varnish or memcached module for nginx and all your db goodies get cached and served by the cache (server/client) until they get modified.

Now, the problem is when you want to serve authorized attachments. Let’s imagine that we want to let our users upload private files, files that should be accessible only by the owner or users designated by the owner.

In this case, a simple nginx rewrite wouldn’t work. We need to authorize attachment requests. Here is a cool way of doing that using nginx and merb’s router. (Expect Rails3 router to do the same).

Let’s start by setting up nginx and create a proxy for couchdb:

Now that this is done, we are going to use Merb’s awesome router to handle the incoming requests. The cool part of this is that we won’t be dispatching requests so, going through the router is almost free. (check on the Merb router benchmarks for more info). Let’s edit our router and set a special route for our assets.

We are using a deferred route which gets executed instead of dispatching the request.

If the attachment route is being matched then we are checking what environment we are currently running in. If we are in production or staging environment then we are sending back a rack response to the webserver. The response is just a forward to the proper couchdb document behind the proxy. Of course, before allowing that to happen, we could authenticate the logged in user, log the request and do a couple of other things. You have full access to your models from the router, so authenticating a session isn’t a big deal. You could even create temporarily urls like AWS s3 does.

If we are not in production or staging mode, then just redirect the request to couch since we assume you have access to the local db. This way, your asset urls will be working in production and dev. In real life, you’ll want to apply the authorization before choosing how to deliver the document/attachment tho as you want it work the same way in development and production.

Planet Merb

MerbAdmin Installation and Overview

Learn About Merb Dependencies and the new Bundler!

Ruby’s Implementation Does Not Define its Semantics

Blocks vs. Procs

So what’s the & thing for?

Mental Models

Metaclasses

Mental Models

Regular Expression Matches

Mental Models

Conclusion

Ruby is NOT a Callable Oriented Language (It’s Object Oriented)

Designing for the Common Case: Calling Methods

Designing for the Common Case: How Blocks Are Really Used

Consistent Method Execution

AbstractQueryFactoryFactories and alias_method_chain: The Ruby Way

The Case of the PerQueryTimingFactoryFactory

The Case: Solved

Postscript: Scala

The Blind Men and the Elephant: A Story of Noobs

Using Bundler in Real Life

You Get a Repo for the First Time

You Update a Repo Using Bundler

You have created a new Rails application

You are ready to share your new Rails application

You have a locked application, and wish to add a new dependency

You want a self-contained application

Conclusion

The Building Blocks of Ruby

Cut to the Chase

So Why Are Ruby’s Blocks Better?

Postscript

notes from yet another rails 3 upgrade

How and why I joined the “suit people”

Why ‘o why?

Corporate America? Are you going to write Java now?

Conclusion

Undo/Redo in MacRuby

Bundler 0.9: Heading Toward 1.0

The Cache

Runtime

Locking

New Commands

bundle pack

bundle lock

bundle install

bundle check

bundle exec …

Environments

SafeBuffers and Rails 3.0

SafeBuffer

Summary

Corporations Cannot Have Natural Rights… Duh

MacRuby 0.5 final is out

How I use Foursquare

The Irony of the iPad: A GREAT Day for Open Technologies

How to detect Cylons with MacRuby

Introducing MacRuby’s debugger!

Controlling iTunes with MacRuby

bridgesupport file

iTunes Documentation

jQuery 1.4 and Malformed JSON

ActiveModel: Make Any Ruby Object Feel Like ActiveRecord

Before I Begin, The ActiveModel API

ActiveModel Modules

The Validations System

Serialization

Others

Amazon Kindle 2

Kindle vs Nook

Kindle vs iTablet

Kindle and tech books/documentation.

The Maximal Usage Doctrine for Open Source

The Playing Field

What I Want When I Write Open Source

Meeting Those Requirements

Lower Barrier to Entry Helps Meet Open Source Goals

Postscript: Linux

The Craziest F***ing Bug I’ve Ever Seen

So what’s the `&` thing for?

9. Everything is an Object, and All Code is Executed and Has a `self`