Just got back from OSCON 2011. There were some really interesting talks, and “the big three” were all over the place: Google, Twitter, and Facebook. I attended the Twitter talk on how they moved from Ruby to the JVM, as well a talk by Rob Pike (of Google) on Go. I use Twitter and Google a lot, and enjoyed hearing about how they work (Facebook, on the other hand, I use begrudgingly).

My first reaction to these talks was “Wow, it sure sounds like a lot of fun to work at these places!” From inventing programming languages to using Scala to scale a massive architecture, it sounds like developer shangri-la. Until recently, I was slinging Spring MVC code in vanilla Java. Sucks for me, right?

This is lazy thinking, and I haven’t fallen victim to it in quite a while. Ask yourself: “Why would Google spend the time and money to get Rob Pike to work for them and invent an entirely new programming language?” Ask: “Why would Twitter spend the time and money to design and deploy an elegant, scalable architecture using the JVM and Scala?”

An awesome new programming language, or an impressive, scalable architecture is not an end to itself. These are tools that serve some other purpose.

In the case of Google, Twitter, and Facebook, that purpose is to serve ads.

Although Steve Yegge’s keynote features him “quitting” his project at Google (which he describes as “sharing cat pictures”) to work on some higher calling (still within Google), he’s just trading one ad-serving system for another.

Google+ isn’t about sharing cat pictures, it’s about serving ads. Twitter’s massive network of 140-character bits of information isn’t about connecting people across the globe or to view current trends in worldwide thinking, it’s about serving ads. Facebook isn’t about entertaining yourself with games or sharing interesting links, it’s about serving ads.

So, why are the best and brightest in the industry busting their asses to work at these places? If all you care about is watching performance metrics increase, or playing with something new, or just watching lots of tests pass, I guess it’s nice to get paid well. But, how can this be truly fulfilling? For me, it wouldn’t be. I need the tools I’m building, the products I work on, and the architecture I’m helping to create to all serve something I’m interested in.

I’m not saying that my code needs to save the world or support humanitarian causes, but it should at least be something that I like, or I would use, or that I care about.

And I can’t fucking abide advertisements. Honestly, I don’t know anyone who does. Except, apparently, a lot of really smart software engineers.

In Java, I almost never make type errors. The type errors that are possible in Java fall roughly into two categories:

1. Using an object of the wrong type (this is caught by the compiler)
2. Casting an object to a more specific type at runtime (this can only be caught by tests or users :)

I’d make error #1 on occasion, but the compiler catches it. As to #2, before generics, I can count on my hands the number of times I got a bona-fide ClassCastException at runtime. After generics? Never.

I don’t mean just that I didn’t experience these runtime type errors, but that they didn’t even make it to the compiler. If you think about how Java syntax works, it’s no wonder:

List<String> someList = new ArrayList<String>();
someList.add("Foo");
someList.add("Bar");

I had to type almost the exact same thing twice. After about 2 days of using generics, my muscle memory literally prevents me from making type errors. To even simulate one requires a pre-generics library call, or some herculean efforts. An arguable win for static typing, if at the price of verbosity.

Of course, in Ruby, I make type errors all the time, especially when using new libraries I’m not familiar with. Ruby libraries rarely document the types of things (though they are frequently liberal with what they will accept). The solution here is just better unit tests. And that’s a pretty good thing. So, a slight negative for dynamic typing that leads us to a better tested system, reduced verbosity, and better productivity once the learning curve is dealth with.

This pretty well illustrates the tradeoffs between dynamic and static typing. Case closed, right?

Enter Scala. With Scala, I make type errors just as much as I do with Ruby. The only difference is that the compiler catches them. Here’s the Scala equivalent to the Java code above

var list = List("foo","bar")

Notce how I haven’t specified a single type? It’s nearly identical to the Ruby version:

list = ["foo","bar"]

These examples are obviously simplistic, but in a more complex system, Scala’s type inferencer tends to be one step ahead of me. While it’s handy that I have a compiler to catch these type errors, the fact remains that, despite Scala being a statically typed language, I’m making far more type errors than I would in Java.

This seems kindof odd, but I think it’s ultimately a win: I get the brevity and productivity of a dynamically typed language, but the safety of the compiler catching my type errors for me.

Scala puts a subtle spin on the “static vs. dynamic” debate, because you aren’t annotating your types nearly as much as with Java, but you still get all the benefits. I’ve certainly heard many criticisims of static typing, but having the compiler check types for you wasn’t one of them.

Of course, sometimes you do need to tell Scala what your types are, but, they seem to be exactly where you’d want them anyway:

/** get users with the given name and age */
def getUsers(name:String,age:Option[Int]):List[User]

This says that we take a requires String and an optional Int and will return a list of User objects. To give the same information in Ruby, you’d need to:

# Gets the users with the given name and age
#
# name - a String
# age - an Int or nil if not searching by age
#
# Returns an array of User objects
def get_users(name,age=nil)
    ...
end

(Sure, you could leave off the comment, but do you really hate your fellow developers (*and* future you) that much?)

Now which language is more verbose? Perhaps the static/dynamic typing debate is really just about entering text?

In a recent blog entry, Cédric Beust calls out scala’s Option class as nothing more significant than null checks. Commenters rightly set him straight that the thesis of his blog post was marred by an ignorance of idiomatic use of the class.

But, it’s hard to really blame Cédric, when you look at what he had to go on. Odersky’s book states that one should use pattern matching with Option, and the scaladoc for Option is just abysmal:

This class represents optional values. Instances of Option are either instances of case class Some or it is case object None.

That is the entire description of the class, no examples, nothing. Worse, the method that commenters called out as idiomatic, flatMap, has the following description:

If the option is nonempty, return a function applied to its value. Otherwise return None.

This is not what Option#flatMap actually appears to do (nor is documented to do; it’s documented to return an Option[B]!):

scala> val s:Option[String] = Some("foo")
scala> val n:Option[String] = None
scala> val f = { (x:String) => Some(x + "bar") }
scala> s.flatMap(f)
Some(foobar)
scala> n.flatMap(f)
None
scala> val result = s.flatMap(f)
r: Option[java.lang.String] = Some(foobar)
scala> r.getClass
res6: java.lang.Class[_] = class scala.Some

It certainly doesn’t call itself out as “the way” to use Option. A simple example in the scaladoc could have gone a long way.

So, angry post + helpful commenters == problem solved, right?

Wrong.

The Option class is, really, an implementation of the NullObject pattern, and a more elegant way to handle optional values. In scala, we might have this method signature:

/**
 * Updates the full name
 * @param lastName the last name
 * @param firstName the first name
 */
def updateName(lastName:String, firstName:Option[String])

This means “update my name; lastName is required and firstName is optional”. In java, this method might look like this:

/**
 * Updates the full name
 * @param lastName the last name, may not be null
 * @param firstName the first name, may be null
 */
public void updateName(String lastName, String firstName) {
    if (lastName == null) {
        throw new IllegalArgumentException("lastName required");
    }
    StringBuffer b = new StringBuffer(lastName);
    if (firstName != null) {
        b.append(", ");
        b.append(firstName);
    }
    this.fullName = b.toString();
}

So, what’s the right way to do it in Scala? According to the commentors:

def updateName(lastName:String, firstName:Option[String]):Unit = {
  val b = new StringBuffer(lastName)
  firstName.foreach( (name) => b.append(", "); b.append(firstName) )
}

Yech. Does anyone else think that calling a method called “foreach” on our “optional value” is just nonsensical? Or that the idiomatic way to treat an optional value is as a collection, e.g. by using the for comprehension? This just feels hacky. Naming is one of the most important (and challenging) things in software engineering, and Option’s API is an utter failure (even its name is wrong; when one has an option, one typicaly has many choices, not just one or nothing. Optional is really what is meant here, so why are we afraid of adding a few more letters? Especially given how “precise” some of the documentation is, mathematically speaking, why are we not being precise with English?). If Option is just shorthand for a “list of zero or one elements”, and we get no better methods than what comes with List, then what’s even the point of the class?

I’m not saying we remove all the collection methods from Option, but how about a throwing us a bone to make our code readable and learnable without running to the scaladoc (or REPL) to see what’s going on? I mean, there’s a method on Option called withFilter whose documented purpose (I’m not making this up) is: “Necessary to keep Option from being implicitly converted to Iterable in for comprehensions”. Am I expected to believe that it’s ok to have this hacky pile of cruft, but we can’t get a readable method for “do something to the contents if they are there”?

class Option[A] { 
  def ifValue[U]( f: (A) => U ):Unit = foreach(f)
  def unlessValue[U]( f: () => U):Unit = if (self.isEmpty) f
}

def updateName(lastName:String, firstName:Option[String]):Unit = {
  val b = new StringBuffer(lastName)
  firstName.ifValue( (name) => b.append(", "); b.append(firstName) )
}

Which would be less surprising? Couple this with some better scaladoc:

/** This class represents an optional value.  
 *
 * To use as a null object:
 * val optional = getSomePossiblyOptionalValue
 * <pre>
 * option.ifValue { (actualValue) => 
 *   // do things with the value, if it was there
 * }
 * </pre>
 * or
 * <pre>
 * optional.unlessValue { log.debug("missing optional value") }
 * </pre>
 * 
 * To use as a Monad or collection:
 * <pre>
 * val first8upper = option.flatMap( (y) => Some(y.toUpperCase) ).
 *                          flatMap( (y) => Some(y.substring(0,8)) )
 * </pre>
 */
class Option[T] {
  // etc.
}

With these examples, you cover the two main uses of this class, show newcomers how to use it, and demonstrate its superiority over null checks.

It’s frustrating to see this, because Scala has so much potential to be a lucid, accessible, readable language, but API usability and learnability are just not prioties. Scala needs to take some lessons from Ruby in terms of API design (and Java in terms of API documentation).

Of course, none of this does save you from null, because Scala is perfectly happy to assign null to anything. It kinda makes the whole thing seem a bit pointless.