So, my post on monads apparently set of a bit of a firestorm over my comments about avoiding null pointer exceptions. To show you what I mean, here’s a link to one of the more polite and reasonable posts.
Ok. So. Let me expand a bit.
The reason that I think it’s so great that I don’t get NPEs when I use an option with something like Option isn’t because it makes me a super-programmer who’s better than the lowly slime who deal with NPEs. It’s because it changes how that I write code, in a way that helps me avoid making mistakes – mistakes that I make because I’m just a fallible idiot of a human.
There are two fundamental things about an option type, like what we have in Scala, that make a huge difference.
First, it narrows the field of errors. When I’m programming in Java, any call that returns a pointer could return a null. The language makes no distinction between a function/expression that could return a null, and one that can’t. That means that when I get an NPE, the source of that null pointer could be anything in the dynamic slice leading to the error. With an option type, I’ve got two kinds of functions: functions that always return a non-null value, and functions that sometimes return a non-null value, and sometimes return a None. That’s incredibly valuable.
Second, it forces me to explicitly deal with the None case. In Java, programmers constantly build code without null checks, because they know that a function won’t return null. And then it does, and ker-splat. With an option type, I have no choice: I have to explicitly deal with the potential error case. Sure, I can forcibly code around it – in Scala, I can use Option.get, which will turn into an error analagous to an NPE. But it forces me to make that choice, and make it explicitly.
Even if I’m going the stupid, brute-force route and assuming that I know, without fail, that a function is going to return a non-null value… Consider an example:
Java: : T g = f.doSomething() g.doSomethingElse() Scala: val g: Option[T] = f.doSomething() g.get.doSomethingElse()
The scala case has to explicitly deal with the fact that it’s dealing with a potentially empty value, and using a statement that asserts the non-emptiness.
But in reality, if you’re a decent programmer, you never use .get to directly access an option. (The only exception is in cases where you call the .get in a context dominated by a non-empty test; but even then, it’s best to not, to avoid errors when the surrounding code is modified.) In real code, you pretty much always explicitly de-option a value using a function like getOrElse:
val f: User = getUser("markcc").getOrElse(new User("markcc"))
As I hope it has become plain, the point of avoiding NPEs through option-like type structures isn’t that somehow it makes the entire category of unexpected result value disappear. It’s that it changes the way that you code to distinguish where those errors can occur, and to force you to deal with them.
I think that ultimately, things like this are really just manifestations of the good-old static vs dynamic type wars. Type errors in a dynamically typed language are really just unexpected value errors. Strong typing doesn’t stop you from making those errors. It just takes a bunch of common cases of those errors, and converts them from a run-time error to a compile-time error. Whether you want them to be run-time or compile-time depends on the project your working on, on your development team, and on your personal preferences.
I find in practice that I get many fewer errors by being forced to explicitly declare when a value might be null/None, and by being required to explicitly deal with the null/None case when it might occur. I’ve spent much less time debugging that kind of error in my year at foursquare than in the 15 years of professional development that I did before. That’s not because I magically became a better programmer a year ago when I joined foursquare. It’s because I’m using a better tool that helps me avoid mistakes.
The other way I find Option[T] helpful is that when the answer is None, I know the it’s not a T, while null can be anynothing.
If that makes sense.
the=>that in previous
We introduced an Optional type in Guava almost a year ago: http://docs.guava-libraries.googlecode.com/git/javadoc/com/google/common/base/Optional.html
Thoughts?
Not sure what kinds of thoughts you want.
Initial reaction: hurrah! It’s a valuable thing. There’s two problems as I see it.
First, it’s not a monad – you don’t have anything like flatMap. That really limits it.
Second, until the next version of Java with lambda comes out, the syntax of using it is going to be very painful. Not your fault obviously, but still a problem.
It sounds a bit like the Java feature of making possible exceptions be part of the interface to a function:
int foo() throws BarException { … }
any code that calls foo either has to handle explicitly BarExceptions:
void baz() {
try { foo() ; }
catch (BarException) { //fail gracefully }
}
or declare that it passes the buck:
void quux() throws BarException {
foo();
}
(naively) calling foo like this:
void floo() {
foo();
}
is a compile time error.
Of course, explicitly catching/forwarding NullPointerExceptions is optional in Java because it would be a pain in the patooty to have to explicitly deal with them when most of the time you “know” that it isn’t going to be a problem.
This is true, stigant. The really neat thing about Option[T] is that it has a .map() method, which makes it a functor!
That is, because Scala allows first-class functions — specifically, functions as arguments — you can write a method on the Option class that handles all of the “most of the time” behavior you mention in a uniform — category theorists would say “natural” — way.
I think this all boils down to case analysis. When you write a program, do you want to think about and handle all cases? Or do you just want to handle the expected case, and have it blow up otherwise?
Generally, I’m pro-case analysis – I like it when the compiler tells me I forgot to handle a case. However, it is admittedly annoying when you’re just trying to throw a demo together, or you’re writing a script for yourself and you can fix corner cases after they actually happen. I think a lot of folks (particularly those writing in scripting languages) want to start out just implementing the happy path and leave the rest until “later”, because if you throw out the code before a case actually arises, later may never come. If we’re honest we should admit there are times when that’s appropriate.
Java is a bit of an odd duck – a lot of verbosity and error-checking, and yet it doesn’t actually solve case analysis much. But null pointer exceptions seem fine for actual scripting languages.
Based on my career so far, I think the best compromise is one where I have to explicitly ignore all the other cases, leaving a marker in the code that indicates that I didn’t think about it hard. “fromJust” in Haskell’s Data.Maybe module is a good example of this; the only way to ignore the possibility of a Nothing is to use a partial function that’s easy to search for.
In this sense, traditional nulls are dangerous, because there’s no easy way to distinguish three cases in my code:
1. I intended to ignore the possibility of null – I’m just prototyping, and if it happens, it’s no big deal.
2. I have shown that the value can’t be null at this point. This could be (e.g.) because I’ve tested it for nullness earlier, or because I’ve got some other proof of not-null in hand.
3. I’ve made a mistake; I ought to be checking for null, but I haven’t yet done so.
Something like an Option type, with a “no case analysis” function like fromJust gets me all three safely; in case 1, I use something like fromJust, and a later grep will tell me which bits of my prototype need modifying to harden them for production. In case 2, I’ve done my case analysis with something like getOrElse, and the compiler will ensure I never get an unexpected null. In case 3, I can’t compile, I get an error telling me to convert the code to case 1 or case 2.
Isn’t flatMap the same think as http://docs.guava-libraries.googlecode.com/git/javadoc/com/google/common/collect/FluentIterable.html#transformAndConcat(com.google.common.base.Function) ?
It takes time to get into the mindset of using Option when you’ve used nulls for decades. When I first started writing Scala code I was skeptical about Option. And I would get NGEs (None.get exceptions) at about the same rate that I got NPEs in Java. That’s because I was treating Option the same way I treated nulls before. Now I almost never get NGEs because I’ve learned how to use Option properly.
And I’ve really come to appreciate Option because, as John Armstrong said, Option is a functor! (And it’s a Monad). Which means that I have a nice API for working with “null” values. So besides imperative matching you can deal with “no values” functionally using the methods on Option and for comprehensions. Also the Option cheatsheet is a great resource for beginners: http://dibblego.wordpress.com/2008/01/16/scalaoption-cheat-sheet/.