How do you evaluate a new language for use in a given project or team? Sometimes you have a particular framework or platform that dictates your choices. If you are developing an application for iOS you probably want to learn Objective-C (I know there are other options, but you get the point). If you really want to try Django, you’re going to learn Python. In those cases things are really simple.
But what if they’re not? What if you want to try out a new language or framework just because you believe, like me, that regular change creates new insights, relieves boredom, and helps a team learn important new methods and skills. Or what if you are finding it difficult to hire people who want to use Delphi, or PL/1 or whatever, and you need to switch to something new before all your old programmers retire? How do you decide then?
This is the situation I found myself in. We have a number of new projects coming up in the next 24 months and I wanted to get some new ideas and methods flowing through the team. In addition, we have a lot of legacy code written in Perl, and regardless of how you feel about Perl as a language, one thing is absolutely true: finding people in Victoria who want to program in Perl is extremely difficult.
I have done a lot of reading and experimenting with Scala and have come to like it. I thought it might be a good choice for our team. But I didn’t want to dictate something as personal as programming-religion (aka. language choice). So I thought we’d try something a little more data-driven.
About three months ago I had everyone on the team suggest the languages they’d like us to evaluate. Fortunately, the resulting list had only ten choices, which felt like a reasonable upper-limit. We created a spreadsheet with a set of evaluation criteria and assigned team members to research and score one or two languages in each category.
Probably your first question is: what languages did we evaluate?
For each language we gave it a score from 1 to 5 (higher is better) in the following categories:
- License (would it work for our business?)
- Build Tools
- Documentation Automation tools
- Patch/Fix techniques (do you have to redeploy everything, or just individual files?)
- Dependency Management Tools
- Scalability / Cluster support (anything special here?)
- Developer productivity (based on whatever metrics we could find online)
- Test frameworks
- WebUI Development frameworks
- REST development frameworks
- Cool language features
- Easy support for Multicore machines
- Ease of learning
- SQL, NoSQL, and ORM support
- AWS Support (existing libraries?)
- Ease of recruitment
- Size and health of the surrounding community
We did our best to score each language and add everything up. Obviously this exercise is not free from bias. The scores depended on the person doing the research, which resources they used, how they felt about type systems and language flavours, etc. However, once all the scores were in, we sat down as a team and evaluated the evaluations. Some marks were adjusted, but many were not.
I think it’s pretty easy to say that any other team, even if the followed the same methods, would almost certainly end up with different results. A concept like Developer Productivity, or Ease of Learning, is heavily influenced by your team’s background and preferences.
Despite all the areas where our methods could be questioned, the activity was useful and the results had value for us. The winner, Scala, is where we decided to start. Details of how that worked out will be in my next post.