Better than Scala but too hard to learn and hire for
November 07, 2019
Better than Scala but too hard to learn and hire for
Engineer in EngineeringInternet Company, 201-500 employees
Score 3 out of 10
Overall Satisfaction with Scala
We use Scala as the main backend language. So this includes everything from the API, to the web app, to the deeper backend services such as for scheduling tasks, running data-workflows, and storing and saving data from our datastores. Our data science team also uses Scala with our Spark architecture. The idea, when the original engineers picked Scala, was that it would be a better version of Java and make our Engineering teams more productive.
- Compatibility with Java: if you are switching off of Java onto a new language, one reason to pick Scala is that it is about 99% compatible with Java, so any Java libraries or code you were using before can be called from Scala (not vice-versa though).
- Great built-in features for managing concurrency (e.g. Futures, Actors, and Akka). Making the most of every single thread on the machines your Scala code is running on is much easier and safer than doing it with Java. Scala abstracts away thread pools and threads quite well with Futures. I wouldn't say Futures are easy to learn though....but they are definitely safer to use than pure threads.
- Null-pointer safety: In Scala, null pointers are rare because most libraries pass around a class called Option when whatever you are referencing could possibly be null. Options are first-class and the functional nature of Scala combined with Options means you can almost always avoid referencing a null directly using Option.map or Option.flatMap (see here for what they do https://www.scala-lang.org/api/current/scala/Option.html). That means you'll almost never encounter another null-pointer exception unless you do something quite stupid and avoidable. Java has Options for helping with this now, but it's not widely used and not nearly as powerful.
- The built-in compiler, scalac, is sssssssssslllllooooowwwwww. I mean like, if you thought the Java compiler was slow, try Scala! The default compiler on my 12k line codebase takes 4 minutes to compile from scratch on my i7 quad-core machine. This can be mitigated through the paid solution of Hydra which compiles your code in parallel. Unfortunately, it's quite expensive and your legal department or finance department may not approve of it. But if they do, for me, it reduced my compile time down to 80 seconds, much more manageable.
- Scala is not going anywhere and support for it is slowly dying. This is the main reason I would not choose Scala for my next company or project. Important Scala libraries such as secure social (which is used for OAuth, a major requirement of every web app) are hardly maintained. Another library that suffers from lack of updates is Slick, the database mapper. There aren't enough engineers working on it to even provide support for the new features that came out in Postgres 9.0 (e.g. JSONb). There is simply not enough of a community to drive Scala forward and keep 3rd party libraries up to date as Java world does it.
- As a corollary of a stagnant community, hiring Scala developers is hard as well. Of the 30 backend engineers we've hired, only 3 came in already knowing Scala. And as I will mention below, this is a BIG problem because learning Scala is really tough.
- The learning curve for Scala is very, very steep. Anecdotally, I came into my current company with strong Java experience. Java is the closest language to Scala but it took me 6 months before I stopped needing to pair program on easy tickets. It doesn't help that Scala has some weird syntax like Map[A, +B] and that it forces you to do functional programming.
- Negative: slow engineer onboarding. As I mentioned before, it took me 6 months to get up-to-speed on Scala and didn't need to bother more senior Scala engineers anymore for help with every ticket. That's hundreds of hours I wasted of myself and other engineer's time.
- Positive: thread safety, no concurrency bug. The ROI on this one is really hard to calculate, but I do believe Scala has saved me hundreds of hours over the past few years by allowing me to never have to worry about deadlocks or race conditions. Scala is simply so safe we've never had race conditions within the JVM before.
- Negative: third-party libraries aren't maintained so we have to fork and update them ourselves. As I mentioned before, we use Securesocial but it stopped receiving updates and there is simply no alternative to it. So, we forked it and put an engineer on it for a month to get it back up-to-date. What a waste of his time!
The Scala community is still pretty active and friendly. Martin Odersky, the creator Scala, and his team are sill quite passionate and gone above-and-beyond to fix bugs and address the need for more features. They also have a company called Lightbend that will help you integrate Scala into your engineering stack. I have heard mixed things about them but never worked with them myself so take what I say with a grain of salt.
Do you think Scala delivers good value for the price?
Are you happy with Scala's feature set?
Did Scala live up to sales and marketing promises?
Did implementation of Scala go as expected?
Would you buy Scala again?
If you are in the data science world, Scala is the best language to work with Spark, the defacto data science data store. I think that is really the main likely reason I would ever recommend Scala. Another reason is if you already have a team of programmers familiar with functional programming, e.g. they all have years of Haskell experience. In that case, I definitely think Scala is a superior and faster-growing language than Haskell and that picking up Scala after Haskell should be quick.