> Scala offers you many design choices that work well in skilled hands
I agree that Scala works well in some skilled hands (see Li's libs), but the people that write the most terrible Scala programs actually know a lot about Scala. Scala has a unique way of encouraging genius programmers to write crazy code. People that know little Scala can just use it as a "better Python" language and actually write amazing Scala code.
> Scala programmers generally frown on programmers that want to use it as a better Python and I think it should be the opposite.
It's hard to make generalizations like this that are meaningful. There are a number of different Scala ecosystems, as you probably know. Spark is really a world of its own. Lightbend's libraries form their own ecosystem, as do Typelevel's and ZIO's.
My own opinion is that using algebraic data types (sealed traits, final case classes) with a sprinkling of type classes (implicits) where needed is pretty easy to work with, and that's the style that Typelevel is using. It is mostly inline with Li's "Least Power" post, though using IO pervasively instead of Future.
Is Typelevel's approach a better Python? I don't know. I don't really know what that means, and I'm not convinced that Python is a great model to follow. I see you posting along similar lines in many Scala threads on HN, so you seem quite invested in the language. I think it would help make your point clearer if wrote up a bit more of your thoughts, perhaps as a blog post.
The only Typelevel library I've used is frameless, which I blogged about here: https://mungingdata.com/apache-spark/frameless-typed-dataset.... Don't think the lib is really usable. Typelevel stuff is not what I'm referring to when I say better Python. I'll try to make another blog to clarify.
I think Scala is a better language than Python. But for some reason it just seems like Scala is full of mind bending code that I can't understand. Full disclosure: I am way less smart than lots of the Scala ppl I've worked with.
Agreed on Frameless. I think Frameless was an experiment that didn't pan out. I don't think Scala 2 is capable of implementing a usable system that does what Frameless attempted to do (a typed data frame). Scala 3 might.
I find what most people mean when they say "simple" is really "familiar". When I first started teaching most programmers weren't familiar with first class functions and they were "complex". Now almost everyone has been exposed to them. Similarly I'm seeing more students are learning FP at university and thus find FP techniques "simple".
I always find it funny how ppl will whine about Scala's complexity but then when Scala's "complex" features get re-implemented in java(usually in some bastardized fashion) they are god-givens
> Scala has a unique way of encouraging genius programmers to write crazy code.
Scala is a popular platform to explore FP concepts without fully understanding their caveats or having enough respect for the rest of the team's ability and willingness to follow, I'll give you that. But that's only because it's a great platform to explore FP concepts in general!
You have to reign in the "geniuses", that's what engineering culture is for. I really don't like that label for people writing crazy code, btw. They shouldn't be on a pedestal just because they lack the empathy to write maintainable code.
The pythonic-Scala approach is certainly gaining steam in the ecosystem. The problem is, it's hard to see because it's boring and doesn't require many special tools – Scala has much of that built-in. So the popularity of this approach is not apparent if you just look at open source activity. But internally companies who work with Scala use it in very benign ways for the most part, in my experience at least.
Still. For some data scientist who knows nothing about programming, or for some CS student getting started with his first programming assignment - which is easier - Python or Scala? I'd say Python, by a lot.
I agree that Scala works well in some skilled hands (see Li's libs), but the people that write the most terrible Scala programs actually know a lot about Scala. Scala has a unique way of encouraging genius programmers to write crazy code. People that know little Scala can just use it as a "better Python" language and actually write amazing Scala code.
I agree that Scala codebases that follow the Principle of Least Power can be amazing: https://www.lihaoyi.com/post/StrategicScalaStylePrincipleofL...
Scala programmers generally frown on programmers that want to use it as a better Python and I think it should be the opposite.