So one of the interesting things that a lot of commenters seem to miss:
Its not the choice of technology, its the deliberate limiting to those choices. The team have said ok, we are choosing these things, and nothing else. This means that yes, some things are harder to do, but it also means that there is less shit to maintain.
If we built bridges like we built tech, we could see the silliness of choosing the components for the developer rather than the product:
the start of the bridge will be made of steel reinforced concrete, the hand rails will be made of carbon fibre, but they are taking too long, so the team switched to wood.
The middle of the bridge will be made from just steel, because one of the developers wanted to learn how to slip weld.
The tail end of the bridge is a suspension bridge, because one other developer decided that they looked cool.
That's not to say that developer efficiency doesn't matter, but efficiency is very different from "that looks cool, lets do it like that" Limiting choice is good, but when you first start, it fees like a straight jacket.
The difference between buildings and software is that generally a building has a defined „done“ state, but a piece of software keeps changing until it has so many scars of unmaintainable mess that it’s cheaper to start a new company than to fix it.
I mean thats the unwritten/implied problem with software project management.
There is no "Done" state, there is no real plan for the future. It is possible to make software sustainable, but for many reasons there's not much incentive to do so.
Partly because its rare that the business understands that product it makes, or if they do understand, leadership get bored and want to try something new.
Your great analogy provoked a thought, and at risk of taking it too seriously:
The bridge you describe sounds like one that was built and would be maintained in a more organic way. If we repaired more, we'd perhaps have more objects built like this. And we'd in turn learn much more about interactions between materials, and have a larger body of knowledge (and maybe folk wisdom) about objects built with such interactions.
But yes, more risk and uncertainty, which is intolerable in the current culture (though I suspect it could become more tolerable if our energy systems and supply chains start to break down, and the current calculus stops making sense)
> What happens when we run an engineering organization by constraining the number of tools in our toolbox “for the greater good”?
I read this and think: ok, I guess this is going to be one of those posts advocating for boring tech (i.e just use Java everywhere).
> the answer is quite short... Clojure for production services, Kafka for asynchronous communication, Datomic as our database for high value business data, Scala for our analytical environment, and Flutter for our mobile app
For longevity I'd put my money on Clojure over Flutter or Kafka (which are younger techs too). And of course Datomic now is effectively in-house software for Nubank and has been conceptually validated besides on its own merits, by the number of kindred DBs around[1].
IME Clojure is boring in a good way vs Java, as the pace of language evolution is slower, the language is simpler, and simplicity is more highly appreciated in the culture of the ecosystem.
I don't think that Kafka is bad per se, it's just that it's often used as a message queue which it doesn't seem optimised for. Half of the stuff you need for a proper MQ (e.g. DLT handling) isn't provided by Kafka proper (although interestingly it is provided by spring-kafka - but that locks you into that tech and just means another abstraction which makes things harder to understand). Also, Kafka doesn't really deal well with long-running consumers, something that might be quite necessary for many async systems.
Voting -1 might have been a better response; alternatively you might have phrased things in a way that requests an answer, or at least articulate your argument.
The Kafka being Java one is the most difficult one to refute I'd say.
Aeron is for a pretty different use case. Kafka is optimized for volume batches, Aeron is for very low latency transactional. I wouldn't use one in place of the other. They have very different operational requirements too.
Both are technologies to transport messages over the network in a topic-oriented producer/consumer fashion, and both provide the ability to record and replay from a specific point.
Love these choices? Wait two years and read again.
More seriously, this content should be read and evaluated divorced from the reader's personal affection for specific tools, otherwise what value did this writeup provide if your opinion was set from the get go?
It's a post about being conservative with technology yet most of the choices seem hipster. Clojure, why not java? Diatomic, why not postgres? Flutter... it's gonna be cancelled.
Though I agree with the message that keeping the library choice small is good. I just think they already lost that war at some point. There are cracks in the argument in the article. Diatomic... "For high value data", implying it did not generalize to low value data, or they have several other dbs around. Shoulda picked postgres, it's more general, but hand was forced by clojure, which was the original hipster in the stack I expect.
I think you're being way more harsh than necessary. While I can certainly see were you're coming from, their choices make a certain amount of sense. Plus, it's about limiting your stack, not choosing dusty tech.
Clojure is functional(valuable for correctness), based on lisp(a proven language), compatible with the JVM ecosystem, and attracts high quality programmers. Janestreet uses Ocaml for similar reasons.
I'm less familiar with Diatomic, but a quick look around seems to suggest that companies like Facebook and Netflix are paying customers, and Nubank are the owners. Plus it's written in Clojure. If it works well for them, seems like it's probably a fine investment.
Flutter might get canned, but also Flutter apps are cheap and high quality. It was probably a good choice, and can easily be replaced if the project dies.
I don't mean to be harsh. Just reconciling the message with the article's observable facts. I agree with the message, if I applied it I would have a different stack comprised of general purpose tech choices so I could minimize dependencies. Not lots of specialized techs which is close to what we have here.
You've misunderstood the point of the article and only seem to be reinforcing that in these messages. The point is not being conservative on the whole, but conservative within the already established technology choices in the company.
Yeah, but you see, that doesn't work. You can't start hipster and switch to conservative. Because you have a number of specialist tools that are hard to reapply. As I said, you can see the cracks in the article. The use of diatomic is caveated.
I feel like someone has had a realization that they need to stop using tailored technologies. This article is them trying to put the breaks on the engineering culture of over-engineering. Or maybe diatomics success means everyone who works there is trying to write a new DB or something.
I'm flipping back and forth on my analysis of the situation as I'm reading your exchange, for what it's worth. So you're both stating your perspectives well, and holding valid positions as far as in concerned.
My original (and common) perspective is to favour dynamism and choice ecologies.
But then I realized there's an interesting analogy here of mimicking how life tends to navigate the explore-exploit tension. For example, in lifecycles or social creatures:
Explore first (like "stupid" and "rash" children make decisions, in breadth first search of solution space) and then slowly ratchet into a more exploitative strategy, capitalizing on learnings. So adapt the decision strategy over time, instead of expecting a continuously applicable strategy. Older organisms adopt this strategy, and it's evolutionarily selected because it works in most environments.
So it's perhaps not unreasonable to make very exploratory hipster choices at the start, and then adapt to become more conservative after childhood :)
Clojure effectively is Java, they share the same ecosystem. The only differences are the syntax, which is really just cosmetic (being much terser than classic Java is nice) and Clojure being highly opinionated about semantic style (ie, you should be coding in the style of a very specific expert developer named Rich Hickey).
This sounds very bitter. Also, having worked in multiple places where Clojure is the main language, I don't agree at all. Most places make their own coding rules / standards, enforce them with linters and custom formatter rules. Also the biggest difference by far isn't the terser syntax, it's that Clojure is a immutable, functional language, and Java is the exact opposite.
What part is bitter? I mainly develop in Clojure, Rich Hickey is a genius and more people should program like him.
> Clojure is a immutable, functional language, and Java is the exact opposite.
Program in an immutable, functional style is possible in any language. It won't be as good as in Clojure, and if someone wants Clojure's style they should use Clojure. But that won't stop anyone who wants specific properties. I personally am happy to be writing code that is Clojureish in any language people want to pay me to use.
Regardless of that, the interop between Clojure and Java ecosystems is as close to perfect as can be achieved; they're basically the same circle on a Venn diagram. There is no risk of Clojure having a different deprecation schedule compared to Java.
It is so not. Clojure is to immutable-functional as Prolog is to logic: a procedural language with fancy syntax and funny execution order, masquerading as the other thing.
It's built on the JVM. How could it be otherwise? Sure, you can program in the immutable-functional style in Clojure, but under the hood, it's procedural. You have to deal with all the same procedural issues: the compiler doesn't really smooth anything over for you.
> This sounds very bitter.
This is bitter. But I thought what you replied to praised Clojure.
The examples you just created are unidiomatic Clojure. Is that supposed to somehow convince me? I have yet to see anyone actually use `while` in Clojure, and redeclaring `def`'s is also not something I've actually ever seen, despite yes, it being possible to write horrible code if you really set your mind to it.
> The examples you just created are unidiomatic Clojure.
Assuredly. Yet, in an immutable-functional language, the examples I gave are unrepresentable. Clojure has nice syntax, like recur, to make it easy to write pure functional programs, but ultimately it's a procedural language full of side-effects, with a fairly simple mapping to JVM semantics.
Clojure is often the right tool for the job, and I'm sure the Clojure programmers reading this are thinking things like “a simple mapping to JVM semantics is not a bad thing!”. But mere usefulness doesn't make it Agda.
Isn't Clojure dynamically typed by default? That would seem like a major difference to Java.
Or is enterprise Java effectively dynamically typed as well, so that the language-level difference matters less? I had only very limited exposure to JEE, and was surprised how much that was nominally unchecked at the Java language level was effectively type-checked by the JEE platform. But maybe this falls apart for non-toy projects?
Clojure is statically typed - in practice everything is a java.lang.Object. Checkmate, type system enthusiasts >:D. Except macros, I suppose. If you try to pass function arguments that aren't a Java object or primitive IMO it won't work. [0]
More seriously, if you want something with type T, Clojure can give you something with type T. It understands types. Doesn't respect them, but it understands what it is ignoring. It isn't trying to replicate Java, but it has all the tools to replicate Java if necessary.
[0] The author acknowledges that this paragraph, while true, is unhelpful. It is offered in a wry tone.
Java doesn't have macros, so everyone uses reflection. That moves errors to run-time. Add to that the typical enterprise patterns like dynamic dependency injection and you end up with very dynamic deployments.
I understand that. The build environment I used had type-checking for CDI annotations, obviously outside of the Java language, if I recall correctly. If the types did not line up, the build would fail. This was part of some JEE training I took out of curiosity, maybe real-world JEE is different?
There's Java interop, a bit like Rust has C interop, but the Clojure sw ecosystem and culture is quite distinct from Java's. So I'd revise your characterisation to: different syntax, different semantics, different ecosystem, different culture.
> It's a post about being conservative with technology yet most of the choices seem hipster. Clojure, why not java? Diatomic, why not postgres? Flutter... it's gonna be cancelled.
Clojure is 16 years old, how is that hipster tech?
> Clojure is 16 years old, how is that hipster tech?
Common Lisp is 39 years old and it’s hipster tech. APL is 57 years old and it’s hipster tech. You can be a hipster at any age. (And it’s not a bad thing.)
They're a poster boy for Latam tech due to their growth.
While this post makes sense -- limiting tech choices -- it's rather contrarian, limiting choices to bleeding edge tech, which can go away or change on a whim.
But their culture has always been like this, I once attended a conference were they touted how they were using 'hexagonal architecture', at the time, it felt like something only Google or Amazon could get away with (inventing tech buzzwords).
For what it's worth, they have a wrap for always talking up how they use bleeding edge/buzz word tech at meetups, interviews and the like. I guess they can get away with being a unicorn in their market (i.e. Latam neobank)
the main difference between java and pg vs closure and datomic (not: diatomic) is "immutability", which implies a different way of thinking about multithreading and multiprocessing and transactions.
instead of locking and mutexes and semaphores you think in terms of branching and merging, much like git or similar VCS systems.
How are they similar, apart from both running on the JVM? One is a Lisp dialect the other one an OOP language...
I guess your argument is "if you want to use the JVM environment (i.e., available libraries and frameworks), why not use Java?" -- which is a fair question. I suppose so, this argument just questions any JVM-language other than Java in general, and is not specifically tailored at the choices of Nubank.
You could as well have asked "why Clojure and not Common Lisp?" - I mean, if you for some reason determined that a lisp is the best language for the job, why would you choose Clojure... But I think the standard answer is that with Clojure, you get the best of both worlds: Java interoperability and Lisp expressiveness.
I personally find Lisps a maintenance nightmare (because of their ability to express complex computations very tightly), so the even more general question could have been: "Why Clojure?" (without comparison to any potential alternative).
> Diatomic, why not postgres?
Datomic kind of follows naturally from chosing Clojure, I suppose.
Apart from that, I do agree with your general analysis.
Lots of words, but the ssl on the website is broken. After all, it is a bank.
The important point: it’s a good thing to choose a strong stack of a few tools and avoid adding more languages and tools as much and as long as possible. It’s true in every scale, 600 or a single developer.
By the end of the day, Clojure is their Cobol and it will stay with them for as long as they exist. Everything else is just how they reason around it.
The specific technology choices they made are a mistake, IMHO. But it doesn’t really matter.
The concept of these new age banks is nice. But I really think they haven't pushed it all the way yet.
For example, Revolut has one-time-use credit cards. Even thinking about the possible features, there is a lot more that could be done. One time use with a predetermined amount (to be used like a gift card), in fact make every transaction a new credit card. An abstraction over the visa and mastercard networks. Also really actually good bookkeeping like with e.g. gnucash. I really feel there is a space for a hacker- and privacy-friendly new age bank product.
Well, I'm a Nubank customer and they have now a 24h card that auto expires. On top of a permanent list of 10 virtual cards that you can rotate manually. That's what I do.
But yeah, Bank of Brazil, a boring public bank has this for ages, just the app is ugly and intuitive only for financial savvy people.
You can set number of transactions, limit per transaction, total limit of that specific virtual card, etc.
Bookkeeping is not something people usually do here. And I hope that OpenFinance, that is a Brazil Central Bank endeavour (that Nubank is also adhering), be more open to the general public, because today is more about exchanging info between banks to provide better intel whenever you need loans, etc.
With OpenFinance you can choose to share your info from one bank to another, but you can't download this information yourself. Only through downloading ofx, CSV, pdf (the existing tools) on each of your banks, a nightmare to automate.
I like this idea, the downside is it would require an overhaul of the entire fintech system, every page that accepts credit cards, etc, because we'd just run out of card number space. So in essence it would amount to starting a financial network from scratch.
There are 16 numbers in a creditcard. That is a lot of potential cards. It will be a while before we run out of cards. And when we do they could just add more numbers. It isnt such a big deal.
You lose some digits as the bank identifier etc, so really you get 12 digits or so. The population of the world right now is a 10 digit number. It sounds like a lot of space at first glance, but it really isn't. "Just add more numbers" is what people thought when IPv4 started to run out and we all know how that is going.
Quite a few websites don't accept my Revolut or Monzo cards at all, despite being VISA/Mastercard. So there is clearly some way for them to detect these neobank cards.
Well if BTC got adopted properly it would have been the answer. We are stuck with the system designed ages ago. No doubt it's good for government. Current Fintech can go any peasant go bust. However, normal folks have zero control over their financial security. Your always under radar especially under cashless society umbrella.
single purchase "virtual" credit cards created on demand are available and kinda standard offering by many companies though I've never checked their availability for consumers, always had them via various B2B
I'm curious about the usage of CockroachDB at Nubank, given it's not mentioned as a "paved road" (or otherwise) in the post alongside Kafka/Datomic/Spark, but is described here: https://www.cockroachlabs.com/customers/nubank/
Its not the choice of technology, its the deliberate limiting to those choices. The team have said ok, we are choosing these things, and nothing else. This means that yes, some things are harder to do, but it also means that there is less shit to maintain.
If we built bridges like we built tech, we could see the silliness of choosing the components for the developer rather than the product:
the start of the bridge will be made of steel reinforced concrete, the hand rails will be made of carbon fibre, but they are taking too long, so the team switched to wood.
The middle of the bridge will be made from just steel, because one of the developers wanted to learn how to slip weld.
The tail end of the bridge is a suspension bridge, because one other developer decided that they looked cool.
That's not to say that developer efficiency doesn't matter, but efficiency is very different from "that looks cool, lets do it like that" Limiting choice is good, but when you first start, it fees like a straight jacket.