If it was a traveler's union, maybe. Cop unions don't result in better outcomes for the general public, and there's no reason a controller's union won't end up just boosting pay and having a rubber room for hacks (referencing NYC schools paying teachers to not work because they're either predators or terrible at teaching, but being unable to fire them).
I wouldn't rely on a single human being to fix this kind of issue. It's only solvable through massive collaboration and communication among those who want to fix it.
Movements that ignore the need for a charismatic leader fail, often spectacularly. It's why for example occupy wallstreet was such a laughable failure. Who was its leader? Is the human megaphone a species of "massive collaboration and communication"? Can you name me one leader from that movement who was nationally recognized as such?
Strong leaders are always required. Such people reduce the cost of messaging and communication which would otherwise be insurmountable to cohere a movement and actually make change. You don't elect a mob. Find leaders you trust and spread your conviction without apology. Roosevelt was not Roosevelt until after his works were done. We don't need some amorphous "massive collaboration and communication" we need to elect leaders who will fight for what we believe. So many of your friends, family and neighbors are willing to elect sell-out leaders. You could start there, that is if you actually want to fix the problem rather than invent new ones.
> It's why for example occupy wallstreet was such a laughable failure.
This claim is enormous. I would instead argue that the movement lacked cohesiveness because it basically complained about too large a set of (correctly identified as interconnected) issues and lost momentum because the surface was too large.
That said, I agree w your point about a face being important. Even in software, where tech can speak for itself, we see this heavily: Torvalds, Matsumoto, van Rossum, Jobs,
...which is typically done by building a movement around a leader who represents the values a movement wants to achieve.
FDR is a good example of an American leader who made substantive, wildly successful, left-leaning policy changes that ushered in decades of prosperity and (in part) last to this very day despite facing heavy opposition from the business elite of the time. They even tried to coup him!
At the time, the long term trends were dire for the American left. Double insulation was strong and getting stronger. Then the Great Depression hit. Around the world, populists and radicals were elected to office, and one way or another they changed things. In America, we managed our reform process without trying to conquer the world and without starving millions. Not Hitler, not Stalin. Roosevelt. I think that's a worthy goal to aim for again this time around.
Perhaps I mean to ask a question then, how did FDR manage to become such a widely heard leader back then with so many less ways for people to talk together? Did it make a bigger difference that he had to exist as someone people spoke to other people about? Shouldn't it be easier to find these leaders with so much more access to everyone nowadays?
Communication friction is only one cost of running a campaign among many, so the structure of parties and campaigns and primary / general elections has largely remained the same. Even if the technological barriers went away, I suspect the human factors would still hold up the structure because only so many people are willing to spend years of their life building legitimacy and promoting a political platform and each voter is only willing to spend a certain amount of time participating and choosing.
Exactly how that may have played out in the last century could be explained by many, many chains of causes and effects. But it wasn't a great leader that made it happen. At the bottom of everything, I believe it was this:
Decades of Famine, Pestilence, War, and Death destroyed not only capital but huge swaths of the labor pool. With labor at a premium, it became more valuable and power shifted.
I think that without a similar apocalypse, it will not happen again.
Yes, economic disaster is the driver (tangential: a lump-of-labor supply shock was not the transmission mechanism), but big political movements always happen from the pieces lying around. Everyone can feel that a disaster of one form or another is coming. We need to make sure the right pieces are lying around.
Out of curiosity, what sort of things have you seen it do that better fit 'autoresearch' than 'autotune' thus far? Optimizations it made that wouldn't be been surfaced by an autotune system, I suppose.
The most recent round of autoresearch (round 2) which decreased "time to GPT-2" from 1.8 hours to 1.65 hours had some examples. I adjusted the program.md to "look at modded nanogpt project and draw inspirations from there for things to try" and it came back with a bunch of tuning, but also tried and implemented new architecture changes, some of which actually helped including the smear gate and the backout skip connection. These are not just hyperparameters, they are new PyTorch code. I'm now working on a more general system that can have a queue of ideas that could be sourced from archive papers, github repos, etc.
Do you have a sense of whether these validation loss improvements are leading to generalized performance uplifts? From afar I can't tell whether these are broadly useful new ideas or just industrialized overfitting on a particular (model, dataset, hardware) tuple.
Did you consider providing the LLM with a framework for automatic hyperparamter tuning? This would free up its capacity to focus on the more important architectural decisions.
I see this critique about autoresearch online often, but I think it’s misplaced.
Here’s a use case that may illuminate the difference, from my own work at Nvidia. Im currently training some large sparse autoencoders, and there are issues with dead latents. Several solutions exit to help here, such as auxk, which I can certainly include and tune the relevant params as you describe. However, I have several other ideas that are much different, each of which requires editing core code (full evaluation changes, initialization strategies, architecture changes, etc.), including changes to parallelism strategies in the multi-rank environment I’m using. Moreover, based on my ideas and other existing literature, Claude can try a number of new ideas, each potentially involving more code changes.
This automated run-and-discover process is far beyond what’s possible with hyperparam search.
It wasn't meant as a critique, I'm legitimately interested in knowing more about where it can push boundaries and where it struggles. I agree that in general it's a truism that "Claude can try a number of new ideas" etc., but the question remains as to where in particular it actually takes advantage of this to push the envelope in a way other tools don't -- since that informs when it makes sense to use something like this.
When you're fighting the same enemy on a dozen battlefields, you won't stand a chance of winning until you understand that fact and go after the root cause.
The whole idea of enshittification is that someone makes a high-quality app (or whatever), outcompetes all other entrants, and locks down the market. Then, having acquired pricing power, they can raise prices or, more often (as these tools aren't 'priced' from the perspective of the consumer, but rather indirectly funded e.g. through ads) lower the quality of the product. The steps in this chain are not inherent to 'making products', they emerge entirely from the confines and incentives of our market-based economy.
And it's not just "centrally planned economies" that avoid this. We see evidence from historical modes of production like artisinal handicraft. Despite there not being a free market of producers (as guilds generally possessed legally-enforced monopolies over saleable production) the general quality of goods thereby produced did not generally trend downwards. Indeed, we can see from the sources that in cases where quality was known to have dropped, popular backlash led to interventions, e.g. the various Parisian bread laws, or hallmarking regulations for goldsmiths. Obviously, similar mechanisms exist today in the form of governmental regulations, but the problem with free market economies is that they produce actors both incentivized and empowered to hamstring the government, capture regulators, and ultimately undermine that self-same free market, to their own benefit.
This feels to me like a false dichotomy. The only alternative to the current way of doing things isn't a planned command economy, no matter what "libertarians" or tankies might argue.
Anything other then capitalism with slightly more regulation is just going from the US to Germany. Great, but they have software updates on cars too.
If you want to change anything more fundamental, you are going to have to do a planned economy.
At best you can say, maybe could be slightly better Germany by having a better political process or something. But even then, software updates in your car are going to be a reality because it solves are problem for manufactures, saves consumers lots of time in many cases and generally the positives outway the negatives.
I bet you 100% that in any planned economy OTA updates would still happen.
At best we can argue for some better practice about OTA Updates in regards to security and functionality. Maybe forcing manufactures to have a 'security only' feed an a 'feature feed'.
> I bet you 100% that in any planned economy OTA updates would still happen.
How so? In a democratically planned economy, we would expect that economic decisions considered by the majority of the population to be unwise/upsetting/etc. would not take place. Yes, many/most decisions would probably happen 'behind the scenes', according to the delegated authority of smaller committees or individual officials, but that's only so long as those decisions don't cause bad results for the broader populace.
More broadly, how exactly would enshittification take place in an economy not based around market principles? The whole idea is that someone makes a high-quality app (or whatever), outcompetes all other entrants, and locks down the market. Then, having acquired pricing power, they can raise prices or, more often (as these tools aren't 'priced' from the perspective of the consumer, but rather indirectly funded e.g. through ads) lower the quality of the product. These steps are not intrinsic to reality, they emerge entirely from the confines of our market-based economy.
And yes, you can argue that in an "ideal market" they wouldn't happen, but a truism of modern economics is that "sufficiently free markets" produce actors with the power and desire to capture/destroy said free market.
Isn't this a depressing thought? Regardless of AI, to think that everything we read would come in the same literary style, conveying little of the author, giving no window through which to learn about who they are -- that would be a real loss.
Ultimately it’s up to the author to make that explicit choice. I think that AI does and will enhance writing and depth and breadth of analysis one could perform. But, to be trustworthy, people will need to either lay out all cards on the table and/or work on other ways to gain trust over time. Maybe people need to provide some context to communicate what model was used and in which ways. What % of final output is AI vs author. I mean, if I see 100% composed by human author stated somewhere then there’s my cue to at the very least learn a little about the author. Certainly more complexity and discernment for readers. Depressing? In some ways maybe; but I’m kind of optimistic. Imagine what Tolkien could worldbuild armed with AI.. but then it wouldn’t be Tolkien.
Not at all depressing. If authors/creators want readers/consumers to know who they are, then they'll do so in whatever way they consider acceptable to them.
Isn't that a good deal more reasonable though? China, as a polity, does indeed have agency. It's strange to suggest they don't, as if only America can do things on the world stage.
Sure, the usages aren't all flawed. But it's far more likely to see "Europe" doing something than "US" doing something in the headlines in similar cases, I feel.
Same goes for China, if a couple of companies do something, often in the headline it's just the general "China" doing it. For example we'll see China doing something with EVs whereas for the US we'd see Tesla doing something with EVs.
You can't just proclaim what incentives should be. We do have a mechanism for changing the incentives of management though: it's called unions.
reply