RAM and storage price increases due to the AI bubble have certainly made the cost of entry more expensive, but once you have the hardware, running models locally does make financial sense, especially if you have access to home solar power that is sufficient to run the hardware. You can't get much lower running cost than free.
> However I'm not going back to asking stackoverflow and pretend that I have nowhere else to find answers.
That's not your only option.
What you're meant to do is understand the tools you're using well enough to not need to ask for help from anyone or anything else. Stack Overflow is useful, but it's a learning tool. If all you were doing before AI was copying and modifying other people's code, it's no wonder that you have taken to AI, because it's just a slightly more convenient form of that.
Once you get good enough at a programming language, you can code a lot from memory and logic. As in, you can think of a design and how to build it without having to look up someone else's code. It's still useful to keep notes to refer back to, and look up information online to jog your memory, but it's not always a question of finding other people's code to modify.
Alright, let's see Codex 5.3 create a competitor to postmarketOS (without just copying the homework of other devs). If you believe in the technology so much, put it to the test, see what it can really do.
Sure, AI has developed quickly, but let's see it take on a real engineering challenge, rather than regurgitating boilerplate code.
Writing device drivers from incomplete specs is much harder than "writing a whole application" where the specs are clearly defined and there's a lot more example code to reference. If you believe in AI so much, and believe that it's unreasonable for postmarketOS to not want to use it, put it to the test, prove the doubters wrong, what have you got to lose?
What does a developer who writes a driver from incomplete specs do? Writes some values in some registers, sees how the device behaves, updates the spec. Rinse and repeat. Sounds exactly the kind of stuff coding agents thrive at - a verifiable loop. And they can do it 24x7 until done.
Haha, are you trying to suggest you'll have lost much by putting an AI tool to the test? You seem to think it's powerful enough to do the work of porting Alpine Linux (or equivalent) to new hardware without human intervention (beyond the initial prompt), what exactly are you losing by trying this out? It's not your time, as you would have spent less time on giving a simple instruction to an AI tool than you spent in talking to me.
Perhaps the reality is that you know AI needs more hand-holding than this, and the tools aren't up to the task you're thinking of setting them.
You are also strangely fixated on today's capabilities, completely missing the exponential we are on.
In a few months will have posts here from device driver writers explaining how they hooked up a phone to an Arduino and a video camera and how the AI is automatically writing device drivers.
> You are also strangely fixated on today's capabilities
I am talking about today's capabilities because this comment thread started with the suggestion that the benefits of AI for coding was no longer avoidable after the launch of Codex 5.3.
> In a few months will have posts here from device driver writers explaining how they hooked up a phone to an Arduino and a video camera and how the AI is automatically writing device drivers.
A few months? Almost zero chance. If it happens in the next 5 years I'd be less surprised, but I suspect it'll take longer.
> Fun that you had to caveat it with some hand wavy homework bull.
Not really. If AI is just copying someone else's code, it's not really designing it is it. If you want it to truly design something, it needs to be designing it using the same constraints that the human engineers would face, which means it doesn't get the luxury of copying from others, it has to design things like device drivers with the same level of information that human engineers get (e.g. device specifications and information gathered through trial and error).
Are you suggesting that a human being writes an OS in a vacuum without seeing any other OS or looking into how it is built. That feels a little facetious, no?
> Are you suggesting that a human being writes an OS in a vacuum without seeing any other OS or looking into how it is built. That feels a little facetious, no?
No, I'm suggesting in order for it to be a fair test, you need to impose the same restrictions that a human engineer would face.
For example, consider the work done by the Nouveau team in building a set of open source GPU drivers for NVIDIA GPUs. When they started out the specs were not so widely available. They could look at how GPU drivers were developed for other GPUs, but that is not going to be a substitute for exploratory work. Let's see how well AI does at that exploratory work. I think you'll find it's a lot harder than common uses for AI today.
Research supported by the Fine Foundation, that has a mission statement that includes "works to sustain Jewish life by combating antisemitism"... Funny how the timing of this "research" coinciding with the new owners of TikTok indicating their new stance on censorship of criticism of Israel.
Cause IQ describes the funders of this research as, "The Melvin and Betty Fine Foundation promotes Zionism, Jewish education, and the US-Israel relationship through various programs that work towards these goals." https://www.causeiq.com/organizations/melvin-and-betty-fine-...
I think it's warranted to be skeptical of the research.
If an AI thinks for you, you're no longer "outsourcing" parts of your mind. What we call "AI" now is technically impressive but is not the end point for where AI is likely to end up. For example, imagine an AI that is smart enough to emotionally manipulate you, at what point in this interaction do you lose your agency to "outsource" yourself instead of acting as a conduit to "outsource" the thoughts of an artificial entity? It speaks to our collective hubris that we seek to create an intellectually superior entity and yet still think we'll maintain control over it instead of the other way around.
1. Do you think it's impossible for AI to have it's own volition?
2. We don't have full control over the design of AI. Current AI models are grown rather than fully designed, the outcomes of which are not predictable. Would you want to see limits placed on AI until we had a better grasp of how to design AI with predictable behaviour?
I'd see Chinese RAM manufacturers like CXMT filling the void left in the market for consumer-grade RAM modules, I appreciate they face challenges (like lack of access to cutting edge EUV machines), but the RAM just needs to be fast enough and affordable enough for the average user for these companies to make significant inroads into the market that Micron, Samsung and SK Hynix are abandoning to chase the AI server market.
Their scale is simply too small to affect the market outside China, majority of their chips will be eaten up by HBM3 production with yet unknown yield rate.
They are forbidden to buy foreign equipment beyond their current process node, which is already obsolete, die size is 40% bigger than Samsung, not to mention lithography, the big 3 are using EUV while they are stuck with lobotomized DUV.
They can start making some decent money now, but vastly expanding capacity as is means enormous losses if the cycle went downward a few years later, that's how all previous makers went bankrupt.
They can squeeze out a bit more performance if they are ready to go beyond their current node using only domestic equipment and be blacklisted by the US government.
But the cap is there, unless they can make a working EUV machine in 5 years, they are doomed to be a minor player, if the current cycle even lasts that long.
They will grow exponentially and catch the western market unawares in 10-15 years with a sudden flood of cheap, effective chips. Just like everything else China makes. Electric vehicles for example.
Sure, if they've got production grade EUV, but right now they don't even have production grade DUV.
I'm also sure they can go as far as 5nm like SMIC if they really wanted to, since it's strategic for China, but the cost would only be justified if the current cycle lasts long enough.
I was corrected elsewhere when I thought RAM was more expensive 10 years ago. RAM was actually cheaper 10 years ago, when it was DDR3/DDR4 too. If any company can replicate the 10 year old SOTA, they can bring prices down.
This is what I expect to happen. 2016's ram was good enough for consumers then and probably still is for a huge class of consumers now. I'd rather 32GB of DDR3 than 8gb of DDR5.
The market seems to stay strange because few believe it can stay strange, but looking back at the netbook industry I feel like a few losers at keeping up for the AI level will realize they may as well make the products that people prefer old prices for and start resuming anything that can bring total system costs down.
This is where China's crazy solar advantage affects real day to day outcomes. When you have electric costs going into 6-8 cents per kwh then you can run older nodes that slurp more electricity. They aren't even done lowering the price. I've thought about this recently. If the dream of meterless electricity came to fruition then that terabytes of DDR3 could essentially be run until it literally burned out and then recycled back into its core components. The sun provides more power than the entirety of society could possibly currently use and so its a shame that the ram is being tossed instead of used.
The problem is that the chances of the bursting of AI bubble seem far more likely than this happening first which you say about 10-15 years.
plus, the ram manufacturer cycle moves and does this all the time.
Atrioc does a really good job explaining these cycles[0]
But the point is that AI demand peaked when the supply was at its lowest which is why we are caught up in this messed up timeline that we live in. And this has sort of happen in the past too and this industries notorious for it (again watch the video, definitely worth it imo)
But still it feels like we are in this atleast for a year or two hard. Micron is iirc like suggesting what hundreds of billions of $ in factory investment right now and saying that the fastest might open in early 2027
Some estimates 2028 idk, I do feel like the chances of AI bubble popping around this time are likely too.
But still for atleast 1-2 years, we either as consumers or as small vps providers (yes the people who create vps providers are same people like you and me) are absolutely f*ed and the question is around that imo.
The thing is, even if the market bursts, prices are already inflated. RAM manufacturers know that everything 5xed and they aren't likely going to rush out and drop the price levels to pre-expansion. Once the AI market bursts, you can expect slow and methodical decreases in price (if any).
And that will ultimately buy China a lot of time to shove their ram into the market cutting ram manufacturers out of most non-US markets.
I think the major memory manufacturers are simply banking on their ability to flood the market if worst comes to worse. That or I could see some standards trickery around DDR6 (or some new BS standard). It'd not shock me if they coordinated with AMD/Intel to keep the standard secret as long as possible simply give themselves a lead in production.
Those were driven up by scalpers during the crypto stuff. The manufacturers were still selling their cards for a reasonable amount, you just couldn't get one because scalpers and crypto farmers had bot armies gobbling up supplies.
The ram pricing is coming directly from manufacturers.
>Graphic cards prices normalized quite quickly after crypto boom
Eh, I don't think we were on the same planet then. Even post crypto pre-AI GPUs were far more expensive than they were before said crypto. We just got used to paying $1000 for a mid tier cards.
likely naive question: why it is a problem? I would be fine if RAM in my PC has 10 times larger physical size if it is overall cheaper.
I guess that larger may have more power draw, but given costs of RAM and electricity and power draw of RAM it sounds unlikely to be a problem.
At a high end it would run into real-estate prices - at some point using half of room for computer stops making economical sense, given costs of rent or buying flat space. But just doubling size of PC does not sound like a bad tradeoff if it would be say 20% cheaper. Or 50% cheaper.
Is it about not fitting existing motherboards?
Is there reason why they cannot just make memories physically larger? It is "only" 40%, not 40000%
For big PC towers it's not that big of a deal, but for smaller laptops, phones and SBC, having bigger chips may not be feasible due to size constraints
Not 10nm, they are producing with 18.5nm and 17nm now, which technically already is in breach of US restrictions, the US government can blacklist them if they feel like it.
3D DRAM is no magic, it will only give them maybe 2 generations' breathing room if they got the required etching equipment figured out. But others will be doing 3D DRAM with EUV by then.
I don't understand why we don't just cut to the chase and elect the Joker in 2028. The same people will vote for him, and he won't be any worse for the country.
> Their scale is simply too small to affect the market outside China, majority of their chips will be eaten up by HBM3 production with yet unknown yield rate.
They don't currently have the tech to compete on HBM3 production, but they can produce DDR5 memory, and they will undoubtedly be scaling up production on this.
Obsolete process node hardly matters when the rest of the market is bottlenecked on production capacity; small overall scale still might. Expanding capacity may or may not make sense; it depends on your prediction of the way the market will go.
> They can squeeze out a bit more performance if they are ready to go beyond their current node using only domestic equipment and be blacklisted by the US government.
Which suits the rest of the world just fine. More for the rest of us, and if the single-digit-percent portion of their market that the US represents wants to lock itself out, no skin off anyone else's nose.
> which is already obsolete, die size is 40% bigger than Samsung, not to mention lithography, the big 3 are using EUV while they are stuck with lobotomized DUV.
And? That's good enough. My daily driver desktop, which I use to do development, plus play a few games (FC5, Dirt Rally, etc) has 16GB DDR3.
For 90% of computers in use today, including laptops, that RAM you call obsolete is fine.
Users aren't going to complain that a document which takes 3s to open now takes 3.5s
What you need as far as RAM goes, to make the computer perform acceptably is capacity. Users get a bigger performance boost by going from 8GB RAM to 16GB Ram and from 16GB Ram to 32 GB RAM than from DDR3 to DDR4 or from DDR4 to DDR5.
>market that Micron, Samsung and SK Hynix are abandoning to chase the AI server market
These three have collectively committed what, approaching $50B towards construction of new facilities and fabs in response to the demand?
The memory industry has traditionally projected demand several years out and proactively scheduled construction and manufacturing to be able to meet the projected demand. The last time they did that, in the crypto boom, the boom quickly turned into a bust and the memory makers got burned with a bad case of oversupply for years. With that context, can you blame them for wanting to go a bit more slowly with this boom?
Sure, the new fabs won't be up and at volume production until late 2027 / early 2028, but committing tens of billions of dollars to new production facilities, including to facilities dedicated to DRAM rather than NAND or HBM, is hardly 'abandoning'. They're pivoting to higher profit margin segments - rational behavior for a for-profit corporation - but thanks to the invisible hand of the (not quite as free as it should be) market, this is, partially, a self-solving issue, as DRAM margins soar while HBM margins compress, and we're already seeing industry response to that dynamic, too: https://www.guru3d.com/story/samsung-reallocates-of-hbm3-cap...
> Sure, the new fabs won't be up and at volume production until late 2027 / early 2028, but committing tens of billions of dollars to new production facilities, including to facilities dedicated to DRAM rather than NAND or HBM, is hardly 'abandoning'.
Look at what happened to Crucial. Why would Micron axe it's whole consumer RAM division if it was just experiencing a temporary drop in DRAM supplies until new fabs were brought online? Samsung and SK Hynix may have changes in priorities in the coming years, and in the case of Samsung I'm sure they'll still make sure to supply sufficient DRAM chips for the devices it manufactures (phones, TVs, etc...) but Micron has made it's current intentions fairly clear. They'll probably work with OEMs, but they're unlikely to return to selling to the general public any time soon.
What makes you think consumers would want to trust them again after they were abandoned previously? Consumers would rather vote with their wallets for companies that are going to continue giving them good deals. The only reason Micron has a chance to pivot back to consumer RAM is because there's not much competition, but that could change if the Chinese RAM manufacturers can make inroads into western markets and continue their rapid technological growth.
There is definitely some room for reputation in the enthusiast/direct-to-consumer market.
Crucial had one of the best marketing positions in the business:
- They had a captive supply chain. There was no risk they'd switch from Samsung to Hynix chips but keep the same SKU, so you could buy a second set later and expect it to have similar timings.
- They had a reputation for being conservative about their offerings. There's a lot of RAM out there that meets rated timings only with a voltage right on the edge of what the memory controllers will start to burn out at.
- They were on a lot of mainboard manufacturer's qualification lists, and yet were easily obtained (at least in the US). There are a fair number of brands and ranges that simply aren't widely distributed.
So they were in a place to say "we can charge 10% more for confidence", and considering enthusiasts willingly pay 30% more for RGB and fancy heatspreaders, that's not a bad message. I mean, I've had competent results with plenty of other brands (I have a Team set in my main rig, and it replaced a G.Skill one before the RAMpocalypse), but I always thought of Crucial as a brand you'd use if you were building a machine for work or a family member and didn't want to deal with surprises.
> Because there isn't room for reputation here. The RAM works, or it doesn't. It costs a certain price. Do you want to spend more money, or less money?
RAM from China already works and as Chinese RAM manufacturers scale up there's no reason they can't be competitive on price. Furthermore, it's more than possible for China to catch up technologically as well. Micron is likely to regret pissing off the majority of their customers if Chinese RAM manufacturers start aggressively targeting the market Micron have left behind, how bad this gets for Micron depends on how quickly they pivot back to the general consumer market.
This is the classic commodities cycle, and it happens everywhere in an economy where aggregate supply is inelastic in the short term but aggregate demand can fluctuate quickly. The reason it’s coming for DRAM first is that memory is the closest part to a pure interchangeable commodity and that recent process nodes have had almost zero improvement to memory density for years now, despite logic density continuing to increase exponentially. That and these companies have been known to fix prices in the past, but in this case the evidence suggests it’s a large aggregate demand shock.
Something I’ve been sort of wondering about—LLM training seems like it ought to be the most dispatchable possible workload (easy to pause the thing when you don’t have enough wind power, say). But, when I’ve brought this up before people have pointed out that, basically, top-tier GPU time is just so valuable that they always want to be training full speed ahead.
But, hypothetically if they had a ton of previous gen GPUs (so, less efficient) and a ton of intermittent energy (from solar or wind) maybe it could be a good tradeoff to run them intermittently?
Ultimately a workload that can profitably consumer “free” watts (and therefore flops) from renewable overprovisioning would be good for society I guess.
First: Almost anything can be profitable if you have free inputs.
Second: Even solar and wind are not really "free" as the capital costs still depreciate over the lifetime of the plant. You might be getting the power for near-zero or even negative cost for a short while, but the power cost advantage will very quickly be competed away since it's so easy to spend a lot of energy. Even remelting recycled metals would need much less capital investment than even a previous-gen datacentre.
That leaves the GPUs. Even previous gen GPUs will still cost money if you want to buy them at scale, and those too depreciate over time even if you don't use them. So to get the maximum value out of them, you'd want to run them as much as possible, but that contradicts the business idea of utilizing low cost energy from intermittent sources.
Long story short: in might work in very specific circumstances if you can make the numbers work. But the odds are heavily stacked against you because typically energy costs are relatively minor compared to capital costs, especially if you intend to run only a small fraction of the time when electricity is cheap. Do your own math for your own situation of course. If you live in Iceland things might be completely different.
Exactly as you'd expect: they make it possible to run the GPUs more hours in exchange for needing additional capital. Those batteries will have an upfront cost and will depreciate over time. You'll obviously also need more solar panels than before, which also further increases the upfront investment. Also note that now we're already straying away from the initial idea of "consuming free electricity from renewable overprovisioning". If you have solar panels and a battery, you can also just sell energy to the grid instead of trying to make last-gen GPUs profitable by reducing energy costs.
Again: it might work, if the math checks out for your specific source of secondhand GPUs and/or solar panels and/or batteries.
This is a problem with basically all "spare power" schemes: paying for the grid hookup and land on which you situate your thing isn't free, as well as the interest rate cost of capital; so the lower the duty cycle the less economic it is.
> top-tier GPU time is just so valuable that they always want to be training full speed ahead.
I don't think this makes much sense because the "waste" of hardware infrastructure by going from 99.999% duty cycle to 99% is still only ~1%. It's linear in the fraction of forgone capacity, while the fraction of power costs you save from simply shaving off the costliest peaks and shifting that demand to the lows is superlinear.
That's probably what is going to happen, it's a strategic opportunity for the Chinese government here, there's a big market demand that can fuel their domestic production capabilities that nobody wants to take.
It would be a strategic opportunity for Intel, if they weren't run by imbeciles. DDR4 doesn't require the latest and greatest nodes. It's boring old technology. Even DDR5 is pretty boring. Intel could clean up fabbing DRAM (like they used to). But alas no. They're part of the semiconductor cartel and uninterested in the supply of DRAM increasing. Prices would drop and the fabs would only make stupid margins instead of disgusting margins.
Intel would be in the very same bind as every other DRAM producer who's trying to expand production today, only far worse because they have practically no experience fabbing DRAM compared to logic. (You can fab eDRAM on logic processes, but you'd only do that out of sheer desperation since the cost per bit is much higher.)
What they could do very easily in this market is bring back frickin Optane and hook it up to a modern PCIe bus with modern PCIe performance.
I truly wish Chinese Ram manufacturers luck to fulfill this market. Seriously, the amount of ram and its downstream effects can be hardly understated imo. it really just starts impacting everything.
They will first fill the local demand for all their electronics manufacturing. Then their massive computer infra and AI. And if any is left, it will be bundled to local PC exporters like Lenovo.
It’s fine if it’s just filling Chinese manufacturing. Low-cost VPS hosts are going to be using brands like Supermicro anyways. It still gets exported.
Except for RAM from YMTC, which the USA gave a near-death sentence to by placing it on the Dept. of Commerce “Entity List” so no USA-associated business can do business with YMTC now.
We use ASRock Rack servers, mainly because the only option for our industry are OEMs like Supermicro and ASRock. Dell and HPE are non-starters, except for our "storage" offering.
Back in 2019, HPE was a good midrange option. Then came ASRock Rack who obliterated HPE with the X470D4U, relegating HPE to high-end enterprise servers. But also made Ryzen-based VPS hosts including yours truly, BuyVM, et al.
I am very hopeful of CXMT. But then then it could take a while for them to ramp up production. Maybe by then, the AI bubble would've burst.
One problem with US sanctions is it could hurt US companies too, like in the case of cutting-edge EUV and CXMT. This is when China is actually a hero and not a villain.
We can certainly do with less plastic junk and fast fashion. But on the high end it hard to argue that cheaper Chinese products are ever a bad thing.
If corporations in western (aligned) countries stopped feeding sovereign wealth funds and private equity with profits and actually invested something maybe they could compete with China more closely, even with whatever shenanigans the CPC get up to with state support.
Neither Void or Arch are a "modern Gentoo". Gentoo is it's own thing. If anything, Gentoo's closest "competitors" in terms of OS customisation would be NixOS or Guix, not Void or Arch, but Gentoo is forging it's own path, it doesn't need to follow any other distro.
To clarify, in PowerShell there is a difference between text between single quotes (e.g. '$test') and double quotes (e.g. "$test"). Single quote strings are literal strings, so whatever text is contained within them is reproduced as written. Double quote strings are expandable strings, which means that certain text inside the string is evaluated before it is returned. If you have double quotes in a literal string, you'll see double quotes within that string, and the same should be true for ampersands.
What do you mean it works in some cases, not others? A literal string is a literal string in all cases.
As for it being a "masterpiece of design", it has it's quirks but compared to common Unix shells (aside from Nushell) it's far better. It doesn't need to have a perfect design in order to be a step above the competition.
I hope to continue to see the growth of Nushell, I can see that becoming the best shell one day.
The "killer app" is that it's easy to string together multiple commands and have them work more reliably than shells that rely on plaintext. In other words, it's something in-between running individual commands and writing full scripts. I've not used Nushell before, but I'm very familiar with PowerShell which is similar, and it's ridiculously easy to manipulate and explore your file system with confidence that you don't have to rely on hacky and hard to read regex and similar suboptimal solutions.
reply