The wait is finally over. One or two iterations, and I’ll be happy to say that language models are more than fulfilling my most common needs when self-hosting. Thanks to the Gemma team!
Strongly agree. Gemma3:27b and Qwen3-vl:30b-a3b are among my favorite local LLMs and handle the vast majority of translation, classification, and categorization work that I throw at them.
I'm using the default llama-server that is part of Gerganov's LLM inference system running on a headless machine with an nVidia 16GB GPU, but Ollama's a bit easier to ease into since they have a preset model library.
I would be inclined to agree with this except that my "most common needs" keeps expanding and increasing in difficulty each year. In 2023 and 2024, most of my needs were asking models simple questions and getting a response. They were a drop-in replacement for Stack Overflow. I think the best open source models today that I can run on my laptop serve that need.
Now that coding agents are a thing my frame of reference has shifted to where I now consider a model that can be that my most common need. And unfortunately open models today cannot do that reliably. They might, like you said, be able to in a year or two, but by then the cloud models will have a new capability that I will come to regard as a basic necessity for doing software development.
All that said this looks like a great release and I'm looking forward to playing around with it.
Not OP but one example is that recent VL models are more than sufficient for analyzing your local photo albums/images for creating metadata / descriptions / captions to help better organize your library.
The easiest way to get started is probably to use something like Ollama and use the `qwen3-vl:8b` 4‑bit quantized model [1].
It's a good balance between accuracy and memory, though in my experience, it's slower than older model architectures such as Llava. Just be aware Qwen-VL tends to be a bit verbose [2], and you can’t really control that reliably with token limits - it'll just cut off abruptly. You can ask it to be more concise but it can be hit or miss.
What I often end up doing and I admit it's a bit ridiculous is letting Qwen-VL generate its full detailed output, and then passing that to a different LLM to summarize.
For me, receipt scanning and tagging documents and parts of speech in my personal notes. It's a lot of manual labour and I'd like to automate it if possible.
I use local models for auto complete in simple coding tasks, cli auto complete, formatter, grammarly replacement, translation (it/de/fr -> en), ocr, simple web research, dataset tagging, file sorting, email sorting, validating configs or creating boilerplates of well known tools and much more basically anything that I would have used the old mini models of OpenAI for.
I think it was achieved by two nuclear armed countries openly amassing their assets in the region for months. Any conflict between peer non-nuclear nations would have probably began with the country in Iran’s position sinking those carriers. Thanks to US and Israeli nukes, they were free to start killing people without fear of getting surprised.
It is unlikely that Iran decided to not sink US carriers because of fear of nuclear retaliation. It is much more likely that before the air attack started, Iran's leadership preferred not to do anything that could make an attack more likely, such as attacking carriers. And after the invasion started, they would have loved to attack carriers but did not have the military capability to do so.
I’m waiting for this to affect more young men from my country of Finland.
Decades of a government monopoly on gambling was dismantled, and the market is being opened up for foreign companies buying licenses to operate.
It feels like there is a global wave of reintroducing every ”vice” that was somehow curbed with laws and restrictions. Nicotine? Cigarettes were so expensive that young people didn’t even bother, until snus, vapes and nicotine pouches (especially the pouchess) took off, and now more young people are hooked on nicotine than ever, and even younger.
Light alcohol beverages were only sold through supermarkets, and the wines and the stronger stuff through the well-equipped national monopoly. Now home delivery and breaking up the monopoly is on the table. Ever stronger stuff is getting moved to supermarkets instead of liquor stores.
Now gambling is next. It’s so bad that even the great Apple Inc. includes betting odds by DraftKings (it says so on the app) on their Sports scores app that’s rated ages 4+. You literally have to go to the app settings to turn them off.
All these rollbacks are made by economically right wing parties in the name of personal liberties. Oddly enough, as with many such reforma by the economic right, the gains are personal, but the losses are collective. Billions of euros a year go into fixing the negative effects of alcohol and nicotine. I’ve no idea about the numbers for gambling, but at least the revenues from the government monopoly funded NGOs and other public services directly.
It’s increasingly maddening seeing imperfect solutions to terrible human problems being replaced with… nothing. Nanny state laws might not work, but neither does my alcoholic neighbour… or my old high school friend, who lost so much money gambling that he refuses to find work, because the debt collectors taking a majority of his paycheck. They have more freedom now, at least.
Why don't you say what country you mean? Sorry, but just writing "in my country" and leaving everyone else to guess is an internet trope I find very annoying.
Edit: Looks like you edited your comment to say Finland. Thanks!
As for the content of the comment, I totally agree. I think the eroding standards of regulation of addictive substances (and addictive behaviors like gambling or social media apps) is a serious mistake that we will come to profoundly regret.
Nicotine is actually good for you (at least no worse than caffeine). It's all the other stuff in tar that's bad for you. And that part of the smoke is bad pretty much whatever you burn.
You have clearly articulated what I’ve personally explained to people. Thank you for that. The nature of the strikes as a part of a thoroughly pre-planned surprise attack lays the entire blame at the planners, approvers and those who executed the strike.
The lack of comprehension some people have baffles me, as I’ve had the displeasure of reading several dozens of online posts asking why kids were at school during the strikes. Even giving these people the benefit of the doubt that they do not know that not all countries observe the same weekday/weekend split as in the case of Iran, how in the world is a teacher or a child supposed to know when to hide from a surprise attack?
The easier it gets to give people the tools and power of lethal force, the more preventable injuries and death happen to innocent people. The cover of military conflict should not protect from consequences in cases like this.
Knowing the demographics of this website, it will not make anyone here safer that there is credible proof of Israel using Whatsapp metadata to source location data of adult men, and executing strikes based on that information. Western media already shared stories of how ordinary cell phone metadata was used to conduct strikes that killed innocent civilians. 15-20 years later the exact same deadly inaccurate methods are being used to quench the leaders’ and planners’ thirst for any results. One day a bomb might fall on any of our homes purely based on some circumstantial proof that wouldn’t even be enough for a traffic violation…
On smartphones? It’s not worth it to run a model this size on a device like this. A smaller fine-tuned model for specific use cases is not only faster, but possibly more accurate when tuned to specific use cases. All those gigs of unnecessary knowledge are useless to perform tasks usually done on smartphones.
This news concerns the biggest shift in Trans-atlantic relations in decades – possibly ever. The US is not losing relationships with single countries such as France. It’s not just scaring a small population like Denmark. The entire ”Western world” has started decoupling from the US. Tech is one of the first targets.
Covid, Russia and the axis of US+Israel has done massive damage to the European psyche.
Covid showed us how economically dependent we are to major manufacturing countries like China. Paper money != ability to manufacture.
Russia broke any notion of peace that can be funded by cheap energy. It will always be a tool used against you, and Russia will not change.
The axis of US+Israel is breaking down the international system of laws and diplomacy. It’s going to be in a state even worse than the heights of the Cold War. Nukes are now a more favored instrument of peace compared to diplomacy.
Is it worth fighting for what we had, or should we fight for something better? Who knows.
(Edit: I don’t think non-Europeans can appreciate the whiplash suffered in our populations. In the span of around two years, European leaders drew red lines on political, economical and cultural decoupling from Russia based on human rights and the rule of law, then had to explain why preventable atrocities happening to civilians in the Mideast is not against our values and laws concerning human rights.)
I could be wrong, but I've experienced the opposite. Seeing Putin and Trump openly undermine and threaten the EU forced countries to address the situation and take action. It's encouraging. I'm looking at this situation from Hungary tho, where Russian influence began 10–16 years ago. It seems Hungary has a chance to get rid of Orbán, and the rest of Europe is also taking measures finally. It's nice.
The war in Ukraine is literally at the EU's border. It could be destabilizing in many ways. It's not just about moral reasons. By the way, I see similarities between Putin and Trump as they both started wars against big countries without thinking ahead more than three days. It's one more reason to strengthen the EU.
Nordics and the Baltics are very pro Ukraine, we have a common enemy now to focus on.
Poland has stepped up too militarlity.
Sweden where I am has seen a HUGE uplift in military spend, and the companies like SAAB and Bofors (heheh)
Germany is the big loser as they had cheap Russian energy and shut down their nuclear plants.
Where in Europe are you from exactly?
EU overall is pro Ukraine except for Hungary...
In summary, what are you on about? And post your passport.
Oh, and Israel is our ally. I am sick of EU being so pro Hamas and pro Iran. Thankfully our government cut down on grants to the MENA and increased to Ukraine!
I’m your neighbour. The damage is done to the system of laws and conventions small non-aligned countries like Finland wished would protect us before and after our combined Nato accession. The very reason we are and continue to be pro-Ukraine is for empathy as well as pragmatism: the rule-breaker has to be shown the line that can’t be crossed, so that ours won’t be either.
You’re more than welcome to disagree with me, but it is quite objectively been a balancing act that we hoped to not have to take between our most militarily and economically powerful ally and values (our own and international).
The times of finlandization were supposed to be behind us, but a united front (militarily and politically) has shown cracks due to the unilateral decisions and threats the US has made these past couple of years. Hypocritical application of international law waters it down even further, and the results are already visible.
Israel is Europe's shame, not ally. The UK, France and the US share full responsibility for the creation of the state of Israel, and every day that Europe refuses to denounce Israel, its culpability for Israel's actions grows.
Funny that you should mention that mandate. A "mandate territory" is simply a transfer of colonial rule from a loser of World War I to a victor of the war. In no way should the term "mandate" here be construed as an agreement that was supported by broad consensus.
A little context: The League of Nations was established by the Treaty of Versailles at the end of WW I. That treaty was fully drafted by Britain, France, the United States, and Italy; the first members of the League included all allied victors of the war, their (former) colonies, and the colonies of the losers of the war, which became the mandates you refer to. Many countries that were neutral in the WW I were extended an invite to join the League, but they were not present during negotiations in Versailles.
So, contrary to what you suggest, the League of Nations never voted on the establishment of these "mandates", as they were simply the result of the World War I victors divvying up the spoils of war -- and for Palestine specifically, the so-called "Principal Allied Powers" explicitly reserved those powers for themselves without any input from the rest of the League [0]:
> The San Remo Resolution passed on 25 April 1920 determined the allocation of Class "A" League of Nations mandates for the administration of three then-undefined Ottoman territories in the Middle East: "Palestine", "Syria" and "Mesopotamia". The boundaries of the three territories were "to be determined [at a later date] by the Principal Allied Powers"
So no, "most of the planet" did not weigh in on the establishment of the Palestinian territory; that burden of responsibility squarely falls back on Britain, France, Italy, Japan, and the US.
Also, your claim that Israel was created by the League Of Nations is an anachronism. The League of Nations was disbanded on 18 April 1946 (subsumed into the United Nations), and Israel did not become a nation state until 14 May 1948. Before that date, the territory was known as the Mandate of Palestine.
Most of the things I've used LLMs for is scripting code for integrations between systems, or scripts that extract and transform data from APIs.
For this specific use case, LLMs and their integrations with tools like VSCode have been excellent. A simple instruction file dictating what libraries to use, and lines about where to look for up-to-date API docs, increases the chances of one-shots significantly.
My favorite part has been that I'm able to use libraries I wouldn't have used previously like openpyxl. A use case like "get data from an API, transform it, and output it to an excel file with these columns" is super fast, and outputs data to a stakeholder/non-techy format.
It made me chuckle when Claude etc. release Excel integrations, since working with Excel files seems to have been at a great stage for people who've already worked with Excel/CSV libraries.
The number 1 suggestion I'd have for people eager to work with text is to use models to learn about old unix tools like grep/sed etc. With these powerful tools + modern tools + code you can build quite complex integration code for many uses. Don't sleep on the classic unix cli commands and download stuff from github to achieve things that have already been solved 40 years ago :)
I use a massive OLED monitor as my workhorse and I’d say money and expectations are better spent on established OLED manufacturers and a large screen vs. a laptop screen. Based on the common job roles HN users have, a large OLED main monitor will probably offer more value than the laptop screen that will probably spend most of its time as a side monitor or just turned off while connected to large monitors. The HDMI 2.1 and other display output gains bring more benefits with pixel output and framerate increases. Just my two cents.
This is something even defense analysts have been hammering about tirelessly. The American pivot to counter China should not come at the cost of the Atlantic relationship. Using Cold war scare tactics against China will not work, because China is not in the sphere of European countries that were under the shadow of Russia. China’s language and culture are so far removed from the rest of the world that the risk of direct conflict would not be a matter outside of Taiwan, and cultural hegemony will not take place in the same way Russia tried and at times succeeded.
It feels doubly stupid that not only did American Business sell out their nations’ economic base to Chinese competitors decades back, they fumbled again and sold out to the guy who (yet again) damaged relations of countries funding the service, finance and defense sectors of the US. So now you lost the manufacturing base and you lost the other money-makers. No wonder they are going all-in on fossil fuels to Europe.
No other company is as clear an example of this double whammy as Tesla. Move manufacturing to China to lose the technological edge, and alienate end-users, to lose the customer base.
reply