That's the point. For things like summarizing a webpage or letting the user ask questions about it, not that much computation is required.
An 8B Ollama model installed on a middle of the road MacBook can do this effortlessly today without whirring. In several years, it will probably be all laptops.
But what you would want to summarize a page. If I'm reading a blog, that means that I want to read it, not just a condensed version that might miss the exact information I need for an insight or create something that was never there.
You can also just skim it. It feels like LLM summarization boils down to an argument to substitute technology for media literacy.
Plus, the latency on current APIs is often on the order of seconds, on top of whatever the page load time is. We know from decades [0] of research that users don't wait seconds.
You don't use it to summarize pages (or at least I don't), but to help understand content within a page while minimizing distractions.
For example: I was browsing a Reddit thread a few hours ago and came upon a comment to the effect of "Bertrand Russell argued for a preemptive nuclear strike on the Soviets at the end of WWII." That seemed to conflict with my prior understanding of Bertrand Russell, to say the least. I figured the poster had confused Russell with von Neumann or Curtis LeMay or somebody, but I didn't want to blow off the comment entirely in case I'd missed something.
So I highlighted the comment, right-clicked, and selected "Explain this." Instead of having to spend several minutes or more going down various Google/Wikipedia rabbit holes in another tab or window, the sidebar immediately popped up with a more nuanced explanation of Russell's actual position (which was very poorly represented by the Reddit comment but not 100% out of line with it), complete with citations, along with further notes on how his views evolved over the next few years.
It goes without saying how useful this feature is when looking over a math-heavy paper. I sure wish it worked in Acrobat Reader. And I hope a bunch of ludds don't browbeat Mozilla into removing the feature or making it harder to use.
And this explanation is very likely to be entirely hallucinated, or worse, subtly wrong in ways that's not obvious if you're not already well versed in the subject. So if you care about the truth even a little bit, you then have to go and recheck everything it has "said".
Why waste time and energy on the lying machine in the first place? Just yesterday I asked "PhD-level intelligence" for a well known quote from a famous person because I wasn't able to find it quickly in wikiquotes.
It fabricated three different quotes in a row, none of them right. One of them was supposedly from a book that doesn't really exist.
So I resorted to a google search and found what I needed in less time it took to fight that thing.
And this explanation is very likely to be entirely hallucinated, or worse, subtly wrong in ways that's not obvious if you're not already well versed in the subject. So if you care about the truth even a little bit, you then have to go and recheck everything it has "said".
It cited its sources, which is certainly more than you've done.
Just yesterday I asked "PhD-level intelligence" for a well known quote from a famous person because I wasn't able to find it quickly in wikiquotes.
In my experience this means that you typed a poorly-formed question into the free instant version of ChatGPT, got an answer worthy of the effort you put into it, and drew a sweeping conclusion that you will now stand by for the next 2-3 years until cognitive dissonance finally catches up with you. But now I'm the one who's making stuff up, I guess.
No, I'm not "fully converted." I reject the notion that you have to join one cult or the other when it comes to this stuff.
I think we've all seen plenty of hallucinated sources, no argument there. Source hallucination wasn't a problem 2-3 years ago simply because LLMs couldn't cite their sources at all. It was a massive problem 1-2 years ago because it happened all the freaking time. It is a much smaller problem today. It still happens too often, especially with the weaker models.
I'm personally pretty annoyed that no local model (at least that I can run on my own hardware) is anywhere near as hallucination-resistant as the major non-free, non-local frontier models.
In my example, no, I didn't bother confirming the Russell sources in detail, other than to check that they (a) existed and (b) weren't completely irrelevant. I had other stuff to do and don't actually care that much. The comment just struck me as weird, and now I'm better informed thanks to Firefox's AI feature. My takeaway wasn't "Russell wanted to nuke the Russians," but rather "Russell's positions on pacifism and aggression were more nuanced than I thought. Remember to look into this further when/if it comes up again." Where's the harm in that?
Can you share what you asked, and what model you were using? I like to collect benchmark questions that show where progress is and is not happening. If your question actually elicited such a crappy response from a leading-edge reasoning model, it sounds like a good one. But if you really did just issue a throwaway prompt to a free/instant model, then trust me, you got a very wrong impression of where the state of the art really is. The free ChatGPT is inexcusably bad. It was still miscounting the r's in "Strawberry" as late as 5.1.
> I'm personally pretty annoyed that no local model (at least that I can run on my own hardware) is anywhere near as hallucination-resistant as the major non-free, non-local frontier models.
And here you get back to my original point: to get good (or at least better) AI, you need complex and huge models, that can't realistically run locally.
You can just look down thread at what people actually expect to do - certainly not (just) text summarization. And even for summarization, if you want it to work for any web page (history blog, cooking description, github project, math paper, quantum computing breakthrough), and you want it accurate, you will certainly need way more than Ollama 8B. Add local image processing (since huge amounts of content are not understandable or summarizable if you can't understand images used in the content), and you'll see that for a real 99% solution you need models that will not run locally even in very wild dreams.
Sure. Let's solve our memory crisis without triggering WW3 with China over Taiwan first, and maybe then we can talk about adding even more expensive silicon to increasingly expensive laptops.
Browser development is done by Mozilla Corporation which is a for-profit entity. It's illegal to donate to it. This is by design of the US tax code.
You can donate to Mozilla Foundation (parent entity of Mozilla Corporation), which is a non-profit. But you can't expressly state that the money go towards browser development.
Do I understand correctly that the parent nonprofit Foundation can decide to use some of its donor money to fund its for-profit Corporation (with the same tax treatment as any other investment, and of the corporation’s profits before they’re returned to the Foundation)? But donors can’t direct their gifts to that use if the donors still intend to deduct them as charitable donations?
And thus I guess Foundation has to do a good amount of conventional non-profitty stuff like “education and advocacy,” otherwise it would just be a flimsy facade for what’s substantially a for-profit endeavor?
Why is the browser arm organized as a for-profit at all?
It's the other way around, Mozilla Corporation is profitable and those profits go directly to the Mozilla Foundation which owns 100% of it.
This idea that Mozilla doesn't have enough money to fund Firefox is just wrong, Firefox development is perfectly sustainable, it earns more money than it spends. If you want to give money to the Mozilla Corporation instead of the foundation, you do the same thing as with any company: you purchase products from them (such as their VPN or MDN Plus, both of which are owned by the corporation).
> Why is the browser arm organized as a for-profit at all?
So that they can make business deals with the likes of Google, which they wouldn't be able to do as a non-profit.
Edit: I really wish there was a single thread about Mozilla here that doesn't devolve into this being like 80% of the comments. Maybe one day.
It is legal. But most for profit corporations don't solicit gifts because it isn't worth the compliance costs and risks. Some were punished when donors took tax deductions. Or the IRS decided their disclosures were inadequate. Or they overlooked a state or province regulation. And they were not associated with non profit foundations with similar names.
Anyone can give Mozilla Corporation money by purchasing services.
The Peter Thiel's part about the current situation not working for young people is right on the money. I am of the age where my kids (and those of my friends) are graduating from college. Every single one of them is having trouble finding work within their profession. Every one.
I've done reasonably well as a developer, having been an architect at several large enterprises. I consider myself a pretty good developer. My kid followed in my footsteps and also became a developer. He is objectively ridiculously good at it. And far faster than I ever was. But it took solid 9 months before he found a job. A one month contract. Which he then parlayed into a full time job, once they saw how good he was.
Compare this with when I graduated. I had a full time job before I received a diploma. It never even occurred to me to worry about finding work.
Yeah. Anyone who still doubts how bad things are: "interview" for a month. Just a month. It will be very apparent but many people who are content in their careers would never otherwise notice.
It's no wonder Gen Z is so cynical. People are talking about how good things are and they can't even get a human to respond to them for their first step in.
I think Thiel's right, though I despise the man immensly for unrelated reasons.
I grew up wildly excited about the capitalist world of promise and meritocracy in which I was going to grow to be a man. I never had grand ambitions of becoming wealthy, but wanted to work hard so that I coulod have a nice life and build a family.
Now that im an adult, I cant think of a single reason to be proud of the current system. Ive worked incredibly hard for a life of 'getting by'. Im incredibly privileged where my friends and family are not, and cannot imagine the struggle and hardship of those with less prospects than me.
I cannot in good conscience witness the world in which I am now an adult and say that this capitalism is a good thing. The people around me are suffering and I firmly believe that it is every man's duty to fight for their family and community. Nothing about fighting for community here involves supporting capitalism as it stands today.
Unfrotunatelly, it happens and I am aware of this. In that case:
a) you are lucky because your CV is not scannable by AI so it's good if you want to keepyoru carieer as far as possible from AI tools ;)
b) you are unlucky: most likely the software recruiters are using to pre-screen applications (and they are using it a lot) is not seeing your CV either so you will be dropped on the first stage, :( work on this if you consider finding new job nowadays
c) if you still want to use my tool consider extracting CV's text to .txt or reformat PDF (this will help you with point b)
I can't wait until someone does this, then autofills 50k rows down, then gets a $50k bill for all the tokens.
Reminds me of when our CIO insisted on moving to the cloud (back when AWS was just getting started) and then was super pissed when he got a $60k bill because no one knew to shutdown their VMs when leaving for the day.
If someone is processing 50k rows, that means they found real value and the UX is working. That's the whole point.
Also, 50k rows wouldn't cost $50k. More like $100 with Sonnet 4.5 pricing and typical numbers of input/output tokens. Imagine the time needed to go through 50k rows manually and math doesn't really work for a horror story.
I've also lived through that era and used OS/2 extensively. Display drivers were always the weak point of the system.
At one point, after a very prolonged struggle, I managed to get 800x600 resolution working (standard was 640x480). The gigantic sense of achievement I felt at that moment hasn't been repeated in 30 years hence.
Warp on, I didn't have video driver issues. I had a personal machine as well as a work machine (actually ran win32 and win16 builds onthe work machine) that never really had an issue.
From the article, it does look like things got better around then. I do recall running 2.1 as well but was more of a try it out thing. I actually miss OS/2
reply