I’m always curious why local models aren’t being pushed more for certain types of data the person is handling. Data leakage to a 3rd party LLM is top on my list of concerns.

I am not as concerned with that with API usage as I am with the GUI tools.

Most of the day gig is structured extraction and agents, which the foundation LLMs are much better than any of the small models. (And I would not be able to provision necessary compute for large models given our throughput.)

I do have on the ToDo list though evaluating Textract vs the smaller OCR models (in the book I show using docling, their are others though, like the newer GLM-OCR). Our spend for that on AWS is large enough and they are small enough for me to be able to spin up resources sufficient to meet our demand.

Part of the reason the book goes through examples with AWS/Google (in additiona to OpenAI/Anthropic) is that I suspect many individuals will be stuck with the cloud provider that their org uses out of the box. So I wanted to have as wide of coverage as possible for those folks.

Worth noting that AWS Bedrock makes it easy to have zero retention with premier claude models. Not quite local, but it feels local-adjacent for security while getting affordable access to top-performing models... GCP appears to be a bit harder to set this up.

IMO Google Vertex is not any harder than AWS. AWS biggest pain is figuring out IAM roles for some of the services (batching and S3 Vectors -- I actually cut out Knowledge Bases in the book because it was too complicated and expensive). Have not personally had as big an issue figuring out Vertex.

I do have a follow up post planned on some reliability issues with the APIs I uncovered with compiling the book so much -- I would not use Google Maps grounding in production!

but they claim your data is private and they will totally not share any of it with their advertising partners!

I’ll die on the hill that the Ultima 7 games are the absolute best RPGs ever made. So beyond their times and hold up incredibly today.

The privacy issue still exists in this scenario.

All doctors (including concierge) use 3rd party services for practically everything from blood work to imaging to application products. It’s safe to say that it’s very likely that they’ll outsource the genetic testing to a 3rd party

All of them need to follow HIPAA. That's as close to a protection as you can get right now.

Sounds like you preferred it when it was controlled by liberal interests.

Next time Dems win elections (if that's ever), it will be. Politicians on both sides are happy about this.

It's only a problem for the GOP if Dems do win elections again. Judging by how things are progressing, it may not happen anymore. They have complete control over everything.

Liberal interests ≠ Democrat government

They aren’t saying that - and you know they aren’t. They’re also correct.

Fediverse, in my actual experience, is overwhelmingly filled with hyper left leaning echo chamber content that I’m not interested in.

I know they aren't? The person hinted at something but didn't say anything concrete, so I don't actually know what they're saying, which is why I asked them to clarify.

I've been using mastodon for nearly 10 years. Fediverse, in my actual experience, is overwhelmingly a really good place to meet and chat with interesting, friendly people. Follow and engage with cool people and you'll have a good time. If you go looking for what you don't like, you'll find it, because it's there just like anywhere on the net. The difference is you don't have to see stuff you don't want to see, because the software and protocol have features that allow this. I'm on a pretty small server and we all are pretty much on the same page about what we expect our social interactions to be like. You know you can just filter out content, right? Your server can totally block or defederate from servers that have objectionable content, too (by whatever your collective definition of "objectionable" is). It works pretty well if you use it the way it's designed to be used.

It's known that reality has a strong left-wing bias. Can you specify further?

Maybe yes? Maybe no? This has been an ongoing situation with the UK and demanding backdoors into US platforms - I’m not convinced them dropping it came down to “striking a secret deal”.

I think this is a bit of a sensational take. The code being executed is all there without obfuscation.

Not at all. The code that you see is not guaranteed the code that curl will receive. And even if you check the curl output, if you run it a second time to pipe to sh it might receive something different.

I've been in infosec for the past 14+ years and hiring these types are pretty nuanced and complex. On one hand, you have a person who shows their ethics are questionable at best. Do you want those folks having the proverbial "keys to the kingdom"?

On the other hand - people make mistakes and learn. And these types of folks are decently effective at what they do - although I will say the fact they got caught demonstrates they're not THAT good.

I'd probably pass on this specific person for the latter reasons.

That terminal and even the associated game look incredible!

Metrix!!! I used to live across the street from them and would go there often. Capitol Hill in Seattle is outrageously expensive.