So your "evidence" is the transcript of an interview that references an indictment containing information that hasn't been publicly substantiated.

There was also testimony to the Congress by its CEO which wasn't very convincing. There is a dude running for congress in Maine who has a troubling political background due to his past associations and this guy could not call him out. I'm pretty sure if the congress hopeful were running as an IND or repub he'd be treated differently by this organization in question.

Even if they're more productive, the amount of work to be done has also increased significantly. Not only is there so much work to be done on AI integrations but all of the other products have bug trackers exploding with new privacy and security vulnerabilities that need to be addressed.

Some of the security people I work with claim that without the productivity gains from AI, there is no way they could keep up with the bugs.

Theoretically, that's true. In practice the gains were limited to the wealthy who use that money to fund increasingly deranged start-ups* until the market crashes. Then rinse and repeat. The failed start-ups effectively waste any increased productivity leaving everyone about where they were before.

*They fund these start-ups to get a good return on investment so they can get even more money. As the economy overheats the number of places to invest with a reasonable return falls so they are left with the high-risk stuff to invest in. I'm not sure what they want the money for, though, since they could already afford most of the things I would find useful...

I think funding deranged startups is a type of consumption, and it does get money back into the economy the same way funding a remodel does. Maybe the reasoning to do so is different, and maybe the deranged startups add more capital than a bathroom remodel, but then again, bathroom remodels probably tank less often than startups, and worst case you can always read a magazine in your new bathroom. hard to do that in a dead startup

How do you handle object disambiguation with YOLO? All the examples I've played with have the problem where if two "cars" get too close to each other then the tracking IDs keep switching between them, meaning we'd need an additional kinetic model for disambiguation.

Ick. The entire article starts from the fundamentally flawed premise that "you want a function that takes a blob of memory as an argument". Then they discuss bytewise access into structures..

Passing around void pointers is simply not a safe thing to do in C++. You can't do anything with a void pointer, so you're probably going to cast it as something else. Use that type instead, so that your caller knows they need to pass a valid pointer to that type. If the pointer has the wrong alignment then that will result undefined behavior. If you need to support multiple pointer types, use templates.

And, unless there are some really weird circumstances, you actually don't want to access your structures bytewise. Offsets can shift with compiler flags/versions. If you want serialization , please use a serialization library that correctly handles all of the odd cases. These can be quite efficient.

I've only actually had to munge bytes in a class once. Somebody decided that a previously POD class that was passed between processors with different memory spaces needed a virtual function, so I had to overwrite the vtable when I received it to make it valid.

In general, you're right.

The exception that I could think of is a "dump memory" function. You take a pointer to something (who cares what it is), and print out the bytes there. That I could see taking a void*.

But that's a really limited case. In general, yes, you do not want to be dealing with blobs of memory as arguments. You want to be dealing with things that are known to be the right kind of thing as arguments.

But parent is right, you have to cast it anyways before reading from it, so might as well take the right type from the beginning.

Anything can be aliased by char, unsigned char, std::byte (as well as signed char in C), and usually uint8_t == unsigned char, thus by extension any valid void pointer can be cast to u8*.

Thus void*+size is usually the right type if ones only care for the memory representation of an object (cstring functions like memcpy, etc.)

Most likely one would have both overloads:

    void Hexdump(const void *p, size_t size); // (1)
    
    template<typename T>  // (2)
    inline void Hexdump(const T &obj) {
         return Hexdump(&obj, sizeof(T));
    }

With (2) being a wrapper to (1) that compilers will almost always inline, avoiding monomorphization costs (and (2) can also accept rvalues as argument).

(1) could also take std::span<const u8>, but (void*, size) is the more common idiom, more convenient to use and to read , as it is unambiguous which overload it is.

Why not both?

Chess games are in their training set, other games are not.

Chess is representable entirely in text as well, and generally speaking the LLM concept of "picking the next best token" fits pretty well for "picking the next best move" where a move is a text token

That representation is also old, incredibly well documented, and used to describe how to reason about chess. There are of course text guides to other games in training data but they rely upon depictions of what’s happening that aren’t purely text so the game harness is always going to have to make novel decisions about represent the game as text.

But there's nothing stopping AI from developing domain expertise. If you fine tune a model based on the records of all previous work (effectively "shadowing" the existing workers) then it can easily learn this domain knowledge. The only difference is that AI companies have gone after programming domain knowledge first. Others will come later.

I don't see how it would be possible for inference costs to dominate training costs, even after amortization.

Training involves multiple passes over the entire training dataset, ideally in large batches where you can perform inference on as many samples as possible simultaneously and then perform backpropagation to adjust the model weights (which is about as expensive as inference).

Let's consider the size of the dataset we're dealing with here. The dataset likely consists of practically every piece of digitized text they can get their hands on (including that extracted from audio and video). We know Google has digitized a large portion of the books in existence as part of their "search book contents" feature and we have no reason to believe they're not using it alongside their cache of 90+% of the internet to train their models. We're talking about 100s of millions of books each with an average of 100,000s of tokens. The internet has 10s to 100s of billions of pages on it with who knows how many tokens on average. This is a huge dataset that we've got to go through hundreds of times.

Second, let's consider the effect of batching and how it sets requirements for our hardware. We know that larger batch sizes converge faster, are more stable, and produce better models. So if you want a good model you need large batch sizes. This means that you need machines several orders of magnitude more powerful than you use for inference. From what I heard Google uses clusters of 100s of the their TPUs all located in a single rack for training. These clusters are organized in a customized computing architecture to maximize memory locality between cores (really critical for efficient back-propagation). Further, you can't use reduced precision weights for training like you can for inference, so there are no shortcuts.

Finally, the initial training stage is followed by reinforcement learning stages - this is key development in how AI models have improved in the past year. This may mean going through a curated set of traces (either synthetic or captured from users) and adjusting the weights based on experienced outcome.

Overall there's so many orders of magnitude more work and more hardware requirements for training that I find it improbable that inference dominates. The number of "inference" steps in training is freaking ridiculous and includes such factors as the "number of words ever written".

It's been a while since I saw a detailed paper on a high end training run, but extrapolating from what I remember, it seems those training runs are in the 10s of trillions of tokens. This already accounts for potentially sampling tokens multiple times during the training run.

That seems like a large number, until you realize that OpenAI claims to have almost a billion weekly users. And OpenRouter shows many models at over a trillion tokens per week.

So in pure token terms, I'd say it is in fact extremely plausible that inference dominates, at least for the popular models.

Not saying you're wrong, but I'll note why inference might dominate despite everything you mentioned.

A given model is trained once but applied N times. A large enough N will dominate training, no matter how complex and costly it was.

But how long is a model useful for? How often will labs need to train new models? Time will tell.

This statement is well known to be incorrect for at least a year.

You forgot the /s.

Or did you forget all the journalists kicked out of the Pentagon, the exclusion of the associated press over their use of the internationally recognized name for the Gulf of Mexico, the threatened cancellation of FCC licenses of ABC and CBS regarding their reporting, etc.