While most of the interview is boring and nothing new has been said by Toner, to me, it feels like huge political fight where sama tried to overthrow her first and then Toner went in attack mode trying to overthrow him. It's clear she was the lead instigator and accidently found Illya as sympathetic co-conspirator. Board was obviously ok and took no action when they weren't informed about release of ChatGPT. It also didn't took action when sama didn't told them about startup fund. Only after Toner's paper and sama's move, all these became important.
Overall, I genuinely believe that the board needs to get out of way. They are not the founder, they are not even technical. They should do oversight for intentional and significant wrong doings but politics and brewing up secret coups is not their job.
I've seen this mentioned a few times, but is this true? I could swear I've heard him mention that he uses ChatGPT for programming stuff. He might not be a super technical software engineer, but is he "non-technical" in that he can't even write some simple python or something equivalent?
> He might not be a super technical software engineer, but is he "non-technical" in that he can't even write some simple python or something equivalent?
1. that's a very low bar, almost to the point of making any distinction meaningless
2. imho, "technical" and "non-technical" are context dependent, and not intrinsic human qualities. Speaking for myself : I'm a technical individual contributor on my current team, but there are plenty of domains (bleeding edge AI research likely being one of them) where my technical acumen would fall short of expectations for an IC, and hence where my most logical role would be non-technical in nature.
> They are not the founder, they are not even technical
OpenAI can't have it both ways. Either it's not a company owned by it's founders (as Sam claims) or it is. A board that does nothing ever is equivalent to the former.
My guess is that Mustafa wanted to sell to MSFT at 10X but MSFT didn’t wanted pay that kind of money. Mustafa was ok with fire sale but VCs were greedy. Mustafa then quite in rage.
They had raised massive amount and not from good patient investors. No traction means Mustafa got fired. This is not surprising though but what is surprising is MSFT picked him up. The guy is not technical, is not even visionary and had just got lucky hanging out with Demis. I would think Satya had better taste.
He also left DeepMind because of allegations of bullying employees (https://en.wikipedia.org/wiki/Mustafa_Suleyman). Between that, what you just brought up, and the strange PR blitz he went on to promote his book, I kind of predicted Inflection would run into major trouble well before OpenAI and Anthropic
There’s a whole extremely famous book series on this very topic, you might have heard about it. The first volume’s title is ‘Paul is bad’ but it’s more widely known as ‘Dune’.
I actually found the portrayal of Paul in the Villeneuve films to be closer to what Frank Herbert described as his intent in interviews when compared to the book.
That is, sincere with terrible consequences for bystanders. Not bad per se, but not conducive to the common good.
I also felt that the books undermined the warnings of tyranny by leaning on prescience to provide an ends justify the means argument. Moreso with God Emperor where the Golden Path seemed to be the ultimate end justifying any act.
Well, as Paul Atreides himself explains in the sequel to Dune, his Fremen Jihad causes the death of over 60 billion people throughout the empire. You can safely guess that the vast majority of them are innocent ordinary people, but just not important enough to pause over on the path to grand prophetic dreams. Basically, from when he takes power again, he becomes a genocidal monster in charge of an army of murderous fanatics, and then he spreads this right across the empire, well beyond Arrakis itself where he already caused all kinds of chaos.
The story is a classic narrative of hoped-for ends being used to justify horrible actual means, but cloaked in space opera.
No, it's in the second book actually, (spoiler alert) which covers the time in which Paul's rule has been firmly developed following his enormously bloody jihad across the Empire, and the time after his death, after which his son Leto succeeds hims and lets himself be symbiotically covered in baby sandworms to turn into the eventual god emperor of the Imperium, ruling for 3000 years. (If I remember correctly)
That would apply probably to most famous entrepreneurs. Whatever I have read about Jobs, Musk, Gates and others, they all are very willing to abuse people to achieve their goals.
It's crazy there's zero accountability for bad behavior in tech. I went through my own story at Google, and seeing them say it was vaguely bad before promoting him to VP mirrors exactly the "intervention" I saw.
The deck is completely stacked against you based on hierarchy. Behavior that a fast food manager would proactively solve in 30 seconds gets ignored in white collar tech. No one above you will even mention it - they know you can't win and they just hope you'll quietly give up.
If someone above you in the informal hierarchy is messing with you, there's massive confirmation bias if you complain. They'll spin it to whoever you complain to make you the bad guy. HR never helps - their job is to investigate, and then give the results to someone 2-3 steps above you to do something with.
The higher ups control the outcome, and they designed the power structure in the first place, their confirmation bias is accept the spin.
If you want to survive, avoid conflict 100% of the time. Let people blame you, fail reviews undeservedly.
My Google career ended from just doing exactly what I was supposed to do in order to get a 3 year delayed project done, that 4 separate VPs had been asking for all those years. I spent 6 months warning my manager fuckery was afoot. Didn't matter. TPM witnessed and defended me, didn't matter. Guy who led it hired his unqualified childhood buddy to replace me. Didn't matter. All on me. Everyone wanted to do it, and gee whillakers, refulgentis went mad and dropped the ball completely for some reason.
Of course, 6 months later they delayed the project a 4th year because they could, documenting the only downside being a strained relationship with a less influential partner team. (my orgs managers didn't realize their...unvarnished...takes were in a doc shared with all of Google)
At the end of the day, HR will funnel you into taking mental health leave -- 6 months worth, exactly long enough that an EEOC complaint can no longer be filed. (took me 6 years to realize why "disgruntled Google employee" news articles always included a bit referencing leave/6 months off as if it was a bad thing. go/mh-leave if you're at Google. You don't actually need to talk to HR, and I don't recommend going to them ever. I didn't for this, but they wouldn't have helped.)
>Guy who led it hired his unqualified childhood buddy to replace me. Didn't matter. All on me. Everyone wanted to do it, and gee whillakers, refulgentis went mad and dropped the ball completely for some reason.
How is that possible at Google, which should have a hiring committee? Managers aren't allowed to just hire rando person.
Did your director or VP not like you or something? I'm curious if there's more to this story.
Why didn't higher ups care? I didn't bother going up higher than skip. My core interlocutor was my skip's peer's report's report, I didn't expect my skip to go to war over slow-drip white collar bullying. Honestly, I was done and planning my exit once year 3.5 of "not this year" hit, going crying to VPs you see once a month / once a quarter felt insane & would have just devolved to he said/she said.
To your point re: seems like a lot. I worked with a couple counselors at same level as my skip over my last year there. (highly recommend G2G if anyone reading is at Google, kept me sane.) #1 said they dealt with less after kissing their VP's wife at an off-site - which is why I got #2, wasn't sure if that one was too skeezy at first.
What would I have done differently? I was honest the whole time, which didn't help because the fact I wasn't happy and it was escalating was clear. ex. with the hiring friend thing, told my manager that I was shocked and didn't expect that kind of thing at a startup.
How do you hire a friend without domain experience as a manager?
There's a core principle that once you've made it through Google interviews, domain doesn't matter, all Google SWEs will excel. Having loose rules with kind intent is awesome, but they're double-edged. That gives you rationale, and combined with moving recruiters into individual orgs., lets you put your thumb on the scale and bring in who you want. Also 2022 through June, Google was desperate to hire, this would have been justified as an awesome referral, and what's a friend, anyway?
> lets you put your thumb on the scale and bring in who you want. Also 2022 through June, Google was desperate to hire, this would have been justified as an awesome referral, and what's a friend, anyway?
Ok, I guess you're saying a manager brought in his friend, who was already hired by google (or passed HC), but they didn't get them hired at Google, right?
>#1 said they dealt with less after kissing their VP's wife at an off-site
Lol please tell me this is an official story I can find somewhere???
Anyway, I've heard Google is extremely slow with firing folks, even ones who are abusive. But I do see people get fired, though usually after many years of a pattern.
> Ok, I guess you're saying a manager brought in his friend, who was already hired by google (or passed HC), but they didn't get them hired at Google, right?
Exactly, you're right, they still went through interviews.
One person’s “politics” is another person’s “demonstrate basic empathy and understand that your point of view is not universal.”
Yes, there are creatures who have no real skills other than navigating political currents. But there are also creatures who can’t understand that technical brilliance is no excuse for utter social cluelessness.
You seem to think that people are either good self promoters or socially total clueless. There are a lot of very good people who have normal social skills but aren't good at self promotion (or just don't want to do it). These people won't go very far in most organizations. Self promoters without real skills will win over them. Best is to have real skill and also be a good self promoter.
Of course there's a spectrum here. But when I read someone complain "I did great technical work but POLITICS," there are several possible scenarios:
1. The engineer is reasonable and the people they complain about are craven self-promoters with no real skills
2. The engineer is unreasonable and the people they complain about have normal, regular, business-normative expectations of engineer conduct.
3. Both - the engineer is bad and the self-promoting people are bad.
Your scenario lines up with #1, i think.
I see all three happening. I see many engineers who are gruff, entitled, lack the ability to talk about anything other than their own work, cultivate perceptions of technical status, and seem to actively want to make everyone avoid them. I also see social climbers (#2), but they are easy to spot and not that common in my environment (though others are of course different, including at previous companies).
That's the secret of higher education! Forget about the coursework, forget about the social aspect, forget about exposure to ideas. The actual value of universities is that you keep kids busy during the time when they are potentially destructive, and when they pop out they are now magically older and can handle some basic adulthood-on-training-wheels expectations.
And yet a few hundred years ago we had 14-year-olds in the mines of Cornwall supporting whole families.
It is especially horrible for new Amazon employee because of their tiered stock vests. You might have left your great job because Amazon offered you half million signing bonus but you won't last long enough to get even 30% of that.
That's not how Amazon's comp works. The sign-on bonus is prorated over the first two years to fill in the "gap" before hitting the normal RSU vesting cadence. Someone hired one year ago leaving today would "only" vest 5% of their RSUs, but they would have received ~35% of their initial RSU value in cash over the last 12 months. Someone hitting their second anniversary would "only" vest another 15% of their RSUs, but they would have received another ~25% of their initial RSU value in cash over the last 12 months.
I have tried using LLamaIndex and APIs are just aweful. It's a nightmare of class hierarchies and abstractions that only author can wrap their heads around. Comments are basically useless classic style of rewriting the function name. Just look at this example:
def get_context(self, response: Response) -> List[Document]:
"""Get context information from given Response object using source nodes.
To be honest - I had the opposite feeling - I found it pretty easy to wrap my head around their abstractions. If anything - I believe their abstractions are quite clean as I was able to navigate their codebase quite easily. Their "context" abstraction was an interesting choice admittedly but that's the only non-intuitive thing I found.
Disclaimer: I worked in this space for a few years as an ML engineer so it might be quite different for new entrants in the field.
Most people don’t know some history. During 1990s, a group of people made a fortune out of consulting gigs where they will be called in by their CTO friends in traditional enterprises to save the late and over budget projects. One of these people was Kent Beck. Kent will use his license to kill to turn things around and eventually generalize his rescue formula and sell it to make 100X more. His crowning glory during those days was XP or eXtreme Programming.
Like with all self-help formulas, Kent will label his solution as magic bullet for all software development problems. He will advertise it as secret medicine that cures all ills. He will be at every conference, write articles after articles, publish books.
Also, like all magic self-help formulas, it wouldn’t quite work. So, Kent will invent something new. His next prescription was TDD and when I first saw it, I thought it was a joke. But people around me started drinking cool aid and if you didn’t join them then you weren’t one of them. Again, Kent and friends will go out on massive marketing spree advertising it as secret talisman. Like all overweight desperate people in need to lose weight, people will enthusiastically start new Kent Beck diet, lose few pounds and endorse the formula. But they will soon find that they had simply traded one problem for another more uglier one.
This went on for long time. For more than two decades, these group of people kept inventing these processes, selling it as magic pill and made millions upon millions in consulting gigs, books, training, certifications and so on. They came up with Agile and 17 people in that group created “agile manifesto”. Their most aggressively marketed prescription was scrum. Like their all previous prescription, world is finally coming off of night of drinking cool aid and feeling severe headache.
I think most of these people have now sort of retired after amassing massive fortunes and hopefully we will not see more of these magic processes pushed to dumb CTOs with promises of curing all ills.
The truth is Scrum was never a magic bullet and it is downright harmful for many projects. It is useful for highly predictable projects where research component is negligible, for example, CRUD websites AND where you are stuck with unmotivated tier-3 talent who failed to get job at insurance company. For everything else, it should never have been used. It is especially going to hurt creativity, originality and novelty if you are in business of making a differentiating unique novel product. It also is very very bad choice if you already had tier-1 highly motivated team.
I think even a tier-1 team could make it work for them, but the key is they would make it work for them (make it their own process).
Once you hire a scrum master to tell you how to do your work, you've sort of lost. They are rarely useful other than as sort of "priest" of the process, who ends up becoming another sort of management, but without management powers (usually).
The meetings etc. can be downright useful, in certain cases, but don't really make sense to follow religiously. I.e. if the devs themselves are running the meetings, for themselves, its a useful form of self-management, and you don't need much management skill to run it (any seasoned dev with any sort of communication capability can do it).
But if its imposed upon you, its just a cookie-cutter sort of management, which, doesn't fit all teams or scenarios.
Second, even XP is not a "magic bullet". Nothing is. It's work that works. (Scrum, on the other hand, is not a "magic bullet" but simply a "bullet". Use it to kill projects very effectively).
Third, at my first real job after uni, we did most of the XP-like practices, and it worked amazingly well. But we didn't know about "XP". Partly because it didn't exist yet, as this was around the same time the Kent Beck started at Chrysler Comprehensive. When the XP books came out it was fun to have a name for what we had been doing so successfully. And also to compare and contrast.
Fourth, I had a great side-by-side natural experiment during my tenure at the BBC. My team did XP-ish things, mostly the technical practices, so test first/TDD, YAGNI etc. Pairing when necessary, but we were co-located around a desk "island" (sort of the way journalist workspaces are organised). My team succeeded far beyond expectations [1]
The team next to us, larger, more important and with way more experienced developers did SCRUM, but not XP. That project had to be rebooted completely after 2 years.
> Their most aggressively marketed prescription was scrum.
I don't think Agile has prescribed this though. Scrum, in my view, is an intermediary 'solution' so non-technical 'bosses' can overlook and micromanage dev teams. I guess it all stems from 'unproductivity' really, those cases you mention, where you end up with sub-par devs trying to deliver complex software products.
If you already know _exactly_ what your code needs to do, you can "just implement it".
I find TDD to be very helpful in the cases where I do _not_ know everything in advance, because it lets me take small steps to explore things and I get very fast feedback if I "misstepped".
All of the methods mentioned are based on a reasonable core. It muddles the waters and make the snake oil marketing - the "this is the cure to it all" discourse - harder to dismiss. Testing is good. Planning is good. Discussing the project is good. But these things are beside the point of op. The point is, these cargo cults are designed to make consultants money. Now they have a lot of inertia because people grow up on it.
It seems like the new generation of software development silver bullets is "microservice", cloud "devops" etc... Managed kubernetes is not a bad thing. Configuration files, software defined infrastructure, etc, not bad things at all. But there is a definite market push in consulting for overtly complicated frameworks as The Way and people who are anxious about their complicated projects gobble it up.
>The truth is Scrum was never a magic bullet and it is downright harmful for many projects. It is useful for highly predictable projects where research component is negligible, for example, CRUD websites AND where you are stuck with unmotivated tier-3 talent who failed to get job at insurance company.
Superconductors are basically perfectly conductive wire. Wires that transfer 100% of power over arbitrary distances and that don't heat up. Obviously there are limits, you can't put arbitrary power over a hair thin filament but as long as you're under that limit you get perfect efficiency.
MRI machines can be made a lot simpler as you no longer need to use liquid nitrogen to cool the superconductors. MRI machines could end up being small and cheap.
Perfectly efficient electromagnets make a lot of problems in fusion reactors simpler, I'm not sure that room temperature superconductors make fusion reactors instantly viable but it's a big step and would reduce the energy requirements for a fusion bottle by a lot.
Basically anything involving electromagnets becomes a lot more efficient. Motors can be made smaller, generators can be made much more efficient for the weight, maglev trains can require very little power to hover. It has effects on almost every industrial process as it fundamentally changes the weight and energy efficiency of anything involving electromagnets.
One neat things would be surgical robots that can work as an MRI while also levitating a small blade in a 3D space. Challenging for sure but when you can replace complicated liquid-nitrogen cooled coils with an array of simple passive coils a lot of options open up.
Superconductors can also be used for power storage, and at room temperature that becomes a lot more viable.
A note about MRI machines: they use liquid helium, not liquid nitrogen. LN2 isn't cold enough. Being able to eliminate liquid helium would be huge, as helium is scarse and quite expensive. Its roughly 10x the cost of LN2 and only going to get more expensive.
Previous improvements in high-temperature superconductors already made it possible to build a MRI machine using LN2 instead of LHe. I think all existing operational units still use LHe, but using LN2 has been demonstrated in lab conditions, and the next generation of machines will almost certainly use it instead of helium.
It still might be worth cooling this with LN2, in many applications, assuming critical current and critical field scale up as temperature decreases as they do with other superconductors.
It takes a long time to validate new stuff for medical devices. Even if this discovery completely pans out, there will be two or three generations of MRI machines based on LN2-cooled superconductors.
They use both liquid helium and liquid nitrogen. The nitrogen is used to cool the helium. On MRI scanners that have come to market in the last few years, helium volume has been reduced at least 100x and is now only a few liters (i.e. previously >1000L and requiring frequent top off to <1L and requiring refill only after emergency/full power loss).
Superconductors are a transferring energy technology, not a storing energy technology. Although they would likely augment the efficiency of storage technologies.
> What kind of energy density could we get using it for energy storage?
Actually, not a lot. The are some very compelling uses of them for storing energy, but they are much more relevant for distribution grid stability and control than for raw energy storage.
There are people here are pushing some really non-compelling use cases (like long distance power distribution), but there are plenty of transformative ones.
(But the thing is that this one on the paper is much less useful than it could be. There is still some work on understanding why and fixing it.)
Don't forget about computer chips that do not emit heat. So much wasted power at the datacenter scale simply to keep things cool. At a personal computer level things get way more efficient, too, resulting in cheaper, smaller, quieter computing devices.
Superconductors change every assumption about how we harness electricity and magnetism. Beyond reducing the cost of electricity transmission, they enable all sorts of fascinating applications:
- They enable low cost, continuous, passively-stable magnetic levitation. Superconductors could replace ball bearings in many applications.
- They enable permanent magnets that are far stronger than any we make from conventional magnetic materials. For example, motors tend to run at high speed and low torque, so as to minimize heat generated from current in the copper windings. Superconducting direct-drive motors could allow for ultra-high-torque actuators without any need for gearing, and with minimal heat generation or losses. So superconducting electromagnets could replace everything from electric motors to hydraulic pistons to simple springs.
- Superconductors allow for very sensitive antennas and magnetic field sensors, allowing for near-field detection of very small signals (such as from neurons firing in the brain). There is a lot of impressive technology that only exists inside research labs where a generous supply of cryogenic liquids are always on hand. Those could make their way into mass-market products.
Something that immediately comes to mind for me in Sweden, is that the country is fairly long in latitude, and most of our electricity production is from hydroelectric power in the northern half of the country, while most of the population is in the southern half of the country. Better energy transmission could help a lot.
It probably wouldn't greatly affect the heat generation in a PC, unless the transistors could themselves be replaced with some superconducting alternative. Harnessing the efficiency from that would probably require that the computer be designed as a reversible computer. It would be its own research avenue.
Unfortunately, as soon as you actually use the result of the computation in any kind of practical manner as an output, you break reversibility, though you could make the heat production happen away from the computation.
The idea of reversible computing is that if you only add heat in a few instructions, you can have a much more economical computer. And magnetronics is a good candidate for implementing this, so yeah, computers that use a lot less power are an application too.
I haven't seen any reversible low power superconducting gate that can credibly operate at a high temperature - not because of the superconductor itself, but because of thermal noise. Again, I haven't read through the literature in this field for a while (and it wasn't that extensive either), but from what I recall what you're proposing is roughly as difficult as making a gate for a quantum computer, and you have to keep your system way colder than your critical temperature from that due to thermal noise. If you have any links for high temperature physically reversible logic gates I'm all ears.
I don't think you actually need reversibility if you don't discard the energy but return it to the power supply?
In other words, "reversibility", but you can actually pool the useless results together, you don't need to separate them later. Or so I read somewhere...
I might be wrong since I've studied this a long time ago, but from what I remember, in order to do that classically, you need to copy the output bits somewhere else before uncomputing your system and recovering the ancilia.
That's technically fine, as long as you have an infinite supply of stably initialized bits onto which to copy your result. Initializing those bits is going to be non-reversible in some way.
Computation inherently generates heat, but if you could make chips that release negligible amounts of heat, you would unlock the third dimension which would help with reducing signal length and enable computers to be significantly faster.
That this as a solution applicable to _personal_ computing is a bonus. The real benefit is in datacenters which could be made smaller, more efficient, and cheaper while simultaneously adding capacity.
Other commenters have science fiction dust in their eyes, and speak of room temp superconductors in general. But this particular discovery is a brittle crystalline structure that cannot be extruded into wires, and does not have the high current capacity required for power transmission or rail-guns.
It's an important, exciting step but it's very far from world-changing at this stage. Or if it is in a limited way. The first transistors were clunky affairs, of limited usefulness, world changing for ship-to-shore communication in the military. But then people discovered how to make them with deposition instead of factories, and they got smaller and faster, and they really did change the world. We're in the "clunky transistor" period.
Exactly. But that clunky transistor was in fact world-changing. It just took a while for the changes to take place but the stage was set when that first device showed that it could be done at all.
Oops, the first practical radios were powered by semiconductor rectifiers, not transistors. The Pickard silicon point detector circa 1906 was used in WWI (btw owning/making a radio was illegal during the war!)
I believe MRIs use superconductivity, so I assume any application of superconductors that doesn't require heavy, large, energy-consuming cooling will benefit greatly.
Perhaps MRIs will become ubiquitous and cheap, something we all get every time we go to the doctor?
Superconduction also has some weird magnetic properties I believe, so there could be benefits regarding maglev transport.
And finally and most basically, the movement of electrical energy across potentially large distances with zero loss would be a great thing.
I have no real idea what I'm talking about but figure 1 has critical magnetic flux curve ranging up to 3000 Oe so... in MRI-speak maybe it tops out before 0.3T? IIRC permanent magnet MRI have already been built in the 0.3T range, but they're very heavy and outclassed by the higher-field scanners. Clinical MRI nowadays typically runs at 1.5-3T (with some clinical scanners at 5-7T).
Having said that there is a resurgence of interest in low-field MRI lately, primarily marketed for use in developing nations and for combination machines that integrate radiation therapy. From what I've heard from diagnostic radiologists, the low-field MRI scanners seem to be of limited diagnostic value on their own.
Anyway that's just my thought that the best/first applications here may not be about generating magnetic fields.
I assume we could make CPUs stupid fast if we didn't have to worry about heat as much, though I'm not sure how much is lost to resistance vs operating transistors.
Below summarizes the epidemic at many labs, especially in AI. If you want funding, GPUs and head count then you must prove yourself to be in top 5. The result is relentless pressure for the results that can create headlines.
It identified a culture where Tessier-Lavigne “tended to reward the ‘winners’ (that is, postdocs who could generate favorable results) and marginalize or diminish the ‘losers’ (that is, postdocs who were unable or struggled to generate such data).”
Most people don’t know this but OpenAI is actually very dysfunctional company and only accidentally successfully because of just handful of right people having a rather lucky hunch (Schulman, Redcliff et al). However, execution wise they are really really poor. You can see this i myriad of problems such as their dataset is still stuck in 2022, plugins was a complete disaster, web browser mode sucks, app is poor, unable to release 32k and multimodal, rate limit even for paid users etc etc.
All thesis problems are now even more aggravated by their recent massive hiring spree of AI doomer crowd and Yudkowsky’s cult members instead of actually doing real research. Now the company is full of doomers whose sole job is to slow things down and be barrier to efforts. Meanwhile Bard has been making amazing progress. It’s free, doesn’t log you off all the time, it always feel latest and very close - if not better than current limited ChatGPT. Given OpenAI’s new staffing composition, they are unlikely to be leader down the road, especially when Gemini comes out. This is unfortunately sama’s second failed execution. He should probably just focus on investments.
Totally agree. Actually a couple of months ago Sam Altman even admitted that they had a very hard time doing proper "engineering" (meaning that they had the right team to create a very good LLM but not the right team to productionize and their models and APIs). Many people are actually finding the OpenAI API very unstable and do not plan to rely on OpenAI for their production workloads.
Overall, I genuinely believe that the board needs to get out of way. They are not the founder, they are not even technical. They should do oversight for intentional and significant wrong doings but politics and brewing up secret coups is not their job.