I do some electrical drafting work for construction and throw basic tasks at LLMs.
I gave it a shitty harness and it almost 1 shotted laying out outlets in a room based on a shitty pdf. I think if I gave it better control it could do a huge portion of my coworkers jobs very soon
I just can't imagine we are close to letting LLMs do electrical work.
What I notice that I don't see talked about much is how "steerable" the output is.
I think this is a big reason 1 shots are used as examples.
Once you get past 1 shots, so much of the output is dependent on the context the previous prompts have created.
Instead of 1 shots , try something that requires 3 different prompts on a subject with uncertainty involved. Do 4 or 5 iterations and often you will get wildly different results.
It doesn't seem like we have a word for this. A "hallucination" is when we know what the output should be and it is just wrong. This is like the user steers the model towards an answer but there is a lot of uncertainty in what the right answer even would be.
To me this always comes back to the problem that the models are not grounded in reality.
Letting LLMs do electric work without grounding in reality would be insane. No pun intended.
You'd have to make subagents call tools that limit context and give them only the tools they need with explicit instructions.
I think they'll never be great at switchgear rooms but apartment outlet circuitry? Why not?
I have a very rigid workflow with what I want as outputs, so if I shape the inputs using an LLM it's promising. You don't need to automate everything; high level choices should be done by a human.
I've been using pyrevit inside revit so I just threw a basic loop in there. There's already a building model and the coworkers are just placing and wiring outlets, switches, etc. The harness wasn't impressive enough to share (alos contains vibe coded UI since I didn't want to learn XAML stuff on a friday night). Nothing fancy; I'm not very skilled (I work in construction)
I gave it some custom methods it could call, including "get_available_families", "place family instance", "scan_geometry" (reads model walls into LLM by wall endpoint), and "get_view_scale".
The task is basically copy the building engineer's layout onto the architect model by placing my families. It requires reading the symbol list, and you give it a pdf that contains the room.
Notably, it even used a GFCI family when it noticed it was a bathroom (I had told it to check NEC code, implying outlet spacing).
I'm going to try to get it to generate extrusions in Revit based on images of floor plans. I've tried doing this in bunch of models without success so far.
You might want to give it some guidance based on edge centers? It'll have a hard time thinking of wall thickness and have it draw points if you're trying to copy floor plans.
for clarity now that I'm rereading: it understands vectors a lot better than areas. Encoding it like that seems to work better for me.
I would really love a magic wand to make things like AVEVA and AutoCAD not so painful to use. You know who should be using tools to make these tools less awful? AVEVA and AutoCAD. Engineers shouldn't be having to take on risk by deferring some level of trust to third party accelerators with poor track records.
I think that, much like LLM’s are specifically trained to be good at coding and good at being agents, we’re going to need better benchmarks for CAD and spatial reasoning so the AI labs can grind on them.
A good start would be getting image generators to understand instructions like “move the table three feet to the left.”
You twisted one "goalpost" into a tangential thing in your first "example", and it still wasn't true, so idk what you're going for. "Using a wrench vs preliminary layout draft" is even worse.
If one attempted to make a productive observation of the past few years of AI Discourse, it might be that "AI" capabilities are shaped in a very odd way that does not cleanly overlap/occupy the conceptual spaces we normally think of as demonstrations of "human intelligence". Like taking a 2-dimensional cross-section of the overlap of two twisty pool tubes and trying to prove a Point with it. Yet people continue to do so, because such myopic snapshots are a goldmine of contradictory venn diagrams, and if Discourse in general for the past decade has proven anything, it's that nuance is for losers.
The problem is how we use it. A human sees not a photo but a video, and has long context before and after, not just that instance, we can also change position, a LLM can't do that at all.
> Remember when the Turing test was a thing? No one seems to remember it was considered serious in 2020
To be clear, it's only ever been a pop science belief that the Turing test was proposed as a literal benchmark. E.g. Chomsky in 1995 wrote:
The question “Can machines think?” is not a question of fact but one of language, and Turing himself observed that the question is 'too meaningless to deserve discussion'.
The Turing test is a literal benchmark. Its purpose was to replace an ill-posed question (what does it mean to ask if a machine could "think", when we don't know ourselves what this means- and given that the subjective experience of the machine is unknowable in any case) with a question about the product of this process we call "thinking". That is, if a machine can satisfactorily imitate the output of a human brain, then what it does is at least equivalent to thinking.
"I believe that in about fifty years'
time it will be possible, to programme computers, with a storage capacity of about 10^9, to
make them play the imitation game so well that an average interrogator will not have
more than 70 per cent chance of making the right identification after five minutes of
questioning. The original question, "Can machines think?" I believe to be too
meaningless to deserve discussion. Nevertheless I believe that at the end of the century
the use of words and general educated opinion will have altered so much that one will be
able to speak of machines thinking without expecting to be contradicted."
Turing seems to be saying several things. He writes:
>If the meaning of the words "machine" and "think" are to be found by examining how they are commonly used it is difficult to escape the conclusion that the meaning and the answer to the question, "Can machines think?" is to be sought in a
statistical survey such as a Gallup poll. But this is absurd.
This anticipates the very modern social media discussion where someone has nothing substantive to say on the topic but delights in showing off their preferred definition of a word.
For example someone shows up in a discussion of LLMs to say:
"Humans and machines both use tokens".
This would be true as long as you choose a sufficiently broad definition of "token" but tells us nothing substantive about either Humans or LLMs.
The turing test is still a thing. No llm could pass for a person for more than a couple minutes of chatting. That’s a world of difference compared to a decade ago, but I would emphatically not call that “passing the turing test”
Also, none of the other things you mentioned have actually happened. Don’t really know why I bother responding to this stuff
Ironically the main tell of LLMs is that are too smart and write too well. No human can discuss the depth of topics they can and no humans writes like a author/journalist all the time.
i.e. the tell that it's not human is that it is too perfectly human.
However if we could transport people from 2012 to today to run the test on them, none would guess the LLM output was from a computer.
That’s not the Turing Test; it’s just vaguely related. The Turing Test is an interactive party game of persuasion and deception, sort of like playing a werewolves versus villagers game. Almost nobody actually plays the game.
Also, the skill of the human opponents matters. There’s a difference between testing a chess bot against randomly selected college undergrads versus chess grandmasters.
Just like jailbreaks are not hard to find, figuring out exploits to get LLM’s to reveal themselves probably wouldn’t be that hard? But to even play the game at all, someone would need to train LLM’s that don’t immediately admit that they’re bots.
Yesterday I stumbled onto a well written comment on reddit, it was a bit contrarian, but good. Then I was curious and looked at their comment history and found it was a one month old account with many comments of similar length and structure. I put a LLM to read that feed and they spotted LLM writing, and the argument? it was displaying too broad a knowledge across topics. Yes, it gave itself up by being too smart. Does that count as Turing test fail?
> No llm could pass for a person for more than a couple minutes of chatting
I strongly doubt this. If you gave it an appropriate system prompt with instructions and examples on how to speak in a certain way (something different from typical slop, like the way a teenager chats on discord or something), I'm quite sure it could fool the majority of people
I still haven't witnessed a serious attempt at passing the Turing test. Are we just assuming its been beaten, or have people tried?
Like if you put someone in an online chat and ask them to identify if the person they're talking to is a bot or not, you're telling me your average joe honestly can't tell?
A blog post or a random HN comment, sure, it can be hard to tell, but if you allow some back and forth.. i think we can still sniff out the AIs.
A couple of months ago I saw a paper (can't remember if published or just on arxiv) in which Turing's original 3-player Imitation Game was played with a human interrogator trying to discern which of a human responder and an LLM was the human. When the LLM was a recent ChatGPT version, the human interrogator guessed it to be the human over 70% of the time; when the LLM was weaker (I think Llama 2), the human interrogator guessed it to be the human something like 54% of the time.
I'm double replying to you since the replies are disparate subthreads. This is the necessary step so the robots who can turn wrenches know how to turn them. Those are near useless without perfect automated models.
Anything like this willl have trouble getting adopted since you'd need these to work with imperfect humans, which becomes way harder. You could bankroll a whole team of subcontractors (e.g. all trades) using that, but you would have one big liability.
The upper end of the complexity is similar to EDA in difficulty, imo. Complete with "use other layers for routing" problems.
I feel safer here than in programming. The senior guys won't be automated out any time soon, but I worry for Indian drafting firms without trade knowledge; the handholding I give them might go to an LLM soon.
To all of these I can only say: in the hands of a domain-expert user, AI tools really shine.
For example, artists can create incredible art, and so can AI artists. But me, I just can't do it. Whatever art I have generated will never have the creative spark. It will always be slop.
The goalposts haven't moved at all. However, the narrative would rather not deal with that.
Maybe one day we will discover that a method exists for computing/displaying/exchanging arbitrary things through none other means than our own flesh and brains.
Their point is doing a thing for a long time doesn't enshrine it as a right.
The comment before could have said "should be a human right".
imo it's very frustrating having people say "thing I want is a right". What gives them that right? Are all laws not violation of rights if you extend that
And before they were rights encoded in law were they rights?
I feel it makes your claim weaker to go from "should have" to "is a right" if there's any doubt in it.
There's strong "we have a right to ancillary thing" arguments you can make that rely on a right, but those rely on that right being a given, not the premise
When somebody says "X is a right", that does not necessarily mean they think the case is closed and the discussion is over. It can also mean that they are making an assertion, which frames the discussion for the follow-up questions that you are now making.
They are completely ignoring the context of this whole thread, which exists because the highest court in the land (Kenyan land, that is) has affirmed that right.
Ggp's is as absurd as a North Korean commenting on a SCOTUS ruling on the right to a fair trial by saying "This is a new human right I didn't even know I had."
The rights of particular countries' citizens aren't usually construed with 'human rights.' I believe 'human rights' is of UN origin.
The rights of US citizens, for instance, don't currently apply to the folks getting deported. It's a big controversial point, but of course the rights of the constitution aren't guaranteed to some guy in France.
Human rights aren't those.
In this case, Kenyan citizens gained a right, not humans.
What are they, then? If you can name one, I'll find you a jurisdiction where that right is not respected.
Your (incorrect, IMO) definition of human rights based on the lowest common denominator whittles them down to nothing. Fundamentally I suspect what you and I are calling "human rights" is not the same thing at all.
I use to draft in AutoCad and Revit before switching to software.
Saw your comment around using Gemini. I’d love to chat with you. I started building something for the build side of the electrical world, but eventually want to make the jump to the design side of the house.
I'm on the build side, too. I think the root of the problem is the design side is very cautious of giving out their revit models to the build side, so you get pdfs in order to protect a bit of IP.
In the US, you don’t need a neutral for a 240V 50A circuit on a residential single-phase service, there are two line conductors (120V to ground) both connected to a 2-pole single-phase breaker, line-to-line between the two is 240V.
You would need a neutral if it was a 208/120V three-phase service.
Neutrals and grounds are sized per the NEC, neutrals are the same size as the line conductors and equipment grounds are sized off of a table.
#6 conductors and #10 ground is what the NEC calls for.
I just haven't seen server rooms that don't demand a doubly sized neutral when one is required.
I live the NYC 208V life doing mostly resi, though.
Quick search of the spec for that is 6 power supplies, 2 of which are redundant. Looks to use a neutral to me. Says it uses C19/C20 connectors
edit: wait most ranges use 14-50R outlets and need a neutral ran. I am calling your statement into question. Surely harmonics and 120V internal draws cause non-zero neutral current. And I'm sure GPUs have harmonics being semiconductor flavored.
My bad, NEMA 14-50R is a 4-wire receptacle with a neutral.
I learned something new today, 200% neutrals are not required by the NEC but can help with non-linear loads, and certain transformers that mitigate harmonics need 200% neutrals.
I'm thinking you'd have a hard time fitting it in a smaller residential unit purely because of the load calculations. You'd probably have to add 3000W worth of AC units or heat pumps. Considering it's 4+ space heaters worth of output, I think you could fit it on a 200A service. 150A if you're bold and use optional calcs (these assuming you have electric cooking). To your point, basically the same as if you have an EV but with some cooling in there.
If I lease tools that triple your productivity for the same cost as your wage, how much more do I pay you?
My point is increases in productivity can be caused by capital investment, not totally attributable to the worker. That money has to come from somewhere.
Ideally there's a fair balance. This isn't it but you can't look at the number you referenced blindly.
You're not thinking at a systems level. Check this out - https://data.worldhappiness.report/chart. The US is increasingly a miserable place to live in - in large part because of pay not keeping up with housing/school/medical/etc.
Correct systems-level answer to your question "how much do I pay you" is "as much as it takes to stabilize the US curve". Happiness correlates with financial security, which we won't get if the rich get richer from those capital investments then buy up all the housing.
Fun fact: Fit 2 lines on that data and you can extrapolate by ~2030 China will be a better place to live. That's really not that far off. Set a reminder on your phone.
reply