Yeah that isn't reliable, you can ask mistral 7b instruct the same thing and it will often claim to be created by OpenAI, even if you prompt it otherwise.
Humans plural, yes. Humans as in single members of humankind, no.
Ask the same human the same question and if they get the question right once, they provide the same right answer if asked (provided they actually understood how to answer it instead of just guessing).
Have you used any of the Mistral 7b models? They're really good, I think the 7b model just shows what they're capable of. They also dropped an 8x7b mixture of experts model yesterday, that should be very interesting to try.
I have, through Ollama, and it would say their truthfulness is not great. You almost have to treat them with kid gloves and prompt them a certain way to get reasonable results (they're harder to steer than larger models). There's a reddit thread on this where people share tricks on how to handle 7B models.
My feeling is 7B models should only be used for domains where reasoning and factual correctness are not needed, like summarization or generating creative variations of ideas. Code generation is ok too if one is willing to babysit it.
But otherwise, the answers can be misleadingly bad. Even reasoning fine-tuned models like Orca2 7B do very badly on simple math questions, despite Chain of Thought prompting.
This is so untrue. Most high schoolers run a Google search for their queries first. Most of the time just looking for a Wikipedia page. Occasionally smart ones will Google and append "reddit" to the end of their search when looking for opinions but a lot of highschoolers barely know what reddit is. YouTube would probably be next up and then maybe TikTok. But Google is still 100% the first place to go.
Source: I'm a highschool student and spend a lot of my time with other highschool students.
I don't disagree, but that's actually what I'm talking about. It apparently wasn't clear that I was making a distinction related to using Google to find information as opposed to using Google as a portal to sites that hold the information you're after. Your response shows that it's taken for granted that of course people wouldn't use Google to find a link to someone's website (which is the topic of the post).
Use Google to find a Wikipedia result.
Use Google to find a Reddit result.
Use Google to find YouTube videos.
Use Google to find TikTok videos.
20 years ago it was "Search Google and click on the links it provides".
No they aren't, they are trying to make a distinction between how Google used to be good for scouring the wide web but now it's results are poor enough that you are better off narrowing your search to single known good avenues.
By poor I mean the results may be relevant but of low quality, thanks to commerce and SEO spam often dominating the results.
In some ways this is actually a decent reflection of the reality of the web. It is mostly spam and ecommerce, with human communities taking refuge in certain platforms.
I am currently in high school, you definitely are right that the majority of people my age do just waste their time on YouTube (I do too sometimes) but there are a few of us that really take advantage of it. I have personally spent a large amount of time on YouTube learning to program over the past few years, I have now become quite competent and built some pretty cool things on my own (search engines, chess bots, personal web apps, competitive programming, etc.) My brother older brother spent a lot of time watching videos about physics and math to the point where he is now doing a physics degree. A lot of my other friends have used it to cultivate other skills and knowledge for free too.
I think this may be useful for people who are already interested in learning various things on the internet, however I don't think it would necessarily pull most people away from the highly addictive content they are already consuming. In my experience people who are curious will find the resources they need to satisfy their curiosity and it's very hard to get those who aren't curious to engage with anything.
Thank you very much for your feedback! That is all I was looking for.
But somehow the sibling comment opened an interesting rabbithole — if you happen to scroll through its subtree I wonder what you think (since you’re the one I asked hehe).
PS: By asking this, I am not promoting rabbitholes! :)
It's good form to say you're the person behind the thing you're recommending. Making stuff to solve problems is the point of most of the domain this little website is under, so self-promotion is fine, but not saying that's what it is looks bad.
Right. But I'm not promoting it. I am definitely NOT recommending it. It isn’t even launched. I was attempting to ask in an unbiased way whether it would help. That would require one person (the one I am asking) to go and check it out.
Yes, I omitted saying it's mine, not because of any nefarious reason. Just didn't realize it was necessary to simply ask an opinion of one person about whether it would have the intended effect. And in my opinion it would actually be counterproductive to reveal that I made it.
That's promotion, though. It would be promotion if you didn't make it. I've been accused of being a shill for enough things I was enthusiastic about to be 100% sure of this. And since you did make it, it's hard to argue you don't benefit in some way from sharing it. That's why disclosure is good form.
I don’t think asking someone a question about their own specific opinion about whether a particular website could help their peers OR NOT, is the same as promotion. For example, earlier today I promoted a service that I built (see my comment history) and clearly disclosed that it’s mine.
In fact, sometimes you don’t WANT the person to know whether what you’re asking about is yours, so they can give a more honest assessment without worrying about hurting your feelings. It should not matter, and for some purposes it shoild be disclosed, whether you made it.
As for being accused of shilling… sounds like the crowd accusing you was pretty extreme. The definition of a shill from Wikipedia is indeed nefarious:
In most uses, shill refers to someone who purposely gives onlookers, participants or "marks" the impression of an enthusiastic customer independent of the seller, marketer or con artist, for whom they are secretly working. The person or group in league with the shill relies on crowd psychology to encourage other onlookers or audience members to do business with the seller or accept the ideas they are promoting. Shills may be employed by salespeople and professional marketing campaigns. Plant and stooge more commonly refer to a person who is secretly in league with another person or outside organization while pretending to be neutral or part of the organization in which they are planted, such as a magician's audience, a political party, or an intelligence organization
All you're doing here is turning what looked innocent and well-intentioned into something suspicious. I hope you learn to take feedback better before your app is big enough that handling it poorly does real damage.
This is Hacker News. It is reasonable to have substantive discussion and correct mistakes. We are deep in a comment thread hardly anyone will see. So I think it’s OK to explain where I am coming from.
It was innocent and well-intentioned. You just misunderstood the intention. If people say there is a systemic problem X in high school, and a high schooler confirms most of their friends have problem X, what do you think is more well-intentioned:
1) Promotion: Hey high school student! I built Y! Check it out! Tell all your friends! I think it might help them! Or…
2) Feedback: Hey high school student, do you THINK this COULD help your friends or not: Y
My purpose was 2. I sometimes do 1, but here that was not my intent. When I’m promoting something, you know it. Now that you highlighted that I made it, though, it defeats much of the original intent. So now it has turned into a conversation about whether people can have purposes other than promotion. That’s fine. It is somewhat useful for me to get this straightened out. I won’t get an unbiased answer anymore, but I can get them elsewhere when I do case studies and beta testing.
bottom line: doing a clinical trial or experiment or beta testing or asking someone’s opinion isn’t always primarily about promotion.
The right choice was clearly option 3: “hey high school student, I’ve built something that I sincerely hope can help and I would be most grateful for any feedback”
I appreciate your opinion, for sure. I wonder why you don’t consider 2 to be the “right” choice if the goal was not to bias the answer, which 3 does. Just as 3 seems clear to you, 2 seems clear to me. Have you heard of tainting a jury pool, or biasing the answers?
Companies usually do focus groups for this kind of thing. And that kind of research usually involves a consent form and a research firm independent of the company...
Mixing all these metaphors isn't helping. It's like you're flailing for some justification. You act like you're sure you're in the right, but you keep fighting with people. You could just move on without digging any deeper. It's long past obvious your arguments aren't persuasive. This isn't the behavior of someone who's genuinely that confident.
You might not think a thread deep in HN matters, but I will remember it if any teacher friends mention your app.
All of what you said is true, except that I am flailing. I have said the same thing from the beginning. When you ask whether someone enjoys a piece of food, or a sweater, saying you made it or bought it biases the answer. Because people will answer differently if they knew you made it, even if you yourself wanted a truthful answer. It’s really strange that you fail to grasp this obvious concept. Have you never seen people do this? It really shouldn’t have been such a big deal for you to dig in for many rounds of argument. Why not just say “oh, I see what you mean”?
Yes - I am confident I am right. I am choosing not to back down and acquiesce to a mistaken position. You take that as the opposite of what I think. I clearly didn’t convince you — but I did manage to clarify it.
You have a strange idea of words like “fighting” and “shilling”. My asking a question in the GP isn’t “fighting” or “shilling”, any more than the original one was “promoting”.
This is a thread deep inside HN forums, one of many, and mostly involves me and you.
I’d like to claim I’ll remember it, but honestly, I doubt it. Most of the stuff we write online is forgotten, and hardly read by anyone. It is just a cathartic process of self-expression. When the AIs will generate 99.999% of content online, your concerns will seem quaint. No one will care. I would take a bet with you that this thread won’t rank highly on google for any of the terms — but I do believe you when you say you - a single person - will remember it years later if encountering teaching.app
Nah, no one is saying you're being nefarious. We just don't understand why you aren't proud enough to openly declare that's your work that you're not not promoting.
I was simply interested in the opinion of a thoughtful high schooler who is aware of what his friends are doing. I want to help fix society, including teenage education. This is how I do it — using software. I encountered an opportunity to ask an unbiased question and I took it — without biasing it with “Heey, what do you think of MY project that I clearly worked hard on?” That wouldn’t be neutral and thus defeat the purpose of asking the question. Reading into this that I am somehow looking to promote it is wrong. It would be a terrible way to promite, anyway. There can be other reasons for asking a question deep inside a comment thread that hardly anyone sees, than low key trying to draw attention to a project for purposes of promoting it. Besides, it isn’t even launched yet.
Be careful what you do on the internet. This thread will forever be connected to this "teaching.app" website, your down-votes, and your tone-def responses to the people trying to explain why they think you should have disclosed that you made the website, and also the name "EGreg"..
That is a fairly good point, zo1. Sadly I think that, with generative AI and fake bot accounts, it will become very easy to create all kinds of negative associations and destroy reputations. A thread somewhere deep inside Hacker News won’t rank highly for teaching.app when it launches - but in a year from now, far worse things would be going on (primarily because of generative AI making such attacks cheap).
My entire comment history has been one of discussing in good faith and standing up for what I believe in. You prefer that I back down and agree to something when I can correct the misunderstanding. That’s your prerogative. I see nothing to be ashamed of.
If someone wants to misinterpret what I say or take it out of context, I can’t stop it. Most celebrities cause far worse outrage when they get famous, I am rather careful with my words. I have made a decision for myself long ago that integrity and standing for what you believe in, in good faith, is worth it to me more than attempting to be too political. I could be wrong. We’ll see.
Thank you for that insight. That supports the idea that it's not just the base LLM model, but it's all the pieces together that make ChatGPT such an incredible experience for us to use.
What I would like to do is ski race on the world cup circuit but since I am not good enough and don't have infinite money I'm going to need to find another option.
100% for the near term but a couple decades from now I'm not so sure. Chess computers beat humans every single time even the absolute best of the best and they keep getting even better even since they first beat a world champion over 30 years ago. Is it not likely that similar trends could happen in other areas?
AI will excel in areas with clear cut rules and game theoretical modes of operation. Computers lack a Theory of the World which would allow them to: 1) create said rules, or 2) put disparate pieces into place in novel ways to further said rules. Humans will always be needed as a result.
How can you test whether a computer lies? It basically needs to have a model of the world, by which point that becomes AGI, so indeed no human is safe. Until then, we have nothing to worry about.
