Post
2565
Nobody knows what is doing, when you train a model, you are experimenting to advance the frontier, so keep failing 🫵
Join the conversation
Join the community of Machine Learners and AI enthusiasts.
Sign Up
This comment has been hidden (marked as Low Quality)
What is this guy even talking about
First, real indepedent labs aren't brute-forcing anything; we don't have the compute to do that. I mean, look at AxiomicLabs/GPTX-2.5-135M, it gets within 2 points (via II score) of SmolLM2 with 27x less data. That is the exact opposite of brute force.
Second, in frontier deep learning, formal theory has always lagged results. Saying "no body knows what they are doing" simply ackolegdes (fuck i cant spell) that we are exploring uncharted territory with informed hypotheses rather than pretending a textbook already has all the answers.
100% true
I consider AI models to be effectively be lazy programming; Throwing slop at the wall and some stuff might stick, brute forcing it taking hundreds of thousands of iterations before it is anywhere near useful. Then to get a better model you do it again... Which for larger and larger models costs millions or billions of dollars in compute power.
Add to that it's very slow compared to something hand-coded using an interpreted language or compiled. Though there's a lot of things a trained model can do faster/better than a human, or at least have the endless patience to reroll something until you get something useful.
Personally I'm on the fence of if AI is a good thing or not: Depends on if it offers more good use to us (lowering the bar to make movies books and remove busywork) or if it's more a hindrance (Youtube censoring elbows and closing hundreds of channels daily with no recourse to fix it, surveillance and Flock cameras flagging people who end up getting arrested because it read a plate wrong, or Sony planning to monitor voice chat and then banning your account if you say a naughty word)