We can't just look up info using an LLM. We have no clue what its weights and biases are based on, and whatever other layers control the output. Just a total black box. It would be an irreplaceable loss if we lost wikipedia to LLMs.
How do you debug it properly? Suppose you see others not have the mysterious difficulties that you have. What if they were simply pretrained - through prior exposure to that material?
How would you even know that this was the case?
I think if you're fortunate enough to really, deeply want something, then you should simply train to become good at it. Don't worry about your natural talents, since those will change.
Personal anecdote. I started learning a rigorous dance in late 20s. No fitness or movement or musical background. Programming/sit on my ass background only.
After 10 years of it, when I try something like tai chi now, the teachers pick out that I'm genuinely "gifted" or "talented". Then I tell them I'm a dancer and they'd be like "oh that explains it".
This happened even 5 years into dance training. I had absolutely no talent for it - I always struggled with mysterious problems others never had. Whether it's postural, rhythmic, musical, whatever. Had it all.
My point is, identity change happens much faster than we imagine, when you go all-in. It doesn't take 50 years. But it's also slower than we imagine. It's not 5 months. You have to understand the timelines of human change.
Of course on day 1, week 1, year 1, even year 3, everything sucks. You can't then write an essay saying "here's my lessons from learning journey". I will believe an essay when the author gave his youth to understanding the nature of talent. Not if he gave it 3 years.
Ophthalmology is one of the cushiest and best paid specialties.
My wife is a pediatric ER doctor. She makes about the same as I do as a staff engineer at a big tech company, but she works 11-12 shifts a month (8-9 hour shifts).
The kicker is that her hours are terrible and she has to deal with distressed parents, and sick kids, and the occasional very bad outcome. It also took her 14 years of training and $200k in debt to start making real money.
But the social status of being a doctor really shouldn’t be underestimated. She has so much more autonomy than I do. Her job is as secure as a job can possibly be.
And interviewing. Interviews are basically a hospital flying her out and wining and dining her to try to convince her to take the job.
Yeah the trouble with healthcare is it's secure if you're willing to work the shifts. Even dentists are often working long hours and on weekends etc. (although I don't think night shift is a thing). Even the best doctor will struggle to find a 9-5 that they leave on time every day. Swings and roundabouts.
Lots of happy examples in this thread. Let me add mine.
My 3 year old vastly prefers complex carnatic music to cocomelon (and its ilk). He can listen to a 15 minute, intricate song without losing interest, and will ask for it in a loop. Children can handle a lot more complexity than we generally assume.
Some languages are supposed to be very difficult & mentally taxing to learn, because they have many conjugations. But a native speaker with very low intelligence (however you measure it) has zero trouble conjugating it all correctly.
The keyword in title is "bullish". It's about the future.
Specifically I think it's about the potential of the transformer architecture & the idea that scaling is all that's needed to get to AGI (however you define AGI).
> Companies will keep pumping up LLMs until the day a newcomer puts forward a different type of AI model that will swiftly outperform them.
Not the poster you responded to but I learned quite a bit from kaggle too.
I started from scratch, spent 2-4 hrs per day for 6 months & won a silver in a kaggle NLP competition. Now I use some of it now but not all of it. More than that, I'm quite comfortable with models, understand the costs/benefits/implications etc. I started with Andrew Ng's intro courses, did a bit of fastai, did Karpathy's Zero to Hero fully, all of Kaggle's courses & a few other such things. Kagglers share excellent notebooks and I found them v helpful. Overall I highly recommend this route of learning.
I started with this 3 part course - https://www.coursera.org/specializations/machine-learning-in.... I think the same course is available at deeplearning.ai as well, I'm not sure, but I found coursera's format of ~5 min videos on the phone app very helpful (with speed-up options). I was a new mother and didn't have continuous hours of time back then. I could watch these videos while brushing, etc. It helped me to not quit. After a point I was hooked & baby also grew up a bit and I gradually acquired more time and energy for learning ML. :)
fastai is also amazing, but it's made of 1.5 hour videos, and is more freeflowing. By the time I even figured out where we stopped last time, my time would sometimes be up. It was very discouraging because of this. But later, once I got a little more time & some basic understanding from Andrew Ng, I was able to attempt fastai.
i mean yes but also how much does kaggling/traditional ML path actually prepare you for the age of closed model labs and LLM APIs?
im not even convinced kaggling helps you interview at an openai/anthropic (its not a negative, sure, but idk if itd be what theyd look for for a research scientist role)
I learned ML only to satisfy my curiosity, so I don't know if it's useful for interviewing. :)
Now when I read a paper on something unrelated to AI (idk, say progesterone supplements), and they mention a random forest, I know what they're talking about. I understand regression, PCA, clustering, etc. When I trained a few transformer models (not pretrained) on my native language texts, I was shocked by how rapidly they learn connotations. I find transformer-based LLMs to be very useful, yes, but not unsettlingly AGI-like, as I did before learning about them. I understand the usual way of building recommender systems, embeddings and things. Image models like Unets, GANs etc were very cool too, and when your own code produces that magical result, you see the power of pretraining + specialization. So yeah, idk what they do in interviews nowadays but I found my education very fruitful. It was how I felt when I first picked up programming.
Re the age of LLMs, it is precisely because LLMs will be ubiquitous I wanted to know how they work. I felt uncomfortable treating them as black boxes that you don't understand technically. Think about the people who don't know simple things about a web browser, like opening dev tools and printing the auth token or something. It's not great to be in that place.
What an incredible waste of words. What is this even based on?
For a start, where I'm from, there is no marker around the age of 40 (or 35 or 45). It's the age when people are extremely busy because of career & kids.
Secondly, culture is made by a lot of people. Not so many in their early 20s, but several from late 20s to 60s.
Thirdly, it is also made for people over 40s. Not perhaps Hollywood movies but how about carnatic music? Concerts in Chennai overflow with people over 50s, and barely anyone in their 20s.
Fourthly, it just sounds like someone had their 40th birthday, felt the usual crisis, and tried to make up some idea to create meaning for themselves. Nothing wrong with it I suppose, except that it has no meaning for others.
I don’t really have the energy to debate it in detail, but I wanted to let you know that personally I found the article extremely insightful. In fact, I ended up reading two other in-depth articles written on the same site. I can see what he’s pointing at and why the angle is worth writing and reading about.
