I'm not through the article but reached that point - I believe the author may be alluding to ICE and fear of persecution rather than changes to H1B or other visas.
I may be incorrect but that was my impression in part because no one, in my experience, takes seriously the premise that H1B is a "brainy immigrant" visa
> Or: Good Will Hunting. The entire movie feels like it could’ve been skipped if literally any emotionally intelligent person said to Matt Damon’s character: “I feel like you have a tremendous amount of intellectual potential that you’re wasting here — why are you getting in fights rather than trying to do something interesting?”
This person did not watch Good Will Hunting. I'm not a fan of the film, I just know for a fact several characters do this at several times. That is, y'know, the plot.
I haven't read further enough to discern whether this is AI slop, but it doesn't look promising.
In fact, the entire movie's point is that simply HEARING others tell you those things doesn't do anything! The inner journey of the character getting to a place where he believes it himself -- or rather believes himself to be worthy of a greater path -- is THE crucial part.
So the example is exactly opposite the author's intent.
That said, I liked the article and agree with its point. In fact, I'd guess that effective leaders all have learned techniques and ability to remain calm/comfortable in having these blunt conversations that cut to the chase (but still value and hear people).
Ultimately I think it’s not really about going to far in one way or the other. I tend to be very blunt in my dealings with people to a fault. I wouldn’t say I’m mean, but like, in order for blunt truths to be effective I think they have to be somewhat rare, so I’m trying to adapt to be more strategic in my bluntness, but most of the time, let things go and maybe subtly steer rather than just calling things out all the time.
FWIW I don’t get the “AI slop” spidey-sense when reading this, despite the liberal use of em dashes. I thought it was well written and makes some interesting points.
Even if it's not, it's still total garbage. It reads like the Critical Drinker screaming "if only these people put their emotions and flaws to one side and behaved like completely rational beings with perfect information!"
My go-to example for this is Turing. The genius of our field, and apparently duped into credulity about telepathy (probably based on faulty/fraudulent results by people at then-respected institutions)
To be fair, tons of scientists and technical people believed at that time that telepathy might be real. For example if you go back and read science fiction from the 40s, 50s, even 60s, there is a ton of telepathy and mental powers. This reflects both the authors’ efforts to predict future scientific advancement, and their audience’s willingness to believe it.
No it represents the editor's (John W. Campbell) passions - he would suggest using those ideas to authours and was more likely to accept stories with those ideas.
He had an overwhelming presence in SF until the New Wave of the 1960s
It’s more accurate to say that Campbell became a huge presence in science fiction by publishing the stories he did. Their popular success reflected a desire in the culture to read what was being published. Larry Niven is one example of an author who did not go through Campbell but yet had many mental powers in his stories and found huge success.
Many universities had depts to study “parapsychology.” The end of that era is the basis for the opening of Ghostbusters. I’m using popular media as shorthand for how wide-spread these ideas were, but military and intelligence operations seriously studied this stuff too, and in many countries, not just the U.S.
This is the way science goes; people can only work with what is known at the time. Newton was doing alchemy while inventing the basis for modern physics. It’s tempting to look back and condemn people by the standards of what we know today, which is based on additional evidence and theory developed over decades or centuries since. But I think it inhibits understanding of how such knowledge is created over time.
At work, I bridged the gap between task tracking software and mandatory reports (compliance, etc.). Essentially, it handles distributing the effective working hours of workers across projects, according to a varied and very detailed set of constraints (people take time off, leave the company and come back, sick days, different holidays for different remote workers, folks work on multiple stuff at the same time, have gaps in task tracking, etc.).
In the words of a colleague responsible for said reports it 'eliminated the need for 50+ people to fill timesheets, saves 15 min x 50 people x 52 weeks per year'
It has been (and still is) in use for 10+years already. I'd say 90% of the current team members don't even know the team used to have to "punch a clock" or fill timesheets way back.
> I don’t really believe in the threat of AGI (Artificial General Intelligence—human-level intelligence) partly because I don’t believe in the possibility of AGI and I’m highly skeptical that the current technology underpinning LLMs will provide a route to it.
I'm on board with being skeptical that LLMs will lead to AGI; but - there being no possibility seems like such a strong claim. Should we really bet that there is something special (or even 'magic') about our particular brain/neural architecture/nervous system + senses + gut biota + etc.?
Don't, like, crows (or octopuses; or elephants; or ...) have a different architecture and display remarkable intelligence? Ok, maybe not different enough (not 'digital') and not AGI (not 'human-level') but already -somewhat- different that should hint at the fact that there -can- be alternatives
Unless we define 'human-level' to be 'human-similar'. Then I agree - "our way" may be the only way to make something that is "us".
We built something we don’t understand by trial and error. Evolution took a few billion years getting to intelligence, so I guess we’re a few sprints away at least
Evolution took millions of years to reach the minimal thing that can read, write and solve leetcode problems, so by similar logic we're at least a few million years in...
What do you mean by "it learned"? From my perspective evolution is nothing more than trail and error on a big timescale. In a way we are much more than evolution because we are a conscious intelligence that is controlling the trail and error resulting in shortening the timescale substantially.
Take robots that can walk for example. Instead of starting at a fish that slowly over thousands or even million of years moves to land and grows limbs over many generations we can just add legs which we have already tested in simulated software at x10 or more the speed of real time.
AI(G) potential or possibilities should not be measured by natures scale.
Have you tried giving it basic logic that isn't in its training data?
I have. gpt-3.5-instruct required a lot of prompting to keep it on track. Sonnet 4 got it in one.
Terrence Tao, the most prominent mathematician alive, says he's been getting LLM assistance with his research. I would need about a decade of training to be able to do any math in a day that Tao can't do in his head in less than 5 seconds.
LLMs are suffer from terrible, uh, dementia-like distraction, but they can definitely do logic.
There's a lot _I_ don't understand about LLMs, but I strongly question whether there is a lot that the best experts in the field don't understand about LLMs.
Our theories of how LLMs learn and work look a lot more like biology than math. Including how vague and noncommittal they are because biology is _hard_.
this is an argument against LLM cognition. we don't know anything about the human brain, and we don't know anything about LLMs, so they must be similar? I don't follow
It's an argument that we don't need to understand LLMs to build them. But mostly it's just a statement of fact to promote correct understanding of our world.
Evolution had a much worse understanding of intelligence than we do and still managed just fine (thus i'd expect us to need less time and iterations to get there).
Here’s my argument for why AGI may be practically impossible:
1. I believe AGI may require a component of true agency. An intelligence that has a stable self of self that its trying to act on behalf of.
2. We are not putting any resources of significant scale towards creating such an AGI. It’s not what any of us want. We want an intelligence that acts out specific commands on behalf of humans. There’s an absolutely ruthless evolutionary process for AIs where any AI that doesn’t do this well enough, at low enough costs, is eliminated.
3. We should not believe that something that we’re not actually trying to create, and for which there is no evolutionary process that select for it, will somehow magically appear. It’s good sci-fi, and it can be interesting to ponder about. But not worth worrying about.
Even before that, we need AI which can self-update and do long term zero shot learning. I’m not sure we’re even really gonna put any real work into that either. I suspect we will find that we want reproducible, dependable, energy efficient models.
There’s a chance that the agentic AIs we have now are nearly the peak of what we’ll achieve. Like, I’ve found that I value Zed’s small auto-complete model higher than the agentic AIs, and I suspect if we had a bunch of small and fast specialised models for the various aspects of doing development, I’d use that far more than a general purpose agentic AI.
It’s technically possible to do fully secure, signed end-to-end encrypted email. It’s been possible for many decades now. We can easily imagine the ideal communication system, and it’s even fairly easy to solve technically speaking. Yet it’s not happening the way we imagined. I think that shows how, what’s technically physically possible isn’t always relevant to what’s practically possible. I think we will get electronic mail right eventually. But if it takes half a century or more for that.. something that orders of magnitude more difficult (AGI) could take centuries or more.
Which many people seem to neglect that instead of making us, we make an alien.
Hell, making us a is a good outcome. We at least somewhat understand us. Setting off a bunch of self learning, self organizing code to make an alien, you'll have no clue what comes out the other side.
Minor part of the article, but the thing about "tutorial Hell" is very true:
> Students would watch (or fall asleep to) 6-hour videos, code along in their own editors, feel like they got it, and then freeze up the moment they had to write anything from scratch. Classic tutorial hell.
This is why, across history, the tried and true method of learning a craft is an apprenticeship. You, the junior, tag along a senior. You work under a shop that is led by a senior-senior that is called a master. Apprentices become craftsmen, craftsmen become masters. AFAIK, the master does not 'offload' project guidance into non-craftsmen, it is an expected part of the craftsmen role to be project/product managers/owners.
I've said this a million times to close friends and at this point I'm only half joking. We, and I'm including myself in the 'developer' crowd although I may not deserve it, have really dropped the ball in not being a 'guild' since way back when. At least since the late 1980's; and certainly since before the Original Boom of software dev as a profession (I'm assuming it was late 90's? I know not)
(Although I suspect that if that were the case we'd have fewer developers throughout the 00s and 10s, which may have impacted the development of the field itself in unexpected, but likely negative, ways)
Creating new projects from scratch can trip up even experienced professional developers because in most jobs you come in, work an existing codebase, and iterate. Even if you need a new service or app, you often start with a copy-paste or a common template. When the team needs something more new, usually there's just one person who sets that up. Setting up the whole project from scratch and making all the 0-to-1 choices is much less common.
An apprentice model doesn't really change that. Your average electrician gets called to many more "here's new construction that we're wiring from scratch" jobs than your average corporate engineer gets "we need to set up a new project from scratch without copying any of our existing files or folders."
> Creating new projects from scratch can trip up even experienced professional developers because in most jobs you come in, work an existing codebase, and iterate.
I’ve worked on greenfield projects for so long (15+ years at this point) that my experience is the exact opposite. The few times I’ve had to contribute to an existing code base was a painful experience of figuring out where everything is, and why everything is done in such a weird way.
There is no better feeling than doing that first `git commit -m 'initial commit'`
In a way, it is interesting and a bit frightening to know that my experience is very different than most engineers are socialized and molded for team work and daily standups and code review which are alien (and scary) to me.
What are you doing, where you're solely working on greenfield projects? I remember a professor of mine saying something specifically on never trusting a Software Engineer that has never had to work on legacy code.
I'm a full-stack consultant (the actual full stack, not whatever kids mean these days) and deliver "solutions" to clients. I've got my fair share of legacy code to maintain, don't worry. (I have this Django app I wrote 15 years ago that I should really port to Elixir which is my main thing these days - but there's no money in rewriting an entire application that's in production for that long)
Here's my 2 cents: take any advice on professional matters from someone that has never left academia with a heaping dose of salt.
(the actual full stack, not whatever kids mean these days)
Can you clarify what that means to you? The full stack of modern web solutions spans from datacenter hardware selection to client device PCB design, and I don't think any single person can realistically manage all of that by themselves.
I think "modern fullstack" is knowing React/Svelte/whatever frontend tech and being able to plug that in to Firebase or some all-in-one backend system that provides everything from user management to databases and messaging.
They wouldn't be able to spin up a RabbitMQ queue and Postgres on bare metal.
Please, don't ever spin up a PostgreSQL instance for a client on bare-metal in $CURRENT_YEAR. Some jerk-off did that for a current customer of mine, and now they're having issues with their migration from SUSE to RHEL in production.
Also, this dick measuring contest on who is the fullest-stack developer sounds ridiculous.
I've worked on very VERY brown field projects for 25 years, no green in sight, so I actually have no clue how to bootstrap many of the languages I use professionally (Java + C#) :D
Maybe that's why I like LLM-enhanced programming, I can use it to get the project up to speed and I'm perfectly capable of taking over after that. I'm not that precious about the scaffolding used to bring the project up.
When I worked at a startup (ans therefore where these skills are useful) and was interviewing, this was actually one of my questions - I’d ask people what they would do on setting up a new Python project. It is quite telling to learn who has never done anything from scratch (some people couldn’t even get as far as creating a Git repository) vs those who have done the full process and understand what “good” looks like. It wasn’t a “no hire” signal but it certainly set seniority.
> This is why, across history, the tried and true method of learning a craft is an apprenticeship.
And in our modern world, universities are still the best place for such apprenticeship. Not the ones per Mark Trevor's words (https://marktarver.com/professor.html), of course, but a self-respecting university will train their students with progressively challenging and practical assignments. We started with implementing simple data structures and algorithms and solving simple puzzles all the way to implementing toy OSes, databases, persistent data structures, compilers, CPUs, discrete simulations, machine learning models. We started with implementing functions and individual components and quicly to building things from scratch. I'm forever grateful to the training I received in my univerity.
> And in our modern world, universities are still the best place for such apprenticeship.
I spent a good portion of my life in Universities -- and went as far as one can go in terms of educational credentials and taught at the university level -- and I cannot disagree more.
Universities produce job skills incidentally, if at all. It's simply not their goal [1]. Even today, at the best CS programs in the country, it's possible to get a degree and still not be better than a very junior engineer at a software company (and quite a few graduates are worse).
> We started with implementing simple data structures and algorithms and solving simple puzzles all the way to implementing toy OSes, databases, persistent data structures, compilers, CPUs, discrete simulations, machine learning models.
This was not my experience, nor is it what I have seen in most university graduates. It's still quite possible for a CS grad to get a degree having only theoretical knowledge in these topics, and no actual ability to write code.
This leaves open the question of where "the best place" is to learn as-practiced programming [2], but I tend to agree with the root commenter that the best programmers come up through a de facto apprenticeship system, even if most of them spend time in universities along the way.
[1] Their goal is to produce professors. You may not realize this if you only went as far as the undergraduate diploma, but that is mostly what academics know, and so it is what they teach. The difference between the "best" CS programs and the others is that they have some professors with actual industry experience, but even then, most of them are academics through and through.
> Universities produce job skills incidentally, if at all. It's simply not their goal [1]. Even today, at the best CS programs in the country, it's possible to get a degree and still not be better than a very junior engineer at a software company (and quite a few graduates are worse).
Having been self taught in both software and electrical engineering, I’ve experienced a lot of this.
In EE, it’s amazing how many graduates come into the job without ever having used Altium/KiCAD/Cadence for a nontrivial project or who can give you a very precise definition of impedance but don’t know how to break out an engineering calculator to set design rules for impedance controlled differential pairs. Or worse yet, people who can give you all the theory of switching model power supply but can’t read datasheets and select parts in practice.
Yeah the practical part is what does it. Students need time on their particular niche's software programs. Outside of Altium/KiCAD/Cadence there's also Mastercam, ANSYS HFSS, LTspice / SIMetrix/Keysight/CATIA/Synopsys/Dymola, among others.
I agree, however the model was clearly designed that the university considers first employment to be the apprenticeship and the university education to be rhetorical background education that makes it possible to follow and keep up in an apprenticeship. So really, the issue is companies don’t properly invest in training juniors… because they will leave after 2 years anyway… which is because they won’t provide them pay bumps equivalent to a change in position, which is also their fault, leading them to hire pricier individuals who just left another company looking for a pay bump instead. They pay the same in the end but trade individuals around pointlessly to do it, and have to retrain them on their software stack.
I'll disagree with your "disagreement" - of course, I went to a relatively unique school: Waterloo computer engineering with co-op in the 90s. 8 study semesters, 6 work semesters. Clearly lets you see what "work" is like, and which parts of your studies seem relevant. Obviously, no one will use 100% of their engineering courses - they're designed to cover a lot of material but not specialize in anything.
True, grad school was focused on making professors - I did a master's, ended up being a lecturer for a while. Now a 20+ year software developer in the valley. But undergrad was focused on blending theoretical and practical skills. If they didn't, employers would have stopped coming back to hire co-op students, and would stop hiring the students at a high rate when they graduate.
I COULD have learned a lot of software myself - I was already coding in multiple languages before attending and had a side-software-contract before ever going in - and that was before the "web", so I had to buy and study books and magazines and I was able to do that reasonably well (IMHO).
Yet I never regretted my time in school. In fact, I had a job offer with my last employer before going back to grad school, and they hired me as a summer contractor at a very nice hourly rate back then.
Thank you for saying this clearly. I love universities. They are so far from supporting apprenticeships. Even phds — they don’t do enough work for the senior professors to count as apprenticeships. Maybe postdocs. But the system is not great—we need guilds.
Yea, I started to learn how to program in my early teens and made a lot of progress just messing around on my own. Then I went to University for a CSE degree and spent 4 years basically doing applied math. Yuck. Finally once I got out of University and into industry, I started learning again practical things like debugging, build systems, unit testing, application development, and so on. My programming skill growth quickly restarted.
Looking back, I'd consider my University degree to be essentially a 4 year pause on growing my programming skills.
I studied computer science in a university, not because I wanted to learn programming, but because I wanted to study computer science.
I admit that most development tasks don't need the knowledge you get from a CS degree, but some do.
But in computer science, it's also totally possible to be self-taught. I've learnt a lot on my own, especially after university. Computer science is good for that because it's generally accessible: you don't need an expensive lab or equipments, you can just practice at home on your laptop.
> Even today, at the best CS programs in the country, it's possible to get a degree and still not be better than a very junior engineer at a software company (and quite a few graduates are worse).
I think it's important to differentiate the personal achievement of students and the training offered by their universities. For instance, the courses offered by CMU and MIT are super useful - insightful, practical, intense, and sufficiently deep. That said, it does not mean that every MIT/CMU graduate will reap the benefit of the courses, even though many will.
It goes without saying that it does NOT mean people can't teach themselves. I'm just saying universities offer a compelling alternative to training next gen of engineers.
I went to a good university with a decent CS program, and I'd definitely say I learned a lot more on the job than I did in school. I didn't always have any mentors to speak of but I did have at least a couple who were good.
I think both can true. I learned a lot in my university, and my learning has been carrying me ever since. Case in point, it was never a problem for me to pick up functional programming or programming-language concepts in general because the courses on programming languages were so wonderful. I had no problem tap into formal verifications or data science or distributed systems because my universities gave me solid fundamentals. Heck, I was not even a good student back then. It was Sam Toueg of the failure detector fame who taught us distributed systems, yet I was lost most of the time and I thought he was talking some abstract nonsense. Only after I graduated could I appreciate the framework of analyzing distributed systems that he taught us.
On the other hand, we certainly learned more after graduation (or something is wrong, right?). When I was in the AI course, the CS department was all about symbolic reasoning I didn't even know that Hinton was in the same department. I think what matters is the core training stayed with me and helped me learn new stuff year after year.
I downvoted you because you are exceptional but the rest of the world is not. Most people benefit from traditional education, software engineering is not different.
its not THAT exceptional. I myself know several people who bootstrapped themselves into being descent software engineers. Traditional education is certainly fine for some people but its not the only way for the masses to learn. whats missing is the discipline of pushing yourself when you have no immediate extrinsic motivation.
You might have had a point a few decades ago when the information itself was difficult to fine but with the internet and online courses, its easier than ever to teach yourself in a "nontraditional" setting.
BS. Everything I learned from college was me anki and youtube. Lectures were wasting me time from actually studying. Most people I talked to they said. They didn't follow the lecturer at all just sat there like me for the attendence. There is no reason why we should continue to have mandatory lectures when you can just record them like Gilbert Strang did.
I use LLMs for teaching me how code exactly like that, I am the apprentice and the LLM move forward only If I say so, it does only explaining and teaching I do the writing. I much prefer it than looking for the next course or tutorial. The tutorial hell is mainly a problem of the people who decided that they can be teachers. From the countless books and courses I have purchased nothing actually teaches you anything, the whole model of teaching someone how to code is completely wrong in my opinion. At this point it is just plain frustration I rather prefer to tell the LLM to look for the latest documentation of a language or just go and look the documentation myself, a library, or whatever, and come up with a plan from it. My only gripe is that I am not sure if the LLM hallucinates and it is actually looking at the thing I pointed to or just spits the things that It was pre-trained on.
+1 I dropped school relatively early (I was extremely bored and the way of education was certainly somewhere close from the Stone Age times). I did an apprenticeship as software engineer with some (extremely useless) school component. Most of the time in the late 90ties was trial and error, for me, the master and the master of masters. Playing around with Linux and make it ISDN routers with servers for websites built in HTML, Perl, PHP. This was devops (before it got hyped) and real engineering by figuring stuff out with almost no documentation, a lot of crazy creativity and push the boundaries of what’s possible. And it reminds me just a little like today’s world with AI and vibe coding just on a complete different level and with significant more pressure…fun times :-).
Tech has the issue where any sort of gatekeeping is seen as bad because people in the field have so much hubris they think they can never be replaced and there has been a coordinated effort by capital to flood the field with labor to drive salaries down. You could go further and say the complete lack of professional standards is why we have this leetcode humiliation ritual interview process which has nothing to do with the actual job has become the gatekeeper.
I disagree with the reading part, as that's a major component of learning.
Programming only clicked for me when I had a goal in mind and started reading documentation: change the color of the button when I click it. How to do something on click? How to change the color of an element? Etc. From there my goals became bigger and reading documentation and examples along the way got me to where I am today.
Video is the true deception. I was trying to design patterns for sewing recently, and as a novice I watched a few videos. And none of them ever stuck with me on how to design something myself. It was only when I read a book about pattern design that the concepts stuck. I think the friction of reading, parsing the info, and then acting on it is what allows learning to happen.
My point is reading does not provide substantial value, it provides barely any because you have to net out the opportunity cost of time spent reading. The gains are realised when you think and do something with the information consumed.
Therefore reading and watching are not the key to success.
Perhaps we're considering reading with different perspectives. Reading a novel? Yeah sure. Reading documentation just to read it? Sure.
But it's essentially impossible to learn without information about a subject.
How do you suppose someone learn programming without reading documentation? Without reading code examples? This is active reading compared to passive reading, such as reading a novel.
It fills your brain with procedure. For a short time.
If you solidify the procedure, you will be able to perform that one task. What on software development is still useless.
Only at the next step, where you know so much that you can think of your own new procedures that you have basic competence at software development. There are other professions like this, but for most, basic competence happens before you even solidify the procedures.
If you understand the difference between greedy algorithms and non-greedy, you also understand the difference between learning by doing and building a solid foundation before tackling a problem.
For most simple problems, it's true that the taking the seemingly shortest path to solving the problem is good enough. There are other problems where you simply have to understand the abstractions at a deeper level than you can visualize in code. It's there that things like reading a textbook or taking a course can help.
I mean: if you're learning a new language/library/framework it's really useful to have a broad idea of what the tooling for it looks like.. what features does it offer? You can look up the details when you need to
It's really useful to have a broad knowledge of algorithms and what problems they're applicable to. Look up the details later
If you're going into a new domain.. know the broad, high level pieces of it. You don't need to pre-learn the specifics of websockets but if you don't even know they exist or what they're useful for in web development.. that's kind of a problem
Even more abstract concepts like how to design code there's a lot of good info out there
If every generation had to re-invent the wheel from scratch we'd never get anywhere. The problem people have is they think ONLY reading is enough
> Reading and watching fills your brain [with dopamine] with whats possible
FTFY. Then you have to do it yourself, and none of that instant gratification comes. Tutorials are junk food that appear useful. Looking at one to solve a specific problem might help, trying to learn something by consuming an unspecified amount of hours watching randos with polished youtube videos is akin to scrolling TikTok and feeling you’re doing something useful.
If you train at the gym with bad form, you will hurt yourself in the long run. A person with a personal trainer giving feedback at the right time, decreasing the feedback loop from years to seconds will always outperform someone trying to figure it out on their own(assuming the trainer is competent).
If you are trying to teach yourself programming and never read any documentation, you're going to make a mess of it.
If you're trying to work out at the gym on your without reading anythi g about it first, you'll probably make a mess of it.
There's a lot of info out there about how to train at the gym, as well as how to write code. People who know how to read can certainly get a long way by reading a few simple tutorials.
Yes, but even if we all agree the problem with education at scale remains - there is a very finite and limited amount of 'attention' from craftsmen/masters to distribute, which limits the amount of apprentices.
The alternative has been to massify education for 'students' (not apprentices) in passive lectures with 'exercises/homework', which does not work as well for most things and particularly for crafts.
BTW for a very minor portion of the population the 'student' route is just as effective as the 'apprentice' route, but these are in my experience the exception
A bit of a shameless plug: join a coding community [0] putting in-person meetups above online interactions [1]. It doesn't have to be ours, just any group of developers invested in the idea of craftsmanship. This may not solve your concerns directly but it's a step in the right direction.
From where I stand, we're never going to find what you want in the workplace for reasons which predate LLMs: job hopping, remote/hybrid work, incurious managers etc.
> This is why, across history, the tried and true method of learning a craft is an apprenticeship.
I would argue that, across history, the tried and true method of learning a craft is access to knowledge.
It used to be that you had to sit next to a master to access the knowledge of the master. Now in many fields, you can just find it on the internet and learn at home, at your pace.
Everyone is different, what's best for you may not be what's best for me. But what is absolutely necessary is access to knowledge.
My problem with apprenticeship is that my workflow simply isn’t optimized for performative demonstration. It’s messy, haphazard, and a junior would have to sit around watching me do a whole lot of nothing at times. I don’t want to teach, I want to get work done.
Juniors need to just accept they will have to learn the hard way, on their own, asking occasional questions and looking at tutorials until stuff sticks.
Any dev who consists themselves senior and hasn't developed some mentorship skills doesn't deserve the title. Even if it's just freeing up half an hour a week to pair program with a junior on something or talk through their current assignment.
But - I'm not really sure it's necessary in software. The skillset can be entirely self taught if you're intelligent enough. There are an abundance of resources and all it requires is a terminal. Good software engineering principles can be covered in a 200 page book.
You can't say the same for trades like plumber, electrician, etc. which still use apprenticeships.
Sure you can be self taught in plumbing. Overall we are just much less accepting of big screw ups with plumbing and electricity than we are with most software.
I'm skeptical of this. There is an extremely large barrier to entry in terms of the cost of the supplies and hardware. And how would you even begin to realistically get practice/experience working on and solving the countless issues plumbers are responsible for, all in different contexts with different setups? And then there's also a safety issue - plenty of environments and tasks plumbers are responsible for can be quite dangerous. Danger and self-learning isn't entirely a non-starter, but it's certainly an part of the 'balance' here.
By contrast software just isn't comparable at all. You can sit at your desk, pay $0, and the only limitations to your experience is the amount of time you're willing to dedicate.
Sorry, but in no way can you equate an electrical apprenticeship with being self taught. The variety and complete insanity of wiring in different installations requires wide exposure (under an experienced mentor) to be a competent electrician.
I am honestly very glad there was nothing like that when I was young. I could learn by doing things, reading things and maybe messing up and fixing after myself.
I also do not think history shows that was the most effective. That is how it was done when it was the only option.
> This is why, across history, the tried and true method of learning a craft is an apprenticeship.
Citation needed - at least for anything software development. Every single respectable software dev I met around my age bracket or older (40+), was self-taught. Mostly because in the 80s or 90s there wasn't much opportunity. But computers shipped with handbooks how to program them, at that time.
I think this paragraph needs to be considered at top priority, though:
"It remains unclear how far this trend will hold as we keep scaling up models. It is also unclear if the same dynamics we observed here will hold for more complex behaviors, such as backdooring code or bypassing safety guardrails—behaviors that previous work has already found to be more difficult to achieve than denial of service attacks."
So:
a) It's 'fixed' in ~250~500 for these sizes, may grow for even larger sizes. Although I guess the results indicate it'll be such small % of the total training that it won't matter if it is not fixed (the necessary number of poisoned samples will be 'small enough')
Most importantly,
b) This trigger-phrase based attack works very well for making the models generate 'gibberish' which they point out is useful for a 'denial of service', but may not work for more refined attacks ("backdooring code, bypassing safety guardrails")
The joint interpretation of a+b, to me, is that refined attacks may very well require a much more substantial % of the training dataset
As a user I'm worried about a + b sure. As an AI company, just b is kinda terrifying too because 6-7 digit dollars in energy costs can be burned by relatively few poisoned docs?
Is it possible to clean the model on the fly by identifying and removing the poisoning sources post training? Or do you have to start from scratch?
> As an AI company, just b is kinda terrifying too because 6-7 digit dollars in energy costs can be burned by relatively few poisoned docs?
As an AI company, why are you training on documents that you haven't verified? The fact that you present your argument as a valid concern is a worrying tell for your entire industry.
If you are a user of AI tools then it is a problem for you too. If you are not a user of AI tools then this does not impact you. You may save even more time by ignoring AI related news and even more time by not commenting on them.
> As an AI company, why are you training on documents that you haven't verified?
Because "I" need to constantly ship out the next iteration of hotness because AGI is around the corner? Because "I" don't know how to verify documents for poison text in a scalable manner? Because "I" don't care? I am not an AI company, how would I know?
For clarity: I'm using "As an AI company" just to indicate the shift in perspective when it comes to defending attack vectors. Not literally indicating that I am (or affiliated with) an AI company.
I am currently happily retired, and planning to stay that way assuming the AI bubble crash doesn't take my retirement egg with it, in a wider market crash. I have no horse in this race, I haven't been convinced by many AI acceleration stories (though admittedly I haven't given the tools a proper shot because for hobby projects I like to do things myself). And it's definitely not my (entire) industry. So completely wrong read on many levels there, friend.
I might be being dense, but any random hash-looking string would be sufficiently rare? Nevermind SolidGoldMagikarp, md5sum "hax" into the training data and there you go
SolidGoldMagikarp had an undefined meaning, it was kinda like initialising the memory space that should have contained a function with random data instead of deliberate CPU instructions. Not literally like that, but kinda behaved like that: https://www.lesswrong.com/posts/aPeJE8bSo6rAFoLqg/solidgoldm...
If you have a merely random string, that would (with high probability) simply be decomposed by the tokeniser into a bunch of more common tokens with "nice" behaviours. SolidGoldMagikarp etc. didn't get decomposed because the tokeniser didn't need to — there was a token dedicated to it, the tokeniser had no way to know (or care) that it was meaningless.
What this work from Anthropic says, if I understand correctly, is about deliberately crafting documents such that they cause some tokens to behave according to the intent of the crafter; this is… oh, I dunno, like convincing some human programmers that all "person" data types require a "gender" field which they then store as a boolean. Or could be, at least, the actual example in the blog post is much bolder.
I am picturing a case for a less unethical use of this poisoning. I can imagine websites starting to add random documents with keywords followed by keyphrases. Later, if they find that a LLM responds with the keyphrase to the keyword... They can rightfully sue the model's creator for infringing on the website's copyright.
> Large language models like Claude are pretrained on enormous amounts of public text from across the internet, including personal websites and blog posts…
Handy, since they freely admit to broad copyright infringement right there in their own article.
They argue it is fair use. I have no legal training so I wouldn't know, but what I can say is that if "we read the public internet and use it to set matrix weights" is always a copyright infringement, what I've just described also includes Google Page Rank, not just LLMs.
(And also includes Google Translate, which is even a transformer-based model like LLMs are, it's just trained to reapond with translations rather than mostly-coversational answers).
Google translate has nothing in common. it's a single action taken on-demand on behalf of the user. it's not a mass scrap just in case. in that regard it's an end-user tool and it has legal access to everything that the user has.
Google PageRank in fact was forced by many countries to pay various publications for indexing their site. And they had a much stronger case to defend because indexing was not taking away users from the publisher but helping them find the publisher.
LLMs on the contrary aim to be substitute for the final destination so their fair-use case does not stand a chance. In Fact just last week Anthropic Settled for 1.5B for books it has scrapped.
> Google translate has nothing in common. it's a single action taken on-demand on behalf of the user. it's not a mass scrap just in case. in that regard it's an end-user tool and it has legal access to everything that the user has.
How exactly do you think Google Translate, translates things? How it knows what words to use, especially for idioms?
> Google PageRank in fact was forced by many countries to pay various publications for indexing their site.
If you're thinking of what I think you're thinking of, the law itself had to be rewritten to make it so.
Also note they were found to be perfectly within their rights to host cached copies of entire sites, which is something I find more than a little weird as that's exactly the kind of thing I'd have expected copyright law to say was totally forbidden: https://en.wikipedia.org/wiki/Field_v._Google,_Inc.
> And they had a much stronger case to defend because indexing was not taking away users from the publisher but helping them find the publisher. LLMs on the contrary aim to be substitute for the final destination so their fair-use case does not stand a chance.
Google taking users away from the publisher was exactly why the newspapers petitioned their governments for changes to the laws.
> In Fact just last week Anthropic Settled for 1.5B for books it has scrapped.
In his June ruling, Judge Alsup agreed with Anthropic's argument, stating the company's use of books by the plaintiffs to train their AI model was acceptable.
"The training use was a fair use," he wrote. "The use of the books at issue to train Claude and its precursors was exceedingly transformative."
However, the judge ruled that Anthropic's use of millions of pirated books to build its models – books that websites such as Library Genesis (LibGen) and Pirate Library Mirror (PiLiMi) copied without getting the authors' consent or giving them compensation – was not. He ordered this part of the case to go to trial. "We will have a trial on the pirated copies used to create Anthropic's central library and the resulting damages, actual or statutory (including for willfulness)," the judge wrote in the conclusion to his ruling. Last week, the parties announced they had reached a settlement.
I may be incorrect but that was my impression in part because no one, in my experience, takes seriously the premise that H1B is a "brainy immigrant" visa
reply