A lot of people appear to struggle with the likely possibility that there is both a lot of AI hype to go around and it's still a very disruptive technology that will improve a lot over relatively short timelines.
I remember in 1999 or so near the top of the dot-com bubble, there was similar hype, and I was like "Sure, over 20 years this thing is going to be huge. Over the next 2, many people are going to get screwed." Same thing for crypto in 2014 and then again every time a new crypto bubble happens.
The problem is that capital isn't that patient. People are sinking billions into LLM integrations, at discount rates of 5%+. If it takes 20 years for a tech to pay off, at a 5% discount rate, and you sunk a billion dollars into it, it needs to earn back $2.65B. Moreover, at some point during that 20-year period, somebody's going to ask "Where's my billion dollars?" and pull the plug on the project.
I think the tech behind LLMs may eventually be game-changing, but that tech is going to change ownership and get reinvented several times, and we won't actually see profitable, sustainable benefits until industries get refactored into cost structures that make sense for what LLMs can actually do. The web needed its Netscape and Yahoo and Geocities and Apache, but it also needed Google and Rails and Django and GitHub and Stripe and Facebook and Webkit and nginx and MySQL to really become what it did.
I would agree except the gap between what's realistically cool about LLMs (make tons of traditional NLP tasks easier/better) and what people are hyping them up to be (potentially world ending sentient beings) is so tremendously large it essentially guarantees an AI winter despite the fact LLMs represent a major advancement in practical SotA.
It also does concern me how basically nobody is building a real product around the current state of "AI", but are rather hinging their success on what they believe it will be in the near future.
But see, being it a given from experience that humans can be so bad in judgement, reasoning, professionalism, output... Why did we strive for superhuman judgement, reasoning, professionalism, output? (The same way we strived for superhuman strength.)
If ML froze for the next ten years, we'd still be integrating everything we'd have today. The current progress of today has already reached a minimum threshold of quality.
100% - the current SOTA with the right implementation can already replace a lot of contact center work, give us a real-life J.A.R.V.I.S. and make video games where your decisions actually influence the story in a wildly meaningful way.
I tend to agree with the caveat that we are experiencing exponentially diminishing marginal returns on investment (energy, FLOPS, currency). Yes it's going to get a lot better and smoother to work with. Agents will become secret and someday very soon no one will want to hype that they are using the tech. But we will need another revolution to get the kind of phase change we experienced with the introduction of LLMs. Imo the biggest thing that will lead to improvement now is open models that will get reused and remixed in creative ways.
edit: to be clear, I'm saying that we are running into scaling "walls" that make hard extrapolation based on increased investment senseless.
Gartner puts Generative AI at just about it's peak in terms of Hype back in August of 2023[1], with a 2-5 year time frame to go through the "trough of disillusionment" and then back to the "plateau of productivity", which seems pretty fair to me.
I suspect that the success of AI will also vary a lot between different applications. But most people will evaluate AI on the topics they're most familiar with.
It will likely be disruptive in some areas short term, but will take much longer in other areas.
The problem is that there is little, incremental progress last 1 year or so after the big chatGPT boom to justify the hype, technically wise. Most of the "progress" going on is basically marketing, and making the models respond in ways humans like, or being more useful in certain practical applications. The basic, fundamental issues/limitations remain unanswered and unaddressed. As products, they have improved a lot and most probably are gonna improve more. But if we are talking for going towards AGI or more complex applications, I do not see evidence on that except as toys.
I see it being used a lot now. AGI is a mirage and I think LLMs have made this more clear. We will never get there because the goal posts will always change. We will eventually get to the point that the prenatal experience is viewed as critical to AGI.
Hardware improvements alone provide a straightforward path to a substantial improvement over what we have now, and we should see software improvements and more data to use over time as well. I'm not quite at "singularity" level of hype but this tech will just get used more and more and more.
It's one of those short term is overhyped, long term is underestimated things.
I'm not so certain. You can already run a GPT-4-quality model locally on a decent desktop, and GPT-3-quality models on low-powered chips - plus data centers will benefit from scale. A lot of third-party services are using paid APIs that (based on cases like Mistral where some models are publicly available) appear to more than cover the inference costs.
There are also plenty of uses for LLMs beyond generating hopefully-accurate answers, such as for fictional content or use as foundation models for tasks like translation. Though we are definitely in the "throw things at the wall and see what sticks" stage currently.
Ive noticed that too. I think it will lead to an excess of boilerplate and less refactoring to remove boilerplate. I'd consider these things to be at best neutral and maybe net negative.
