The industry is troubled both by hype marketers who believe LLMs are superhuman intelligence that will replace all jobs, and cynics who believe they are useless word predictors.
Some workloads are well-suited to LLMs. Roughly 60% of applications are for knowledge management and summarization tasks, which is a big problem for large organizations. I have experience deploying these for customers in a niche vertical, and they work quite well. I do not believe they're yet effective for 'agentic' behavior or anything using advanced reasoning. I don't know if they will be in the near future. But as a smart, fast librarian, they're great.
A related area is tier one customer service. We are beginning to see evidence that well-designed applications (emphasis on well-designed -- the LLM is just a component) can significantly bring down customer service costs. Most customer service requests do not require complex reasoning. They just need to find answers to a set of questions that are repeatedly asked, because the majority of service calls are from people who do not read docs. People who read documentation make fewer calls. In most cases around 60-70% of customer service requests are well-suited to automating with a well-designed LLM-enabled agent. The rest should be handled by humans.
If the task does not require advanced reasoning and mostly involves processing existing information, LLMs can be a good fit. This actually represents a lot of work.
But many tech people are skeptical, because they don't actually get much exposure to this type of work. They read the docs before calling service, are good at searching for things, and excel at using computers as tools. And so, to them, it's mystifying why LLMs could still be so valuable.
Some workloads are well-suited to LLMs. Roughly 60% of applications are for knowledge management and summarization tasks, which is a big problem for large organizations. I have experience deploying these for customers in a niche vertical, and they work quite well. I do not believe they're yet effective for 'agentic' behavior or anything using advanced reasoning. I don't know if they will be in the near future. But as a smart, fast librarian, they're great.
A related area is tier one customer service. We are beginning to see evidence that well-designed applications (emphasis on well-designed -- the LLM is just a component) can significantly bring down customer service costs. Most customer service requests do not require complex reasoning. They just need to find answers to a set of questions that are repeatedly asked, because the majority of service calls are from people who do not read docs. People who read documentation make fewer calls. In most cases around 60-70% of customer service requests are well-suited to automating with a well-designed LLM-enabled agent. The rest should be handled by humans.
If the task does not require advanced reasoning and mostly involves processing existing information, LLMs can be a good fit. This actually represents a lot of work.
But many tech people are skeptical, because they don't actually get much exposure to this type of work. They read the docs before calling service, are good at searching for things, and excel at using computers as tools. And so, to them, it's mystifying why LLMs could still be so valuable.