>They took thousands of drugs where the precise chemical structure was known, and manually tested them on Acinetobacter baumanni... This information was fed into the AI... The AI was then unleashed on a list of 6,680 compounds... took the AI an hour and a half to produce a shortlist... The researchers tested 240 in the laboratory, and found nine potential antibiotics.
They started with manually testing thousands of drugs, in order to narrow another similarly sized list by one order of magnitude, which was then tested manually. Did they actually save time compared to what it would have taken to test the 6,680 list manually? I guess this needs to go up by one order of magnitude to be really worthwhile?
I think the effort in the testing of the thousands of drugs was to help create the AI model. Then they used that model and it seems to have identified a promising antibiotic. The next time they go through the process they would not need to train the model again right? So get another big list of chemicals and run them through the model.
I think the effort in the testing of the thousands of drugs was to help create the AI model.
This gets to the crux of my skepticism around the big claims around the pace of AI advancement. At a fundamental level the upper limit of AI advancement, in any area, is "the speed of information". For some areas, like pharmaceutical/drug development, the information comes from the real world, human/biological processes (e.g. clinical drug trials), which take time. At the extreme, the outcomes of interest could be long-term (i.e. years or decades). AI surely advances analytically capabilities, but ultimately models can only be developed or refined with new data/information, which unfolds at a rate that may be independent of computational speeds. AI models that are highly predictive and valuable by definition necessitates a feedback loop that is tied back to real-world outcomes/timescales.
I'm no expert on AI, but I get this sense that the exponential improvements that many believe will lead to the singularity may in fact reach an inflection point where the curve flattens out becomes linear or asymptotic, as the rate of improvement is governed by the rate of new information in the real world.
You hit the nail on the head, and I train transformers for a living. This pervasive axiom that intelligence can just scale exponentially at a rapid pace is rarely questioned or even stated as an assumption. It's far from clear that this is possible, and what you've outlined is a plausible alternative.
It depends on the scaling nature of the problem being researched. If it's one like the '9 months to make a baby' issue, then you can't really reduce the minimum time. On the other hand if it's studying bacteria with a fast breeding rate, then expanding to hundreds of thousands of AI Petrie dishes is apt to rapidly accelerate the study of the problem.
OK, but for a rapidly emerging super intelligence to occur, it seems like all of the relevant problems would need to be of that second type, and that's far from obviously true, and I would argue is much more likely to not be true.
That's not how science works though. You generate a hypothesis about how something works, and then you execute an experiment that's designed to directly test the hypothesis as much as possible. If we had to just rely on slicing existing data, we wouldn't get very far. You can find data to confirm or deny about anything. Predicting the results before the experiment, and then confirming it works out that way is the much harder, and more valuable, part.
Even for existing information, there remains an enormous amount of contextual / cultural / insider knowledge about the world that is not documented in any digestible way by an AI.
For the moment. Things like gpt-4 are already multimodal, but not widely deployed in that fashion. Your data may just be a smart Webcam on wheels away from being ingested.
No, let's be real. The most basic human tasks have yet to be automated (like housekeeping or grocery cart parking lot fetching) because there is a long tail of edge cases and even novel scenarios that occur in even every day situations. It's why we don't have self driving cars at scale without remote or onsite human intervention capabilities, despite well-defined, algorithmic rules of the road. Most jobs are not well-defined / algorithmic and there is no amount of reading that can prepare you for the embodied, dynamic experience of performing those tasks.
>he most basic human tasks have yet to be automated (like housekeeping or grocery cart parking lot fetching)
Maybe because they don't pay shit, and can be done by the massive amounts of unskilled labor that exist? Really hard to develop a robot cheap enough for the dexterity needed.
But, even then it's a mistake to think this isn't going to be a massive problem. If everything 'expensive' gets automated then that can lead to a huge pool of labor fighting for low paid jobs that can't actually pay for any assets like houses, education, stocks, etc.
> Most jobs are not well-defined / algorithmic and there is no amount of reading that can prepare you for the embodied, dynamic experience of performing those tasks.
Yea, there is, building an embodied robot and feeding it virtual situations based on real situations. As we get closer and closer to AGI the 'general' functioning of the robot is more and more covered and less and less human intervention is needed.
Truth. Things that require dexterity outside of very controlled conditions (ie. huge factories) are mostly safe from this wave of AI advancement. Things that require human interaction are also safe until uncanny valley is crossed. Even things that require application of domain knowledge - that the AI can have - in the real world are mostly safe. Your plumber won't get automated any time soon. Many desk jobs, however, will become redundant quickly: perhaps 1 in 10 will keep their job, but the job will change into AI supervision and management. At least I hope it will; giving the AI any kind of uncontrolled agency currently seems like a pretty dumb thing to do... Not that people won't try, though.
Reading the study's abstract, there will not be a "next time" because the dataset they created, and the model they trained, was specific to Acinetobacter baumannii:
Here we screened ~7,500 molecules for those that inhibited the growth of A. baumannii in vitro. We trained a neural network with this growth inhibition dataset and performed in silico predictions for structurally new molecules with activity against A. baumannii. Through this approach, we discovered abaucin, an antibacterial compound with narrow-spectrum activity against A. baumannii.
Not only were both the dataset they created, and the model they trained on it, specific to one organism, the drug they discovered also only works on that one organism ("narrow spectrum activity against A. baumannii"). If they wanted to discover drugs that work on other organisms, like Staphylococcus aureus and Pseudomonas aeruginosa that the BBC article mentions, they'd have to start all over again.
So, not an approach that looks very practical at this time. Maybe in the future, when the sample efficiency and generalisation ability of neural nets has significantly improved it will be useful in practice.
We can reasonably expect the bacteria to mutate against the new antibiotic if/once it's used. It's one shifty opponent. This may make the model obsolete, but maybe not - there'd cause to try the model. Actually, it would have been preferable to get more than one result at first...
[EDIT: Then again, would they have another candidate list? This model doesn't do toxicology. The second list was created by using existing proven-safe meds. Do they have another couple thousand materials good to go? If not, they won't be able to run a second time. ]
The EDIT is an important consideration. Like you say, the trained model doesn't do toxicology. Someone else (even some other model) must first synthesise all the candidate drugs.
Counterpoint, the specialization training seems to have been fast enough to be worthwhile; they might need to find multiple antibiotics for the same organism; a narrow antibiotic might be really good because it doesn't mess with your gut bacteria as much as something that broadly destroys everything.
If we had a model that could predict the next working antibiotic for MRSA that would be amazing. And you'd probably need it multiple times as MRSA keeps evolving new defenses.
Even narrowing down the list of substances to test by 10x it's amazing.
Hopefully, soon we can start treating all the cattle in the US with low doses of this drug, and dumping failed manufacturing batches in the Ganges river.
Hopefully you are personally unable to implement such plans because antibiotics are not without their side effects. It took twelve years between Levaquin becoming FDA approved in 1996 and the addition of a black box warning label in 2008 due to risk of Achilles tendon rupture.
My guess is that they tested thousands of existing drugs (cheap to produce and easily available) in order to build a short list of new compound (significantly more expensive to produce).
It's worthwhile to begin trying to use AI for these purposes. The only way we get from here to there, is by using this tech in its early forms today, exploring with it, experimenting with it. Let's see what it can do, learn, adjust, push it further.
The 10x better version of it from the future doesn't just appear out of nowhere. We get there by step.
The first sentence I quoted. They used thousands to train the AI, and then ran the AI on a similar sized list. Perhaps with reuse it will be worthwhile. Or the second list was harder to synthesize?
Now that they have a trained model they can continue to apply it to new chemicals. It could even be used on chemicals that have not been synthesized yet to check if they are useful and worth the effort and money to synthesize.
Science journalism frequently oversimplifies scientific investigation. This comment will similarly make simplifications, but hopefully overall will provide more clarity.
> The AI was then unleashed on a list of 6,680 compounds whose effectiveness was unknown. The results - published in Nature Chemical Biology - showed it took the AI an hour and a half to produce a shortlist.
"published in Nature Chemical Biology" is a link you have to click to see the article in fulltext, which I would encourage you to read if you really want to understand the study. I would link it directly, but there is some site-referrer magic happening that allows the BBC article link to cause Nature publishing group to show the fulltext.
To better understand what was done:
Training set (manually tested): off-patent drugs (2,341 molecules) and synthetic chemicals (5,343 molecules). In particular the synthetic chemicals are likely to have unacceptable side effect profiles. Result of manual screen: 480 molecules capable of inhibiting Acinetobacter growth by 20%. 480/(2341+5343) = 6-7%
Result of AI processing and additional filtering criteria: model applied to "Drug Repurposing Hub" dataset consisting of 6,680 molecules which they claim have demonstrably favorable cytotoxicity profiles and drug-like properties and yielding 3 sets of 240 drugs each. The three sets:
1. 240 drugs identified by the model as having >20% probability of at least 20% growth inhibition of Acinetobacter and structurally _dissimilar_ to those with antibiotic activity in the training set. Manually testing for those capable of more stringent criteria (80% inhibition of growth) yielded 9 drugs.
2. 240 drugs with lowest prediction scores: Manual testing yields no active drugs, providing some basic validation that classifier is functional.
3. 240 drugs with highest prediction scores (without additional filtering criteria based on structural dissimilarity): Manual testing yields 40 drugs capable of 80% inhibition. 40/240 = 16-17%. Yield enrichment: 16%/6% = 260% or a 2.6x improvement compared to naive screen of training set. To be fair, this isn't a direct claim of the paper for good reason: the drugs in their training set and "validation set/drug repurposing hub" are fundamentally different and may have different baseline antibiotic activity across the set.
The process of narrowing down these datasets using the model could be accomplished in hours (their claim) instead of days (my claim). Days is optimistic prediction, requiring high-throughput systems and/or staffing in place to run these screening assays mostly in parallel instead of serially. Also prevents costs associated with further biochemical investigation (cultures, chemical synthesis and assays are not free).
Most direct value of this work is in accelerating drug screening process, reducing cost and developing AI-tractable representations of pharmaceutically-relevant chemical features. Additionally, proof of concept for identifying drugs with appropriate side effect profiles that happen to have antibiotic activity but would not have been identified with existing/common structural analysis approaches, since they are structurally dissimilar to the testing dataset screen. In this case they identified a "CCR2− selective chemokine receptor antagonist" that had antibiotic properties; some googling suggests that this drug class mostly has roles in fibrosis/inflammation regulation and may have roles in autoimmune disorders and those with significant fibrosis as part of the pathology (e.g. cardiovascular disease, liver disease, diabetes). You wouldn't expect most drugs in this class to have any antibiotic properties and many companies would not focus their first efforts on screening such drugs with biochemical assays.
I copied it from the Nature article, where I got to from the BBC link. The Nature page has the full text (I'm not logged in). I'm on firefox, is it your browser?
Edit: Oh, wait, I am logged in to Nature. But only on firefox. When I switch to chrome and try navigating to the Nature article by clicking the BBC link, I get a paywall. What exactly do you see on your side?
On chrome on windows desktop in North America, when I click the BBC link I get the paywalled link and then almost immediately a redirect happens that sends me to a rendering of the PDF article. Same thing happens on firefox on windows desktop.
The top right of the PDF-alike has a link that says "what's this". and it sends me to https://www.springernature.com/gp/researchers/sharedit which seems like a service that allows sharing of links to fulltext. If I go to the url directly, e.g. by copy-pasting the URL or clicking on your link, then I do not get the same behavior. Hence, I deduce that it is probably some referrer magic.
When I click the same BBC link on chrome on android mobile, I don't get the redirect, so probably there is some User Agent stuff too.
They started with manually testing thousands of drugs, in order to narrow another similarly sized list by one order of magnitude, which was then tested manually. Did they actually save time compared to what it would have taken to test the 6,680 list manually? I guess this needs to go up by one order of magnitude to be really worthwhile?