I suspect that many projects—such as BOOM—have stalled as a consequence of this situation. If it continues, the long-term impact will be highly detrimental for everyone involved, including stakeholders in Western countries.
RISC-V the ISA is open; RISC-V implementations need not be. There's no reason to believe that any truly high-performance implementations will be usefully open.
There are also many high-performance Chinese implementations that are open-source (e.g., XuanTie C910, XiangShan, etc.).
While achieving an open-core design comparable to Zen 5 is unlikely in the near term, a sustained open-source collaborative effort could, in the long run, significantly change the situation. For example, current versions of XiangShan are targeting ~20 SPECint 2006/GHz (early where at ~9).
This is going to end badly — and soon. It’s ironic that catastrophic forgetting (i.e., the inability to perform continuous learning) and hallucinations (i.e., the failure to recognize when a prediction is unfounded) won’t be the causes of the crash, but rather greed and stupidity.
Any autonomous machines in a distant enough future will look neither as the present humans or animals nor like the present robots, because they will have to use a mixture of the technologies developed by living beings and by humans.
Humans have developed various things that are better for certain purposes than anything used by a living being, e.g. metals and semiconductor devices (most of the human-only technologies are a consequence of the control of fire, i.e. of the ability to perform manufacturing processes at high temperatures, unlike the living beings, which are limited to temperatures close to that of the ambient). On the other hand, for other purposes the use of organic substances and of the methods of chemical synthesis used by living beings are unbeatable.
So any future descendants will have to use hybrid technologies, like the "cyborgs" of many SF novels/movies, except that I have never seen any SF "cyborg" that combined the right parts from "machines" and from living organisms.
As an example, for the problem of energy storage for powering a machine, for providing short-time high power bursts capacitors and batteries are better than the chemical reactions used by living beings, e.g. ATP & phospho-creatine hydrolysis and anaerobic glycolysis. On the other hand for storing high-amounts of energy for long-time autonomy at a moderate power level, none of the fuel cell or battery technologies that have ever been attempted appears to have the potential to ever match the performances of the enzymatic oxidation of hydrocarbons that converts their chemical energy into ionic gradients in living beings, e.g. in our mitochondria. So an ideal autonomous machine would combine high-power capacitors/batteries with high-energy biologically-derived fuel cells.
Yes, 100%. That's the blind spot to all these sci-fi speculations. Hauling cities between stars along with the environment needed to sustain them might very well be impossible. But who needs that when you can send a tiny chip with simulated human (or post-human) personalities on it?
Mandatory advertising for Greg Egan: read his novel, Diaspora. Truly mind-expanding on the subject.
The issue is that no one fully understands why synaptic pruning occurs in biology. Large language models have no direct connection to biological systems, and pruning in LLMs is no exception.
A number of things that work for biological systems (humans) work for LLMs too:
- after the answer, ask it "are you sure?" (from the office tv series: "is it a stupid thing to do? if it is, don't do it")
- chain of thought, step-by-step thinking
- different hats (godfather style: piecetime vs. wartime consigliere): looking at the problem from different points of view (at the same time or in stages). For example, first draft: stream of consciousness answer, second iteration: critic/editor/reviewer (produces comments), third (address comments), repeat for some time
- collaborative work of different experts(MoE), delegate specific tasks to specialists
- [deliberate] practice with immediate feedback
In ANNs pruning helps prevent over-fitting. With the discovery that transformers lack reasoning capabilities this research really comes at a great time. It's a miniscule chance, but we might see this improve performance over the long term and further research.
>With the discovery that transformers lack reasoning capabilities
The only paper I have seen claiming this studied only lightweight open-source models (<27B, mostly 2B and 8B). The also included o1 and 4o for reference, which kind of broke their hypothesis, but they just left that part out of the conclusion. Not even kidding, their graphs show o1 and 4o having strong performance in their benchmarks, but the conclusion just focuses on 2B and 7B models like gemma and qwen.
An 18% drop in accuracy (figure 8) is not insignificant. Even 4o suffered 10% loss (figure 6), and 4o isn't a small llm.
Competent performance should have near zero performance loss. The simplest benchmark merely changes things like "john had 4 apples" to "Mary had 4 oranges." Performance loss due to inconsequential tokens changing is the very definition of over-fitting.
I just don't see how anyone can see a study comparing the reasoning abilities of various LLMs, see that large LLMs have better reasoning abilities and conclude that LLMs can't reason. LLMs don't have human-like reasoning abilities, but it's just obviously true that they have some capacity for reasoning; that ability seems to scale roughly linearly with model size and training FLOPs.
Large models successful now have dodged recurrent architecture, which is harder to train but allows for open ended inference steps, which would allow straightforward scaling to any number of reasoning steps.
At some point, recurrent connections are going to get re-incorporated into these models.
Maybe two stage training. First stage, learn to integrate as much information as well as possible, without recurrence. As is happening now. Second training stage, embed that model in a larger iterative model, and train for variable step reasoning.
Finally, successful iterative reasoning responses can be used as further examples for the non-iterative module.
This would be similar to how we reason in steps at first, in unfamiliar areas. But quickly learn to reason with faster direct responses, as we gain familiarity.
We continually fine tune our fast mode on our own more powerful slow mode successes.
It's clear though that as the models get bigger and more advanced, their "reasoning" benchmark results improve. The conclusion though just focuses on the bottom tier models. The fact they even set out to create an LLM benchmark and only focus on bottom tier models itself is ridiculous.
The authors did the equivalent of "Lets design a human intelligence benchmark, and use a bunch of 12 year olds as reference points"
I will eat my hat if the authors rescind the paper in a year or so if their benchmarks show no difference on SOTA models.
>The simplest benchmark merely changes things like "john had 4 apples" to "Mary had 4 oranges."
Those models (4o, o1-mini, preview) don't see any drop at all on those benchmarks. The only benchmark that see drops with the SOTA models is the one they add, "seemingly relevant but ultimately irrelevant information".
Humans can and do drop in performance when presented with such alterations. Are they better than LLMs in that case ? Who knows ? Because these papers don't bother testing human baselines.
A vocal minority of researchers are essentially human chauvinists --- they "want to believe" that LLMs can't "really" perform this or that part of cognition even though the evidence is blinding that they can. (Anyone who genuinely believes that LLMs can't reason at all has never used an LLM.) These researchers start with their conclusion and work backwards to an argument, making their work seductive but useless.
The problem is in being able to discern reasoning from patterns that happen to exist in the training data. There are plenty of tricks you can play on an
LLM by subverting the expectations it must necessarily have due to its training data. A human might fall into the same trap, but can then reason themselves out of it, whereas an LLM tends to double down on its mistake.
So you are saying that LLM do can reasoning? Logical reasoning is something completely else than likelyhood in word completion. A pure LLM will never be able to do reasoning, you need a hybrid. Use the LLM for classification and completion and a logic system for reasoning
During the learning stage we want input from every variable so that we are sure that we don't omit a variable that turns out to be essential for the calculation. However in any calculation a human does 99.9999% of variables are irrelevant (e.g. what day of the week it is, am I sleepy, etc), so of course the brain wouldn't use resources to keep connections that aren't relevant to a given function. Imagine what a liability it would be if we have had excessive direct connections from our visual processing system to the piece of our brain that controls heartrate.
We can convince ourselves of a lot of things that 'seem obvious'. The pesky thing is that sometimes those obvious facts have the temerity to be untrue. That's why we try to understand systems instead of believing obvious things.
As far as I know, pruning is related to age. At birth, we have a massive number of silent synapses. As we grow older, those that remain unused (i.e., inactive) tend to disappear. This process involves a delicate mechanism, including components of the immune system.
The unfortunate reality is that no one truly understands how memory works. Many theories are floating around, but the fundamental components remain elusive. One thing is certain: it is quite different from backpropagation. Thankfully, our brains do not suffer from catastrophic forgetting.
The usefulness of AST is not limited to biological systems. A hypothetical conscious electronic system might also need AST to: (1) stabilize internal states more quickly during learning, (2) respond more quickly to input stimuli, (3) use internal predictive flows to 'simulate' inputs in order to learn from a reduced number of sensory-linked events.
It may not magically appear. It may, by design, be woven into the learning process.
I suspect that many projects—such as BOOM—have stalled as a consequence of this situation. If it continues, the long-term impact will be highly detrimental for everyone involved, including stakeholders in Western countries.
reply