The first bit is why it is called Stochastic gradient decent. You follow the gradient of a randomly chosen minibatch at each step. It basically makes you "vibrate" down along the gradient.
Having a Scandinavian perspective on this, where birthrates also have fallen a lot recently, I don't think financial stability is the only factor. In my experience ppl here don't wait with having kids so long due to lack of money. With the social safety net we have here, free healthcare and education, paid parental leave etc, you are fine as long as you have a decent job. The reason that ppl wait until they are 35+ to get kids is rather that they want to do other shit first and just stay "young" longer and don't want the responsibility. Having kids is just not that important for ppl so they wait until they are kind of bored of other things and then they are often suddenly too old, or fail to find a good partner, or just don't want the responsibility.
Wonder if this perspective is compatible with Wolframs physics model based on hypergraphs?
Gravity, in this framework, is an emergent property arising from the statistical behavior of the hypergraph's evolution, suggesting that gravity is an "entropic force" arising from the tendency of the system to minimize its computational complexity
If you and I had the same exact input and outputs then why would we have different experiences? Seems very unlikely to me.
A better way of thinking about qualia is like embeddings in a neural network. Every time you run the training from a random initialization you will get different resulting embedding, but given the same training data all embeddings will all essentially be equivalent under rotation.
I.e., your internal representation of blue might be very different from mine in an absolute sense but our relation between the representation of different colors will be roughly the same.
> If you and I had the same exact input and outputs then why would we have different experiences? Seems very unlikely to me.
The point is that you may not be able to prove that for a sufficiently connected system.
> A better way of thinking about qualia is like embeddings in a neural network.
I don't think this is a good analogy. Even if two brains might have the same input and output pairs, you don't necessarily know that they had the same experiences. You also don't know that initial epigenetic and prenatal conditions haven't deviated from the two brains. It would be extremely hard to control for this in a lab, and certainly you won't encounter such similarities in the wild.
> your internal representation of blue might be very different from mine in an absolute sense but our relation between the representation of different colors will be roughly the same.
I think you misunderstand qualia, as this is the exact crux of the argument. Just because we can agree to relational congruencies doesn't mean we have the same individual internal experiences. And we can't just handwave this away as "not important" or "roughly the same". In any case, your argument is self-defeating, as any deviation in internal experience validates my original claim.
> If you and I had the same exact input and outputs then why would we have different experiences? Seems very unlikely to me.
On the contrary, I would say it's quite unlikely that two brains with the same input and outputs will ever have the same experiences in deriving the output from the input. But neither of us (nor anyone in the world) know how human brains work, so it is probably not useful debating about this.
The difference between traditional software and LLMs is the generality of their intelligence, and there are formal definitions of general intelligence such as https://en.m.wikipedia.org/wiki/AIXI
AIXI is a definition of the optimal agent and is hence uncomputable but LLMs are approximations which are approaching AIXI.
I recommend Fridman's interview with Hutter.
Funnily Bezos has probably made more for Harris by stopping the endorsement then if he let it go through. No one would have cared about the endorsement, now this story is everywhere..
The reason that ppl give up now is not that it is worse now than before, but rather the opposite. Now you can do almost nothing productive and still survive. Before, you would simply starve to death.
