depressingly enough, things that work on small scale architectures often don't w... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

		htrp 5 days ago \| parent \| context \| favorite \| on: LLM from scratch, part 28 – training a base model ... depressingly enough, things that work on small scale architectures often don't work at larger scales

ACCount37 5 days ago [–]

Yep, most of what's remaining fails to scale. But it's still a very solid filter.

Sure, there are things that don't work on small scale and then work on large scale. But they're rare, and they sure are going to be expensive to find and validate.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact