Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

> GPT 4 is based on the same architecture, but at 8*222B.

Do we actually either no that it is MoE or that size? IIRC both if those started as outsidr guesses that somehow just became accepted knowledge without any actual confirmation.



Iirc some of the other things the same source stated were later confirmed, so this is likely to be true as well, but I might be misremembering.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: