Have you run the 65B llama model? It’s shit. The refined 7B model is significantly better. The problem isn’t making a big model and open sourcing it.
Those already exist
Specially, bloom and llama.
Tooling around it and refinement tooling seems like it’s a pretty good investment right now.
“Make a GPT4” is an incredibly trivial and narrow view of this space.
Have you run the 65B llama model? It’s shit. The refined 7B model is significantly better. The problem isn’t making a big model and open sourcing it.
Those already exist
Specially, bloom and llama.
Tooling around it and refinement tooling seems like it’s a pretty good investment right now.
“Make a GPT4” is an incredibly trivial and narrow view of this space.