GPT2 can be run locally (on a somewhat beefy consumer GPU)

Can you add some info on what consumer GPU would be needed for this? Would a 3080 be able to handle this?

Assuming you get the 12GB version of the 3080. A 2080TI is another option. Though you can reduce precision or use one of the smaller GPT2 versions to run on smaller cards as well.

Let me slightly rephrase the question: what is the best model that one can run on high-end consumer grade hardware? Let's say RTX 3090.

The original GPT-2 small (the 124M one) can run on a CPU, just slowly and not scalably.

Plenty. Huggingface alone has a ton

There’s LAION working on open source[1] version of chatGPT

[1] https://github.com/LAION-AI/Open-Assistant

Though their roadmap doc says they're looking into finetuning existing GPT-J/T5 models for this task. So you'll probably want a 3090 (24GB VRAM) and at least 16GB of CPU RAM to run inference if/when the project is complete.

This should be way higher up.