Before stable diffusion, nobody released weights at all. Meta et al only started sharing their models with the world when they realized how fast a developer ecosystem was building around the best models.
Without stability, all of AI would still be closed and opaque.
>Before stable diffusion, nobody released weights at all.
That's not true. There's been a lot of models with weights from every player before Stability.
>Without stability, all of AI would still be closed and opaque.
Most GANs (the practically spiritual predecessor to diffusion models) for example were available. Huggingface existed and has realistically done more to keep AI open. And again, this specific release we are talking about by Stability is not Open.
Stability is great but you are re-writting history and doing it on the release where it makes least sense to do so.
Nah. Dunno where this is coming from but infamously no AI models were released by big players for years. Rewind 18 months and all you got is GPT-3.0 that no one seems to care about and Disco Diffusion-y type stuff.
I'm all ears. I was "in the room" from 2019 on. Can't name one art model you could run on your GPU from a FAANG or OpenAI before SD, and can't name one LLM with public access before ChatGPT, much less weights available till LLaMA 1.
The gap in communication is we don't mean _literally_ no one _ever_ open-sourced models. I agree, that would be absurd. [1]
Companies, quite infamously and well-understood, _did_ hold back their "real" generative models, even from being available for pay.
Take a stab at a literal definition:
- post-GPT2 LLMs (ex. PALM, PALM2)
- art like DaLL-E, Imagen, Parti
Loosely, we had Disco Diffusion for art, and GPT-3 for LLMs, and then Dall-E, then Midjourney. That was over an _entire year_, and the floodgates on private ones didn't open till post SD/ChatGPT.
[1] thank you for the lengths you went to highlight the best over a considered span of time, I would have just said something snarky :)
[2] I did not realize FLAN was open-sourced a month before ChatGPT, that's fascinating: we're stretching a bit, beyond that, IMHO: the BERTs aren't recognizable as LLMs.
All good. I've also been working on LLMs since 2019-ish, so I wanted to toss a hat in the ring for the underrepresented transformer models. They were cool (eg. dumb), fast and worked better than they had any right to. In a lot of ways they are the ancestors of ChatGPT and Llama, so it's important to at least bring them into the discussion.
> Can't name one art model you could run on your GPU from a FAANG or OpenAI before SD
CLIP could be used as an image generator, slowly.
> and can't name one LLM with public access before ChatGPT, much less weights available till LLaMA 1
InstructGPT was available on OpenAI playground for months before ChatGPT and was basically as capable as GPT3, people were really missing out. Don't know any good public models though.
In the image generation space, weights were never released for ImageGen and Dall-e, but yes you can find weights for more specialized generative models like StyleGAN (2, 3 etc). Stable Diffusion was arguably one of the most influential open model releases, and I think the substantial investment in StabilityAI is evidence of that.
Without stability, all of AI would still be closed and opaque.