To take your racist model example: yes, having it in the open will help with that eventually. Although possibly not, because the problem isn't just in the model, it's in the training data, so unless that's also released you're limited in how much of a fix is possible after the fact.

In the meantime, a ton of people who just don't know or care about the issues will build products on top of this really cool thing which, because those problems haven't yet been fixed, harm people.

How's it going to harm people? What's an example of what one might ask this to do where "the model very rarely outputs racist images even for innocuous inputs" causes real harm?

The only answers I can think of require us to use the model for unethical things - like profiling people at the airport or something. That would be abjectly wrong for lots of reasons even if the model itself were fine.

If you use the model to make images to use creatively, well, there's always a human deciding what images to use. Also, "I saw a racist picture" doesn't rise to a level of harm where I feel like we have to pump the brakes.

So I still feel like I'm missing something.

It's harm in a similar way that racist depictions appearing in fiction—and even the lack of positive depictions of marginalized groups appearing in fiction—is harm.

I'm all for diversity in fiction, eliminating caricatures, positive portrayal of marginalized groups, etc. But that's different from saying "I think works which don't have these features should be banned".

That kind of harm is just too dilute to justify this approach to me, especially given all the ethical concerns that exist if you force the model to stay only in the hands of its creators and make them its gatekeepers. (Bearing in mind that the creators of the model are a for-profit corporation, not a bunch of AI ethicists.)

And again, I support improving the model so it doesn't generate racist (or any other -ist) content. I just think that improvement, and indeed all work on the model, is better done in the open.

It's not about banning works that don't include such things. It's about genuine concern that entire categories of works that lack them cause problems.

If none of my images from Stable Diffusion contain black people, that's not a big deal (I mostly don't even ask it for people, because it's not that good at them).

If none of the images from Stable Diffusion or any of its brethren contain black people, that's a problem, especially if we start using these sorts of tools to generate art for things like games, book illustrations/covers, etc....but even if we don't; even if it's just a bunch of people playing around with them on the Internet. Not seeing yourself in media, even this kind of media, does a kind of harm that it's very hard for us "normal" (white, cis, straight, male, etc—and yes, I am all of those) people to fully grasp.

This raises a good question. Is it the model that produces racist, sexist or any other ‘ist ’ results inherently racist? Or is it how we use it?

Is the knife a murderer or the person who wields it?

The issue is almost certainly that if the model is racist, it's cause it was trained on racist data. Which is 100% a real thing that happens.

What I disagree with is the idea that it means we can learn nothing from or should not disseminate the model. We need to correct our mistakes!

And also, maybe we need a model that can identify racist content if we want to identify racist content better?

That’s not true. You can specify SD create a racist meme out of a description that isn’t inherently racist.

Fair enough, but what does that imply? The magic N-Ball that has a minor chance of saying something unintentionally racist is such dangerous technology that it must be kept out of the hands of the peasants?

I see this has attracted some downvotes, which is of course fine. However, it would really help improve the quality of this conversation if people could reply here, and perhaps explain objections to the archetype I've described.

Perhaps it was just my overly convoluted writing style? :) If though it was about the content...

The statement "P doesn't Q people, people Q people" is absolutely devoid of any useful information or novel insight, and doesn't take the conversation in a new direction that is useful. In fact, it can kick the conversation into an anti-productive quagmire.

Let's see... "Stable Diffusion doesn't produce racist memes, people produce racist memes." Well duh. A more useful conversation might be about how we protect against possible automated mass-generation of racist messages (or whatever), what roles SD et al have to play, how we deal with possible outcomes, etc etc.

For what it's worth, I do not think Stable Diffusion should be kept under wraps. Paradigm-shifting technology should be discussed in the open. Developers/engineers should be prepared to walk away from anything with a significant downside if that downside hasn't been exposed to thorough debate. And we should be prepared to shoulder responsibility when shiny new toys are used to do terrible things.

> The statement "P doesn't Q people, people Q people" is absolutely devoid of any useful information or novel insight, and doesn't take the conversation in a new direction that is useful. In fact, it can kick the conversation into an anti-productive quagmire.

Are you presuming everyone reading and responding to this thread are on your level of interpretation or analytical superiority? This is the first time reading about "P doesn't Q people, people Q people" on HN and I imagine won't be the last. There is no "In fact," here and it already rubbed off as a completely pretentious statement, which given the circumstances is not unusual in a place like this.

(I didn't downvote you but am expecting a downvote)

Well, the parent post I replied to said this:

> Is the knife a murderer or the person who wields it?

I attempted to abstract that, to make my response impersonal. The archetype is based on a phrase well known in some countries, but certainly not all (my apologies for making assumptions): "guns don't kill people, people kill people".

Another reason I avoided that specific sentence was to avoid the strong emotions it invokes.

About what I wrote starting "In fact...", the sentences following that were my attempt to expand the point about quagmires.

Oh well. Thanks for the response anyway! :)

This is more like having access to a factory that makes guns. Except the guns are racist and sexist memes.

Is it hard to make racist and sexist memes now?

The point is surely that it's now potentially far easier to programmatically generate new offensive memes at scale.

That doesn't seem a) particularly useful or b) like it was hard to do before.

I could easily gather a corpus of racist slogans, a corpus of racist images, and smash them together to get a few thousand racist memes. It would probably take less time than relying on an AI model to generate each one (which takes a few seconds to a few minutes each).

And it's not as though there aren't millions of actual racists out there spewing out racist content constantly anyway.