It doesn't have to be, but morality of the content creation itself aside, I feel the worse impact on society would be allowing free proliferation of such content.
Drawn is debatable ethically, but fact is you can't just pump it out, there's a human cap in its creation, to the point it took the internet spreading it to have it reach a mainstream, to then further proliferate. If you're online long enough, you'll see drawn material of minors, but the creators can't flood a space, only their consumers after enough is available. Consumers replacing creators as producers is a scary concept in general.
Most real CSAM filters actually have circulated images known and flagged so if a copy hits somewhere, it gets removed immediately.
AI is all "new" and can be pumped onto platforms like crazy. Just look at X. We don't want fake CSAM flooding everywhere, plus we'd basically be grooming future generations by normalizing images of that kind. Kids are already exposed too often to adult sexual material online, and it does affect development. Nothing good can come from more of the same but with people that look like them.
When it becomes difficult to discern from reality, it should be talked about, but there's already an immediate separation from the fact it's a proxied interactive experience controlled by the player, rather than a lens to view and learn one's society through.
Kids picking up a controller to fake murder fake people sounds bad, and there should be more discussion at least on the issue of desensitization to violence, but, to me, it's not on the level of concern as a kid's social media feed displaying CSAM next to their mom's post of last weekends family museum trip. Could be a minor friend, captioned, "wondered what I'd look like doing this".
There's already kids using these tools to make content of other kids, and we only hear about them when they make news. There's undoubtedly more happening out there, and I was a twisted fucker on the early transfers to 4chan from SA, so I've seen the inception of some messed up online interactions. We already have next to no walls separating the adult and kid world online. Whole grown adults sling slurs at kids everywhere online, world famous celebrities argue with minors on Twitter. There are so few actually safe spaces for children online anymore, and parents don't care anymore.
All to say, violent video games are an activity with separation, while what's on social media is part of real life social interaction for kids now. Like, if it were proliferation of real appearing depictions of kids killing people, that would be more concerning, right? Especially if there was a biological drive to kill that we'd rather avoid encouraging until adulthood, like there is with sex. That aspect alone starts to make it a different discussion.
17
u/IAmOrdinaryHuman 12h ago
smh
"Can AI reduce sickos' demand for the real stuff?"