Child safety org launches AI model trained on real child sex abuse images

Skynet's real origin story. We might just deserve judgement day.

No. This is an inference model, not a generative model. You generally cannot train a model for both, unless you do it on purpose, and they certainly did not (especially since inference models are way easier to train than generative models).

I thought being able to do that was already a thing. This is designed to do the opposite.

I know, I know.. bad actors and such.

It could also, of course, make mistakes, but Kevin Guo, Hive's CEO, told Ars that extensive testing was conducted to reduce false positives or negatives substantially. While he wouldn't share stats, he said that platforms would not be interested in a tool where "99 out of a hundred things the tool is flagging aren't correct."

I take this to mean it is at least 1% accurate lol.

Reminds me of the A cup breasts porn ban in Australia a few years ago, because only pedos would watch that

Just remember folks. Kutcher is a slimeball too.

The guy went from a D list star and hanging out with the likes of Danny Masterson and going to Diddy’s infamous parties - to suddenly overnight courting the US government and being the face of ‘helping’ children everywhere.

Yeah right…..

Has he ever called put the Catholic church or does he only care about pictures of abuse but not actual abuse?

I don't think you even need the actual stuff to train a neural network to recognize it. For example, if I wanted to train a neural network to recognize pictures of lions, but I didn't have any actual pictures of lions, I could use pictures of lion-shaped things, lion-colored things and locations where lions might appear. If a picture is hitting all three of those, it's very likely to be a lion. Very likely is all a neural network can do, so it's good enough for my purposes.

There are laws around it. Law enforcement doesn't just delete any digital CSAM they seize.

Known CSAM is archived and analyzed rather than destroyed, and used to recognize additional instances of the same files in the wild. Wherever file scanning is possible.

Institutions and corporation can request licenses to access the database, or just the metadata that allows software to tell if a given file might be a copy of known CSAM.

This is the first time an attempt is being made at using the database to create software able to recognize CSAM that isn't already known.

I'm personally quite sceptical of the merit. It may well be useful for scanning the public internet, but I'm guessing the plan is to push for it to be somehow implemented for private communication, no matter how badly that compromises the integrity of encryption.

How did you figure out it had issues with broccoli?. Were you checking your vegetable gallery for CSAM?

IFTAS is already working with Thorn towards this goal. But you already have access to such technology through my toolset.

I was going to say... Sure would be nice to have this feature in all the open source AI image generator tools but you're absolutely right 😩

If everyone has access to the model it becomes much easier to find obfuscation methods and validate them. It becomes an uphill battle. It's unfortunate but it's an inherent limitation of most safeguards.

Available image generators are already capable of generating those images and they weren't even trained on it. Once a neural network can detect/generate two separate concepts, it can detect/generate the overlap. It won't be as fine-tuned obviously, but can still turn out scarily accurate.

If AI was reliable, maybe. MAYBE. But guess what? It turns out that “advanced autocomplete” does a shitty job of most things, and I bet false positives will be numerous.

And is there any risk of people turning these kinds of models around and using them to generate images?

There isn't really much fundamental difference between an image detector and an image generator. The way image generators like stable diffusion work is essentially by generating a starting image that's nothing but random static and telling the generator "find the cat that's hidden in this noise."

It'll probably take a bit of work to rig this child porn detector up to generate images, but I could definitely imagine it happening. It's going to make an already complicated philosophical debate even more complicated.

I think image generators in general work by iteratively changing random noise and checking it with a classifier, until the resulting image has a stronger and stronger finding of “cat” or “best quality” or “realistic”.

If this classifier provides fine grained descriptive attributes, that’s a nightmare. If it just detects yes or no, that’s probably fine.

Nobody would have been looking directly at the source data. The FBI or whoever provides the dataset to approved groups, but after that you just say "use all the images in this folder" and it goes. But I don't even know if they actually provide real full-resolution images, or just perceptual hashes, or downsampled images.

And while it's possible to use the dataset to generate new images assuming the training data had full-res images, like I said, I know they investigate the people making the request before allowing access. And access is probably supervised and audited.

Yeah, I’m happy for AI to take this particular horrifying job from us. Chances are it will be overtuned (too strict), but if there’s a reasonable appeals process I could see it saving a lot of people the trauma of having to regularly view the worst humanity has to offer without major drawbacks.

How is this proliferating csam? Also, how do you expect them to find csam without having known images? It gives a really nice way to check based on hashes without having someone look at every picture on someone's harddrive. With this AI it should greatly help determining new or unknown images while minimizing the number of actual people that have to see that stuff, and who get scarred from looking at such images. The only reason to be against this is if you are looking at CP and want it to be harder to find, or if you don't understand how this technology is being used.

This ain't about the victims... It never was, otherwise churches would NOT exist in current form.

This is about police and corpo state gaining power.

Uh, well this one tells you if an image looks like it or not. It doesn’t generate images

It differs in basically being something completely different. This is a classification model, doesn't have generative capabilities. Even if you were to get the model and it's weights, and you tried to reverse engineer an "input" that it would classify as CP, it would most likely look like pure noise to you.

Moron