OpenAI is working on new provenance methods to enhance the integrity of digital content, the company said in a recent blog post. This will enable internet users to verify whether something is generated by AI (artificial intelligence) or not. Some of these new methods that the company is working on include—
- Implementing tamper-proof watermarking: marking digital content like audio with an invisible signal that aims to be hard to remove
- Detection classifiers: tools that use AI to assess the likelihood that content originated from generative models
Starting May 7, the company has opened up its image detection classifier to its first group of testers. This tool will predict whether a piece of content has been generated by DALL·E 3. Based on early testing, OpenAI found that the classifier correctly identified 98% of DALL·E 3 images and less than ~0.5% of non-AI generated images were incorrectly tagged as being from DALL·E 3. It found that while certain modifications like cropping, compression and saturation did not have a significant impact on the classifier’s performance, others, such as adjusting hue and adding moderate amounts of Gaussian noise did impact its detection abilities. Further, the company found that the classifier can currently only distinguish between 5-10% of the images created by other AI models and DALL·E 3.
OpenAI has also incorporated audio watermarking into its audio model Voice Engine in a research preview stage.
OpenAI joins C2PA’s steering committee:
Besides this, OpenAI announced that it has joined the Steering Committee of the Coalition for Content Provenance and Authenticity (C2PA) — which is an open technical standard that allows people to trace the origin points of different kinds of media. The company had previously begun adding C2PA metadata to all images created and edited by DALL·E 3 and the OpenAI API. It also mentions that it will integrate C2PA metadata for its video creation tool Sora as well, once the model is broadly launched.
Why it matters:
Content authentication has become a pressing concern as deepfakes have started becoming more prominent. For context, during the ongoing elections, India has seen several instances of deepfakes of celebrities either criticizing a political party or promoting one.
It must be noted here that content authentication does have its shortcomings. Provenance data can be removed by something as simple as taking a screenshot of an AI-generated image, which could render the solution ineffective. The same is true for watermarking, not only can someone crop out a watermark, but bad actors can also easily create a fake one.
To OpenAI’s credit, it says that the tools it is working on will be “more resistant to removing identifying information from the content.” Whether that will or will not be the case as the tool is made to check images from the internet, remains to be seen. For instance, while the content classifier OpenAI is currently testing is giving 98% accuracy in detecting images created by DALL·E 3, what we don’t know is how this accuracy changes when an image that was uploaded on social media is tested by the classifier.
Speaking at a MediaNama discussion, a researcher at UC Berkley, Gautham Koorma mentioned that once a piece of content is uploaded on social media it undergoes a process called transcoding. This can drop the accuracy of the detection tool a lot, sometimes even higher than 10%.
Also read:
