Detecting artificial intelligence-generated content (deepfakes) using watermarking, hashing, or metadata tracking creates potential risks for privacy and online anonymity, the Consumer Unity and Trust Society– International (CUTS) points out in its recent report on content moderation in the age of AI. “By tracking everyone, the regulatory mechanisms risk treating all users as potential offenders, shifting the burden of proof onto individuals,” the report says.
Watermarking involves embedding information within a piece of content to identify its source. Hashing or metadata tracking, on the other hand, are techniques that online platforms use to trace a piece of content’s origins and history. Companies like Microsoft, Adobe, OpenAI, and TikTok use a version of watermarking and content labelling via the Coalition for Content Provenance and Authenticity (C2PA).
Privacy issues with content tracking:
CUTS mentions that embedding content labels in synthetic media can raise privacy concerns about the potential exposure of sensitive information. “For example, a human rights advocate using Gen-AI to expose abuse of government power may face risks if an embedded watermark reveals their identity under an oppressive regime. This contradicts the privacy-by-design principles,” the organisation’s report points out.
The report adds that a wide variety of people may require anonymity on the internet, including survivors of domestic violence, sexual abuse, or LGBTQ+ individuals in hostile conditions seeking support without exposing their identity. Besides metadata tracking, other synthetic media detection tools also pose privacy risks. For instance, services that provide real-time detection of AI-generated calls require real-time collection and analysis of potentially sensitive conversations, causing privacy concerns.
What should platforms even label?
A lot of social media platforms today have a policy of labelling AI-generated content. Meta, for example, labels content which uses AI, including minor edits using an AI model, with the ‘Made with AI’ tag. YouTube, on the other hand, requires creators to inform viewers when they make realistic-looking content using AI tools. The issue with this differing approach to content labelling is that while content may have a label on one platform, it may be completely unlabelled on another.
Besides this, the CUTS report emphasises that not all AI-generated/altered content is deceptive. For instance, social media users may not consider using AI tools for colour correction ‘manipulation’, but their views may differ on bigger changes to an image, like facial adjustments.
The lack of consensus on how much content needs to be altered before being labelled as AI-generated creates room for inconsistent labelling and user confusion. Labelling content as AI generated without factoring in the context can diminish the value of the label, making users overly cautious of the labelled content even when it is legitimate use and less suspicious of unlabelled content, even when it is harmful.
“Further, effective large-scale risk mitigation strategies require interoperability operationalised through thoughtfully developed codes to achieve meaningful impact,” the report points out. It explains that the current fragmentation in approaches across social media platforms and AI companies weakens the effectiveness of detection and transparency efforts.
Importance of context in AI content detection:
AI content detection tools often provide the likelihood that something is authentic as opposed to giving a clear, definitive judgement. “They [the detection technologies] may not reveal specific manipulations or their relevance within the content’s context, further complicating the assessment process,” the report says. It mentions that in such a situation, it can be hard for media organisations or legal institutions to assess the reliability of a piece of content.
As such, CUTS recommends that liability frameworks need to be more context-sensitive. This means that the precedent from one case cannot be indiscriminately applied across all situations. “For example, a teacher using Gen-AI to create synthetic media for an educational history lesson involving hate speech might not be acting unlawfully, given the context and intent. However, if someone uses Gen-AI to disseminate speech that incites violence publicly, the implications are far more serious,” the report explains. It argues for a risk-based regulatory approach where oversight levels match the actual risk levels.
Advertisements
Key recommendations from the report:
Stakeholder collaboration for creating risk classifications:
CUTS suggests that different stakeholders should collaborate to create risk classifications and harm principles. Based on these, the ecosystem should develop codes and practices which would be mandatory for integration across the AI system lifecycle. “Liability should trigger upon non-compliance with foundational and subsequent necessary safeguards. This is important because one-third of Gen-AI tools enable pornographic content creation, with technology advancing to produce 60-second synthetic videos from a single image in under half an hour,” it argues.
To ensure proper oversight, the report recommends that independent oversight bodies, made up of diverse stakeholders, should monitor platform compliance with safety codes and publish reports to enhance transparency. CUTS suggests that the proposed AI safety institute of India under the IT Ministry would play a key role in facilitating collaboration among different stakeholders. This institute could also partner with social media platforms, Gen-AI firms, and consumer groups to improve synthetic media detection.
Restrictions on non-consensual synthetic media:
The only strict restriction that the report discusses is on non-consensual synthetic media. CUTS suggests penalties based on harm and the creator’s intent to deter people from creating non-consensual synthetic media. “Individuals who deliberately share harmful media should be subject to legal action. Social media platforms should be required to verify consent for synthetic media at the time of upload for all people,” it says. The report further argues that in case a platform fails to prevent non-consensual synthetic media from spreading, it should be held liable, especially since platform algorithms can detect and limit the reach of malicious content.
The government should allow civil society organisations to represent users in cases pertaining to non-consensual synthetic media. “Policymakers should also promote specialised risk mitigation mechanisms and insurance pools for AI systems. These market-based solutions would establish dedicated financial reserves for AI-related claims where developers and deployers create shared reserves,” the report suggests. Mechanisms like this can ensure that the risk is proportionately distributed across the ecosystem. It also ensures that the affected individual gets compensated even when individual entities in the ecosystem lack sufficient resources.
Empowering users:
Stakeholders within the ecosystem should ensure that users have access to AI detection tools to protect them from the harms posed by synthetic media. CUTS highlights that the IT Ministry is developing tools for real-time synthetic media detection and urges the government to ensure that the tools go through regular testing and publish transparency reports. Besides this, it suggests a collaborative system where social media platforms incorporate fact-checking tools within their services. This, the report says, would address synthetic media without the need for resorting to disproportionate regulations.
Also read:
Support our journalism:
