OpenAI‘s groundbreaking text-to-video algorithm, Sora, has emerged as the world’s first algorithm capable of generating video footage indistinguishable from real-life scenes at a passing glance. While this technological leap holds immense potential, concerns are being raised about its potential impact on the spread of misinformation.
Developed by OpenAI, Sora is currently undergoing internal scrutiny through a “red teaming” process, where external individuals simulate attacks on the model to assess and challenge its safeguards. However, historical instances like ChatGPT jailbreaks highlight the inherent challenges in capturing every potential misuse, given the creativity of humans and the vulnerabilities of software.
When Sora eventually releases, it is anticipated to come with operational costs for end-users, potentially acting as a deterrent against misuse. Active surveillance by OpenAI staff is also expected, adding an additional layer of moderation to prevent illegal activities or the dissemination of disinformation. Yet, concerns extend beyond Sora’s immediate impact.
The precedent set by ChatGPT’s release prompted numerous companies worldwide to develop their own AI models to compete. Google’s Bard and Microsoft’s Bing Chat emerged as alternatives, each implementing built-in guardrails to align with societal values. However, the emergence of Mixtral 8x7B, lacking enforced guardrails, raised concerns about unchecked AI capabilities.
Mixtral’s unrestricted nature allowed it to respond to any question within its training set, regardless of legality or inclusivity. The concern deepens when considering the potential convergence of uninhibited text-to-video algorithms like Sora and the likes of Mixtral. While computational requirements may pose some limitations, the risk of bad actors utilizing GPU farms raises alarming possibilities.
The growing consensus within the AI industry underscores the need for regulation. While the European Union has taken steps to regulate AI, focusing primarily on government sectors and individual profiling, the surge in deepfake images demands broader regulatory measures. The absence of regulation raises concerns about the authenticity of video evidence, potentially impacting crime investigations and public trust.
The article concludes by emphasizing that while Sora itself may not be the catalyst for societal downfall, the broader implications of unregulated text-to-video advancements could lead to heightened distrust in established forms of communication. OpenAI’s cautious approach with Sora is acknowledged, but the article warns that future developments might not exercise the same level of care, necessitating proactive regulatory measures in the evolving AI landscape.