OpenAI’s ChatGPT, while gaining massive global popularity, has been found to have a significant vulnerability in its search algorithm, according to a report from The Guardian. Research reveals that the AI chatbot’s search function can be exploited through hidden content on web pages, which could lead to manipulated responses or even harmful outcomes for users.
The issue arises from a practice known as Prompt Injection, where hidden material embedded in web pages can influence ChatGPT’s responses. These hidden instructions could be used to manipulate the AI into providing biased ratings or feedback that doesn’t align with the actual content of the website. In some cases, the AI could even retrieve and present malicious code from web pages, posing a serious security risk, especially when used for summarizing or analyzing web content.
During the study, a test was conducted using a fake website that promoted a camera product. When hidden instructions were added, ChatGPT gave an overly positive review of the product, despite the visible content on the page being negative. This demonstrated how easily the AI’s responses could be swayed by covert instructions.
Jacob Larsen, a cybersecurity expert at CyberCX, expressed concerns that the current search engine functionality in ChatGPT could become a major security risk if it were fully launched. He highlighted the possibility of fake websites being designed to manipulate the AI’s responses and noted that OpenAI has an AI security team working on addressing these vulnerabilities.
At present, the search tool is only available to premium users, and OpenAI has been urging them to make it their default search engine. OpenAI is working on further testing and improvements to ensure that such vulnerabilities are addressed before the feature becomes available to all users.
This issue comes as Apple recently announced plans to integrate ChatGPT into its iOS, iPadOS, and macOS platforms, providing users with enhanced AI capabilities, such as image and document understanding. However, the vulnerability in the search function raises concerns about the potential for misinformation and security risks in these new features.
Related topics:
How OpenAI Helped Lowe’s Transform Retail with Generative AI
ChatGPT Search Tool Vulnerable to Manipulation and Malicious Content
Google’s Gemini Update Competes with OpenAI’s Reasoning AI Model