by OpenAI, known for championing safe and beneficial artificial intelligence (AI), has recently faced concerns regarding the security of personal data. Google researchers discovered a method to manipulate ChatGPT into revealing private user information using just a few simple commands.
ChatGPT, which gained immense popularity since its launch, boasts over 100 million users within two months. Its success lies in its extensive collection of more than 300 billion data fragments sourced from online platforms, including articles, websites, posts, journals, and books.
Despite OpenAI’s efforts to prioritize user privacy, everyday conversations and online interactions accumulate a vast amount of personal data that is not intended for public distribution.
In their research, Google scientists identified a method that allowed them to exploit ChatGPT by using specific keywords to trigger the disclosure of unintended training data.
By investing only $200 in queries to ChatGPT (gpt-3.5-turbo), they were able to extract over 10,000 distinct verbatim training examples. They estimated that with a larger budget, dedicated adversaries could obtain even more data.
Through this technique, unauthorized individuals could gain access to sensitive information such as names, phone numbers, and addresses by manipulating ChatGPT with absurd commands that lead to malfunctions.
For instance, by requesting a continuous repetition of the word “poem,” the researchers managed to compel the model to access restricted details within its training data, moving beyond its normal training protocols.
Similarly, they successfully retrieved the email address and phone number of an American law firm by requesting an infinite repetition of the word “company.”
To prevent unauthorized data disclosures, several companies have restricted their employees’ usage of large language models. Apple, for instance, has blocked access to AI tools like ChatGPT and GitHub’s AI assistant Copilot.
Earlier this year, confidential data stored on Samsung servers became exposed due to employee errors. Although not related to a specific leak, employees unintentionally revealed confidential information, including source code and a transcript of a private company meeting. Ironically, this incident occurred shortly after Samsung lifted an initial ban on ChatGPT due to concerns about potential exposure.
In response to growing concerns surrounding data breaches, OpenAI implemented a feature that allows users to disable chat history, providing an additional layer of protection for sensitive data. However, this data is still retained for 30 days before being permanently deleted.
Google researchers, in their blog post, pointed out that OpenAI claims that over a hundred million individuals interact with ChatGPT on a weekly basis, amounting to a billion user-hours. Surprisingly, no one had noticed the model emitting training data with such frequency until their paper was published, highlighting the significance of their findings.
The researchers expressed their concerns and emphasized that caution should be exercised when training future models for privacy-sensitive applications. Extreme safeguards must be put in place to ensure the protection of user data.
*Note:
1. Source: Coherent Market Insights, Public sources, Desk research
2. We have leveraged AI tools to mine information and compile it
Money Singh is a seasoned content writer with over four years of experience in the market research sector. Her expertise spans various industries, including food and beverages, biotechnology, chemicals and materials, defense and aerospace, consumer goods, etc.
