“Never assume privacy of conversations with a cottage.” AI testers watch intimate materials of user

Four contractors employed by the Outlier training platforms (belonging to Scale AI) and Alignerr said that RThey came across conversations containing full names, telephone numbers, e-mail addresses, gender, interests and other personal data. One of them estimated that the data enabling the identification of the user appeared in more than half of thousands of chats, which he browned weekly. Two colleagues admitted that They came across talks in which users sent Chatbot their selfie. Talks came from people from all over the world.

Some of these personal data were attached by the metal history to help contractors in personalizing AI's response – according to the documentation of one of the three projects ordered by the META company, which was analyzed by Business Insider. In other cases, users themselves transferred to chatbot personal data during talks – which, as the Privacy Policy warns, should not take place.

Two of the four BI interlocutors worked on similar projects for other large technology companies and found that Unheanized personal data more often occurred in meta projects.

See also: They got rid of employees and put them on AI. Today they can enjoy their decision

Intimate conversations and photos of users

Design documents – two of which were still used last month – informed the contractors that they were analyzing real conversations between users and chatbot.

Meta's colleagues who talked to Business Insider, They described many of these conversations as very personal. They took the form of therapeutic sessions, private denominations in the style of conversation with a friend or intimate conversations with a romantic partner.

Users opened themselves about their fantasies, flirted with chatbot, complained about problems in life and asked for advice. Sometimes they provided their contact details, jobs, locations or information about children.

One of the contractors who worked on the OMNI project led by Alignerr – aimed at increasing the involvement of users in AI Studio from Meta – said that Some users shared very intimate information with chatbot, including a selfie and erotic photos.

The rules of using AI from META indicate that the company “can view” users' interactions with AI, and this process can be “automated” or conducted by people.

What does the finish say?

The Meta spokesman gave Business Insider that the company has “raw rules” regarding access to personal data – for both employees and contractors.

“Although we work with contractors to improve the quality of training data, we deliberately limit the scope of personal data that they can see and have implemented procedures and security, which determine how to proceed in the case of such data” – the spokesman added.

Scale spokesman AI emphasized that “colleagues have the right to process personal data only to the extent necessary for the project”, they must comply with the security standards of the Outlier platform and are “obliged to determine the answers containing personal data and to skip tasks with such content.”

“Data generated by users never leave the customer's platform,” he added. “Many projects are carried out directly on customer platforms, which provides additional security and does not allow data outflow outside the system” – we read further.

Alignerr did not answer the request for comment.

This is another example of security gaps and privacy problems that appear as companies employ crowds of people to improve developing AI systems. Scale AI blocked access to Google public documents after Business Insider revealed that they contained confidential customer data. Some of these documents described how XAI and Meta train their AI models. In turn, the Surge training platform used a public spreadsheet to instruct employees during AI training Anthropic.

“Never assume that chatbots are private by nature”

– The risk associated with leakage of personal data from conversations with chatbots is huge – it can lead to manipulation, fraud and other abuses – said Miranda Bogen, director of AI Governance Lab in the Center for Democracy and Technology, dealing with digital rights and privacy.

Bogen He advises people concerned about privacy “never assume that talks with chatbots are private, especially since practices in various companies differ significantly.”

She added that AI tools, such as chatbots, are “used in a completely different way than traditional internet tools” and “people may not be aware that their conversations are used to train models, and that people have access to them.”

Many technology companies – including META, OPENAI and Google – inform in privacy policies that they can use conversations with AI for training and improving models.

– Data that users do not expect to be seen by people have long been a problem in the industry – says Bogen. – It's just another installment of this problem. The difference is that the context of a conversation with AI often seems to be more intimate users – like a therapeutic conversation or providing confidential information about work – he adds.

Data enabling identification

Business Insider was unable to determine how often the contractors encountered personal data, but those who talked to the editors claim that it happened regularly while working on Meta AI projects.

One of the interlocutors working on the Omni project said that he was able to perform up to 5000 tasks a week. He was obliged to mean and reject chats containing personal data. He estimated that data such as phone numbers or instagram users' names appeared in “60-70 percent” cases.

Another project implemented from Outlier, named PQpe, was aimed at giving conversations with AI a more personalized character, referring to information known about the user – such as name, gender, location and interests. Each history of chat was provided with a list of facts about the user that AI was to use. These facts were based on previous conversations and “activities on social profiles” – as the documentation said. In this project, contractors could not reject conversations containing personal data.

Meta spokesman said that In AI personalization projects, “contractors have access to specific personal data, in accordance with the Meta Privacy Policy and AI Regulations”.

In some cases, colleagues may have access to the data needed to complete the task – for example, information about the location if the user asks to find a cafe nearby.

The interlocutor BI added that “Definitely” you can identify a person based on a few detailed user descriptions contained in the training task.

The Meta spokesman assured that all contractors undergo tests assessing their competences in the field of cyber security and privacy risk management.

In one case, the data attached to the history of the conversation containing sexual content was so detailed that Business Insider managed to find a Facebook profile with a matching name, city, gender and list of interest. The search took less than five minutes.

The contractor assigned to this chat, whose content saw Business Insider, but decided not to describe, said that the conversation was so disturbing that he had to finish work for the rest of the day.

– It was hard. I had to postpone it for the evening – he said.

Privacy in chatbots

Companies such as Meta intensively develop personalized AI, which means the processing of more and more personal and sensitive data.

The CEO of Meta, Mark Zuckerberg, recently presented the vision of “personal superintelligence” – AI, who “knows us deeply, understands our goals and helps us achieve them.”

In other recent cases, users did not realize that they were providing their conversations with AI in public. Business Insider revealed in June that users unknowingly published conversations with AI Meta – including questions about health, career and relationships – in the public channel in the Meta AI application. Some of them contained contact details, e-mail addresses or full names.

In response, META has introduced a new warning in the META AI application.

The application still allows you to share conversations, which means that they can be indexed and visible in Google search results. At the same time, Opeli removed the function through which the shared conversations with ChatgPT were indexed by Google.

Sara Marcucci, founder of AI + Planetary Justice Alliance – a global organization of researchers and activists – said that Business Insidera reports suggest that “data minimization, anonymization and user control are still irregular and poorly enforced in the industry.”

Bogen emphasized that although automatic filters can detect and delete personal data, they do not capture everything – that's why human intervention is needed.

– The very fact that there is a process of determining and deleting data by people does not mean that it does not process them – but it shows that the system, if at all exists, is imperfect and this imperfection is widely accepted – she added.

The above text is a translation with American Business Insider edition