AI clicks are a revolution but also a threat. Expert:: they collect a lot of data that can hit the wrong hands

Data from AI Chatbots can get into the hands of cyber criminals, foreign services or advertisers and be used to the detriment of users – warns Mateusz Chrobok, a cyber security expert in an interview with PAP. He recommended the use of digital hygiene and skepticism towards generated content.

Mateusz Chrobok, an expert of cyber security and AI, who runs an educational platform and a YouTube channel observed by 157 thousand. people pointed out that Technology companies developing artificial intelligence chatbots often obtain data about users from the Internet, including social media and based on information that they enter in chatbots.

“On this basis, we are usually profiled by a given company so that Czatbot's answers are more suited to us,” he explained.

The expert indicated that From the prompts, i.e. inquiries that a given user enters in Chatbot, companies can read information about the language he uses, appearance – in the case of attached photos, about views, family situation, problems, etc. Some companies additionally ask for access to information from the user's device, e.g. contacts, locations. “Chinese Deepseek also collected a way of writing on a keyboard, from which you can pull out a lot of data, e.g. age, either whether we are tired or sleepless on a given day,” noted the Brave.

“It happens that we are tempted to give our data in exchange for free access to chatbot, a better model, etc. So we really give our privacy for some benefits” – he noted. He indicated that the use of chatbots is associated with the risk of leakage of sensitive data. The Brave recalled the situation from two years ago, when OpenAI confused the Indexes of ChatgPT users. This resulted in a situation in which, after logging in to their account, a person had access to the history of another person's conversations. “The problem has affected 1.2 percent of users, but in practice they are millions of people” – he emphasized.

Another threat to the security of chatbots is the ATO attack, i.e. Account Takeover (takeover of the account – PAP) – pointed out the expert. After taking over the account, the cybercriminter may gain access to data from the history of user conversation, e.g. name, surname, telephone number, credit card number, if he accidentally entered it or made available in a document – warned the Brave. “Studies show that models store this type of information and if they are trained on it, it is impossible to easily erase. There are also ways of how to pull this data out of chatbot,” the expert pointed out.

He added that some companies allow users to disable the option of saving history or model training on their data. “For very sensitive information, it is safest to install a local model on your device or server. Then there is the best chance that our data will not leak” – he emphasized.

Chrobok indicated that the collected data can also be used to create profiled ads. “Let's imagine that we have a bad day or we are struggling with depression, obesity or other problem and advertisers use this information to influence our purchasing decisions, which will not necessarily be beneficial for us. We enter here into the gray world of manipulation. What, we think that it would be good for us, it is not necessarily good for the creators of AI models and companies that optimize profit,” he assessed.

According to the expert, the Chatbota country may influence the level of user protection level. “Deepseek, a Chinese company, creates great models such as R1, but it is blocked in many places, because according to Chinese law, its creators must transfer data about users to the authorities, and these, for example, can give it to services” – he pointed out. He gave an example of a hypothetical situation in which an American official writes with Chinese chatbot and tells him about his problems with his family or business. “The collected information could be used by China to recruit this person on a spy or for another influence. I think that by revealing our weaknesses, we are more susceptible to manipulation in such a situation,” he emphasized.

Chrobok pointed out that research shows that the models reflect the views of their creators. “For example, Chinese Deepseek has a negative sentiment (tone of the generated content – PAP) when statements about American soldiers, and the Openai models are quite the opposite” – he pointed out and added that “every model is a certain information bubble”. “They are not neutral, even if some creators are trying to make them like that. It is worth remembering” – he emphasized.

The expert, when asked about the safety of using chatbots as emotional support or the therapist – which makes some users, recalled a situation that took place in 2023 in Belgium. A man who was very aware of the problems related to global warming wrote about this with chatbot. At some point, Ai told him that if she wanted to reduce the amount of CO2 he generates, it is best if he was not there, as a result of which the man took his life. It was the first registered suicide after talking to artificial intelligence – the expert pointed out.

“This is an extreme, which, however, shows what can threaten us when we ask AI for advice, we share our mental state and views with it. The answers she gives are based on statistics, which means that they will not always be accurate. Thanks to the progress of technology, they are increasingly so, which makes us trust them, and this can sleep, and he added that more specially” The issue, all the more – at least at the moment – the models of artificial intelligence are wrong.

“This can be significantly improved by technical methods such as Deep Research (an advanced method of information analysis – PAP), but not everyone knows how to use them. That's why I encourage skepticism to the content that generates chatbots” – emphasized the expert.

Chrobot, when asked about threats related to chatbots in the workplace, gave an example from 2023, when Samsung's employee threw a presentation with secret data in ChatGPT, which, as a consequence, leaked and the competition could have gained them. “This is what many companies and managers are afraid of and it happens that they completely prohibit employees with the use of artificial intelligence. In my opinion, this is not a good way,” said the educator. According to him, without this technology, companies will be less competitive. “You just need to know how to use it safely,” he noted.

According to Chrobok, the process of safe implementation of AI in companies should start by developing “AI hygiene”, i.e. the education of the superiors themselves and later employees. “Secondly, technical means are important,” he pointed out and explained that there is a whole field of solutions called Data Leak Prevention (PAP protection – PAP) in which the employers can invest. It includes, for example, models that assess what data can leave the company and which do not. The expert pointed out that in each organization there should also be the rules of using AI, in which, for example, what data can be thrown into Czatbot, which of them are sensitive and which should be marked as generated with the help of artificial intelligence, because, for example, this requires the law.

The expert asked whether – according to him – despite the risks, artificial intelligence should be implemented in every company, stated that forcing all entrepreneurs to AI would be “inhuman”. “But it is better to know this technology, because it can help find yourself on the labor market or improve the work itself so that it is less and that it would be more effective” – ​​he assessed.

“We are at a moment that some define with evolution, revolution or use other big words. This is not the moment where Ai is overwhelming us, although in some respects it is certainly so” – he pointed out. He emphasized that human intelligence is limited. “AI models also have their limitations and threats, but they are more and more accurate and in the future they will become better than people. Our skills and abilities will simply be weaker,” he said.

He noted that he is worried about the approach of some companies to employees in connection with the leakage of the president of the president of a company in which he wrote that “before you employ a man, test several AI models”. “Perhaps such a future awaits us if there is no other solutions along the way. After all, I am a fan of implementing artificial intelligence to improve our work. Talking about +temporary fashion +, or +hajpie on AI +, in my opinion, sleeping an important moment” – emphasized the Brave.

Monika Blandyna Lewkowicz (PAP)

mbl/ mick/ mhr/