The EU is pouring asphalt on AI. Law vs. “Black Mirror” scenarios

The AI ​​Act philosophy is based on the so-called risk pyramid.

— The law divides artificial intelligence systems into four categories: unacceptable risk [zakazane całkowicie — red.]through high and limited, to minimal. Let's start from the top of the pyramid – technologies so dangerous that they received a red card in the European Union – explains Dr. Szymborska.

To understand what this is about, it is worth remembering one of the episodes of the series “Black Mirror” entitled “Nosedive” (“On the head, on the neck”). Society functions around an application in which people constantly evaluate each other. The main character, after a series of unfortunate events, loses her “points”, which in turn cuts her off from renting a good apartment or buying a plane ticket.

What we saw in 2016 as dystopian fiction is becoming reality in some places around the world – citizen assessment systems (so-called social scoring) operate, for example, in China (fortunately for citizens, they do not operate – yet – on such a scale). This scenario will not come true in Europe.

— This is guaranteed by Article 5 of the AI ​​Act, which lists prohibited practices in the field of artificial intelligence. Pursuant to paragraph 1 lit. c)it is categorically prohibited to use AI to assess and classify social behavior — the final, binding text of the AI ​​Act won by the European Parliament extended this ban also to private entities. This is a key change from the point of view of citizens – it means that banks, insurers and private corporations are also absolutely prohibited from creating “social scoring” systems. No “digital result” will deprive us of access to an office, doctor or school – explains Dr. Dota Szymborska graphically.

The prohibitions do not end there. Art. 5 section 1 letter f outlaws emotion recognition systems in workplaces and educational institutions.

– It means that the ultimate end to the idea of ​​algorithms that could fire an employee just because a CCTV camera thinks they look tired or bored. In short, our emotions are still our private matter, says the expert.

Read also: Artificial intelligence has put the “secret market” in a trap. Money trapped

Algorithm in the office and in the bank – “high risk” systems

There are also systems that are completely legal and very useful, but have a powerful impact on our lives. The AI ​​Act calls them high-risk systems and takes a closer look at them (Article 6 and detailed Annex III).

The AI ​​Act describes, among others: technologies used in employment (recruitment) and granting access to basic services (such as bank loans).

Why is it so important?

– Let's look at loud scandal from the Netherlands – the so-called Toeslagenaffaire, i.e. a benefits scandal. Due to errors and built-in biases, the self-learning model used by the local tax office to profile child benefit applicants falsely accused tens of thousands of innocent families, often with dual citizenship, of benefit fraud. People had to return huge sums, lost their life's work, and all because of the “black box”, the decisions of which no one could explain. – says the expert.

The AI ​​Act makes it clear: it won't happen again.

— From now on, models deciding whether to reject a CV or loan application will have to be transparent and fed with non-discriminatory data. However, the most important is Article 14. It introduces the obligation of human supervision [human oversight — red.]. The bank or employer will no longer be able to throw up their hands helplessly and say: “We rejected the application because the computer said so, we don't know why.” Ultimate responsibility and control must always rest with humans – says Dr. Dota Szymborska.

Virtual reality and disinformation – who wrote this text?

Most of us don't take out a loan every day, but we read or watch digital content every day. Here, as Dr. Dota Szymborska says, we are entering a limited risk zone.

The flood of false information generated by AI is already our everyday life. We saw this during the recent elections in Slovakia (September 2023), where, at a key moment in the campaign, a prepared audio recording (deepfake) compromising one of the candidates was posted online.

Every day we also come across false advertisements with the voices of famous journalists encouraging shady investmentsand on hotlines we get frustrated talking to bots that pretend to be real advisors.

— This is where Article 50 of the AI ​​Act comes in handy, which imposes transparency obligations. Thanks to it, we gain a fundamental “right to knowledge”. If you call the hotline or write in the chat and there is artificial intelligence on the other end, the system must inform you about it immediately – says the expert.

Similarly with online content: images, sounds and videos generated by AI (including the mentioned deepfakes) must be clearly marked with either a watermark or an appropriate label. — The law is supposed to protect us from the digital illusion in which we no longer distinguish truth from fiction – emphasizes the lecturer at WSB Merito University

Citizen protection or a noose on European innovation?

However, there is no shortage of critical voices about the AI ​​Act. Business representatives and European technology companies – such as the French Mistral AI or the German Aleph Alpha – have long been warning that the EU and the AI ​​Act will inhibit the development of new technologies.

In their opinion, a thicket of regulations, stringent certification requirements, expensive technical audits may cause young European start-ups to simply go bankrupt, will flee to the USA, leaving Europe at the tail of the global technological race.

Are they right? — From an ethical perspective, the answer is clear: technological innovation cannot develop at the expense of basic human rights. As was the case with the GDPR, which initially terrified companies and today is a global reference point for privacy protection, the European Union is counting on the so-called Brussels effect – emphasizes Dr. Dota Szymborska.

In her opinion, companies from Silicon Valley or China who want to sell their products on the lucrative European market will have to adapt to our standards anyway.

— In the long term, consumer confidence [tzw. Trustworthy AI — red.] may become Europe's greatest competitive advantage and most important export product, says the expert.

“Kowalski” Law in the digital world

Therefore, according to Dr. Dota Szymborska, the AI ​​Act will not stop the development of artificial intelligence, block innovation or “turn off” servers.

— His goal is to civilize this rapid process. The digital Wild West in which the strongest players did whatever they wanted with our data is coming to an end. The European Union is starting to pour asphalt and put road signs on the artificial intelligence highway, the expert emphasizes.

However, for this system to work, we all need awareness.

— The law only protects us if we understand it and can enforce it. The next time we apply for a job, ask for a loan or get irritated with the “consultant” on the other end of the phone – let's remember that we are not defenseless against machines. We have new, specific rights. It's worth starting to use them. Thanks to this, sci-fi series will remain in the realm of fantasy, he warns.

And as he emphasizes, the AI ​​Act is not an instruction manual for a small group of programmers, it is a digital protective shield for all of us. – These regulations will determine whether we will get our dream job in the future, whether the bank will grant us a loan for an apartment and how much we will be able to trust what we see while scrolling the Internet on the tram – explains the expert.

Dr. Dota Szymborska points out that there are also many discussions on AI Omnibus, and lawyers talk about the need for product liability regulations, i.e. Agentic Law. — In practice, a new branch of civil law may be created that will regulate the “rights and obligations” of autonomous systems replacing humans in commercial contracts – says the expert.