President of the Personal Data Protection Office points to the need to regulate the use of AI systems in employment

Mirosław Wróblewski submitted a request to the Minister of Family, Labour and Social Policy to undertake work on provisions that will provide additional protection against discrimination against candidates and employees due to the use of artificial intelligence systems. They tend to perpetuate existing prejudices present in the data on which they have been trained.

The President of the Personal Data Protection Office pointed out that he is receiving an increasing number of reports about the use of AI-based tools in recruitment processes. They allow for processing of personal data on a large scale, including sometimes special (sensitive) categories of data. Therefore, in the opinion of Mirosław Wróblewski, additional regulations are necessary to prevent the discriminatory effects of the use of such systems.

The GDPR provides for the possibility of introducing such additional safeguards in national labour law. In turn, the Labour Code establishes an obligation to treat employees equally regardless of gender, age, disability, origin, religion, political opinions or sexual orientation. Meanwhile, AI systems pose a serious risk of replicating biases existing in training data. In addition, they can also analyse much more extensive information than is provided for in the catalogue of data that the employer may request from the candidate for work - referred to in Article 22¹ of the Labour Code.

The President of the Personal Data Protection Office also draws attention to the right of the data subject not to be subject to a decision based solely on automated processing of data by algorithms. However, this provision introduces an exception - the possibility to allow the use of such automated processing in national law. However, if profiling is to be used or permitted by law, it should guarantee suitable measures to safeguard the data subjects' rights.

In the EU Regulation on artificial intelligence (the AI Act), AI tools used in recruitment have been classified as so-called high-risk systems. According to Article 86(1) of that Regulation: 'Any affected person subject to a decision which is taken by the deployer on the basis of the output from a high-risk AI system listed in Annex III, with the exception of systems listed under point 2 thereof, and which produces legal effects or similarly significantly affects that person in a way that they consider to have an adverse impact on their health, safety or fundamental rights shall have the right to obtain from the deployer clear and meaningful explanations of the role of the AI system in the decision-making procedure and the main elements of the decision taken.'

Furthermore, should the Ministry wish to introduce provisions allowing the use of such AI systems in employment, this should be preceded by a data protection impact assessment. This will identify potential risks and make it easier for entities using the AI tools to demonstrate that the processing of personal data complies with the GDPR. At the same time, such an assessment will determine whether the solutions adopted provide for appropriate safeguards for the rights and freedoms of data subjects.

The obligation to perform an assessment of the impact on fundamental rights the use of high-risk system may produce (including the right to privacy) follows directly from Article 27 of the AI Act. However, it should be stressed that it is only intended to complement and not replace the data protection assessment. High-risk systems should be transparent, i.e., they should guarantee access to information on how AI models work and a meaningful explanation of both the technical processes of such a system and the justification of the decision taken as a result of their use. In addition, those who use them must be aware that they are dealing with artificial intelligence. The legislation should therefore set out the objectives to be pursued by the use of such systems. The regulations should also determine who is the data controller and what safeguards are in place to protect the rights of data subjects.

Article 26 of the AI Act stipulates that high-risk systems must comply with legal requirements such as: maintaining a risk management system (Article 9), data and data governance (Article 10), technical documentation (Article 11), recording of events over the lifetime of the system (Article 12), transparency and provision of information to users (Article 13), human oversight (Article 14), robustness, accuracy and cybersecurity (Article 15 AI Act). The guarantees set out in the EU AI Act should be reflected in national legislation with the rank of an act.

The President of the Personal Data Protection Office offered to the Ministry of Family, Labour and Social Policy the support of his expertise in the analysis of the proposed provisions from the perspective of their compliance with the GDPR.