ARTIFICIAL INTELLIGENCE AND DISCRIMINATION – STRENGHTS AND WEAKNESSES
OF THE CURRENT EUROPEAN ANTI-DISCRIMINATION LEGAL FRAMEWORK
Aleksandar Mihajlović, LL.M.
We are witnessing an enormous development of artificial intelligence (AI) which boosts economic productivity, creates new job opportunities, and gives hope that human life will be more prosperous. On the other side, AI, as a new system, that is undiscovered and unpredictable, creates ethical and legal dilemmas and threats to human rights violations, in the context of the principle of equality, the rule of law, and democratic principles, if it is used in an inappropriate way. The subject of the paper is discrimination in the process of AI application in different fields of people’s everyday lives. The aim of this investigation is to analyze provisions in the recently adopted European Union (EU) AI Act and the Council of Europe Framework Convention which are expected to prevent discriminatory treatment through an AI life cycle, as well as of the position of the Serbian national authorities in that regard.
The authors will provide a critical and comparative analysis of these two instruments governing the AI application. Subsequent to that, the paper is focused on the position taken by the Serbian authorities in order to examine the level of its readiness to stay in line with the legal challenges the AI implementation brings, and illustrates a current example which is related to the implementation of the Social Card Law. The methodological framework includes doctrinal, comparative, and descriptive methods.
Keywords: artificial intelligence, anti-discrimination, European Union AI Act, Council of Europe Framework Convention, Serbia, Social Card Law.
Monographs
Bartoletti, I. & Xenidis, R. 2023. Study on the impact of artificial intelligence systems, their potential for promoting equality, including gender equality, and the risks they may cause in relation to non-discrimination. Strasbourg: The Council of Europe. Available at: https://rm.coe.int/study-on-the-impact-of-artificial-intelligence-systems-their-potential/1680ac99e3 (10. 7. 2024).
European Union Agency for Fundamental Rights. 2022. Bias in Algorithms – Artificial Intelligence and Discrimination – Report. Luxembourg: Publications Office of the European Union. Available at: https://fra.europa.eu/sites/default/files/fra_uploads/fra-2022-bias-in-algorithms_en.pdf (10. 7. 2024).
Leslie, D. et. al. 2021. Artificial intelligence, human rights, democracy, and the rule of law: a primer. The Council of Europe. Available at: https://rm.coe.int/primer-en-new-cover-pages-coe-english-compressed-2754-7186-0228-v-1/1680a2fd4a (7. 7. 2024). https://doi.org/10.2139/ssrn.3817999
Prlja, D. Gasmi, G. & Korać, V. 2021. Veštačka inteligencija u pravnom sistemu EU. Beograd: Institut za uporedno pravo. Available at: https://iup.rs/wp-content/uploads/2021/05/2021-Ve%C5%A1ta%C4%8Dka-inteligencija-u-pravnom-sistemu.pdf (22. 9. 2024).
Prlja, D. Gasmi, G. & Korać, V. 2022. Ljudska prava i veštačka inteligencija. Beograd: Institut za uporedno pravo. Available at: https://iup.rs/wp-content/uploads/2022/09/Ljudska-prava-i-ve%C5%A1ta%C4%8Dka-inteligencija-Prlja-Gasmi-Korac.pdf (10. 7. 2024).
Rao, S. A. & Verweij, G. 2017. Sizing the prize – PwC’s Global Artificial Intelligence Study: Exploiting the AI Revolution – What’s the real value of AI for your business and how can you capitalize? PwC. Available at: https://www.pwc.com/gx/en/issues/analytics/assets/pwc-ai-analysis-sizing-the-prize-report.pdf (7. 7. 2024).
Rejeski, D., Reynolds, L. & Wright, S. 2018. When Software Rules: Rule of Law in the Age of Artificial Intelligence. Washington, D. C: Environmental Law Institute. Available at: https://www.eli.org/sites/default/files/eli-pubs/when-software-rules-web.pdf (8. 7. 2024).
Renard-Castets, C. & Eynard, J. 2023. Un Droit de L’intelligence Artificielle: Entre Règles Sectorielles et Régime Général: Perspectives Comparées. Belgium: Bruylant.
Sannerholm, R. 2021. Responsibility and Accountability: AI, Governance, and the Rule of Law. In: Collona, L. and Greenstein, S. (eds.), Law in the Era of Artificial Intelligence. Stockholm: Stiftelsen Juridisk Fakultetslitteratur (SJF) and The Swedish Law and Informatics Research Institute (IRI) and Law Faculty, Stockholm University, pp. 223-246. Available at: https://irilaw.org/2022/02/16/new-publication-nordic-yearbook-of-law-and-informatics-2020-2021/ (8. 7. 2024). https://doi.org/10.53292/208f5901.40d940a1
Weerts, H. et al. 2023. Algorithmic Unfairness through the Lens of EU Non-Discrimination Law: Or Why the Law is not a Decision Tree. In: 2023 ACM Conference on Fairness, Accountability, and Transparency (FAccT’23), June 12–15, 2023, Chicago, IL, USA. New York : ACM, pp. 805-816. https://doi.org/10.1145/3593013.3594044
Scholarly Articles
Bathaee, Y. 2018. The Artificial Intelligence Black Box and the Failure of Intent and Causation. Harvard Journal of Law & Technology, 31(2), pp. 889-938. Available at: https://jolt.law.harvard.edu/assets/articlePDFs/v31/The-Artificial-Intelligence-Black-Box-and-the-Failure-of-Intent-and-Causation-Yavar-Bathaee.pdf (8. 7. 2024).
Chen, Z. 2023. Ethics and discrimination in artificial intelligence-enabled recruitment practices. Humanities and Social Sciences Communications, 10(567), pp. 1-12. Available at: https://www.nature.com/articles/s41599-023-02079-x (10. 7. 2024). https://doi.org/10.1057/s41599-023-02079-x
Fornasier Oliveira de, M. 2021. Artificial Intelligence and Democratic Rule of Law. Revista de Estudos Constitucionais, Hermenêutica e Teoria do Direito (RECHTD), 13(3), pp. 351-369. https://doi.org/10.4013/rechtd.2021.133.06
Ćorić, V. & Knežević Bojović, A. 2020. Autonomous Concepts and Status Quo Method: Quest for Coherent Protection of Human Rights before European Supranational Courts. Strani pravni život, 64(4), pp. 27-40. https://doi.org/10.5937/spz64-30165
Greenstein, S. 2022. Preserving the rule of law in the era of artificial intelligence (AI). Artificial Intelligence and Law, 30, pp. 291-323. https://doi.org/10.1007/s10506-021-09294-4
Heinrichs, B. 2022. Discrimination in the age of artificial intelligence. AI & Society, 37(1), pp. 143-154. https://doi.org/10.1007/s00146-021-01192-2
Huq, Z. A. 2021. Artificial Intelligence and the Rule of Law. Public Law and Legal Theory Working Paper Series, No. 764, pp. 1-14. Available at: https://chicagounbound.uchicago.edu/cgi/viewcontent.cgi?article=2194&context=public_law_and_legal_theory (8. 7. 2024).
Kaur, I. & Puri Gopal, C. 2021. Impact of Artificial Intelligence on Legal Industry. International Journal of Law Management & Humanities, 4(2), pp. 346-354.
Kleinberg, J. et al. 2018. Discrimination in the Age of Algorithms. Journal of Legal Analysis, 10, pp. 113-174. https://doi.org/10.1093/jla/laz001
Korać, V. Prlja, D. & Gasmi, G. 2021. Challenges brought on by Artificial Intelligence. Archaeology and Science, 17, pp. 159-165. https://doi.org/10.18485/arhe_apn.2021.17.10
Korać, V. Prlja, D. & Gasmi, G. 2022. Influence of Artificial Intelligence on Human Rights. Archaeology and Science, 18, pp. 279-292. https://doi.org/10.18485/arhe_apn.2022.18.19
Korać, V. Prlja, D. & Gasmi, G. 2023. European Approach to Artificial Intelligence. Archaeology and Science, 19, pp. 209-216. https://doi.org/10.18485/arhe_apn.2023.19.14
Lütz, F. 2024. The AI Act, gender equality and non-discrimination: what role for the AI office? ERA Forum, 25, pp. 79-95. https://doi.org/10.1007/s12027-024-00785-w
Miasato, A. & Silva Reis, F. 2019. Artificial Intelligence as an Instrument of Discrimination in Workforce Recruitment. Acta Universitatis Sapientiae, Legal Studies, 8(2), pp. 191-212. https://doi.org/10.47745/AUSLEG.2019.8.2.04
Nemitz, P. 2021. Democracy through law The Transatlantic Reflection Group and its manifesto in defence of democracy and the rule of law in the age of “artificial intelligence”. European Law Journal, 29(1-2), pp. 237-248. https://doi.org/10.1111/eulj.12407
Schmidt, N. & Stephens, B. 2019. An Introduction to Artificial Intelligence and Solutions to the Problem of Algorithmic Discrimination. Conference on Consumer Finance Law (CCFL) Quarterly Report, 73(2), pp. 130-144.
Surden, H. 2019. Artificial Intelligence and Law: An Overview. Georgia State University Law Review, 35(4), pp. 1305-1337.
Xenidis, R. 2020. Tuning EU equality law to algorithmic discrimination: Three pathways to resilience. Maastricht Journal of European and Comparative Law, 27(6), pp. 736–758. https://doi.org/10.1177/1023263X20982173
Legal Sources
Regulation (EU) 2024/1689 of the European Parliament and of the Council of 13 June 2024 laying down harmonised rules on artificial intelligence and amending Regulations (EC) No 300/2008, (EU) No 167/2013, (EU) No 168/2013, (EU) 2018/858, (EU) 2018/1139 and (EU) 2019/2144 and Directives 2014/90/EU, (EU) 2016/797 and (EU) 2020/1828 (Artificial Intelligence Act)Text with EEA relevance, OJ L, 2024/1689, 12.7.2024 Available at: https://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=OJ:L_202401689 (15. 7. 2024).
Council of Europe Framework Convention on Artificial Intelligence and Human Rights, Democracy and the Rule of Law, CM(2024)52-final, 17 May 2024. Available at: https://search.coe.int/cm#{%22CoEIdentifier%22:[%220900001680afb11f%22],%22sort%22:[%22CoEValidationDate%20Descending%22]} (7. 7. 2024).
Strategy of Development of Artificial Intelligence in the Republic of Serbia for the period 2020-2025, Official Gazette RS, No. 96/2019.
Ethical Guidelines for the Development, Application and Usage of Reliable and Responsible Artificial Intelligence, Official Gazette RS, No. 23/2023.
Social Card Law, Official Gazette RS, No. 14/2021.
Law on the Prohibition of Discrimination, Official Gazette RS, No. 22/2009 and 52/2021.
Commission decision of establishing the European Artificial Intelligence Office, Brussels, 24. 01. 2024 C(2024) 390 final. Available at: https://digital-strategy.ec.europa.eu/en/library/commission-decision-establishing-european-ai-office (16. 7. 2024).
Online sources
Algorithms in credit scoring: Discrimination based on use of statistical data in Finland. Equinet. 8 July 2021. Available at: https://ai.equineteurope.org/library/algorithms-credit-scoring-discrimination-based-use-statistical-data-finland (10. 7. 2024).
(Anti) Social Card. Available at: https://antisocijalnekarte.org/en (17. 7. 2024).
Dasti, J. Insight – Amazon scraps secret AI recruiting tool that showed bias against women. 11 October 2018. Available at: https://www.reuters.com/article/us-amazon-com-jobs-automation-insight/amazon-scraps-secret-ai-recruiting-tool-that-showed-bias-against-women-idUSKCN1MK08G/ (10. 7. 2024).
Dutch childcare benefit scandal an urgent wake-up call to ban racist algorithms. Amnesty International. 25 October 2021. Available at: https://www.amnesty.org/en/latest/news/2021/10/xenophobic-machines-dutch-child-benefit-scandal/ (10. 7. 2024).
Goodman, R. Why Amazon’s Automated Hiring Tool Discriminated Against Women. 12 October 2018. Available at: https://www.aclu.org/news/womens-rights/why-amazons-automated-hiring-tool-discriminated-against (10. 7. 2024).
Grobelnik, M. Perset, K. & Russell S. 2024. What is AI? Can you make a clear distinction between AI and non-AI systems? 6 March 2024. Available at: https://oecd.ai/en/wonk/definition (7. 7. 2024).
Güçlütürk Gazi, O. Understanding Council of Europe’s Draft Framework Convention on AI, Human Rights, Democracy, and Rule of Law. 17 January 2024. Available at: https://www.holisticai.com/blog/europe-committee-artificial-intelligence-draft-framework-convention (25. 7. 2024).
Smith NMTC Associates. Housing Discrimination: Big Data, AI, and Algorithmic Models. SMITH NMTC ASSOCIATES, LLC. 10 October 2023. Available at: https://smithnmtc.com/housing-discrimination-big-data-ai-and-algorithmic-models/ (10. 7. 2024).
Lück-Matz, N. 2011. Framework Agreements. In: Oxford Public International Law. Available at: https://opil.ouplaw.com/display/10.1093/law:epil/9780199231690/law-9780199231690-e703#:~:text=A%20framework%20agreement%20can%20be,agreements%20between%20the%20parties%2C%20usually (16. 8. 2024).