Researchers from law, political science, psychology, philosophy and computer science gathered to exchange definitions during a recent workshop “Mathematical Quantifications and other models on the quality and fairness of (automatic) decision making” held at TUK in March.

It’s unusual to have so many disciplines in one room learning each other’s jargon, and it is proving quite a challenge to find common ground.

“It is not clear if we can find a connection, whether we can relate what good and fair judgement in court is to what we do in mathematics and computer science,” said Professor Katharina Zweig, who heads the Algorithm Accountability Lab at TUK.

Computer algorithms process datasets, finding trends and patterns, and then models apply those established relationships to new data. For example, identifying who in a population are likely to buy a certain product. While it seems relatively straightforward, there are many ways bias can be introduced, including how computer scientists design the algorithm based on the goal or thresholds, and how they weight different criteria or factors. The datasets the algorithms are initially trained on can also perpetuate existing imbalances. 

“Most of the algorithms have little knobs and levers and you have to decide which levers are on or off,” Zweig said.

This might not cause great concern for shopping and advertising, but algorithmic decision making systems (ADMs) are being introduced to spheres where the stakes are much higher. For example, governments in Poland and Austria are using ADMs to decide how much financial assistance to give unemployed citizens. The U.S. is trying an ADM system to evaluate if criminals eligible for parole are likely to commit another crime. 

As algorithmic decision making systems (ADMs) spread, Zweig and colleagues are seeking to clarify what makes a fair ADM system, and investigate if ADMs have any place in criminal justice systems. The multi-year study, funded by the German Federal Ministry of Education and Research (BMBF), is highly interdisciplinary.

Zweig had anticipated establishing a common definition of good and fair decisions would be relatively clear-cut — taking a global view and applying it to algorithmic design. As with most things, it turned out to be much more complex.

The key difference, it appears, is that computer scientists can only evaluate the fairness of their algorithm based on the final output, while other fields judge decisions based on the process, less so on the actual outcome. 

For example, government decisions are often accepted as legitimate, even if one disagrees with it, if it went through the democratic process. Citizens who disagree can provide feedback by protesting and voting for different lawmakers in the next election.

“The real value of a decision lies in the process itself,” said Professor Georg Wenzelburger, a political scientist at TUK.

The law is also very focused on fair processes and procedures, with clearly defined structures and options for making decisions. Judges strive to make fair decisions, but if an error is made, the system is set up for an appeal.   

In psychology, decisions are extremely context dependent. Evaluating if someone made a good decision depends in large part on the objective, making systematic evaluation difficult.

“What is a good decision? For us, it is highly subjective,” said Professor Anja Achtziger, a psychologist at Zeppelin University. “The starting point is ‘what do you want to achieve?’"

Fairness is even more subjective in psychology, defined as what individuals perceive as fair.

The philosophy of utilitarianism, defining the best decision as one that maximizes benefits for the greatest number of people, might be the closest match for computer algorithms, but there was still much debate if that is an appropriate analogy. 

The team will continue to grapple with this subject to develop a unified theory of fairness and quality measures for ADMs. Zweig emphasized that it important to start here, so that algorithms can be fairly evaluated, as well as offer fair decisions. 

“At the moment everyone is fearful of ADMs, but it is just based on feeling,” Zweig said. “We are trying to introduce facts into the discussion.”