Imperfect algorithms threaten democracy

Do we want algorithms that we can’t understand or question to influence how we get to live our lives?

Unfortunately, as Cathy O’Neil, well-known mathematician and the author of the blog mathbabe.org, notes, our lives are already affected by them, as they are actively used in all sorts of places.

This wouldn’t be a problem if those models are good, but many of them are not actually helping people, but unfairly punish certain parts of the population (e.g.poor people, black people, etc.). They are used as a form of social control, she says.

The main problem is that these algorithms are secret, opaque, and the people who are targeted by them don’t understand how they work and are discouraged from understanding and contesting them – even when they obviously don’t work as they should (i.e. help people).

In a talk she gave last year at the Personal Democracy Forum, she offered some examples of these algorithms and how difficult it is to get to know how they work. Still, she points out, it’s imperative that we do, if we want to live in a democracy.

“Part of living in a democracy is understanding the rules, and these algorithms are secret rules,” she notes.