Robodebt: when the machine accuses

From 2015-2016, Australia entrusted an automated system with detecting welfare overpayments. The method, which came to be nicknamed "Robodebt," averaged the annual income reported to the tax office and spread it evenly over the whole year, as if everyone earned the same amount every fortnight. For people with irregular incomes, the calculation manufactured debts that did not exist. It was then up to recipients to prove they owed nothing.

According to published estimates, hundreds of thousands of erroneous debts were issued before the system was abandoned. At the end of 2020, a class action led to a settlement: more than A$720 million refunded and nearly A$112 million paid in compensation.

On July 7, 2023, the report of the royal commission chaired by Catherine Holmes was made public. It concluded that Robodebt was a crude and cruel mechanism, neither fair nor legal, that made many people feel like criminals. The report made 57 recommendations, including the creation of a body to oversee automated decisions.

If the system fell, it is because recipients, lawyers, journalists and activists refused to take the machine at its word. The lesson goes beyond Australia: wherever an algorithm calculates what vulnerable people owe, there must be a human able to explain, to doubt and to correct.