When a software program successfully targets people desperate enough to pay 18 percent a month, those raking in the profits think it’s working just fine.
Introduction
‘Weapons of Math Destruction’ exposes the negative effects of Big Data in various fields. These fields range from insurance to mortgages and from recruitment to social media. Cathy O’Neil defines a Weapon of Math Destruction (WMD) as an algorithm whose workings are opaque, that operates at scale and which causes significant damage. Some examples of these WMDs are college rankings, insurance premiums and recruitment.
Human beings make choices to create WMDs. Human prejudice and biases become part of the models, even if made with the best possible intentions. As they are models of the real world, they are simplifications of it. Models are based on past data and on the assumption that patterns will repeat.
Since models can grow exponentially, the damage caused scales equally. At a group level, the predictions may be correct, but at an individual level, they can be devastating. Since these WMDs are affecting us more and more, it behooves us to be more cognizant of their impact and call for more stringent standards in their usage.
Cathy O’Neil is an erstwhile academic in mathematics. She has worked on Wall Street and technology firms. She worked in DE Shaw during the financial crisis where she observed the financial impact of the Wall Street algorithms on people. She then worked in a technology firm where she understood the pervasiveness of WMDs on day-to-day life. She evangelizes this topic by blogging. She now calls for a more ethical approach to the design and implementation of the algorithms.
Opinion
‘Weapons of Math Destruction’ was eye-opening. I have read about the people getting caught in the ‘system’. I realized the mammoth impact these WMDs have on people only reading this book. Cathy O’Neill has used many real life examples of affected people, to drive home the impact.The poor, as always, bear the brunt of it. The rich are not immune either. Cathy O’Neil warns us of a dystopia where algorithms run everything. WMDs talk to each other. Your credit score, insurance premium, job opportunities – these are driven by these algorithms and affect each other. How will you fight for justice when the whole system is loaded against you?
Since these algorithms designed for efficiency, their success is measured by the money generated or saved. They do not take welfare or justice or morality into account. The question is how to make “ethical” algorithms.
The author was prescient in her chapter on Facebook and how it could influence politics. I found this chapter eerily similar to what has happened with Trump’s election.
It is time, those of us who work in the IT industry, think about the implications of what we are working on and not get seduced by the thought of Big Data. There is a pledge that Cathy O’Neill has included in her book (which was created by two mathematicians after the 2007 financial crisis) which can be a good starting point for us.
I will remember that I didn’t make the world, and it doesn’t satisfy my equations.
Though I will use models boldly to estimate value, I will not be overly impressed by mathematics.
I will never sacrifice reality for elegance without explaining why I have done so.
Nor will I give the people who use my model false comfort about its accuracy. Instead, I will make explicit its assumptions and oversights.
I understand that my work may have enormous effects on society and the economy, many of them beyond my comprehension.
I recommend ‘Weapons of Math Destruction’ for all. People interested in or working in Big Data applications should definitely read this book.
Comments