A few months ago, Om wrote a great post about society’s move toward a quantified world and the effects that can have on people. His post highlighted the plight of Uber drivers, AirBnB hosts and anyone else whose livelihoods — and very reputations — hang in the balance based on metrics such as user reviews. It’s an important consideration, perhaps no more so than when were talking about real life-and-death situations.
IEEE Spectrum featured a thoughtful post on Monday from Daniel Castro, a senior analyst with the Information Technology and Innovation Foundation, about the difficult decisions that come along with using predictive analytics to decide the order of organ transplants for children. It’s the kind of post that makes you think even harder about just how powerful data can be in a society that’s driven by it. In fields like medicine, people who control the data — or at least the means to understand it — aren’t just agents in a system of natural selection; they’re kind of playing god.
Castro highlighted the case of a 10-year-old girl whose parents had to file a federal lawsuit in order to get her a lung transplant. The reason: under current guidelines, age is the data point that matters most. If you’re over 12, lung donations happen based on which prospective donees have the highest statistical likelihood of a successful transplant. If you’re under 12, it’s pretty much first come, first served — provided there are lungs available from donors under 12 years old.
Somewhere, some time ago, someone decided this was the way lung donations would work, and there’s no doubt many lives have been affected by this decision. Whether or not that’s a fair system is up for debate, but it might look a lot less fair when you consider that most lung donations come from older donors. According to Castro, that means a kid under 12 isn’t too likely to get a replacement lung at all.
Castro is adamant that big data is the answer to figuring out a fairer, less error-prone model for assessing priority, but he also recognizes that relying too heavily on any one factor is also asking for ethical trouble:
“Better prediction means that we will get the answer wrong less frequently, but it does not mean we will ask the right question. For example, should we allocate organs to those who have waited the longest, those who are most likely to die while waiting for a transplant, or those who have the highest chance of survival? And what happens if the rules systematically exclude certain individuals? Underserved populations who receive inadequate information about transplants may not be added to the waitlist until long after their diagnosis.
I’m not envious of people who have to make such decisions, and I assume most people who work with regular business or web data aren’t either.
The advent of big data has made it easy enough to quantify aspects of our lives, build models around them and assign scores, but I doubt it has made ethics easier. The doctor telling a patient she won’t be getting a transplant, or a manager having to fire an employee, can always say “Sorry, it’s the numbers.” That must be hard, but someone upstream had to play god and decide which numbers matter.
That might have been even harder.
Feature image courtesy of Shutterstock user Monkey Business Images.