The algorithm had four guiding principles:
- Increase # of high school students starting after 8am
- Decrease # of elementary school students dismissed after 4pm
- Accommodate the needs of special education students
- Generate transportation savings
Unprecedented opposition to the algorithm’s solution. Angry parents signed petitions and stormed the school committee. BPS dropped the solution.
- Younger students were being forced into earlier start hours.
- Politically connected families were trying to get better the start times, as opposed to equitable distribution amongst neighborhoods.
- Black/brown parents tend to have lower-wage jobs that are inflexible to schedule changes. \(\approx\) 85% would be affected.
It was people who made the final call. This was a fundamentally human conflict, and all the computing power in the world couldn’t solve it. [David Scharfenberg]
To Gmail, Most Black Lives Matter Emails Are ‘Promotions’
The program was used to apportion home care assistance. The underlying problem is insufficient resources. The algorithm aims to divvy up what is available as equitably as possible, without falling to the subjectivity of care assessors.
The needs assessor administers an annual questionnaire, and the algorithm sorts patients into levels of need, with each level affording a standard number of hours of care.
However, there were flaws, e.g. failing to factor in cerebral palsy or diabetes (although Fries' theoretical considered these, the 3rd party software was not updated with the developments); double amputees being marked as mobile because of wheelchairs, etc. The algorithm was also unstable for people at the margins.
Arkansas' in-house system was opaque, and in court proceedings, the data was found to be deeply flawed and mostly discarded. Its decisions were inexplainable even to people handling appeals. Its abrupt introduction led to undesirable drastic change. The system was discontinued.
Computers Can Solve Your Problem. You May Not Like the Answer. What happened when Boston Public Schools tried for equity with an algorithm. David Scharfenberg. The Boston Globe. https://apps.bostonglobe.com/ideas/graphics/2018/09/equity-machine/ . Sep 21, 2018.
To Gmail, Most Black Lives Matter Emails Are 'Promotions'. Adrianne Jeffries; Leon Yin. The Markup. https://themarkup.org/google-the-giant/2020/07/02/to-gmail-black-lives-matter-emails-are-promotions . Jul 2, 2020.
What Happened When a 'Wildly Irrational' Algorithm Made Crucial Healthcare Decisions. Erin McCormick. https://www.theguardian.com/us-news/2021/jul/02/algorithm-crucial-healthcare-decisions . Jul 2, 2021. Accessed Jul 4, 2021.
Brant E. Fries, PhD. Brant E. Fries. https://sph.umich.edu/faculty-profiles/fries-brant.html . https://scholar.google.com/citations?hl=en&user=JGsLs_AAAAAJ&view_op=list_works . Accessed Jul 4, 2021.
A Healthcare Algorithm Started Cutting Care, and No One Knew Why. Colin Lecher. https://www.theverge.com/2018/3/21/17144260/healthcare-medicaid-algorithm-arkansas-cerebral-palsy . Mar 21, 2018. Accessed Jul 4, 2021.