Companies and organizations have been increasingly using mathematical models to streamline important decisions, and these new decision-making processes have been largely unaudited. Indeed, they have been assumed to be fair and objective simply by dint of their mathematical nature.
We at ORCAA call this the “authority of the inscrutable.” It’s real. But it’s an undeserved authority that is being increasingly questioned. Lawsuits are being filed against the use of dubious algorithms like personality tests used for hiring as well as recidivism risk models used for sentencing, parole, and bail.
ORCAA’s mission is two-fold. First, it is to help companies and organizations that rely on time and cost-saving algorithms to get ahead of this wave, to understand and plan for their litigation and reputation risk, and most importantly to use algorithms fairly.
The second half of ORCAA’s mission is this: to develop rigorous methodology and tools, and to set rigorous standards for the new field of algorithmic auditing.
I’ve been tooling around with the slightly infamous BISG methodology lately. It’s a simple concept which takes the last name of a person, as well as the zip code of their residence, and imputes the probabilities of that person being of various races and ethnicities using the Bayes updating rule.