Are we training racist ML systems, or are they training us to be racist?

Luciano Floridi brought up that information as a resource may not always be under the model of “the more the better”. I wholeheartedly agree with his point on how informational deficiency can be for a good cause. Under the guise of Rawl’s veil of ignorance, we can all benefit from discarding age-old biases that may be inaccurate in the modern age.

By US law, credit and loan decisions cannot discriminate on the basis of race and this is an overt approach to shielding races from data analysis for loan processing. However, financial institutions still go around this informational deficiency by utilizing other factors to estimate their clientele’s race which would then affect their overall decision. There is some level of irony in the situation as the few blacks in mostly white neighborhoods are be affected in terms of their “perceived credit” and vice versa.

This shows that modern technology has evolved to the point whereby veils of ignorance are not sufficient because of the Bayesian models of probability that these algorithms use. In this regard, Floridi’s suggestion may be too idealistic because of a multitude of factors that can be used to trace one’s race. As such, we may need to take a more radical stand with regards to how we move forward with machine learning systems that process our data.

Philip Brey championed that the embedded values approach holds that computer systems are not morally neutral. Thus, we should identify tendencies in them to promote equality of treatment across all races.

I would suggest that these institutions run backtests on alternative sources of data while removing these racial factors. This mode of logging in different data is essential as it will eventually allow these institutions to change their value perspective on races.

Only when the core intentions behind these programs change can we see an actual shift in the market for lending to low-income families. Otherwise, the Pygmalion effect will just continuously affect the number of “successes” and feed into this vicious cycle.

By then, it would not be a surprise that the most advanced machine learning system becomes more and more racist due to a lack of racially diverse data. This would, in turn, encourage more racist values as “successful” in this particular sector. Information deficiency is no longer sufficient. If we want change, we need to aim for diverse data sets.