Notable Real-World Failures of Data and Machine Learning Algorithms Over the Past Decade

Insights from data and machine learning algorithms can bring invaluable benefits to organizations, but mistakes can lead to serious consequences. Below are some striking errors from the past decade that illustrate this possibility.

There was a court case involving ChatGPT. This generative AI technology has huge potential for predictive analysis, but a lawyer used it in a case and submitted a non‑existent precedent, causing the case to stall.

Machine learning algorithms can also make mistakes in diagnosing and classifying patients. Errors in labeling training and test data have caused accuracy problems, negatively impacting patient diagnoses.

Machine learning algorithms applied to the real‑estate market can also be problematic. Zillow’s algorithm errors led to inaccurate home‑price predictions, ultimately resulting in layoffs and massive inventory write‑downs for the company.

Another example is the UK public health agency’s issue when reporting COVID cases. Due to data limits in spreadsheet software, thousands of cases were not reported, making contact tracing more difficult.

In healthcare, some predictive algorithms tend to recommend white patients over Black patients for high‑risk care management, leading to racial inequality.

Additionally, some companies’ algorithms have caused problems in social media and hiring. Microsoft’s chatbot Tay posted racist tweets on Twitter, and Amazon’s hiring algorithm favored male candidates.

These examples show that while insights from data and machine learning algorithms can be priceless, errors can have severe consequences. Therefore, organizations must be especially cautious when applying these technologies, ensuring data accuracy and algorithmic neutrality.