Ethical Implications of AI in Health Insurance: Transparency and Regulation

This piece originated as academic coursework at UW-Madison and is republished here with light edits for the web.

Many of the largest health insurance companies in the U.S., including UnitedHealthcare, Humana, and Cigna, have recently faced lawsuits for using AI algorithms such as NH Predict and PxDx to deny health care coverage to millions of people. These companies utilize these algorithms to review claims in batches, evaluating thousands of cases in seconds. For example, Cigna used PxDx to deny more than 300,000 claims for medically necessary services, spending just 1.2 seconds on each case (Hendricks-Sturrup et al., 2024). The use of AI and algorithms by health insurance companies to deny health insurance claims and medical care is only morally permissible if these algorithms are fully transparent and regulated by state and federal governments to ensure they are fair and unbiased.

Since programs like NH Predict and PxDx are proprietary, the specific factors and data they use remain unknown. Generally, these programs suggest decisions on medical claims by analyzing millions of medical records to match patients with others who have similar diagnoses and characteristics. Variables such as age, race, and health conditions are used to predict the type of care a patient will need and for how long. However, because these algorithms are proprietary, they cannot be reviewed for biases. Consequently, there is a risk that the algorithms may not have enough data on minorities, potentially leading to discriminatory practices.

There have been countless examples of patients who clearly still needed medical services, but their insurance claims were denied. One such case involves Judith Sullivan, who was recovering from major surgery at a nursing home. She was informed that her Medicare Advantage insurance plan would no longer cover her care because she was supposedly well enough to go home, despite not being able to walk more than a few feet and still needing assistance with a colostomy bag (Jaffe, 2023). According to patient advocates, these algorithms fail to account for the patient’s individual circumstances. As David Lipschutz, associate director of the Center for Medicare Advocacy, noted, “While the firms say [the algorithm] is suggestive, it ends up being a hard-and-fast rule that the plan or the care management firms really try to follow. There’s no deviation from it, no accounting for changes in condition, no accounting for situations in which a person could use more care” (Jaffe, 2023). This reliance on algorithms, while excluding human judgment, leads to the denial of care that doctors have deemed necessary.

One policy proposal that could improve outcomes for patients is requiring AI technologies such as PxDx and NH Predict used by insurance companies to be fully transparent. Currently, there is little knowledge about how these algorithms were built and what data they use to make suggestions on claims. Transparency would allow these algorithms to be peer-reviewed to identify potential biases and discrimination. Peer review would also enable regulators and patients to monitor the performance of these algorithms and propose improvements, such as providing better or more comprehensive data for reviewing claims. Since many claim denials are appealed, transparency would also improve the appeals process, giving patients, caregivers, and insurers better information and creating a system perceived as fair and trustworthy.

Having full transparency could also open other opportunities. Currently, one area of waste in the healthcare insurance industry is dealing with denied claims and their appeals. It is estimated that one out of every seven claims is rejected in the U.S., leading to a loss of approximately $262 billion annually for hospitals (Johnson et al., 2023). This financial loss is accompanied by substantial administrative burdens, including paperwork and man-hours spent on claim denials and appeals. Administrative tasks like these significantly contribute to the high cost of healthcare in the U.S. For instance, Chris Comfort, the chief operating officer of Calvary Hospital, stated, “We take patients who are going to die of their diseases within a three-month period of time, and we force them into a denial and appeals process that lasts up to 2.5 years” (Hendricks-Sturrup et al., 2024). Dealing with denied claims also causes significant stress for patients, as they are uncertain if their care will be covered by insurance or if they will end up with large medical bills. This stress can negatively impact their recovery process.

In the article “Responsible Artificial Intelligence in Healthcare: Predicting and Preventing Insurance Claim Denials for Economic and Social Wellbeing,” the authors propose a potential solution to this problem. They argue that by using AI algorithms, it is possible to predict when claims may be denied before they are submitted to insurers. This process could improve profitability for both hospitals and insurers while also supporting patient well-being (Johnson et al., 2023). The study developed an AI model that predicted whether a claim would be denied with an accuracy rate of 83.5%, primarily addressing issues such as billing and coding errors or the medical necessity of the claim. If these systems were fully transparent, AI models could potentially interact, achieving even higher accuracy rates and providing more specific reasons for claim denials. Transparency could lead to numerous possibilities for improving the administrative process, allowing resources to be redirected towards actual healthcare.

While AI and algorithms can be powerful tools that provide efficiencies, it is crucial to maintain a human element in the decision-making process. Each individual’s case is unique, and human review is essential to avoid mistakes and minimize biases and discrimination. Making decisions in batches, with only seconds spent on each case, can lead to significant errors and perpetuate biases.

From a consequentialist perspective, establishing the proper regulatory environment and requiring more transparency is the morally correct approach because it would likely produce better outcomes. These policies will likely result in better outcomes for patients by reducing the stress associated with uncertainty about insurance coverage, allowing them to focus more on their recovery. Additionally, hospitals may recover some of the billions lost annually due to denied claims and achieve greater efficiency in administrative costs, enabling them to allocate more resources toward improving healthcare outcomes.

From a deontological perspective, transparency and a proper regulatory framework are also morally justified because the action of providing better healthcare to individuals is the right thing to do. Denying individuals necessary healthcare is inherently immoral, especially when doctors recommend the care. Ensuring transparency and regulation can help more people access the healthcare they need, provided by the insurance they already pay for, is the right thing to do.

While researching this topic, it was challenging to find arguments against transparency and regulation. Many of the health insurance companies involved in lawsuits regarding these algorithms have not commented on their use. Those that have generally provide generic statements, arguing that criticisms are biased or incomplete. They claim that requiring manual reviews of claim rejections would necessitate hiring many more medical directors, leading to inefficient administrative expenses (Johnson et al., 2023).

Another argument against transparency and regulation is that it conflicts with free-market principles, which often make the most economic sense. These are private companies, and they have the right to proprietary information that allows them to operate competitively. However, healthcare is a unique industry, heavily influenced by government involvement, particularly with Medicare payments. Because public money is at stake, transparency is essential to ensure fairness, non-discrimination, privacy, and the prevention of fraud. Furthermore, there is limited competition in the healthcare insurance market, so these companies require more regulation and oversight than a typical market would. One could also argue that healthcare is more of a right or public good than a commodity that should be in a for-profit marketplace, further justifying the need for full transparency and regulation in the health insurance industry.

Implementing full transparency and regulation over the use of algorithms in the health insurance industry is the morally correct course of action. By ensuring transparency, we could see improvements in healthcare outcomes by allowing patients to focus on recovery rather than worrying about potential claim denials. Additionally, peer review and open-source improvements to AI algorithms could promote fairness and non-discrimination. At the same time, administrative efficiencies could be gained on both the provider and insurer sides, reducing costs and allowing more resources to be devoted to better healthcare.

Bibliography

1. Hendricks-Sturrup, R., Vandigo, J., Silcox, C., & Oehrlein, E. M. (2024). Best Practices For AI In Health Insurance Claims Adjudication And Decision-Making. Health Affairs. https://www.healthaffairs.org/content/forefront/best-practices-ai-health-insurance-claims-adjudication-and-decision-making

2. Jaffe, S. (2023). Feds rein in use of predictive software that limits care for Medicare Advantage patients. KFF Health News. https://medicalxpress.com/news/2023-10-feds-rein-software-limits-medicare.html

3. Johnson, M., Albizri, A., & Harfouche, A. (2023). Responsible Artificial Intelligence in Healthcare: Predicting and Preventing Insurance Claim Denials for Economic and Social Wellbeing. Information Systems Frontiers, 25, 2179-2195. https://doi.org/10.1007/s10796-021-10137-5

4. Angwin, J., Larson, J., Mattu, S., & Kirchner, L. (2016). Cigna’s AI Algorithm Rejects Claims Without Review. ProPublica. https://www.propublica.org/article/cigna-pxdx-medical-health-insurance-rejection-claims