A scoping review of fair machine learning techniques when using real-world data Yu Huang, Jingchuan Guo, Wei-Han Chen, Hsin-Yueh Lin, Huilin Tang, Fei Wang, Hua Xu, Jiang Bian Abstract: The integration of artificial intelligence (AI) and machine learning (ML) in healthcare to aid clinical decisions is widespread. However, concerns about AI and ML fairness and bias persist, as AI tools may have disparate impacts, potentially exacerbating health inequities. This scoping review aimed to summarize existing literature and identify gaps in tackling algorithmic bias and optimizing fairness in AI/ML models using real-world data (RWD) in healthcare domains. Methods: A thorough review of techniques for assessing and optimizing AI/ML model fairness in healthcare when using RWD was conducted. The focus was on quantification metrics for fairness, publicly accessible datasets for ML fairness research, and bias mitigation approaches. Results: Eleven papers focused on optimizing model fairness in healthcare applications. Current research on mitigating bias issues in RWD is limited in terms of disease variety and healthcare applications, as well as the accessibility of public datasets for ML fairness research. Existing studies often indicate positive outcomes when using pre-processing techniques to address algorithmic bias. There remain unresolved questions within the field that require further research, including pinpointing the root causes of bias in ML models, broadening fairness research in AI/ML, and exploring implications in healthcare settings. Conclusion: This paper provides useful reference material and insights for researchers regarding AI/ML fairness in real-world healthcare data and reveals the gaps in the field. Fair AI/ML in healthcare is a burgeoning field that requires heightened research focus to cover diverse applications and different types of RWD. The review identified 35 review articles discussing fair ML along with fairness assessment and bias mitigation approaches. Table 1 provides an overview of the array of reported algorithmic bias mitigation techniques. The search for individual studies yielded 11 distinct studies centered around the utilization of RWD for healthcare applications on the topic of fair ML. Fairness assessment involves quantifying bias generated from various sources that affect the model. Ten metrics were summarized for evaluating algorithmic fairness in healthcare. These metrics include overall accuracy equality, equality of opportunity, predictive parity, predictive equality, statistical or demographic parity, disparate impact, equalized odds, intergroup standard deviation, conjunctive accuracy improvement, and generalized entropy index. Algorithmic bias mitigation techniques were categorized into three groups: pre-processing, in-processing, and post-processing. Most studies used pre-processing methods to mitigate identified bias in clinical and biomedical applications. Pre-processing techniques included reweighing, imputation strategies, and eliminating sensitive attributes. Post-processing techniques included multicalibration and post-hoc recalibration. In-processing techniques included training and representation alteration (TARA). The review identified several health care applications in fair ML research using RWD. The existing studies have highlighted the transformative potential of fair ML in differentA scoping review of fair machine learning techniques when using real-world data Yu Huang, Jingchuan Guo, Wei-Han Chen, Hsin-Yueh Lin, Huilin Tang, Fei Wang, Hua Xu, Jiang Bian Abstract: The integration of artificial intelligence (AI) and machine learning (ML) in healthcare to aid clinical decisions is widespread. However, concerns about AI and ML fairness and bias persist, as AI tools may have disparate impacts, potentially exacerbating health inequities. This scoping review aimed to summarize existing literature and identify gaps in tackling algorithmic bias and optimizing fairness in AI/ML models using real-world data (RWD) in healthcare domains. Methods: A thorough review of techniques for assessing and optimizing AI/ML model fairness in healthcare when using RWD was conducted. The focus was on quantification metrics for fairness, publicly accessible datasets for ML fairness research, and bias mitigation approaches. Results: Eleven papers focused on optimizing model fairness in healthcare applications. Current research on mitigating bias issues in RWD is limited in terms of disease variety and healthcare applications, as well as the accessibility of public datasets for ML fairness research. Existing studies often indicate positive outcomes when using pre-processing techniques to address algorithmic bias. There remain unresolved questions within the field that require further research, including pinpointing the root causes of bias in ML models, broadening fairness research in AI/ML, and exploring implications in healthcare settings. Conclusion: This paper provides useful reference material and insights for researchers regarding AI/ML fairness in real-world healthcare data and reveals the gaps in the field. Fair AI/ML in healthcare is a burgeoning field that requires heightened research focus to cover diverse applications and different types of RWD. The review identified 35 review articles discussing fair ML along with fairness assessment and bias mitigation approaches. Table 1 provides an overview of the array of reported algorithmic bias mitigation techniques. The search for individual studies yielded 11 distinct studies centered around the utilization of RWD for healthcare applications on the topic of fair ML. Fairness assessment involves quantifying bias generated from various sources that affect the model. Ten metrics were summarized for evaluating algorithmic fairness in healthcare. These metrics include overall accuracy equality, equality of opportunity, predictive parity, predictive equality, statistical or demographic parity, disparate impact, equalized odds, intergroup standard deviation, conjunctive accuracy improvement, and generalized entropy index. Algorithmic bias mitigation techniques were categorized into three groups: pre-processing, in-processing, and post-processing. Most studies used pre-processing methods to mitigate identified bias in clinical and biomedical applications. Pre-processing techniques included reweighing, imputation strategies, and eliminating sensitive attributes. Post-processing techniques included multicalibration and post-hoc recalibration. In-processing techniques included training and representation alteration (TARA). The review identified several health care applications in fair ML research using RWD. The existing studies have highlighted the transformative potential of fair ML in different