INTRODUCTION: Generative Artificial Intelligence (GenAI) models are rapidly transitioning from prototype tools to active participants in clinical workflows. Despite their promise in documentation, triage, and diagnostic reasoning, concerns regarding hallucinations, biased outputs, and unreliable reasoning remain a major barrier to safe clinical adoption. This systematic review synthesizes the current evidence on safety risks associated with GenAI in medicine and evaluates the effectiveness of emerging mitigation strategies.

METHODS: A systematic search of PubMed, Scopus, Web of Science, and IEEE Xplore was conducted for studies published between January 2015 and October 2025. Eligible studies empirically evaluated the clinical use, safety, or mitigation strategies for hallucinations or biased outputs in GenAI models. Two independent reviewers screened articles, extracted data, and assessed methodological quality using the PRISMA guidelines. Mitigation strategies were categorized into model-level, workflow-level, and human-in-the-loop interventions.

RESULTS: Across recent evaluations of AI models in healthcare, hallucination and omission rates vary widely by task. In one large clinical summarization study involving 12,999 clinician-annotated sentences, AI systems produced hallucinations in 1.47% of sentences and omissions in 3.45%, with structured prompting reducing major errors substantially. Other evaluations of clinical case vignettes have reported hallucination rates ranging from 50% to over 80%, with prompt-based mitigation lowering rates from 66% to 44%. Bias has also been documented across domains; for example, approximately 40% of radiology-reporting studies identified hallucinations or misdiagnosis events, and broader reviews note performance disparities linked to race and gender. Effective mitigation strategies across studies include grounding outputs in verified clinical sources, retrieval-augmented generation, structured prompts, and explicit uncertainty expression review to prevent error propagation in clinical documentation and decision support.

CONCLUSIONS: The current evidence demonstrates that hallucinations and biased outputs remain substantial risks in GenAI-supported clinical practice, but several emerging mitigation approaches show promise for improving reliability. Standardized evaluation frameworks and transparent reporting of model uncertainty are needed to support safe integration of GenAI into healthcare.