Improved Prediction of Chemical Carcinogenicity Based on Hybrid In Silico/In Vitro Methodology

Introduction: In silico methods such as quantitative structure–activity relationship (QSAR) modeling and read-across are widely used for rapid and cost-effective carcinogenicity assessment. However, their predictive reliability is often restricted by applicability-domain (AD) limitations, particularly for structurally diverse compounds.

Methods: We developed a hybrid framework integrating multi-source in silico and in vitro-derived data to improve human carcinogenicity prediction and reduce AD limitations. A graph neural network (GNN) was pretrained on mouse carcinogenicity data using multitask learning with rat carcinogenicity as an auxiliary task. The model was enriched with graph-level features derived from supporting bioassay endpoints and compound-induced gene expression embeddings. It was then fine-tuned on a human carcinogenicity dataset while excluding an independent test set of 166 compounds absent from the animal datasets. The test set included 28 carcinogens and 138 non-carcinogens. An ensemble combining GNNs with descriptor-based gradient boosting models was constructed. Applicability-domain assessment was performed through confidence estimation, where prediction probabilities between 0.4 and 0.6 were considered outside the AD.

Results: The baseline mouse-trained GNN showed limited transferability to human data, achieving a Matthews correlation coefficient (MCC) of 0.19, with an average of 49.3 compounds outside the AD. The integrated and fine-tuned ensemble improved the MCC to 0.50 while reducing out-of-domain compounds to an average of 6.8, substantially increasing chemical-space coverage. For comparison, extrapolation of conventional 2-year rodent carcinogenicity assays to humans on the same dataset achieved an MCC of 0.62 and 82% accuracy.

Conclusions: Our hybrid in silico/in vitro method improved prediction accuracy and significantly reduced applicability-domain limitations. Combining biologically relevant bioassay and gene expression features allows for broader and more cost-effective carcinogenicity assessments while decreasing dependence on animal testing, highlighting its promise as a New Approach Methodology.

Funding: This work was supported by the Science Committee of RA (#23LCG-1F002).