Prostate cancer (PCa) is the second most common cancer in men and has a significant health and social burden, necessitating advances in early detection, prognosis, and treatment strategies. Improvement in medical imaging has significantly impacted early PCa detection, characterization, and treatment planning. However, with an increasing number of patients with PCa and comparatively fewer PCa imaging experts, interpreting large numbers of imaging data is burdensome, time-consuming, and prone to variability among experts. With the revolutionary advances of artificial intelligence (AI) in medical imaging, image interpretation tasks are becoming easier and exhibit the potential to reduce the workload on physicians. Generative AI (GenAI) is a recently popular sub-domain of AI that creates new data instances, often to resemble patterns and characteristics of the real data. This new field of AI has shown significant potential for generating synthetic medical images with diverse and clinically relevant information. In this narrative review, we discuss the basic concepts of GenAI and cover the recent application of GenAI in the PCa imaging domain. This review will help the readers understand where the PCa research community stands in terms of various medical image applications like generating multi-modal synthetic images, image quality improvement, PCa detection, classification, and digital pathology image generation. We also address the current safety concerns, limitations, and challenges of GenAI for technical and clinical adaptation, as well as the limitations of current literature, potential solutions, and future directions with GenAI for the PCa community.
Prostate cancer (PCa) is the second most common cancer in men. According to the latest statistics, the American Cancer Society estimates that there will be 313,780 new PCa cases with 35,770 associated deaths in the United States in 2025.1 While PCa often progresses slowly, it remains a significant health burden, particularly in patients with aggressive disease which can result in metastasis and poor survival outcomes. The economic and social impact of PCa is substantial, necessitating advances in early detection, prognosis, and treatment strategies.2
Early detection of PCa relies primarily on prostate-specific antigen (PSA) testing, prostate biopsy, imaging techniques such as multiparametric magnetic resonance image (mpMRI) of the prostate, and positron emission tomography (PET).3 A Cochrane review and meta-analysis of five randomized trials involving 341,342 men found no statistically significant reduction in PCa-specific mortality due to PSA screening.4 A definitive diagnosis is established through a prostate biopsy, typically guided by transrectal ultrasound (US) or mpMRI.5 However, biopsies are invasive and can miss clinically significant cancers due to sampling errors.6 Medical imaging, particularly mpMRI and PET, has significantly improved PCa detection and characterization, and computed tomography (CT) is used for treatment planning.7 However, interpreting imaging data is complex and prone to variability among experts.8 Artificial intelligence (AI), particularly deep learning and machine learning algorithms, have emerged as a transformative tools with the potential to enhance the accuracy and efficiency of PCa detection and prognostic predictions.9-11 While substantial advancements have been achieved by deep learning models, limited data and expert annotation are still a challenge in medical imaging for PCa diagnosis and treatment planning.
Generative AI (GenAI) is a relatively new domain of AI that can create synthetic content, such as text, images, or even medical insights, based on patterns it has learned from existing data. GenAI has helped to overcome some limitations of deep learning techniques by enabling innovative approaches for enhancing medical imaging, including data augmentation, image synthesis, and image-to-image translation.12 By utilizing learned disease representations, these models have the potential to predict the future progression of a patient’s condition, enabling doctors to foresee and prepare for possible complications.13 Although some specific models have shown promising results in medical imaging14, their application remains limited in capturing the complex heterogeneity of diseases like PCa.15 As AI becomes an increasingly important component of the medical ecosystem, it will be key for radiologists and physicians to understand the underlying principles of GenAI and how it can be used to improve sampling, disease detection, and more. In this narrative review, we explore leading frameworks and their applications in PCa imaging. We assess the existing literature leveraging GenAI for purposes such as synthetic image generation, image-to-image translation, image quality improvement, disease detection, prognosis prediction, and treatment planning for PCa.
GENERATIVE AI NETWORK ARCHITECTURES
GenAI refers to a subset of techniques in the AI space that create new data instances, often to resemble patterns and characteristics of the training data. These systems are often well-suited for tasks that involve image synthesis, augmentation, or quality enhancement, though new use cases are evolving rapidly as AI becomes more widely available.16 From a more technical perspective, these generative systems operate by learning the underlying probability distributions of a dataset and use this information to generate samples that are probabilistic but not necessarily recognizable as equivalent to any true sample used in training.17 This is particularly relevant in the medical imaging context, as AI can potentially produce synthetic images18, improve image quality19, and aid in tasks like object segmentation20 or diagnosis.21 Further, GenAI may have the potential to supply or augment data for studies, circumventing the risks of using identifiable personal information.22 As this technology is rapidly being introduced into medical research and clinical practice, it is important for radiologists and physicians to understand not only the existing applications of GenAI in PCa imaging but also the existing technical frameworks that may impact the pitfalls and biases of such tools moving forward. In Figure 1, a schematic overview is shown, highlighting the application of GenAI in PCa medical imaging. In this section, we discuss the basics of four common technical frameworks for GenAI, i.e. generative adversarial networks (GANs), variational autoencoders (VAEs), transformers, and diffusion models being applied in PCa imaging, and emerging frameworks such as hybrid models which combine existing techniques. Figure 2 shows the schematic overview of key generative frameworks and their simplified structures.
Generative adversarial networks (GANs)
GANs23 are a subset of GenAI models composed of two neural networks called a generator and a discriminator. One of the common applications of generative models, in medical imaging, is to generate synthetic data, where the generator creates synthetic data by learning from the real data, and the discriminator identifies the data as real or synthetic. In an adversarial relationship, the models provide feedback in training until the discriminator is unable to identify or struggles to identify differences between synthetic and real inputs.
GANs have demonstrated significant utility in imaging, medical imaging, and specifically PCa imaging. As discussed in more depth in Section III, key applications such as synthetic MRI generation15 lesion detection20, resolution enhancement19 , and more for PCa imaging have leveraged GANs. Despite their widespread use since their introduction in 201423, GANs are not without shortcomings. GANs face challenges such as model collapse, where the generator produces a limited variation or subset of the data in order to more easily trick the discriminator.
Variational auto-encoders (VAEs)
VAEs are another type of generative models that use probabilistic frameworks to learn patterns of data. VAEs were first introduced by Diederik P. Kingma and Welling24, and have shown significant success in various medical image applications.25 Unlike GANs, VAEs learn how to represent data or information in a compressed “latent space” and decode it back to the original space from a random sample. The key difference lies in that the VAE’s focus on reconstructing data from this latent space which adheres to a predefined distribution such as a Gaussian or normal distribution.
Using this method, VAEs are particularly adept at feature extraction and dimensionality reduction, which can make them more useful depending on the application. For example, they may be more useful in patient-specific applications where an image can be reconstructed from specific data’s features. On the other hand, there may be a tradeoff in terms of the accuracy of high-resolution (HR) reconstructions due to reliance on reconstruction loss. Overall. VAEs have seen qualitatively fewer applications in PCa compared to GANs, although they do hold promise in areas such as unsupervised anomaly detection and multi-model data integration.26 Future studies could explore the potential for synthesizing prostate imaging data on a patient-specific level. For example, one study circles this approach in the context of PCa-specific foundation model using prostate mpMRI for csPCa detection.26
Transformers
Transformers, a network structure originally designed for natural language processing (NLP) tasks by researchers at Google27, have recently been adapted for GenAI in medical imaging. These architectures are particularly effective at identifying which portions of an image to pay “attention” to, making them useful for complex spatial patterns in medical imaging. Unlike VAEs and GANs, transformers have the advantage of being able to generally handle larger datasets, allowing for the generation of more diverse outputs. Due to their ability to parallelize, transformers can provide better scalability while avoiding some of the pitfalls of training GANs.
In PCa transformers have been applied more commonly outside of the GenAI domain. Several models have been developed for segmentation28, detection29, or classification30 of prostate lesions in both radiologic and pathologic imaging. However, there are examples of transformers being used for generating captions or reports from images, perhaps due to their NLP origins.31,32 There may be a role down the road for transformers to play in multimodal GenAI in PCa imaging, such as generating images based on a text report.
Diffusion models
Diffusion models are arguably the most up-and-coming subset of GenAI models which aim to generate synthetic data through a process that iteratively denoises random Gaussian noise introduced into data.33 These models learn to reverse a diffusion process that progressively corrupts data to greater degrees and can generate high-fidelity outputs by learning this reverse process. In medical imaging, generative diffusion models have shown promise in tasks like image reconstruction and quality enhancement and there is some belief that they generally produce higher-quality results and are more stable during training compared to GANs.33 A recent survey showed exponential growth in research on the application of diffusion models in the medical imaging domain.34 Although their applications for PCa imaging are still relatively at earlier stages, initial studies show promise for generating synthetic MRI and enhancing images by denoising.35,36 With significant progress in other medical imaging fields, it is very possible we will see increasingly popular diffusion GenAI models with similar goals in PCa imaging.
Hybrid models
More recently, cutting-edge models have begun to combine features of the models discussed in singular applications. For example, combining GANs with VAEs may mitigate the shortcomings of each approach (such as the instability of GANs and the blurriness of VAEs).37 Similarly, diffusion models can enhance reconstruction but may still benefit from the adversarial approach of GANs to stabilize generated outputs. Hybrid models are few and far between in the PCa context but are becoming increasingly common in neighboring disease spaces. Given these frameworks, each seems to provide advantages and disadvantages in producing high-fidelity generated images, and since their underlying mathematical foundations are not necessarily mutually exclusive, it is possible that hybrid models will prevail as the most useful in PCa imaging moving forward.
GENERATIVE AI FOR PCa IMAGING IN CLINICAL APPLICATIONS
GenAI applications in PCa imaging are quickly evolving as this field is in its infancy. Now that we have covered existing and emerging frameworks for GenAI in medicine, we will discuss and summarize how they have been implemented and the corresponding results, as well as where there may be room for further development.
Synthetic medical image generation
In this section, we discuss the applications of GenAI in synthesizing PCa imaging modalities. Table 1 summarizes the discussed literature in this section.
MR imaging
One of the most common use cases for GenAI in prostate imaging is to synthesize MRIs, an important source of clinical information for clinically significant PCa. GenAI models, such as GANs and diffusion models, have shown significant potential in generating high-quality synthetic MRI for various applications related to PCa.18,38 Xu et al.18 generated synthetic prostate MRI data using a single natural image GANs network from conventional T2w MRI which was further used to train a deep learning semantic segmentation model. This model aimed to segment the prostate boundary on 2D MRI slices and achieved an accuracy of 0.9988. The same work also included multi-reader studies where they asked readers to evaluate 122 images (51 conventional and 71 synthetic MRI), and nearly half (47%) of the synthetic images were mistakenly evaluated as conventional MRI. This study found that board-certified radiologists did not significantly differentiate between conventional and synthetic images in the context of the mean quality of synthetic and conventional images.
In another study, Huang et al.39 used the “Pix2Pix” algorithm.40 Another GAN-based approach, to train a model using 4 MRI sequences, i.e., T1-weighted imaging (T1w), T2w, diffusion-weighted imaging (DWI), and apparent diffusion coefficient maps (ADC), to generate synthetic contrast-enhanced MRI scans. The comparison between Synthetic and acquired contrast-enhanced images showed high similarity with a multiscale structural similarity index (MS-SSIM) ranging between 0.69-0.82 when tested on a total of 323 patients’ images from 3 different test sets. Three radiologists with 3, 6, and 10 years of experience, independently scored synthetic and acquired contrast-enhanced T2-weighted and DW MRI using PI-RADS, version 2.1, and showed excellent reader agreement of PI-RADS scores (Cohen κ: 0.96; 95% CI: 0.94, 0.98). Such results are promising in avoiding contrast-enhanced MRI in patients with contraindications for gadolinium-based contrast materials; therefore, further validation of these new GenAI techniques is quite critical.
Approaches to MRI synthesis leveraging diffusion models have also been explored. Saeed et al.41, proposed a conditional diffusion probabilistic model42 which generates synthetic multi-sequence prostate MRIs conditioned on text and image sequence to control lesion presence and sequence. A single expert clinician was able to identify synthetic ADC-T2W MRI with an accuracy of 59.4% (where random chance would be 50%). Further, an expert reader with 4 years of experience in reading urological MRI was able to identify lesions for real images on average 60.4% of the time, compared to a slight improvement of 62.6% using the synthesized images. For lesion detection, the average accuracy for real ADC and T2W images is 66.65%, whereas on synthetic ADC and T2W, no difference was observed for T2W-DWI (56.33%) between real and synthetic pairs. Authors also trained a lesion detection AI model using AlexNet43 which achieved an accuracy of 76.2% while training on both real and synthetic sequences in comparison to 70.4% for models trained on the real sequences.
Another interesting work by Salmanpour et al.44 used the “Pix2Pix” algorithm40 to generate MRI from US images. The authors investigated the radiomics features (RF) via the Spearman’s correlation coefficient between synthetic and real MRIs and 76 out of a total of 186 RS were identified in the synthetic images, whereas half of the features were lost. The synthetic images were reviewed by 7 experienced physicians. All of the experts could distinguish synthetic images from the original image despite an average structural similarity index measure (SSIM) >0.85 for synthetic images and found that synthetic images lack good quality anatomical information, have artifacts, resolution, and contrast discrepancies, difficult to diagnose PCa using synthetic images. This raises questions on the use of the parameter SSIM as a measure of evaluation of such synthetic data.
These studies illustrate the potential for synthesized images to improve prostate segmentation and lesion detection tasks as well as the importance of readers to evaluate GenAI in oncologic imaging. The variations across these three studies demonstrate the existing heterogeneity in model frameworks and evaluation metrics in such studies. In Saeed et al.44 and Xu et al.18, readers were asked to evaluate whether they thought MRIs were synthetic or conventional, whereas, in Huang et al.39, readers were asked to assign PI-RADS scores based on synthetic or conventional MRIs. Both of these tasks may be valuable and beg the question: what is the best way to evaluate synthetic data in the context of PCa? As GenAI in prostate imaging evolves, it will be important to pay attention to how these evaluations evolve to compare the success of models across different frameworks (diffusion vs. GAN) and patient populations. In their current state, it is quite difficult to evaluate model differences.
Diffusion-weighted MR imaging
Another application of GenAI is generating synthesized ADC maps and DWI from T2w images. Yang et al.45 proposed a multi-modal sequential GAN model trained with semi-supervised learning for synthesizing spatially aligned pairs of ADC and T2w images. This study used two datasets, one was collected locally, and another one was the public PROSTATEx dataset46, consisting of 360 patients’ data, including 226 patients with non-clinically significant PCa (non-csPCa), and 134 patients with clinically significant PCa (csPCa). They reported inception score (IS), Fréchet inception distance (FID)47 parameters of 1.95 ± 0.11 and 263.78 ± 8.21, 2.61 ± 0.24, 234.89 ± 9.62 for synthetic ADC and T2w MRI, respectively, outperforming other state of the art models. They also trained a csPCa and non-csPCa classifier model incorporating the generated synthetic images. The classifier achieved an accuracy of 93.00 ± 0.45 % in comparison to the classifier model trained on real data, only with an accuracy of 93.40 ± 0.40%. This study included 3 radiologists with 1, 8, and 23 years of expertise to blindly evaluate the quality of these synthetic images mixed with real images. They reported that the synthetic images generated by their technique are 5.6% ~ to 50.2% lower than those of real images, indicating that there remain differences between synthetic images and real images to radiologists.
Wang et al.48 proposed a technique to generate synthetic high-quality ADC-T2W MRI from mpMRI for csPCa with the help of semi-supervised and adversarial learning techniques. They used paired ADC-T2w prostate images for the supervised part of the training process with pixel-wise reconstruction loss minimization and unpaired pairs for the unpaired ADC-T2w images with random latent vectors for the unsupervised training process to teach the marginal distributions of the real images via W-distance minimization in an unsupervised manner and the different visual features of csPCa by teaching maximum (max) auxiliary distance (AD) between of csPCa in unsupervised training. The authors also proposed a new technique named StitchLayer to boost GAN performance to synthesize images with a greater size. This study used two datasets, one locally collected dataset49 and PROSTATEx dataset.46 The proposed semi-supervised synthesis method is compared with two state-of-the-art methods CoGAN50 and a method by Costa et al.51 They showed their approach with AD maximization, outperforming other models with the lowest FID of 178.2 ± 3.7, and higher IS52 2.24 ± 0.03 for ADC and 2.10 ± 0.05 for T2W and higher slice-level classification accuracy (SCA) of 94.4 ± 0.5. They also trained a weakly-supervised csPCa localizer model from the literature49 using a combination of synthetic and real images and compared it with a model trained using only real images and they reported a performance boost of the model with synthetic images with a 1.0 non-lesion localization fraction (NLF)53 sensitivity of 0.80 in compare to the NLF sensitivity of 0.76 for a model trained with real images. They conducted a multi-reader study with 3 radiologists with clinical reading experience of 1 year, 8, and 23 years of experience and showed that the SCA between synthetic and real images is very close to the real values (in between 1-7%) and even slightly better for the most experienced radiologist, however, there is a difference of 4-14% in sensitivity when detecting non-csPCa from the synthetic images.
Conventionally, paired ADC-T2w MRIs are used to train GenAI models.48 However, unpaired ADC-T2w images provide more possibilities for the generative models to learn and create diverse physiological characteristics and produce fewer artifacts. A recent study by Ozyoruk et al.54 proposed an AI-ADC model, a GAN-based model to synthetically generate ADC maps from unpaired prostate T2W MR images and compared them with state-of-the-art models like CycleGAN55, contrastive unpaired translation (CUT)56, and StyTr2.57 The proposed model outperformed other models with a higher mean SSIM of 0.863 and FID of 31.992 compared to SSIM of 0.855, 0.797, 0.824 and FID of 43.458, 179.983, 58.784 for CycleGAN, CUT, and StyTr2, respectively, indicating its superior performance in generating ADC maps with less style and artifact issues in comparison to the other models and better visibility of the hypointense cancer suspicious lesions. However, these generated ADC maps are sensitive to the prostate-boundary segmentation on T2w MRI. However, this challenge can be easily overcome by integrating a prostate-segmentation model in the pipeline before generating the synthetic ADC maps.
Xiaodan et al.58 proposed ProstateGAN, which they used to synthesize focal prostate DWI of the corresponding Gleason grade (GG). They combined the concept behind deep convolutional GAN (DCGAN)59 and conditional generative adversarial nework (CGAN)60 to overcome the limitation of GAN to not be able to use the annotated images fully. They provided the diffusion image with paired GG as an embedding with random noise as input to create synthetic image diffusion images that exhibit characteristics indicative of PCa. They showed that their synthetic images showcase similar PCa features as observed in the real DWI of PCa with GG of six or higher, i.e., the abnormally darkened regions. However, they did not report how their model performed in low GG pattern generation. This is an interesting approach to teaching the GenAI model, image patterns with embedded GG scores, to better generate a wide spectrum of DWI with PCa of various GG. These generated synthetic images with indicative PCa can be used as an augmentation technique for deep-learning model development.
Hu et al.61 worked on synthesizing high-b-value (b = 1500 sec/mm2) DWI of the prostate using GAN from acquired standard-b-values of 800 sec/mm2 and 1,000 sec/mm2. This study used a multicenter cohort of 395 and only 96 of the patients were used to develop the GAN model, further optimized by using denoising and edge-enhancement techniques. They compared the synthetic DWI by this model and the cycle GAN model and found GAN-generated DWI with a superior signal-to-noise ratio of (0.819 vs. 0.793), structural similarity (0.901 vs. 0.873), and feature similarity (27.40 vs. 24.67). The authors conducted a muti-reader (2 readers) study to evaluate the diagnostic quality of the synthetic DWI with acquired and traditionally calculated DWI with the high-b value of 1500 sec/mm2 using a five-point Likert scale for imaging features, such as the suppression of normal or benign prostate tissue, anatomic distortion, artifacts, and improved overall image quality. They also conducted a multi-reader blinded study (two 3rd-year radiologic residents and two radiologists with more than 20 years of experience with prostate MRI) on PCa detection assessments using the axial T2-weighted MRI with ADC maps and DWI sets at four different time points with an interval of 2 weeks. The mean area under the curve (AUC) score was calculated from the five-point Likert score of the final indication of the reader’s confidence in the diagnosis of PCa. The study showed that even though the AUC for the resident radiologists was better for synthetic DWI from the GAN model compared to the other two types (acquired and calculated DWI), there is no significant difference in the AUC for the experienced radiologists. They also showed that the ADC value derived from synthetic DWI had a higher AUC than the other two types (acquired and calculated DWI).
Based on the studies by Yang et al.45 and Wang et al.48, it is observed that the generated ADC map enhances the performance of classifier models for the detection of csPCa and non-csPCa from ADC-T2w MR images. As shown by Wang et al.48, GenAI can generate synthetic data with diverse clinically meaningful and distinguishable csPCa-relevant features, and correct paired relationships between synthetic ADC and T2w, which is quite promising. Use of unpaired ADC-T2w MR images to generate synthetic images as shown by Wang et al48 and Ozyoruk et al.54 shows the potential to overcome the paired data limitation and generate more diversity in the synthetic images. The generated DWI and ADC images have the potential to be used as a means of augmentation for PCa detection and classification tasks using deep learning models. The current literature evidence indicates that GenAI has the potential to generate ADC maps and DWI images with meaningful PCa information. While such novel GenAI approaches are critical in creating good quality functional MRI data for PCa evaluation, their net benefit on the performance of radiologists is not clear yet. More multi-reader studies are required to validate the quality of these images which will answer if these synthetic ADC and DWI images have the potential to be incorporated into clinical diagnosis or its just suitable for computer visions and not for human readers, a question we leave to the PCa community to be answered.
CT imaging
Synthetic contrast-enhanced CT image generation from MR images is another application of GenAI in the medical imaging domain.62 An example of synthetic contrast-enhanced CT generation from MR images is shown in Figure 3a. Although CT image synthesis has been a subject of interest in recent medical literature, CT images are not particularly useful for diagnosing or evaluating localized PCa. However, they are still used for treatment planning for radiation therapy to assess anatomy and direct radiation therapy to the correct areas. Therefore, several models have explored the use of synthetic CT images for treatment planning in radiotherapy. To evaluate the clinical evaluation of the generated images by these models, a dose-volume histogram (DVH)63 for planning target volume (PTV) and organs-at-risk (OARs), dose difference (DD), and gamma passing rate (GPR) with 1-3% DD and 1-3 mm distance-to-agreement (DTA)64 criteria at a dose threshold are used. On the other hand, for technical evaluation of the image quality, mean absolute error (MAE), mean error, and dice similarity coefficient (DSC)65, FID47 were used.
Liu et al.66, for example, used CycleGAN55 to generate pelvic synthetic CT (sCT) from MR images for prostate proton beam therapy treatment planning from 17 patients with co-registered CT and MR pairs using the leave-one-out cross-validation technique. The model achieved a DSC of 0.85 ± 0.05 for the bone mask and an average MAE of 51.32 ± 16.91 Hounsfield unit (HU) for the body outline in comparison with the other two state-of-the-art architecture deep convolutional neural network (DSC = 0.81 ± 0.06, MAE = 58.98 ± 18.64) and GAN (DSC = 0.81 ± 0.06, MAE = 74.66 ± 19.96). Less than 1% relative differences in DVH for PTV, -0.07% ± 0.07%, mean values of DD, and 0.23% ± 0.08%, absolute DD was observed between sCT and original CT. However, DVH matrices showed large discrepancies for the rectum and bladder. 92.39% ± 5.97%, 97.95% ± 2.95%, and 98.97% ± 1.62% of mean GPR of 1%/1mm, 2%/2mm3%/3mm, in DD/DAT criteria with 10% dose threshold, also reported, which are comparable based on the reported pelvis proton study from literature, respectively.
Another example is using synthesized CTs focused on the segmentation of pelvic organs and structures. Luu et al.67 Introduced a framework called “CycleSeg,” designed to generate sCT images from MR images and generate segmentation of the prostate, left and right femoral head, rectum, bladder, and penile bulb on the sCT and MR images. The primary aim of this approach was to eliminate the need for CT acquisition in radiation dose calculation by converting MR images into sCT. CycleSeg leverages CycleGAN as the backbone architecture. For the secondary task of segmenting, the framework employed unsupervised domain adaptation through a pseudo-labeling strategy and feature alignment in the semantic segmentation space, utilizing both anatomical segmentation on acquired CT and MRI. CycleSeg outperformed other state-of-the-art models, including CycleGAN: Cycada68, SynSeg-Net69, SIFA70, and DDA-Net71, achieving the lowest MAE of 102.2 and FID of 13.0.
Maspero et al.72 used Pix2Pix to generate sCT images from in-phase, fat, and water MR images of the whole pelvis for dose calculation from 32 PCa patients. The study showed that it takes 5.6 and 21 seconds to generate sCT for a single patient volume using a graphics processing unit (GPU) and central processing unit, respectively, which suggested the feasibility of the use of these sCTs for an accurate MR-guided radiotherapy workflow. Another interesting aspect of this work was to fill up the air cavities in sCT images and bulk assigned (-1000 HU) as located on MR images73, to avoid inconsistent depiction of air between the MR and sCT images on the generated sCT images. They reported that this technique reduced the MAE and ME when compared to sCT generated without air-pocket inserted CT for PCa patients. Apart from PCa patients, the study also included rectum and cervix cancer patients and showed that even though the model was developed on PCa patients, it was feasible for the entire pelvis.
Another work on sCT generation for MRI-guided adaptive radiotherapy (MRgART) in PCa by Hsu et al.74 proposes a method using a CGAN60 with a multi-planar approach to generate sCT images from low-field MR images to improve MRgART. Their study involved 57 patients who received MRI-guided radiation therapy. Dosimetric accuracy was further evaluated by recalculating clinical treatment plans on the sCT images within the MRIdian treatment planning system. DVHs for PTVs and OARs, as well as dose distributions using gamma analysis, were analyzed. The dosimetric differences for all PTV and OAR metrics were, on average, less than 1%. Treatment plans demonstrated excellent agreement, with GPR of 99% for 1%/1 mm in DD/DTA criteria and 99.9% for 2%/2 mm in DD/DTA criteria. This study highlights the feasibility of using sCT images generated by a multi-planar CGAN method from 0.35T MRI TrueFISP images for MRgART in PCa treatment.
Across these four works, we see a trend suggesting that the potential for replacing acquired CTs used for treatment planning with sCTs may save time and resources with minimal compromise on accuracy. Further exploration, especially across various cohorts, is necessary before such tools can be implemented routinely in the clinic. We see that GANs continue to lead the charge in synthesizing CTs for PCa. There are limited diffusion-based or hybrid models, which could be a potential future direction.
PET imaging
PS membrane antigen (PSMA)-targeted PET/CT imaging has significantly transformed the diagnosis and prognosis of PCa and showed superior performance over traditional imaging techniques for detecting biochemical recurrence and metastasis for PCa.75 Unlike other modalities, the application of GenAI in PSMA-PET/CT imaging is still limited but has shown potential in addressing clinical challenges like improving image quality, generating synthetic data, automating lesion analysis, and mitigating artifacts, which has been broadly discussed in a recent review article by Islam et al.75 PET images suffer from an attenuation problem due to the absorption in the body and scattering of the radiation, and the attenuation correction is normally done by computing an attenuation map of the density difference of the body from CT images. GenAI can be used to synthetically generate attenuation-corrected (AC) PET images from non-AC (NAC) PET images. Ma et al.76 developed a 2D Pix2Pix GAN-based model to synthetically generate AC-PET images from NAC-PET images. They used 302 patients, paired AC and NAC 18F-DCFPyL PSMA PET-CT in their study. The generated AC-PET images showed median MAE, SSIM, and peak signal-to-noise ratio (PSNR) of 3.59%, 0.891, and 26.82, respectively. The study also reported an interclass correlation coefficient of 0.88 and 0.89 for max standard uptake value and mean standard uptake value, indicating a great correlation between the real and synthetically generated AC-PET images. Figure 3b shows an example of the real NAC-PET and synthetically generated AC-PET images using GenAI.
As PET and MR images have a significant contribution to PCa detection and prognosis, separately, combining them in the PET-MR modality helps in understanding the structural and functional soft tissue morphology from MR images with metabolic activity from PET images together. However, like PET/CT imaging, PET-MR images also face the PET image AC challenge. Pozaruk et al.77 developed a GAN-based augmentation method to synthetically generate attenuation maps by computing deformation fields from Dixon-MR and corresponding CT images, and used it to reconstruct the AC 68Ga-PSMA PET images. This study used images from 28 PCa patients. The model generated pseudo-CT attenuation maps with over 4.5% accuracy compared to the conventional MR-based technique. The PET reconstruction error was 2.2% by the GAN technique compared to 10.3% for the conventional MR-based technique. This study showed that, with an augmented training dataset, the GAN model generated more accurate attention maps for PET reconstructions.
Based on these two studies, it can be said that the application of GenAI for AC PET image generation is promising. The study by Pozaruk et al.77 uses both MR and CT images to develop its model. However, in a practical setting, corresponding CT and PET-MR are not always available. Secondly, it’s trained on a very small sample size of 28 patients. So, more investigation is required to understand the clinical implications of this approach. In conclusion, as discussed in the review article by Islam et al.75 there are other possible applications of GenAI for PET images, which need more exploration.
Digital twin concept
The digital twin is a relatively new concept in medical imaging, and researchers have started to leverage the potential of GenAI to generate an intermediate imaging modality from two different modalities containing physiological information of both modalities by the combined use of GAN and diffusion models. A recent study by Ma et al.78 used a 3D Schrödinger bridge-based diffusion model79 with GAN to autonomously translate both prostate MR and US images into an intermediate pseudo-modality. They compared the model with another existing state-of-the-art full modality translation technique called UNSB.80 The study showed that their pseudo modality has similar textures in comparison to the original image and showed a reduction of 33.67% and 15.86% in the FID, and 54.90% and 17.65% in the Kernel Inception Distance by their proposed method and UNSB method, respectively. Furthermore, the study showed significant enhancement of the downstream registration task between two modalities using a mutual information registration strategy, by exclusively employing modality-translated results to derive the warping map, along with conducting the actual warping on the original MR and US images. Even though this is an interesting concept, more research is required to evaluate the usability of such an intermediate modality in medical diagnosis.
Image quality improvement
Medical images are often affected by noise, artifacts, and reconstruction errors, resulting in poor-quality and resolution images. Another major application of GenAI in PCa diagnosis is improving the quality of medical images. Han and Huang35 proposed a discrete residual diffusion model (DR-DM) to synthetically generate HR MRI from low-resolution (LR) MRI using a super-resolution (SR) technique. The proposed DR-DM uses a vector quantized variational autoencoder approach to represent LR MRI pairs of images to create discrete image tokens. The study has used Prostate-Diagnosis81 (n = 92) and PROSTATEx46 datasets (n = 346) for model development and validation, where the model achieved SSIM in the range of 0.71 to 0.76.
Majdabadi et al.19 also used PROSTATEx46 and Prostate-Diagnosis81 datasets to develop a multi-scale gradient (MSG)-CapsGAN model82 for generating a prostate MRI SR from LR MRI. The authors used MSG as a generator to upscale the LR MRI and then used CapsNet with a discriminator as a classifier to identify real or fake images. Another important addition to this proposed model is the use of features extracted by a CheXNet model in reconstructing the image. The model uses a 32% smaller number of trainable parameters in comparison to the conventional technique. The model achieved a PSNR of 9.77, an SSIM of 0.60, and a MS-SSIM of 0.79.
Megavoltage CT (MVCT) imaging is performed before PCa patients undergo helical tomotherapy.83 However, these images are affected by noise and reduced contrast in the reconstructed images. Lee et al.84 proposed a method to improve the quality of MVCT by generating planning kilovoltage CT (kVCT)-like images from MVCT of 11 patients with PCa going through helical tomotherapy using cycleGAN. They showcase the model’s performance with and without augmentation technique and reported average MAE, root-mean-square error, PSNR, and SSIM values were 18.91 HU, 69.35 HU, 32.73 dB, and 95.48 using augmented method, respectively, whereas cycleGAN with non-augmented data showed inferior results (19.88 HU, 70.55 HU, 32.62 dB, 95.19, respectively). The study showed that the use of affine transformations for the CT image pairs as an augmentation technique with CycleGAN improved the quality of synthetic kVCT by enhancing the contrasts while reducing the noises presented in MVCT images. However, when they compared the HU values between the synthetic kVCT and real planning kVCT for the soft tissues, they reported large differences due to different image characteristics in anatomical structures. It is important to match the HU values of the MCVT with planning kVCT, for the calculation of accurate patient dose. Even though it is an interesting use of GenAI, limited data is a challenge for this study and requires more investigation.
Artifact detection is also a possible application of GenAI. PCa detection from MRI can be greatly affected due to anomalies in the imaging technique and GenAI can be used to overcome the anomalies and artifacts from the images. Hu et al.85 initially generated rectal artifact-pattern adversarial noise from the pattern of the real rectal artifacts and created proprietary adversarial samples with the generated noise. This study showed how these artifacts impact the performance of the PCa classification model. Afterward, they proposed targeted adversarial training with the proprietary adversarial samples (TPAS) strategy and trained the DenseNet121 network using T2WI and DWI with its derived ADC maps with and without the TPAS strategy. The study showed that the TPAS technique demonstrated improved results in PCa classification at patient, slice, and lesion levels.
Based on these literature discussions, GenAI showed great performance in improving medical image quality and generating HR images from LR images, removing noise, and removing artifacts.
DIAGNOSIS AND PREDICTION
Lesion generation, detection and segmentation
GenAI has also shown promising performance in generating synthetic prostate lesions on MR images13, which can be used to increase the sample size and bring diversity in samples for AI training. Patsanis et al.20 evaluated six state-of-the-art GAN models e.g. f-AnoGAN, HealthyGAN, StarGAN, StarGAN-v2, Fixed-Point-GAN, and DeScarGAN, for PCa detection from T2W MRI using 961 in-house patients for model development and PROSTATEx dataset with 199 patients as an external set. They conclude that out of all 6 models, Fixed-Point-GAN achieved the best AUC of 0.73 and 0.77 on internal and external test sets, which was identified as a promising GAN for the detection of PCa on T2W MRI. Kitchen and Seah86 used DCGAN to generate synthetic, realistic Prostate lesion MR images using the SPIE ProstateX Challenge 2016 dataset.87 They used three modalities that were aligned and utilized: T2, ADC and Ktrans. The authors claim that simultaneously generating all modalities together benefits them to be coherent with each other. However, the dataset is quite small to evaluate its implication in realistic prostate lesion generation on these 3 modalities. Birbiri et al.88 used CGAN, cycleGAN, and U-Net models to develop models for the detection and segmentation of prostate tissue in 3D multi-parametric MRI scans (DWI, T2W, and ADC) from 40 patients. These models were trained and evaluated on MRI data from 40 patients and these models were tested on a clinical dataset annotated for this study and on a public PROMISE12 dataset.89 This study adopted three data augmentation schemes from the literature, the super-pixel approach90, the Gaussian noise addition approach91, and the moving mean approach to compensate for the limited training data. The CGAN model outperformed U-Net and cycleGAN models with a DSC of 0.78 and 0.75 on the private and the PROMISE12 public datasets, respectively.
GenAI has shown significant performance in generating synthetic PCa lesions in MR images from smaller datasets. This approach can be very beneficial to increase the sample size and can be used as a data augmentation approach for training deep learning models for PCa lesion segmentation or classification tasks. It will add value to the PCa community to see if GenAI has the potential to generate PCa lesions with clinical significance and how well it can replicate the heterogeneity of the PCa lesions from a smaller dataset.
Prostate segmentation
Literature has shown significant improvement in prostate gland segmentation model performance with the help of GenAI. Wang et al.92 showed with the help of GenAI, prostate gland segmentation performance can be improved even with difficult prostate MR images that had blurred borders and heterogeneous distributions of pixel intensity inside and outside the prostate. This study proposed the SegDGAN model, inspired by SegAN93, introducing a dense block in the generator network, to automatically generate segmentation of the prostate using MRI from an experimental cohort of 220, 135 cases from multiple open access datasets (Decathlon94, NCI-ISBI 201395 and QIN-PROSTATE-Repeatability96 and PROMISE1289 dataset used to test the model. The model performance was compared with the state-of-the-art segmentation network U-Net97, the FCN98, and SegAN.93 The model achieved the highest DSC of 0.92 and 0.86 and lowest volumetric overlap error of 15.28 and 13.60, and the lowest average surface distance of 0.51, and 1.02, Hausdorff distance values of 11.58 and 7.75 on the experimental cohort and PROMISE12 cohort, respectively.
Fernandez-Quilez99 used DCGAN-based architecture100 to generate whole gland (WG) segmentation masks from T2w prostate MRI and Pix2Pix-based architecture40 to translate the synthetic WG prostate MRI mask to a T2w image. They used the PROMISE12 dataset89 in their study. They used the synthetic data to train a vanilla U-Net97 based WG prostate segmentation model and compared it with various conventional augmentation techniques. The model achieved the highest DSC of 73.77, mean volumetric DSC of 69.36, and lowest mean surface distance of 1.16. Both studies showed significant improvement in the performance by using GenAI models for lesion segmentation outperforming state-of-the-art deep learning models.
Prostate segmentation task using deep learning models is already a well-studied domain. We have seen that, GenAI outperforms the state-of-the-art deep learning models in the prostate segmentation tasks while training with the same sample size. It would be interesting to observe how GenAI performs in segmenting different regions of the prostate gland, which can be beneficial for the clinical application.
Digital pathology
There are various applications where GenAI has been used in histopathological image analysis, e.g. stain normalization101, segmentation102, data generation and augmentation103, etc. Similar to these, the use of GenAI has also been observed in the literature for prostate pathology and biopsy images.104,105 Karimi et al.104 used GAN and DCGAN for creating prostate pathology patches based on GG as a data augmentation technique for a CNN-based classification model for accurately grading PCa in histopathology images with limited data. Their classification model achieved an accuracy of 92%, for the detection of patches with PCa and non-PCa, and an accuracy of 86% in classifying GG 3 to GG 4 and 5 patches.
Golfe et al.103 proposed a conditional progressive growing GAN with stain normalization as post-processing to synthesize prostate histopathological tissue patches by selecting the de-sired GG cancer pattern in the synthetic sample where conditional GG information was provided to the model through the embedding layers. The study reported an FID metric of 81.86 for GG3, 49.32 for GG4, and 108.69 for GG5 after post-processing stain normalization.
Rana et al.106 trained CGAN to generate synthetic hematoxylin and eosin staining (H&E) stained prostate core whole slide RGB image (WSRI) from non-stained WSRI. They also proposed a destaining model to generate non-stained WSRI from H&E stained WSRI of the sample biopsy. The synthetic images were then compared with the real image to observe structural and anatomical details of the prostate core, and the staining and destaining model achieved an SSIM of 0.68 and 0.84, respectively.
Booven et al.107 used DCGAN models to produce high-quality synthetic images of different PCa grades from a total of 33 radical prostatectomy (RP) and needle biopsies (ND). The model achieved a similarity index score ranging from 0.8 to 1 for both the RP and ND sections. Afterward, they used the synthetic image patches alongside the original image patches to train an EfficientNet CNN model for the classification of GG of the digital histology sections and compared the results with another trained AI model using only original image patches to grade the same sections. They reported the model’s performance accuracy improved for GG 3 from 0.53 to 0.67 (p = 0.0010), in GG4 from 0.55 to 0.63 (p = 0.0274), and in GG5 from 0.57 to 0.75 (p < 0.0001) when trained with the combination of original and synthetic image patches compared to the model trained solely with original data.
Ho et al.108 proposed a new framework named stable diffusion (SD) with self-distillation from separated conditions (DISC) to generate multiple GG maps with GG-guided masks, which is used to conditionally train a latent diffusion model to generate synthetic histology tiles containing multiple GGs by leveraging pixel-wise annotations in input tiles. They tested their proposed scheme with the baseline Carcino-Net109 and Carcino-Net trained on tiles generated by the proposed scheme using the SICAPv2110 and LAPC111 datasets. They reported an improvement in precision of 0.027 on the SICAPv2 dataset from the baseline and on the LAPC dataset. Carcino-Net trained with only SD obtained the best precision at 0.7863 ± 0.0547, compared to the Carcino-Net baseline, and 0.7431 ± 0.1452 for SD-DISC-CoTrain provided more stable performance across classes.
These studies showed that there are ample opportunities to use GenAI to synthetically generate histopathology images for PCa and overcome the limited data problem in AI research in digital pathology in PCa. Furthermore, creating synthetic staining from non-stained images is another great potential area of study for GenAI models. In Figure 3c, we have shown an example of an in-house approach to synthetically generate H&E-stained images from phosphatase and tensin homolog (PTEN) immunohistochemistry (IHC) images using GenAI. The same process can be done in reverse order (PTEN IHC image generation from H&E images). As PTEN IHC is not part of the clinical workflow, generating synthetic PTEN IHC from H&E images using GenAI can be very beneficial to have additional prognostic information for the pathologists.
CHALLENGES, LIMITATIONS AND FUTURE DIRECTIONS
The applications of GenAI in PCa imaging have made significant strides in a very short period of time, but there are still several barriers and challenges that continue to hinder widespread adoption in the clinical environment.
Technical challenges
From a technical perspective, different generative models each have varying limitations, and it can be challenging to weigh their advantages and disadvantages across the frameworks as they are often applied to differing datasets and clinical scenarios. GANs, for one, are susceptible to training instability and model collapse where a subset of the image domain is learned extremely well, and other parts are ignored.112 This can hinder the synthesis of high-quality representative datasets, limiting their application in self-supervised training and other medical contexts. Using diverse clinical cohorts with various imaging techniques and a heterogeneous patient population can be a potential solution to this.
Computational requirements present yet another demand and challenge. With high-performance GPUs often being necessary, they may not always be available in clinical environments or research settings. Although certain frameworks like transformers may have an advantage when it comes to data efficiency, there will always be push and pull between computational constraints and the increasing complexity and size of models and data sets. As GPUs become cheaper and more readily available, we are likely to continue to see advancements in terms of computational capabilities and the compromise between volume and efficiency.
Lastly, a pressing technical limitation in the field continues to be data imbalances. With inconsistent and imbalanced datasets across centers and over time, models continue to produce flawed or unrepresentative data with the potential to amplify patterns present in the original dataset. To overcome this issue, careful data curation, and various augmentation techniques, algorithms with attention maps towards the minority classes to learn underrepresented features with more caution, and attention by the GenAI, are some of the potential solutions. Even though these generative models are normally developed for natural images, and afterward, modified for medical images, dedicated generative algorithms need to be developed with consideration of the above-mentioned challenges in the medical imaging domain.
As the field progresses rapidly, we are likely to see significant strides in technical developments. With frameworks such as GANs and diffusion models being relatively new, it is likely that new frameworks will emerge as AI becomes an increasingly popular field across various domains. We have already begun to see emerging hybrid models as discussed in section one, with plenty of room for further exploration in the field of PCa imaging.
Clinical challenges
One of the major challenges GenAI faces is having clean clinical data for learning. Biased, unclear representation and lack of heterogenicity of disease can lead GenAI to generate false and unrealistic data, which can be misleading to clinical decision-making. It is very important to train these models with datasets that have been evaluated by experts of respective domains. So far to encourage and help AI-based research with cleaner data, many publicly available datasets have been released, which is also benefiting the GenAI model development. For PCa also, many open-source data are publicly available46,81,87,94-96 and we have seen in this literature review, that researchers are taking advantage of these datasets to develop the models. However, these datasets do not always have good quality, and it is recommended to do some quality assessment and pre-processing before using them. Furthermore, more public datasets are required from larger cohorts with multi-modal imaging.
Another major challenge in the clinical domain faced by GenAI is workflow integration. Current imaging archiving systems like picture archiving and communication systems (PACS) either cannot incorporate any AI module, or it is not easy to implement these models into the system due to regulation of the existing system. However, many commercial AI-supported PACS are now available, which will be also suitable for GenAI models. As GenAI is a newer field, and its assessment of clinical application is still under research, integrating these models into a protected system like PACS also raises many patient privacy and data protection concerns. Unlike other fields i.e. text generation, and normal image generation, the applicability of GenAI in the medical imaging domain still faces biases due to the data it is trained on. In this review, we identified contradictory results between GenAI-generated images, and in many cases, they helped to boost the performance of the deep learning models, when used as an augmentation method, it is still unclear how these models will impact the diagnosis process for the medical imaging physicians for PCa. More multi-reader studies are required to understand how these generated images or artificially improved image quality affect the expert’s decision-making process. GenAI trained on medical images from diverse clinical cohorts from different scanning techniques, with multi-reader studies consisting of various levels of expertise can be a potential solution to build trust and reliance in this new technique for the PCa community.
Security and privacy concerns
The last but one of the major challenges with GenAI models are security and privacy concerns. While GenAI has transformative potential, it also introduces significant privacy and security risks due to its extensive data requirements and lack of transparency.113 Consequently, processes such as data collection, model training, and system implementation carry inherent privacy and security concerns. As generative AI transitions from research to clinical application, a cautious approach is necessary to identify and mitigate potential vulnerabilities. De-identification of patients’ personal and sensitive information is a must and needs to be strictly followed for medical images to be used with GenAI. More caution needs to be taken if the models will be publicly available.
In conclusion, we see ample opportunities for GenAI in the prostate medical imaging field. Even though we have highlighted many applications of GenAI in PCa imaging, these still reflect the early stages of the research to understand the clinical usability of GenAI in PCa diagnosis and treatment planning. We can establish that GenAI has enhanced the performance of the deep learning models, with providing synthetic data as a data augmentation method, which is a major challenge in the PCa medical imaging domain. It would still be interesting to see how deep learning models trained on smaller cohorts with synthetic data created by GenAI, will perform, in comparison with conventional deep learning models trained on larger and more diverse cohorts. We also see a need for a larger multi-modal PCa imaging dataset to develop a more robust model. Multi-reader studies are a necessity at this phase to evaluate the clinical applicability of these models for PCa diagnosis. Finally, the use of GenAI in treatment planning and response prediction for PCa, from medical imaging requires more attention from the research community.
Authorship Contributions: Concept- F.H., B.D.S., S.A.H., B.T.; Design- F.H., B.D.S., B.T.; Supervision- S.A.H., B.T.; Fundings- B.T.; Materials- F.H., B.D.S., B.T.; Data Collection or Processing- F.H.; Analysis or Interpretation- F.H., B.D.S.; Literature Search- F.H., B.D.S., K.B.Ö., B.T.; Writing- F.H., B.D.S., K.B.Ö., S.A.H., B.T.; Critical Review- K.B.Ö., S.A.H., B.T.
Conflict of Interest: The authors declare that they have no conflict of interest.
Funding: The authors declared that this study received no financial support.
Copyright © 2025 - Balkan Medical Journal
