1. Introduction
Safe and efficacious, assisted reproductive technology (ART) has the potential to enable millions more people globally to have a family, yet it continues to be underutilized [1,2,3,4,5,6,7,8,9,10]. For patients contemplating the use of in vitro fertilization (IVF) with their own eggs, IVF with donor eggs, intra-uterine insemination (IUI), or a wait-and-see approach, understanding realistic treatment success probabilities (e.g., excellent and poor) is expected to empower patient decision making [11,12,13]. Indeed, delivering personalized prognostics, an important part of patient-centered care, is challenging for providers in all areas of medicine, even though prognostics counseling is one of the pillars of Western medicine dating back to the Hipppocratic tradition [14,15,16,17,18]. IVF, as used broadly to include the use of ovarian stimulation with or without ICSI, is the most prevalent form of ART and is the focus in the rest of this article.
We established a globally applicable framework for localizing IVF prognostics reports to support patient–provider counseling and enable validated, data-driven treatment decisions. Previously, we reported that validated, AI/ML center-specific (MLCS) models produced more accurate and higher IVF live birth probabilities (LBP) compared to age-based prognosis [19,20,21,22,23]. Data requirements, model training, and validation methods for creating MLCS models that compute a patient’s LBP based on her and her partner’s own health data were described [20,21,22,23]. Recently, we further showed that MLCS models created using individual centers’ data outperformed national registry-based models in their ability to accurately identify more patients with higher IVF live birth probabilities [24]. Further, using AI/ML product design-to-deployment best practices and in collaboration with fertility providers treating patients from diverse demographics and payer mix (including self-pay, government, and private health plans), we developed widely applicable data-model pipelines, methods, and locally adaptable clinical workflow and implementation [19,20,21,22,23,24,25].
Other researchers have reported IVF pre-treatment models designed specifically for counseling patients prior to starting their first IVF cycle or after one or more failed cycles. For example, online calculators have been developed using the US Society for Reproductive Technology (SART) or UK Human Fertilization and Embryology Authority (HFEA) national database without center-specific validation [26,27,28]. Alternatively, other researchers—Qiu et al., 2019; Liu et al., 2013; Cai et al., 2024—have also reported using machine learning to train and validate IVF pre-treatment models with excellent, center-specific validation results [29,30,31]. Finally, some IVF prediction models such as that reported by Wen et al., 2022 contributed to our insights but would not practically be usable for patient counseling in the pre-treatment context, since they required data that are not available prior to starting IVF (e.g., number of oocytes, number of blastocysts, etc.) [32]. A discussion of the similarities and differences of the above-mentioned IVF pre-treatment models necessarily relate to model design, training techniques, validation, and performance metrics, which are beyond the scope of this article but are provided in a review by Yao et al., submitted [33]. Aside from AI/ML applications in patient counseling, there has been significant development in AI applications using unstructured data such as images or videos to support embryologists in selecting embryos most likely to result in successful pregnancies, and early prototypes have been reported to support optimization of various aspects of the IVF treatment protocol [34,35].
Though IVF success prediction model development has been reported by others, we are not aware of scientific reports of real-world clinical usage of provider–patient prognostic counseling supported by such models and their impact on patients’ treatment decisions [26,27,28]. However, unlike therapeutic intervention, which is evaluated in a progressive, structured set of clinical trials, the development of an AI/ML platform delivering provider- and patient-facing prognostics information must satisfy real-world utilization requirements varying across cultures, socioeconomic environments, and geographies, in addition to scientific validation, before it can be subject to prospective evaluation. Challenges notwithstanding, the reporting of real-world, early-stage clinical usage of AI tools in the medical research literature is important in engaging clinicians to partake in and contribute to technology evolution in medicine. The DECIDE-AI guidelines ensure appropriate and responsible reporting while recognizing requirements—such as customization of implementation in different locations or addressing the diverse needs of clinical teams working at different organizations—unique to technology implementation [36]. For reference, other guidelines adapted for AI-related studies (e.g., TRIPOD-AI, STARD-AI, PRISM-AI, CONSORT-AI, and SPIRIT-AI) address the scientific aspects of prediction modeling or clinical trials unaffected by customized, multisite implementation variations [37,38,39,40,41]. Also, the DECIDE-AI guidelines can apply to diagnostic, prognostic, or therapeutic AI tools and do not impose a particular study design [36].
Current Research Objectives
The overarching goal of this study was to share real-world, early-stage clinical usage of a patient counseling report conveying personalized IVF LBP validated by ML-trained prediction models for each participating center (i.e., the Univfy® PreIVF Report, a.k.a. Univfy report). Specifically, we aimed to report the relationship between Univfy report usage and first IVF and/or IUI usage (a.k.a. IVF or IUI conversion) at three different time intervals after the new patient visit and assess the relationship between Univfy report usage and IVF and/or IUI conversion. Using the DECIDE-AI guidelines, we conducted this retrospective cohort study with seven unrelated North American fertility centers operating in 17 locations across seven US states and Ontario, Canada. In aggregate, these centers performed ~3500 IVF cycles (excluding embryo transfers) and evaluated ~12,000 new fertility patients annually during the study period spanning 2016–2022. We also describe a stand-alone method for quantifying treatment conversion rate that can be applied to any center to facilitate reporting and comparisons.
2. Materials and Methods
2.1. De-Identified Data and Data Sources
De-identified data (including clinical service utilization, treatment, and outcomes data) previously linked and processed as part of Univfy client services were entered into the Univfy research database as per the research protocol, which has an exempt status determined by Institutional Review Board (IRB). The overall time period covered by this study is 2016–2022, within which each center had 4–5 years of post-launch usage.
The original data sources provided by each center included any combination of electronic medical record (EMR) systems, appointment data, other third-party software, the Univfy report database, and SART CORS, the US national registry database managed by the Society for Assisted Reproductive Technology (SART) [42]. Figure 1 summarizes the sources of raw data, data processing steps, and the processed and merged data for analysis in addition to additional details. Of the seven centers, one was a Canadian center, and six were American centers that reported to SART.
2.2. Univfy Report and MLCS Model Life Cycle
The Univfy report and MLCS model life cycle, including methods used for model training and validation, were reported elsewhere [19,20,21,22,23]. Each center’s own historical outcomes data were initially subjected to model training and validation, as part of HIPAA-compliant, paid services performed independent of and prior to the current research study.
IVF Outcomes Data, used for the development and validation of the prediction model (Figure 1), typically contains age, BMI, at least one ovarian reserve test (e.g., Day 3 FSH, serum anti-mullerian hormone (AMH) levels, antral follicle count (AFC)), clinical diagnoses, male partner history and/or any male factor diagnosis, reproductive history, prior IVF-ET outcomes, and any additional available variables that providers wish to test as model predictors. Of note, AFC was not routinely used or available at the time of IVF pretreatment patient counseling. Other than the patient’s age, no other variables were strictly required for model building, as each center typically has recorded sufficient data for an adequate set of variables to support MLCS model development and validation from consecutive years of data. Subsets of these clinical variables showed relative importance as model predictors, though the predictor subsets and their relative weighting vary slightly across centers [20,23]. Specifically, the important role of AMH and AFC was reported by Nelson et al., 2015 [23].
Related but separate from IVF Outcomes Data requirements is the mandatory versus optional data entry for generating a Univfy report for a patient. Whether a variable is required or optional or not even requested depends on its relative importance in the model and clinical workflow (e.g., whether that clinical variable would realistically be available at the time of patient counseling). For example, prior to 2022, each center was using its own first version of the Univfy report (launched at various times in 2016–2019). Some of those first-version Univfy reports used Day 3 FSH as the only ovarian reserve test or used one or both of Day 3 FSH and AMH. That design reflected (i) IVF Outcomes Data not having sufficient AMH availability from the years prior to the Univfy report launch (i.e., a predictor cannot be used if that predictor was not adequately used in the historical datasets); (ii) AMH not being routinely covered by health insurance plans. Currently, fertility centers are using second-version Univfy reports that all require AMH to be entered, which reflects the adequate AMH usage in the IVF Outcomes Data used to build and validate current models and the current, routine usage of AMH in clinical practice.
The Univfy report provided the personalized probability of having a baby from one, two, or up to three IVF cycles. In addition, the fertility center can elect to also provide the probability of having a baby from an IUI cycle or from one, two, or up to three IUI cycles. The IUI success rates may be cited from a reference article typically used by the providers in that context, or they can be generated from the center’s own IUI data, if available. The Univfy report did not recommend a decision between IUI and IVF. Instead, the report gave prognostic information about both treatment options (if the patient was clinically eligible) to support counseling by the provider and decision making by the patient. Additionally, the Univfy report showed patients how their age, BMI, and ovarian reserve compared with other IVF patients at the same center. Finally, the Univfy report also showed the IVF fees (customized for each center) and, if applicable, showed a cost comparison between pursuing “shared risk” (also called the IVF refund program) and paying fee-for-service.
2.3. Defining Event History, Time Interval, and IUI or IVF Treatment Utilization
Using the Event History Data Set, which comprised merged data from Patient Visit Data (including office visits, IUI and IVF treatment dates), Univfy Report Data (including Univfy report usage and associated clinical data), and IVF Outcomes Data (see Figure 1), a chronological Event History was created for each de-identified patient (“patient” hereafter) that had a new patient visit (NPV) during the time period analyzed. The Event History was mapped for 3 timed intervals for analysis: 180 days post-NPV (180D), 360 days post-NPV (360D), and “Ever”. Here, we defined the IVF cycle as having started ovarian stimulation with the intention to perform oocyte retrieval, IVF, or ICSI and subsequent fresh and/or frozen embryo transfer(s). “Conversion” was used to signify the first usage of IUI or IVF following NPV at each center.
IVF conversion included IVF cycles using one’s own or one’s partner’s or third party’s eggs or uterus, egg freezing, and freeze-all and IVF cycles in which oocyte retrieval and/or embryo transfer was cancelled for any reason, outside of illness and other non-fertility related reasons. IUI treatment was taken as-is from the appointment data and included IUI cycles using a partner’s or donor sperm, natural cycle, ovulation induction, ovarian stimulation, ovulation trigger, and/or luteal phase support.
2.4. Defining Patient Groups, Univfy Report Usage, Event Groups, and Time Periods for Analyses
Patients were retrospectively allocated to Patient Groups (Univfy Group vs. No Univfy Group) and Event Groups (IUI, Direct IVF, Total IVF, No IUI, or IVF). The Patient and Event Groups, contemporaneous to each center, and Univfy usage rates were mapped for each of three time intervals: 180D, 360D, and “Ever”. IUI or IVF conversion analyses were performed for each of the three time periods (Figure 2).
Patient Groups: The Univfy Group comprised patients who received a Univfy report prior to the first IUI or first IVF usage or had no IUI or IVF subsequent to receiving the Univfy report, within the analysis period (Figure 3A,B). The No Univfy Group comprised patients who did not receive a Univfy report prior to their first IUI or IVF, or at any time during that same analysis period.
Event Groups: Per time period, each patient took one of four mutually exclusive care navigation paths: (1) IUI Only (IUI is her first treatment, and there is no IVF in the same time period); (2) Direct IVF (IVF is her first treatment); (3) IUI-IVF (had IVF after one or more IUIs); (4) “No IUI or IVF”. Event Groups, non-mutually exclusive, were defined by the following navigation paths: (1) IUI conversion (includes IUI Only and IUI-IVF conversion); (2) Direct IVF; (3) Total IVF (includes Direct IVF and IUI-IVF conversion); and (4) “No IUI or IVF” (Figure 3.)
2.5. Univfy Report Usage Indications and Contraindications
All 7 centers chose to provide the Univfy report to their patients at no additional cost. Univfy recommended using the Univfy report if the following 2 indications were met and there were no contraindications: (1) the patient must be 18–45 years of age; (2) patient was determined by her provider to be clinically eligible to receive IVF treatment and was interested in being counseled on the benefits and limitations of IVF using their own eggs. Contraindications of the Univfy report implemented during this study period were as follows: (1) patient was in perimenopause or menopause, (2) Day 3 FSH > 15 mIU/mL, or (3) had a prior failed IVF cycle with no blastocysts or euploid blastocysts for transfer, and/or the oocyte or embryo quality was extremely poor. (Univfy offered a separate model addressing patients with past poor embryology outcomes, which is beyond the scope of this article.).
As part of the standard data processing and model development processes, we present the steps and results pertaining to the model and model performance, including indications and contraindications, to the clinical leadership at the center for discussion, any revisions, and final approval. These and any additionally necessary contraindications are then stated explicitly in the Instructions for Use (IFU) manual and then taught to center personnel at the time of training.
2.6. Providers’ Autonomy of Univfy Report Usage and Real-World Limitations
Univfy did not impose any additional usage rules. Centers and providers had complete autonomy to choose the extent of Univfy report usage, including the prioritization of Univfy report usage to patients who, in the provider’s clinical judgement, should consider IVF as a first-line treatment for a variety of clinical reasons. During the study period, automated EMR-Univfy data transmission and staffing by advanced practice providers (APP) were not in place at most centers. Therefore, Univfy report usage was limited by staffing and patients’ completion of diagnostic tests.
It was not possible to compare the pre-treatment clinical data between all patients in the Univfy vs. No Univfy Groups. Specifically, baseline pre-treatment clinical data for certain patients would have required de novo EMR data extraction, an activity not permitted under the IRB-exempt study protocol. Patients affected by this data limitation were No Univfy Group patients that took the care navigation paths of IUI Only or “No IUI or IVF”.
2.7. Statistical Analyses
Chi-square tests of independence were used to test actual versus expected proportions for Univfy vs. No Univfy report usage and the difference in IUI, Direct IVF, and Total IVF treatment conversions between Univfy and No Univfy Groups at each center. A Wilcoxon signed-rank test was used to test the mean difference in IUI, Direct IVF, and Total IVF treatment conversions across the 7 centers. NS indicated a non-significant p-value ≥ 0.05.
Two logistic regression analyses were performed. First, logistic regression was applied to data from all seven centers to test the effect of center and Univfy report usage on the likelihood of each of IUI, Direct IVF, and Total IVF usage. Separately, logistic regression was used to test the effect of center and patients’ age on the likelihood of IVF usage among Univfy Group patients. Per research protocol requirements, the data available for this analysis were limited to Univfy Report Data available between 1 July 2019 and the end of the study period for 5 of 7 centers. We did not extend the regression analysis to test the effect of AMH on IVF conversion, because not all the centers’ first version of the Univfy report (designed and launched at various times in 2016–2019) required AMH usage, as the Univfy report accommodated the use of Day 3 FSH and/or AMH at that time (see Section 2.2).
This study is reported according to the STROBE statement and DECIDE-AI guidelines [36,43].
3. Results
3.1. Univfy Report Utilization
Univfy report utilization and treatment conversion analysis was performed for all seven centers. The total numbers of unique patients meeting criteria of the three timed analyses were 20,512 (180D), 17,124 (360D), and 24,238 (Ever) in aggregate across the seven centers. The mean Univfy report utilization rates across seven centers for the 180D, 360D, and Ever time periods were 17.4% (95% CI 12.4–22.4%), 20.1% (95% CI 14.5–25.7%), and 24.3% (95% CI 18.3–30.3%), respectively. Since chi-square tests of independence showed that the Univfy report utilization rate was different across centers in each time period assessed (p < 0.001), subsequent results were reported for the aggregate data and were either on a per-center level or accounted for “center” as a proxy for center-specific attributes.
3.2. Comparison of IUI and IVF Conversion Rates between the Univfy and No Univfy Groups
We tested whether Univfy report usage was associated with differential conversion rates for the Event Groups IUI, Direct IVF, and Total IVF in each timed analysis. Compared to the No Univfy Group, Univfy report exposure increased the odds of IUI conversion by 195% (OR = 2.95, CI 2.63–3.31, p < 0.001), Direct IVF conversion by 213% (OR = 3.13, CI 2.83–3.46, p < 0.001), and Total IVF conversion by 241% (OR 3.41, CI 3.09–3.75, p < 0.001) at 180 days post-new patient visit. Similar findings were observed for the 360D and Ever time periods (Table 1).
Wilcoxon signed-rank tests showed that across centers, the Univfy group showed higher conversion rates for IUI, Direct IVF, and Total IVF (Figure 4A). At the per-center level, Direct IVF and Total IVF conversion rates were significantly higher in the Univfy Group in each of the seven centers, and IUI conversion rate was significantly higher in the Univfy Group in five of seven centers (Figure 4B). Using regression models accounting for “center” as a proxy for center-specific attributes, we confirmed that Univfy report usage was a significant predictor for each of IUI, Direct IVF, and Total IVF conversion in each of the time periods assessed (Table 1).
3.3. Assessing the Influence of Age on Treatment Decisions within the Univfy Group
Although it was not possible to analyze both Univfy and No Univfy Groups in their entireties with respect to patients’ clinical characteristics, we were able to test within the Univfy Group the potential influence of age on treatment decisions. Specifically, for the Univfy Group, we applied logistic regression to analyze the relationship between center and age on the probability of Direct IVF or Total IVF treatment conversion—using Patient Group and Event Group assignment in the Ever time period. Overall, 2468 patients from five centers in the Univfy Group had data available for this analysis (see restrictions explained in Section 2.7 and Figure 1, Sources of Raw Data for PreIVF Report Data). For context, the Direct IVF and Total IVF conversion rates for those 2468 patients were 31% and 45%, respectively. Using “center” to account for any center-specific attributes, age increased the odds of pursuing Direct IVF (OR = 1.05, CI 1.03–1.07, p < 0.001) and Total IVF conversion (OR = 1.03, CI 1.01–1.05, p < 0.001). For example, across centers, the average probability of Direct IVF conversion (without doing IUI first) was 30% at age 34 and 37% at age 40, whereas the average probability of Total IVF Conversion (regardless of IUI usage) was 45% at age 34 and 49% at age 40.
4. Discussion
Here, we discuss the unique contributions, limitations, and potential biases of this study in the context of addressing patients’ decision support needs using IVF prognostics tools. In addition, we offer insights to improve IVF usage, including ways to evaluate complex care navigation using treatment utilization metrics.
First, we review the intended clinical usage and relevance. The Univfy report was intended to provide pre-treatment IVF success probabilities to support decision making by each patient/couple, with a focus on treatment success expectations. The Univfy report conveyed the personalized IVF success probability based on the IVF cycle start, accounting for cycles that were unsuccessful due to cancellation of egg retrieval, poor ovarian response to stimulation, no embryos to transfer, and pregnancy loss. Though national registries have provided a bird’s eye view of IVF outcomes to inform policy and maternal and newborn safety guidelines, the Univfy report enables providers to communicate the IVF live birth probability based on each patient’s own health data with a prediction model that has been validated for data from their specific fertility center. The practical clinical benefits of using the machine learning, center-specific approach specific to model performance are discussed in other reports and are beyond the scope of this article [19,20,21,22,23,24,25,29,30,31,33].
There were limitations to this study. This study’s retrospective and non-randomized setting likely allowed for a mix of intended and unintended tendencies to use the Univfy report in patients who were more motivated in completing diagnostics and scheduling follow-up visits. Given the constraints of the exempt IRB protocol, it is beyond the scope of this current study to explore full EMR data exports to conduct the most appropriate comparison—Univfy vs. No Univfy Groups—to determine the differences in clinical characteristics of both groups, each in its entirety. Nevertheless, we surmised that even if the size of the “No IUI or IVF” Event Group within the No Univfy Group decreased by 20%, the Univfy Group would still show higher IVF conversion rates. There is also a small risk that not all Univfy reports that were generated were necessarily given to patients or used directly in patient counseling conversations, though providers’ informally self-reported that this issue occurred infrequently.
Potential bias notwithstanding, we learned that of the 2468 patients exposed to the Univfy report prior to their first IUI or IVF (or no treatment), the magnitude of patient’s age as a predictor for IVF conversion was limited after accounting for center as a proxy for all other center-specific attributes. In that regression analysis, we did not analyze AMH effect in addition to age and center effects, because clinical practice changed from using Day 3 FSH to AMH for ovarian reserve evaluation during that time (see AMH usage detailed in Section 2.2). We hypothesize that the use of a the MLCS-based counseling report increases IVF conversion rates by acknowledging personalization, center validation, and transparency in addition to its IVF prognostics delivery. Future studies should test additional factors such as IUI failure and IVF insurance coverage for any independent impact on the IVF cycle start decision. In addition, multicenter analyses and single-center analyses are both important in gaining insights that are generalizable to many centers and recognizing actionable items (a.k.a. low-hanging fruit) specific to each center. These examples highlight the need to shift from AI/ML model building to deployment including efficient and effective use of data to realize the potential of AI/ML to improve medicine and healthcare access and delivery [44]. Whereas most therapeutics can be tested using a “purist” approach using standardized administration (e.g., oral, IV, injections, etc.), studying the clinical impact of AI/ML tools requires additional considerations in the early technology adoption phase. The Univfy report utilization rates showed patterns of cautious adoption of new technology, as typically seen in the broader healthcare field [45].
In hindsight, it may seem limiting to not have full EMR data exports to support the Univfy vs. No Univfy analysis. However, during the study period (2016–2022), this was intentional to focus on establishing model validity, implementation, provider trust, and provider-patient satisfaction. Activities that were not “mission critical” for the model-report usage would have required yet more administrative burden from fertility centers and their clinical teams. The need to prioritize provider effort and overall costs is a necessary discipline in technology development and, in turn, keeps the cost of usage affordable, which directly relates back to making IVF more accessible. Further, there is ongoing adaptation to meet clinical needs. The historical and economic contexts related to AMH usage (Section 2.2) and its impact on model design illustrate the dynamic interactions between clinical practice patterns and AI/ML health technology evolution. The various constraints that we experienced turned out to be quite typical of early-stage studies of AI/ML-based tools implemented to support clinical care. In fact, the DECIDE-AI guidelines were established specifically to provide a framework for appropriate reporting under those constraints [36]. Finally, it would not have been appropriate to initiate a prospective trial back in 2016–2020, before confirming real-world implementation and clinical usage.
At the time of writing, the AI/ML platform offers a white glove service that manages and generates Univfy reports for centers including EMR-integrated automation of MLCS-based prognostics report generation; more fertility centers are also using a wider range of providers, including advanced practice providers (APPs), to scale the accessibility of IVF [35,46]. These types of support, which help to streamline provider team flow and improve patients’ experience, were not in place during the study period. Those and many other support features have assisted providers in adopting routine usage of Univfy reports and are essential for successful technology adoption in the current environment where understaffing is the norm.
Although most fertility centers have internal reporting of IVF utilization rates, there have been no standardized metrics, making inter-center discussions void of “apples-to-apples” relevance. By implementing benchmarks (e.g., Univfy report and conversion metrics such as 180D, 360D, and Ever), fertility centers with very different workflows and demographics could optimize patient-centric care navigation tailored to their teams’ and patients’ needs, while reporting metrics applicable across diverse locations. Also, the Univfy report and benchmark infrastructure further allow us to explore provider–patient shared decision-making in IVF and generate hypotheses for testing. There is an urgent need to understand patient decision making to realize the full potential of fertility care.
5. Conclusions
Altogether, this study’s findings suggest that the use of an MLCS-based, IVF prognostics counseling report is associated with higher IVF conversion rates. Among patients having an MLCS-based counseling report, older age was a small yet independent predictor of IVF conversion after accounting for center as a predictor. The retrospective nature of the study did not allow us to test for possible biases that might have caused some patients to receive or not receive the Univfy report. This study’s findings informed the design and implementation of a prospective trial to evaluate the impact of MLCS-based provider–patient counseling on IVF conversion. Further research to study factors influencing treatment decision making and real-world optimization of patient-centric workflows utilizing the MLCS reports is warranted.
6. Patents
Univfy Inc. owns or is the licensee of issued and pending patents related to the subject matter of this study, including Patent Number 9,458,495B2, foreign counterparts and patents issued.
Conceptualization, M.W.M.Y., E.T.N., T.S., X.C., M.G.R., L.A.G., S.C., J.E.N., J.F.P., M.O., K.C., J.M., B.W.D., J.B.III, M.P., R.B., G.N., J.S.B.J., and D.K.W.; methodology, M.W.M.Y. and E.T.N.; software, E.T.N., X.C., and T.S.; validation, E.T.N., T.S., and X.C.; formal analysis, E.T.N.; investigation, M.W.M.Y., and E.T.N.; resources, M.W.M.Y.; data curation, E.T.N., T.S., X.C., M.G.R., L.A.G., S.C., J.E.N., J.F.P., M.O., K.C., J.M., B.W.D., J.B.III, M.P., R.B., G.N., J.S.B.J., and D.K.W.; writing—original draft preparation, M.W.M.Y. and E.T.N.; writing—review and editing, M.W.M.Y., E.T.N., M.G.R., L.A.G., S.C., J.E.N., J.F.P., M.O., K.C., J.M., B.W.D., J.B.III, M.P., R.B., G.N., J.S.B.J., and D.K.W.; visualization, M.W.M.Y. and E.T.N. All authors have read and agreed to the published version of the manuscript.
The study was conducted in accordance with the Declaration of Helsinki and was determined by the Institutional Review Board (Sterling IRB) to have exempt status (protocol code 10010-MYao) on 21 April 2022.
Patient consent was waived due to exempt status given by the Institutional Review Board.
The datasets used in this study are not readily available due to privacy reasons, as the data were obtained from and owned by different third parties. The corresponding author does not have authorization to share data but will direct your dataset access requests to the appropriate person to obtain permission. The authors welcome requests to support reproducibility testing of methods in other datasets.
The authors thank the following individuals for their assistance, editing, advice, insightful comments, and contributions to the present research: Tina Manley (Atlantic Reproductive Medicine Specialists); Faith Ripley (PREG); Nicole Condon (TRIO Fertility Partners); Patrick McCarthy (Poma Fertility); Amanda McCarthy (Poma Fertility); Shirley Sinclair (My Fertility Center); Vincent Kim (Univfy Inc.); Anjali Wignarajah (Univfy Inc.); Marco Menabrito (Univfy Inc.); Candice Ortego (Univfy Inc.); Chris Hinrichs (Advisor); Angela Xie (Advisor); Athena T. R. Wu (Editing).
M. Yao, E.T. Nguyen, T. Swanson, and X. Chen report employment by and receipt of stock options from Univfy Inc. during the time period when the research was conducted. M. Yao reports being board director and shareholder of Univfy Inc., and she is also an inventor or co-inventor on Univfy Inc.’s issued and pending patents. The other authors declare no conflicts of interest.
Footnotes
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.
Enlarge this image.Figure 1. Summary of raw data sources, data processing steps, and the resulting processed and merged data for analysis.
Enlarge this image.Figure 2. Event History mapping for 3 timed analyses: (A) 180 days post-NPV (180D), 360 days post-NPV (360D), and “Ever”. (B) Examples showing the mapping of Event History to Patient Groups and Event Groups as per Methods 2.4.
Enlarge this image.Figure 3. Patient care navigation to treatment conversion by Patient Groups and Event Groups. (A) Retrospective patient cohort aggregated from 7 centers and their Event Groups (bolded) based on the Ever timed analysis: new patient visit (NPV)-IUI conversion (IUI Group), NPV-IVF conversion (Direct IVF Group), usage of 1st IVF cycle with or without one or more preceding IUIs (Total IVF Group), and “No IUI or IVF” Group. (B) The patient cohort from (A) separated into 2 Patient Groups—the Univfy Group (green) and the No Univfy Group (gray).
Enlarge this image.Figure 4. Comparison of IUI, Direct IVF, and Total IVF conversion rates between Univfy Group and No Univfy Group in 3 timed analyses. (A) The conversion rates across 7 centers were compared between Univfy Group and No Univfy Group for IUI, Direct IVF, and Total IVF conversions in each timed analyses, 180D, 360D, and Ever, using Wilcoxon signed-rank test. * p < 0.05. (B) The per-center IUI, Direct IVF, and Total IVF conversion rates are compared using chi-square tests * p < 0.001, NS not significant.
Enlarge this image.Figure 4. Comparison of IUI, Direct IVF, and Total IVF conversion rates between Univfy Group and No Univfy Group in 3 timed analyses. (A) The conversion rates across 7 centers were compared between Univfy Group and No Univfy Group for IUI, Direct IVF, and Total IVF conversions in each timed analyses, 180D, 360D, and Ever, using Wilcoxon signed-rank test. * p < 0.05. (B) The per-center IUI, Direct IVF, and Total IVF conversion rates are compared using chi-square tests * p < 0.001, NS not significant.
Summary statistics (mean and 95% confidence interval (95% CI)) of the IUI, Direct IVF, and Total IVF conversion rates across 7 centers for Univfy Group and No Univfy Group and the odds ratio of increased conversion in the Univfy Group compared to the No Univfy Group for each timed analysis. Χ2 ORs and p-values are based on chi-square test of independence using the aggregate data from 7 centers. Regression OR of Univfy report and p-values are based on regression models analyzing the relationship between center and Univfy report exposure on IUI, Direct IVF, and Total IVF conversions.
| Conversion | Mean Conversion Rates across 7 Centers (95% CI) | Χ2 Odds Ratio | Regression OR Univfy Report | ||
|---|---|---|---|---|---|
| Univfy Group | No Univfy Group | ||||
| 180D | IUI | 20.5% (16.4–24.6%) | 8.9% (5.8–11.9%) | 2.95 (2.63–3.31, p < 0.001) | 3.74 (3.46–4.05, p < 0.001) |
| Direct IVF | 24.6% (10.8–38.5%) | 8.3% (3.8–12.7%) | 3.13 (2.83–3.46, p < 0.001) | 2.53 (2.33–2.75, p < 0.001) | |
| Total IVF | 28.4% (13.8–43.0%) | 8.8% (4.1–13.5%) | 3.41 (3.09–3.75, p < 0.001) | 3.81 (3.37–4.31, p < 0.001) | |
| 360D | IUI | 26.3% (21.1–31.4%) | 12.5% (9.0–16.0%) | 2.94 (2.63–3.29, p < 0.001) | 3.61 (3.20–4.06, p < 0.001) |
| Direct IVF | 28.9% (14.9–43.0%) | 11.1% (5.9–16.3%) | 2.89 (2.63–3.17, p < 0.001) | 2.67 (2.44–2.92, p < 0.001) | |
| Total IVF | 39.0% (23.6–54.4%) | 12.2% (6.5–17.9%) | 3.81 (3.49–4.16, p < 0.001) | 4.14 (3.80–4.51, p < 0.001) | |
| Ever | IUI | 25.3% (22.5–28.1%) | 16.2% (10.9–21.5%) | 2.05 (1.89–2.23, p < 0.001) | 2.43 (2.23–2.66, p < 0.001) |
| Direct IVF | 26.1% (14.0–38.2%) | 12.6% (7.0–18.2%) | 2.04 (1.90–2.20, p < 0.001) | 2.50 (2.31–2.70, p < 0.001) | |
| Total IVF | 36.2% (23.3–49.2%) | 14.2% (7.9–20.5%) | 2.78 (2.59–2.98, p < 0.001) | 3.56 (3.31–3.84, p < 0.001) | |
References
1. European Society of Human Reproduction and Embryology (ESHRE), 2022. ART Factsheet. Available online: https://www.eshre.eu/Press-Room/Resources (accessed on 29 April 2024).
2. Human Fertilisation & Embryology Authority (HFEA), 2019. Multiple Births in Fertility Treatment 2019. Available online: https://www.hfea.gov.uk/about-us/publications/research-and-data/multiple-births-in-fertility-treatment-2019/#:~:text=The%20multiple%20birth%20rate%20has,3%25%20across%20all%20patient%20ages (accessed on 29 April 2024).
3. International Committee Monitoring Assisted Reproductive Technologies (ICMART), 2018. ICMART Preliminary World Report 2018. Available online: https://www.icmartivf.org/reports-publications/#articles (accessed on 29 April 2024).
4. Society for Assisted Reproductive Technology (SART), 2022. More Than 73 Thousand Babies Born from Assisted Reproductive Technology Cycles Done in 2020. Available online: https://www.sart.org/news-and-publications/news-and-research/press-releases-and-bulletins/more-than-73-thousand-babies-born-from-assisted-reproductive-technology-cycles-done-in-2020/ (accessed on 29 April 2024).
5. World Health Organization (WHO), 2023. 1 in 6 People Globally Affected by Infertility: WHO. Available online: https://www.who.int/news/item/04-04-2023-1-in-6-people-globally-affected-by-infertility (accessed on 29 April 2023).
6. Herndon, C.N. Need for expanding insurance coverage for in vitro fertilization in the United States. Fertil. Steril.; 2019; 112, pp. 37-38. [DOI: https://dx.doi.org/10.1016/j.fertnstert.2019.04.040] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/31277766]
7. Klitzman, R. How much is a child worth? Providers’ and patients’ views and responses concerning ethical and policy challenges in paying for ART. PLoS ONE; 2017; 12, e0171939. [DOI: https://dx.doi.org/10.1371/journal.pone.0171939] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/28207807]
8. Rich, C.W.; Domar, A.D. Addressing the emotional barriers to access to reproductive care. Fertil. Steril.; 2016; 105, pp. 1124-1127. [DOI: https://dx.doi.org/10.1016/j.fertnstert.2016.02.017]
9. Vassena, R. Moonshots and last miles: What it may take to treat infertility for all. RBMO; 2024; 48, 103642. [DOI: https://dx.doi.org/10.1016/j.rbmo.2023.103642] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/38884012]
10. Skedgel, C.; Cubi-Molla, P.; Mott, D.; Gameiro, S.; Boivin, J.; Al-Janabi, H.; Brazier, J.; Markert, M.; Andersson, F.L.; Jofre-Bonet, M. Unmet parenthood goals, health-related quality of life and apparent irrationality: Understanding the value of treatments for infertility. Pharmacoecon Open; 2023; 7, pp. 337-344. [DOI: https://dx.doi.org/10.1007/s41669-023-00402-5] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/36920719]
11. Van Eekelen, R.; Van Geloven, N.; Van Wely, M.; Bhattacharya, S.; Van der Veen, F.; Eijkemans, M.J.; McLernon, D.J. IVF for unexplained subfertility; whom should we treat?. Hum. Reprod.; 2019; 34, pp. 1249-1259. [DOI: https://dx.doi.org/10.1093/humrep/dez072] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/31194864]
12. The Guideline Group on Unexplained Infertility Romualdi, D.; Ata, B.; Bhattachara, S.; Bosch, E.; Coastello, M.; Gersak, K.; Homburg, R.; Mincheva, M.; Norman, R.J. et al. Evidence-based guideline: Unexplained infertility. Hum. Reprod.; 2023; 38, pp. 1881-1890.
13. Reindollar, R.H.; Regan, M.M.; Neumann, P.J.; Levine, B.S.; Thornton, K.L.; Alper, M.M.; Goldman, M.B. A randomized clinical trial to evaluate optimal treatment for unexplained infertility: The fast track and standard treatment (FASTT) trial. Fertil. Steril.; 2010; 94, pp. 888-899. [DOI: https://dx.doi.org/10.1016/j.fertnstert.2009.04.022] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/19531445]
14. Thomas, J.M.; Cooney, L.M., Jr.; Fried, T.R. Prognosis Reconsidered in Light of Ancient Insights-From Hippocrates to Modern Medicine. JAMA Intern. Med.; 2019; 179, pp. 820-823. [DOI: https://dx.doi.org/10.1001/jamainternmed.2019.0302]
15. Van Empel, I.W.H.; Aarts, J.W.M.; Cohlen, B.J.; Huppelschoten, D.A.; Laven, J.S.E.; Nelen, W.L.D.M.; Kremer, J.A.M. Measuring patient-centerdness, the neglected outcome in fertility care: A random multicenter validation study. Hum. Reprod.; 2010; 25, pp. 2516-2526. [DOI: https://dx.doi.org/10.1093/humrep/deq219]
16. Dancet, E.A.F.; Van Empel, I.W.H.; Rober, P.; Nelen, W.L.D.M.; Kremer, J.A.M.; D’Hooghe, T.M. Patient-centerd infertility care: A qualitative study to listen to the patient’s voice. Hum. Reprod.; 2011; 26, pp. 827-833. [DOI: https://dx.doi.org/10.1093/humrep/der022]
17. McMahon, C.; Hammarberg, K.; Lensen, S.; Wang, R.; Mol, B.W.; Vollenhoven, B.J.N. What do women undergoing in vitro fertilization (IVF) understand about their chance of IVF success?. Hum. Reprod.; 2024; 39, pp. 130-138. [DOI: https://dx.doi.org/10.1093/humrep/dead239]
18. Miron-Shatz, T.; Holzer, H.; Revel, A.; Weissman, A.; Tarashandegan, D.; Hurwitz, A.; Gal, M.; Ben-Chetrit, A.; Weintraub, A.; Ravhon, A. et al. ‘Luckily, I don’t believe in statistics’: Survey of women’s understanding of chance of success with futile fertility treatments. Reprod. Biomed. Online; 2021; 42, pp. 463-470. [DOI: https://dx.doi.org/10.1016/j.rbmo.2020.09.026]
19. Jenkins, J.; van der Poel, S.; Krüssel, J.; Bosch, E.; Nelson, S.M.; Pinborg, A.; Yao, M.W. Empathetic application of machine learning may address appropriate utilization of ART. Reprod. BioMedicine Online; 2020; 41, pp. 573-577. [DOI: https://dx.doi.org/10.1016/j.rbmo.2020.07.005]
20. Banerjee, P.; Choi, B.; Shahine, L.K.; Jun, S.H.; O’Leary, K.; Lathi, R.B.; Westphal, L.M.; Wong, W.H.; Yao, M.W. Deep phenotyping to predict live birth outcomes in in vitro fertilization. Proc. Natl. Acad. Sci. USA; 2010; 107, pp. 13570-13575. [DOI: https://dx.doi.org/10.1073/pnas.1002296107]
21. Choi, B.; Bosch, E.; Lannon, B.M.; Leveille, M.C.; Wong, W.H.; Leader, A.; Pellicer, A.; Penzias, A.S.; Yao, M.W. Personalized prediction of first-cycle in vitro fertilization success. Fert. Steril.; 2013; 99, pp. 1905-1911. [DOI: https://dx.doi.org/10.1016/j.fertnstert.2013.02.016] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/23522806]
22. Lannon, B.M.; Choi, B.; Hacker, M.R.; Dodge, L.E.; Malizia, B.A.; Barrett, C.B.; Wong, W.H.; Yao, M.W.; Penzias, A.S. Predicting personalized multiple birth risks after in vitro fertilization–double embryo transfer. Fertil. Steril.; 2012; 98, pp. 69-76. [DOI: https://dx.doi.org/10.1016/j.fertnstert.2012.04.011] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/22673597]
23. Nelson, S.M.; Fleming, R.; Gaudoin, M.; Choi, B.; Santo-Domingo, K.; Yao, M. Antimüllerian hormone levels and antral follicle count as prognostic indicators in a personalized prediction model of live birth. Fertil. Steril.; 2015; 104, pp. 325-332. [DOI: https://dx.doi.org/10.1016/j.fertnstert.2015.04.032]
24. Nguyen, E.T.; Retzloff, M.G.; Gago, L.A.; Nichols, J.E.; Payne, J.F.; Ripps, B.A.; Opsahl, M.; Groll, J.; Beesley, R.; Neal, G. et al. Predicting IVF live birth probabilities using machine learning, center-specific and national registry-based models. Submitted.
25. Lavin, A.; Gilligan-Lee., C.M.; Visnjic, A.; Ganju, S.; Newman, D.; Ganguly, S.; Lange, D.; Baydin, A.G.; Sharma, A.; Gibson, A. et al. Technology readiness levels for machine learning systems. Nat. Commun.; 2022; 13, 6039. [DOI: https://dx.doi.org/10.1038/s41467-022-33128-9]
26. McLernon, D.J.; Steyerberg, E.W.; Te Velde, E.R.; Lee, A.J.; Bhattacharya, S. Predicting the chances of a live birth after one or more complete cycles of in vitro fertilisation: Population based study of linked cycle data from 113 873 women. BMJ; 2016; 355, i5735. [DOI: https://dx.doi.org/10.1136/bmj.i5735]
27. Ratna, M.B.; Bhattacharya, S.; McLernon, D.J. External validation of models for predicting cumulative live birth over multiple complete cycles of IVF treatment. Hum. Reprod.; 2023; 38, pp. 1998-2010. [DOI: https://dx.doi.org/10.1093/humrep/dead165]
28. McLernon, D.J.; Raja, E.A.; Toner, J.P.; Baker, V.L.; Doody, K.J.; Seifer, D.B.; Sparks, A.E.; Wantman, E.; Lin, P.C.; Bhattacharya, S. et al. Predicting personalized cumulative live birth following in vitro fertilization. Fertil. Steril.; 2022; 117, pp. 326-338. [DOI: https://dx.doi.org/10.1016/j.fertnstert.2021.09.015]
29. Qiu, J.; Li, P.; Dong, M.; Xin, X.; Tan, J. Personalized prediction of live birth prior to the first in vitro fertilization treatment: A machine learning method. J. Transl. Med.; 2019; 17, 317. [DOI: https://dx.doi.org/10.1186/s12967-019-2062-5]
30. Liu, X.; Chen, Z.; Ji, Y. Construction of the machine learning-based live birth prediction models for the first in vitro fertilization pregnant women. BMC Pregnancy Childbirth; 2023; 23, 476. [DOI: https://dx.doi.org/10.1186/s12884-023-05775-3]
31. Cai, J.; Jiang, X.; Liu, L.; Liu, Z.; Chen, J.; Chen, K.; Yang, X.; Ren, J. Pretreatment prediction for IVF outcomes: Generalized applicable model or center-specific model?. Hum. Reprod.; 2024; 39, pp. 364-373. [DOI: https://dx.doi.org/10.1093/humrep/dead242]
32. Wen, J.Y.; Liu, C.F.; Chung, M.T.; Tsai, Y.C. Artificial intelligence model to predict pregnancy and multiple pregnancy risk following in vitro fertilization-embryo transfer (IVF-ET). Taiwan J. Obstet. Gynecol.; 2022; 61, pp. 837-846. [DOI: https://dx.doi.org/10.1016/j.tjog.2021.11.038]
33. Yao, M.W.M.; Jenkins, J.; Nguyen, E.T.; Swanson, T.; Menabrito, M. Patient-centric IVF prognostics counseling using machine learning for the pragmatist. 2024; submitted.
34. Jiang, V.S.; Bormann, C.L. Artificial intelligence in the in vitro fertilization laboratory: A review of advancements over the last decade. Fertil. Steril.; 2023; 120, pp. 17-23. [DOI: https://dx.doi.org/10.1016/j.fertnstert.2023.05.149]
35. Hariton, E.; Alvero, R.; Hill, M.J.; Mersereau, J.E.; Perman, S.; Sable, D.; Wang, F.; Adamson, G.D.; Coutifaris, C.; Craig, L.B. et al. Meeting the demand for fertility services: The present and future of reproductive endocrinology and infertility in the United States. Fertil. Steril.; 2023; 120, pp. 755-766. [DOI: https://dx.doi.org/10.1016/j.fertnstert.2023.08.019]
36. Vasey, B.; Nagendran, M.; Campbell, B.; Clifton, D.A.; Collins, G.S.; Denaxas, S.; Denniston, A.K.; Faes, L.; Geerts, B.; Ibrahim, M. et al. Reporting guideline for the early stage clinical evaluation of decision support systems driven by artificial intelligence: DECIDE-AI. BMJ; 2022; 377, e070904. [DOI: https://dx.doi.org/10.1136/bmj-2022-070904]
37. Salybekov, A.A.; Wolfien, M.; Hahn, W.; Hidaka, S.; Kobayashi, S. Artificial Intelligence Reporting Guidelines’ Adherence in Nephrology for Improved Research and Clinical Outcomes. Biomedicines; 2024; 12, 606. [DOI: https://dx.doi.org/10.3390/biomedicines12030606]
38. EQUATOR Network. What Is a Reporting Guideline?. Available online: https://www.equator-network.org/about-us/what-is-a-reporting-guideline/ (accessed on 20 February 2024).
39. Sounderajah, V.; Ashrafian, H.; Aggarwal, R.; De Fauw, J.; Denniston, A.K.; Greaves, F.; Karthikesalingam, A.; King, D.; Liu, X.; Markar, S.R. et al. Developing specific reporting guidelines for diagnostic accuracy studies assessing AI interventions: The STARD-AI Steering Group. Nat. Med.; 2020; 26, pp. 807-808. [DOI: https://dx.doi.org/10.1038/s41591-020-0941-1] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/32514173]
40. Tripod Statement. Available online: https://www.tripod-statement.org/ (accessed on 20 February 2024).
41. Liu, X.; Rivera, S.C.; Moher, D.; Calvert, M.J.; Denniston, A.K. SPIRIT-AI and CONSORT-AI Working Group. Reporting guidelines for clinical trial reports for interventions involving artificial intelligence: The CONSORT-AI Extension. BMJ; 2020; 370, m3164. [DOI: https://dx.doi.org/10.1136/bmj.m3164]
42. Curchoe, C.L.; Tarafdar, O.; Aquilina, M.C.; Seifer, D.B. SART CORS IVF registry: Looking to the past to shape future perspectives. J. Assist. Reprod. Genet.; 2022; 39, pp. 2607-2616. [DOI: https://dx.doi.org/10.1007/s10815-022-02634-6] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/36269502]
43. Vandenbroucke, J.P.; von Elm, E.; Altman, D.G.; Gøtzsche, P.C.; Mulrow, C.D.; Pocock, S.J.; Poole, C.; Schlesselman, J.J.; Egger, M. STROBE Initiative. Strengthening the Reporting of Observational Studies in Epidemiology (STROBE): Explanation and elaboration. Plows Med.; 2007; 4, e297. [DOI: https://dx.doi.org/10.1371/journal.pmed.0040297] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/17941715]
44. Zhang, A.; Xing, L.; Zou, J.; Wu, J.C. Shifting machine learning for healthcare from development to deployment and from models to data. Nat. Biomed. Eng.; 2022; 6, pp. 1330-1345. [DOI: https://dx.doi.org/10.1038/s41551-022-00898-y] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/35788685]
45. Davenport, T.H.; Glaser, J.P. Factors governing the adoption of artificial intelligence in healthcare providers. Discov. Health Syst.; 2022; 1, 4. [DOI: https://dx.doi.org/10.1007/s44250-022-00004-8]
46. Adeleye, A.J.; Kawwass, J.F.; Brauer, A.; Storment, J.; Patrizio, P.; Feinberg, E. The mismatch in supply and demand: Reproductive endocrinology and infertility workforce challenges and controversies. Fertil. Steril.; 2023; 120, pp. P403-P405. [DOI: https://dx.doi.org/10.1016/j.fertnstert.2023.01.007]
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.
Details
; Retzloff, Matthew G 2 ; Laura April Gago 3 ; Copland, Susannah 4 ; Nichols, John E 5 ; Payne, John F 5
; Opsahl, Michael 6 ; Cadesky, Ken 7 ; Meriano, Jim 7 ; Donesky, Barry W 8 ; BirdIII, Joseph 8 ; Peavey, Mary 4 ; Beesley, Ronald 6 ; Neal, Gregory 2 ; BirdJr, Joseph S 8 ; Swanson, Trevor 1 ; Chen, Xiaocong 1 ; Walmer, David K 4 1 Department of R&D, Univfy Inc., 117 Main Street, #139, Los Altos, CA 94022, USA
2 Fertility Center of San Antonio, San Antonio, TX 78229, USA
3 Gago Center for Fertility, Brighton, MI 48114, USA
4 Atlantic Reproductive Medicine, Raleigh, NC 27617, USA
5 Piedmont Reproductive Endocrinology Group, Greenville, SC 29615, USA
6 Poma Fertility, Kirkland, WA 98034, USA
7 TRIO Fertility Partners, Toronto, ON M5G 2K4, Canada
8 My Fertility Center, Chattanooga, TN 37421, USA