The impact of experimental designs & system sloppiness on the personalisation process: A cardiovascular perspective

Abstract

To employ a reduced-order cardiovascular model as a digital twin for personalised medicine, it is essential to understand how uncertainties in the model’s input parameters affect its outputs. The aim is to identify a set of input parameters that can serve as clinical biomarkers, providing insight into a patient’s physiological state. Given the challenge of finding useful clinical data, careful consideration must be given to the experimental design used to acquire patient-specific input parameters. Model sloppiness—where numerous parameter combinations have minimal impact on model predictions, whilst only a few parameters significantly influence outcomes—is a critical concept in this context. In this paper, we conduct the first quantification of a cardiovascular system’s sloppiness to elucidate the structure of the input parameter space. By utilising Sobol indices and examining various synthetic cardiovascular measures with increasing invasiveness, we uncover how the personalisation process and the cardiovascular system’s sloppiness are contingent upon the chosen experimental design. Our findings reveal that continuous clinical measures induce system sloppiness and increase the number of personalisable biomarkers, whereas discrete clinical measurements produce a non-sloppy system with a reduced number of biomarkers. This study underscores the necessity for careful consideration of available clinical data as differing measurement sets can significantly impact model personalisation.

Introduction

The concept of digital twin (DT) originates in the 1960s with NASA creating a virtual representation in the Apollo 13 moon exploration mission. There are now many definitions of DT and one comprehensive definition is “a set of virtual information constructs that mimics the structure, context and behaviour of an individual or unique physical asset, which is dynamically updated with data from its physical twin throughout its life-cycle and that ultimately informs decisions that realise value" [1]. In the realm of medicine, the potential of a DT is profound, particularly in enhancing patient care and outcomes. In the context of healthcare, a regularly updated digital representation of an individual’s anatomy, physiology or diseases holds immense promise. It could empower healthcare professionals to simulate and predict a patient’s disease trajectory enabling intervention and treatment to be delivered in a timely and effective way [2].

Notably, in cardiology, the adoption of heart and circulatory DTs has gradually gained momentum and trust within the clinical community, evidenced by several proof-of-concept studies [3–5]. Traditionally, clinical diagnosis and patient trajectories in cardiology rely heavily on a clinician’s expertise and population-based averages [6]. However, the emergence of DTs in cardiology signifies a shift towards a more personalised approach. These DTs integrate mechanistic (physics-based) models, grounded in physiological understanding of the heart, human circulation, and related physiological processes such as baroregulation [7, 8], with dynamic clinical data collected over time or immediate data available in a clinical setting [9]. This integration enables the DT tool to provide tailored predictions and assist in clinical diagnosis, catering for the unique characteristics of each patient. Virtual representations of a patient’s full cardiovascular health in differing states are referred to as their “physiological envelope" [10].

Clearly, the choice of mechanistic model utilised for a cardiovascular DT is vital to ensure the correct set of physiological relevance while also maintaining some set of clinical interpretability. Lumped parameter models (LPM) offer a unique ability to examine both cardiac function and global haemodynamics. LPM provide a simple approach in which all the main characteristics of the blood flow (i.e. blood pressures, flows and volumes) are captured. Typically, an LPM is constructed of a heart chamber (acting as a blood pump), a presentation of the mechanical nature of heart valves and a series of elements representing the various vascular networks in which blood can be transported through the body. This class of model is usually represented as a system of differential algebraic equations; the size of which depends on the complexity of the system investigated (full body circulation or anatomically detailed models of specific vessels) [11].

Each LPM or compartment can be represented as a combination of resistors, capacitors and inductors which are parameterised by numerical values R, C and L, respectively. For a generic vessel or organ located in a larger circulation network, R, C and L represent haemodynamic dissipation, vessel distensibility and the inertial effects of the blood flow, respectively [12]. Along with the input parameters of the heart chambers and valves, these parameters form a set of clinical biomarkers, which when personalised to a patient, by integrating patient-specific clinical data, provide the insight that a cardiovascular DT aims to achieve [13].

Useful clinical data are scarce resources, thus the requirement to identify and choose what data are needed to personalise a LPM (in order to create a useful cardiac DT) is a complex one [14]. Within a clinical setting, there are often a range of both continuous and discrete measurement data. But the process of obtaining insightful and diagnostically useful clinical data often requires a series of invasive tests being conducted on the patient. With any data collected (e.g., blood pressure, flow and volume for each compartment), one then generates a series of clinical metrics: ejection fraction [15], to quantify heart failure; pulse pressure [16], to diagnose arterial stiffening; maximum blood velocity [17], to evaluate heart valve stenosis; cardiac output [18], to measure overall heart health and the observation of various clinical time series waveforms [19]. These metrics can then be amalgamated into a DT, enriching the model’s predictions with additional detail and validity. However, given the plethora of available clinical tests, each carrying its own risks to patients, determining which metrics are indispensable in creating a faithful virtual representation of a patient becomes a challenging task. Each set of measurements collected and utilised to perform DT related tasks is denoted an experimental design [20].

The integration of clinical data into an LPM to form a DT, is a task denoted ‘the personalisation process’ (or ‘model personalisation’ or ‘model calibration’). Mathematically, this is also known as the ‘inverse problem’ [21]. One can think of the solution to the personalisation process as an input parameter set that locates the global minimum of a response surface, spanned by the combination of input parameters of the mechanistic model and the available clinical measurements. Thus, we obtain a set of unique clinical biomarkers [22], i.e., we have found a point in the input parameter space such that the outputs of our mechanistic model most closely match the clinical measurements of a patient. This is the point in the input parameter space which describes a patient’s patho/physiological state. However, calibration alone is insufficient to establish model validity. Rigorous validation across multiple operational points beyond the calibration condition is essential to ensure the DT will behave as the physical entity is extended to the broader patho/physiological state space. Without such a validation, one cannot distinguish between a genuinely predictive model and one that merely exhibits over-fitting to a single calibration point. This knowledge is vital as a DT must update this personalisation point every time new information is received. Therefore, the validation serves as a pillar of DT creation.

Despite progress, there are still many open questions surrounding the personalisation process, which we distil as explicit questions below:

1. What clinical data must be acquired in-vivo to obtain insightful, patient-specific biomarkers?

2. Does the set of biomarkers obtained remain consistent in the presence of new and varying experimental designs?

3. What is the computational cost associated with finding the solution of the personalisation process under different experimental designs?

4. Should DTs be built to encapsulate a patient’s ‘physiological envelope’ or should DTs be targeted to specific conditions?

5. What are the best practices involved in model personalisation under uncertainty?

This study investigates to what extent the above questions can be answered. We aim to provide a fundamental understanding of the sloppiness present within such models before being able to deploy a functioning DT in future work. Before proceeding, it is important to note that all investigations in this work are conducted with forward generated model data, in order to understand and extract clinical biomarkers from the model in an ideal setting (i.e. to eliminate any confounding effects of noise in clinical data). The synthetic data generated from our model here are guided by clinical practice and reflect the type and key features of data obtained in clinic. The investigations performed here reflect the common clinical pathways that are taken within the hospital. Without this critical, off-line investigation, misleading parameter selection from inappropriate experimental design which could then lead to ill-informed clinical decisions.

The structure of this paper is as follows. In section background, we: (i) review relevant literature, (ii) introduce concepts germane to the personalisation process in both extraction and optimisation of clinical biomarkers, (iii) detail the position this type of investigation has in the personalisation process and (iv) summarise the principal contributions of this work. In section methods, the mathematical detail is provided for each tool used for model analysis (Sobol indices, identifiability analysis and sloppy analysis). We also present the various experimental designs utilised within this work. Section results declares our results from different computational experiments. Discussion of the impact of varying experimental designs on the personalisation process is given in section discussion.

Foundations and state-of-the-art

Model personalisation is synonymous with the base concepts of input parameter sensitivity, identifiability and sloppiness [23]. From the discussion in section introduction, it is prudent to review terminologies, prior art and state how this work will provide a novel insight into the study of model personalisation.

Terminologies

When attempting to personalise a cardiovascular model, it is important to discuss the prerequisite properties corresponding to input parameter influence (sensitivity), uniqueness (identifiability) and response surface structure (sloppiness).

Sensitivity.

An input parameter’s effect needs to be influential on the output response surface, if this is the case an input parameter is regarded as sensitive [24], i.e., a change in the input parameter space causes a detectable change on the desired output. Thus, the said sensitive input parameter may serve as a clinically insightful biomarker for personalisation (in the creation of a digital twin), due to the ease of capturing the biomarkers’ effects in clinical outputs. One can distinguish locally and globally sensitive input parameters, with respect to the measurements. Locally sensitive input parameters are those eliciting the steepest gradient in the output about the model base operating point [25]. Globally sensitive input parameters are potential bio-markers which operate within a physiologically realistic value range. Input parameters are said to be most globally sensitive when they cause the greatest influence on the outputs, for the prescribed parameter ranges [26]. Different methods exist to calculate the sensitivity of input parameters, with the most common being the variance based methods, which we adopt in this work (see section sensitivity analysis). The personalisation process and the use of cardiovascular DTs is a global process, because we need a virtual representation of patients in a range of physiological and pathophysiological conditions. Thus, global sensitivity analysis presents itself as an insightful tool in the search for of clinical biomarkers.

Identifiability.

Personalisation of models now entails the pursuit of an identifiable model and identifiable input parameters (an optimal subset, denoted as clinical bio-markers). The analysis of identifiability in a cardiovascular system model requires three distinct examinations: structural, sensitivity-based, and practical identifiability. Structural identifiability (theoretical) assumes abundant and noise-free target output data, rendering a model’s structural identifiability largely academic in clinical terms. However, this assumption overlooks the possibility that inability to identify input parameters may stem from the model’s structure rather than data issues [27]. Naturally, if a model lacks structural identifiability, practical attempts at its utilisation are inherently limited. Sensitivity-based identifiability analysis involves the identification of sensitive and orthogonal input parameters, under synthetic data generated by the model, to ascertain which input parameters are identifiable under ideal circumstances [28]. Practical identifiability analysis takes into account of the quality of patient data, where noise and sampling rates may impact the identification of unique input parameters [29]. For complete personalisation, each stage must be executed sequentially. Within this work, we examine the identifiability of input parameters through their average influence across output space.

System sloppiness.

System sloppiness is a term used to characterise the structure of the input parameter space [30]. As discussed above, the main aim in many areas of systems biology is to optimise a dynamical system’s input parameters to available experimental data. This is normally performed by minimising a cost function, to obtain a point in the input parameter space corresponding to a global minimum of the cost function J [31], of the form

where is the input parameter set, represents the ith experimental measurement available and represents the ith dynamical system output (obtained from the model), which the experimental data are compared against.

Consider an example dynamical system with two input parameters p1 and p2. When optimising such a system to experimental data, contour plots displaying the closeness of fit are generated as in Fig 1. Here we see that moving up (in the direction of p2) and left in the parameter space rapidly changes the value of the cost function, i.e., indicating how good a fit is obtained by a specific value of p2. This direction is denoted a stiff direction in the input parameter space. Conversely, if one was to travel up and right (in the direction of p1), one could visit a range of p1 values without incurring changes of the cost function values. This means the manifold generated by p1 is largely linear whereas the one generated by p2 has steep gradients leading to a unique global minimum. Thus the sloppy direction controlled by p1 would not make for a good biomarker to calibrate a model due to the limited impact of p1 to the selection of model outputs. The converse is true for p2.

[Figure omitted. See PDF.]

Moving up and left would lead to rapid changes in the contour where as moving up and right would lead to slow changes.

However, most models in systems biology and in cardiovascular modelling cannot be visualised through a two dimensional contour map. Mathematical sloppiness analysis of our cardiovascular models can be found in section System Sloppiness.

The final stage of personalisation is the optimal estimation of the selected input parameters, fitted to patient-specific clinical data. This estimation may be iteratively updated within a DT as new information and data from the patient become available. Before the optimisation step takes place, the quantification of the stiffness/sloppiness of the system’s parameter space provides insight into the complexity of the system, and in turn facilitates the choice of an appropriate optimisation routine (e.g. gradient descent, particle swarm genetic algorithms, unscented Kalman filter etc.) [32, 33]. A sloppy system tends to have elongated or even flat valleys around the minimum, meaning that the minimum is not well-defined. Of course, when applying this method to patients whose measurements are noisy, the sloppiness of the model can vary significantly due to influence from the noise.

The related concepts of identifiability and sloppiness provide different but insightful information about the personalisation process [34]. Identifiability is a binary situation, whilst sloppiness quantifies the difficulty associated with obtaining precise identifiable input parameters. The sloppiness analysis of a model can either distinguish stiff and sloppy regions of the input parameter space, or show that the whole system under investigation can be regarded as sloppy. Note, most system biology models belong to the latter category [35, 36]. Within this work, we examine the sloppiness associated with the sensitivity matrices, which are defined by input parameter effects on the chosen outputs. Therefore, we can also establish a secondary aim of investigating the effects of differing experimental design on a cardiovascular system sloppiness.

Relevant literature

Cardiovascular model personalisation has been attempted in many clinically important areas such as, congenital heart disease, fetal circulation or whole heart multi-scale modelling [37–39]. In the majority of works, standard optimisation routines are used to obtain a set of input parameters which are representative of the experimental data [40–42]. Outside of the standard optimisation routines, data assimilation methods, namely ensemble and unscented Kalman filters, have developed traction as an efficient way to estimate patient specific input parameters, at a reduced cost, compared to the optimisation methods [33, 43, 44]. Another area which attracted research community’s attention recently is the utilisation of sensitivity analysis to guide the search and selection of an optimal input parameter subset for simpler and more efficient parameter estimations. Colunga et al. [45] applied this technique to incorporate invasive right heart data to obtain the personalisation of a model of pulmonary hypertension with 25 parameters. Where as Strocchi et al. [46] applied global sensitivity analysis to a 117-parameter cell-to-organ lumped parameter 4-chamber heart representation, reducing the model down to 45 personalisable parameters. Schafer et al. [47] examined how the sensitivities in a 1D model of the carotid artery change with respect to age and sex, highlighting how the input parameters for personalisation do not remain constant.

As discussed above, it is also important to understand the identifiability of input parameters, because this provides reassurance that any optimised input parameters are unique to a patient. Casas et al. [48] performed a profile likelihood analysis of a LPM to personalise flow in the systemic circulation. In comparison, other researchers such as Pironet et al., [49, 50] performed a structural and sensitivity identifiability analysis on a LPM to highlight what outputs would be required to obtain unique input parameters. In addition, there have been developments of experimental approaches from Marquis et al., de Bournonville et al. and Sala et al., who used invasive patient data to make good first estimates of model input parameters before optimisation which ensured more input parameters within the model are identifiable [51–53].

Another popular approach was to optimise input parameters in an iterative manner [54]. Bjordalsbakke et al. [55] applied an iterative step-wise reduction scheme in which, guided by sensitivity analysis, they began to optimise a group of parameters with increased number, each time with different cost functions to examine the closeness of fit. Bjordalsbakke [55] found that cost functions constructed from waveform data, as opposed to common clinical metrics, produced the smallest errors. Hann et al. [56], took a similarly structured approach in reducing the number of available outputs to reduce the complexity associated with the model. They demonstrated that differing measurement sets consisting of continuous and discrete measurements allowed for each subsection of the model to be optimised with minimal error. The impact of varying outputs have also been examined by Eck et al. [57] through uncertainty quantification in the arterial wall models and concluded that continuous time series Sobol indices gave a more insightful look into the process.

All studies above indirectly examined the impact of changing outputs on obtaining personalisable input parameter sets. However this was not the primary objective of their investigations. The most notable publication to date which investigated the impact of experimental design on a cardiovascular model was that of Colebank et al. [58]. They studied the impact of 4 different experimental designs on the ventricular function and found easily identifying biomarkers of the ventricular function, when a practical identifiability analysis is performed (including continuous data from both the left and right side of the heart).

Another aim of our study is to reveal and analyse the impact of experimental designs on system sloppiness. Sloppiness is a property which has been known in system biology models for over a decade [35]. Most studies have been focused on pharmacokinetic models [59] and examinations of the route cause of sloppiness. More recently, a formal definition of sloppiness was given and the concept was used to obtain a minimum set of outputs to ensure identifiability [60, 61]. The impact of experimental designs on system sloppiness has been understood through the lens of model identifiability (i.e., varying the model and the data shown to the model can induce different intensity of model sloppiness which in turn impacts the overall identifiability of the model parameters) [62–64]. In terms of cardiovascular modelling, sloppy analysis has been applied to electrophysiology modelling with the focus on calcium and potassium channel modelling. Whitterker et al. [65], utilised sloppy analysis to provide a method to simplify complex models of ion channels that improves parameter identifiability which will aid in future development in voltage-gated ion channels. Sloppy analysis has been applied to other classes of biochemical models [66, 67]. For example, Bravo et al. [36] applied sloppy analysis to a Bayesian model of an electrophysiological process to highlight how the identification of stiff parameter combinations made the model personalisation much simpler. As far as we are aware, sloppy analysis of a mechanical LPM of the cardiovascular system has not been performed, nor has the impact of varying experimental designs on sloppiness been studied.

Rationale and contributions

DTs in cardiovascular medicine offer profound promise in improving patient care. In order to advance the application of DTs in clinic, further study into the impact of experimental designs on the personalisation process must be understood. We investigate the impact of varying clinical metrics, both continuous and discrete, in an ideal scenario (without the bias of measurement noise). The main contributions of the work are:

1. Stability of identifiable parameters: Through varying experimental designs, we investigate changes in the identifiability ranking of input parameters.

2. Sloppy analysis of an LPM: We perform and report the first global sloppy analysis of a cardiovascular LPM to aid our understanding of the personalisation process.

3. Clinical metric guidelines: By evaluating different clinical metrics, we provide insight into the set of clinical data (and therefore measurements) needed for the effective personalisation of cardiovascular models.

By rigorously assessing the impact of varying clinical metrics and performing the first insightful sloppy analysis of a cardiovascular model, revealing the structure of the input parameter space, our work furthers the investigation and understanding needed in the personalisation of cardiovascular DTs.

Methods

Here, we examine the methods used for analysing a lumped parameter 4-chamber cardiovascular circulation model. In section methods, we present the model, explain the computational framework and provide a full parameterisation with their initial conditions. We then detail the clinical measures utilised in this work, which map to the different measurement sets in section Clinical Measures. The global sensitivity method of Sobol indices and the related Fisher information matrix are given in section sensitivity analysis and section FIM respectively. In section average influence, we then examine an average metric for input parameter influence. We detail how to examine system sloppiness in an n–dimensional space in section sloppy analysis. Finally, section workflow is devoted to highlight the iterative workflow we devised to examine the effects of varying experimental design on input parameter influence and sloppiness.

Cardiovascular model

Our LPM can be expressed in a standard state space formulation:

(1)

in which denotes an input parameter vector, represents the set of state variables of the system, is a function describing the system (usually this is an collection of differential algebraic equations), is the measurement function where the forward model synthetic measurements are generated, using the computed state variables , and represents the measurements of interest.

The model declared in its electrical analogue form in Fig 2 is a system-set, differential algebraic equation based, electrical analogue cardiovascular model, after Comunale et al., [68], with 4 heart chambers and a representation of both the systemic and pulmonary circulations. The model was first developed to model both physiological and pathophysiological states. The state variables of each compartment are specified by its time-dependant dynamic pressure P (mmHg), inlet flow Q (mL/s) and volume V (mL):

(2)

where la denotes the left atrium, lv denotes the left ventricle, sa the systemic arteries, svb the systemic vascular bed, sv the systemic venous system, ra the right atrium, rv the right ventricle, pa the pulmonary arteries, pvb the pulmonary vascular bed and pv the pulmonary venous system, Formally, t is a continuous time variable.

[Figure omitted. See PDF.]

Both left and right atria and ventricles are represented by the Shi double cosine model [69]. Heart valves are assumed to have Ohmic behaviour allowing no back flow. The systemic and pulmonary circulations are represented by a CRRCR combination with all parameters are given in Table 1.

In generic form, the equations relating to the passive compartmental state variables all take the form:

(3)

Above, the subscripts respectively represent the proximal, present and distal system compartments; (mL) denotes the circulating (stressed) volume [70] and C_k (ml/mmHg) and R_k (mmHgs/mL) denote compartmental compliance and the Ohmic resistance between compartments k, . Flow in and out of the active heart chambers are controlled by Ohmic diode valves:

(4)

where representing the resistances across aortic, mitral, pulmonary and tricuspid valves.

The active heart chambers can be represented by time varying elastances E_k(t) (mmHg/ml), which determines the change in pressure for a given change in the volume [70]:

(5)

where & represent the chamber unstressed and stressed volumes for the two ventricles and atria. E_k(t) is written following [69]:

(6)

where is the activation function for the heart chamber and is parameterised by the end systolic and end pulse timing parameters and respectively.

Table 1 provides the parameterisation for the model to produce physiological results for the lumped parameter 4-chamber model in which the input parameter space has a dimensionality of 36. To construct the model, the cardiovascular LPM Julia package CirculatorySystemModels.jl [71] is utilised which reduces the system to 8 ordinary differential equations in volume with the initial volumes also given in Table 1. The model is solved utilising the Vern7 [72] algorithm with tolerances of . For 30 cardiac cycles the model computes in 0.064s.

[Figure omitted. See PDF.]

Clinical measures

When varying the experimental design, we generate synthetic data based on medically accurate measures utilised in the diagnoses of cardiovascular diseases. This enables us to perform such a large computational investigation, and at the same time, to follow the standard clinical pathway through an iterative scheme. To investigate the effect of experimental design, we devise an additive algorithm: each time we move to the next measurement set, the new measurement is added to the previous output set and therefore defining the new output space of an increased dimension, for the analysis of input parameter effects. Practically, the below model outputs represent the conventional medical tests which a patient may be subjected to, with increasing invasiveness to assess their cardiovascular health. While we do not target a specific health condition in this work, the increasing output sets represent further and deepening knowledge about a patient’s physiological envelope.

In Tables 2, 3, and 4, we display the measurement sets for the cases of discrete, continuous and mixed measurements. These will then be utilised to perform the variety of investigations as described below and pictured graphically in Fig 3.

[Figure omitted. See PDF.]

[Figure omitted. See PDF.]

[Figure omitted. See PDF.]

[Figure omitted. See PDF.]

Discrete measurements.

In the discrete case in Table 2, we utilise only single point metrics. These metrics can be obtained through just 3 clinical tests:

1. Blood Pressure (BP): This can be readily obtained through a sphygmomanometer reading while a patient is in hospital. In our chosen model, this measurement is obtained by calculating and corresponds to set 1.

2. Ejection Fraction (EF): This can be obtained through an echocardiogram. In our model, we calculate EF for the left and right ventricle then the left and right atria as . For this corresponds to sets 2A, 2B, 2C and 2D.

3. Max(Q_i) - Maximum flow: This could be obtained from either an echocardiogram or a cardiac MRI. This is calculated for the systemic arteries, pulmonary arteries, aortic valve, mitral valve, pulmonary valve and tricuspid valve.

To highlight the additive process of the experiment in the discrete setting, for example, the full output set for 3A is defined as follows:

Continuous measurements.

For the continuous measurements displayed in Table 3, each continuous waveform obtained is made up of 150 time points. This metrics can be obtained though 4 clinical metrics below.

1. Q_i - Flow rate: This can be obtained through a Doppler ultrasound, for the systemic, pulmonary, aortic valve, mitral valve, pulmonary valve and tricuspid valve.

2. - Chamber volume: can be obtained through a cardiac MRI, for the two ventricles and two atria.

3. - Left heart pressures: These invasive diagnostic measurements can be obtained through catherterisation for the left ventricle and systemic artery.

4. - Right heart and circulation pressures: These invasive measurements can be obtained by performing Swan-Ganz catherterisation and wave form pressures are collected in the right heart for the right ventricle, pulmonary artery, right atrium and pulmonary vein (which can also be seen as a surrogate for left atrial pressure).

Mixed measurement sets.

The previous two measurement settings will reveal the difference between continuous and discrete metrics. The mixed measurement set combines both the discrete and continuous measurements but represents a standard diagnosis procedure with increasing invasiveness, i.e., in clinic, a patient would not be subject to invasive chamber pressure measurements unless deemed necessary. Apart from one additional measurement (noisy blood pressure) which will be introduced bellow, all other metrics and corresponding measurement sets are the same as the ones defined in sections Discrete and Continuous Measurements.

* BPN - noisy blood pressure: This set is added to represent the situation of a patient taking their own arterial blood pressure measurement at home, with noisy reading due to potential human error and lower device accuracy. The analysis of noisy output data will provide insight into how global sensitivity indices alter in the presence of noise.

BPN is calculated as

In this setting, set 2 represents an arterial blood pressure measurement obtained in hospital and is assumed to not be subject to noise. Slightly different to the measurement sets introduced in sections Discrete and Continuos Measurements, here set 2 will replace set 1 instead of adding to it and set 2 will be used in all increasing measurement sets for the later sets.

Sobol indices

Given a model of the form in eqn 1 with Y (a continuous or discrete output), a variance based first order or total order effect can be calculated for a generic input factor p_i. denotes the complementary set, i.e., all other model inputs excluding p_i. Performing a Sobol analysis provides the quantification of the input parameter effect against a specific output [73]. Both the first and total order sensitivity indices return a matrix of the form:

(7)

where n and m represent the number of input parameters and output measurements, respectively. In this work, we handle a fixed high-dimensional input parameter space (n = 36). However, when the experimental design changes, m varies and also the output measurement space. For example, in the discrete measurement setting, the largest value of m is 11. In comparison, in the mixed measurement setting, the largest output set (set 7D) results in m = 1609, so the resulting sensitivity matrix is of size .

The first and total order sensitivity indices can be written as:

(8)

where S_1,i, S_T,i denote the first and total order indices’ vectors for an input parameter p_i against the specific output Y. In order to quantify the effects continuous measurements have on the calculation of sensitivity indices, we typically average this sensitivity waveform. Rather than averaging across a time range (which process regions of low variance equally to those of high variance), one seeks to expose differential sensitivities by examining variance-weighted averages:

(9)

where TAS_i is the time averaged first/total order effect of an input parameter i and represents the approximated continuous measurement at time step k [57].

The sensitivity indices can be interpreted as:

i.e., for a given input parameter p_i, the total order indices are the first order indices (p_i’s independent effects) plus all higher order interactions. This study utilises the total order sensitivity matrix to quantify an input parameter’s full impact on the outputs. To ensure convergence, we used 75,000 samples with the Jansen estimator [74, 75] with a bootstrapping sample of 1000 [76]. In this work we prescribe a uniform distribution on the inputs with the upper and lower bounds for the parameters being at , from the values shown in Table 1, and we generate a quasi-Monte carlo Sobol sequence to improve the convergence of the Sobol indices [75].

Fisher information

Another important matrix derived from the sensitivity matrices is the square () Fisher information matrix (FIM) [77]:

(10)

The FIM is a symmetric matrix representing the information one can extract on parameters from the model outputs (i.e., the available measurements [78]). We choose to construct the FIM from the total order Sobol indices eqn 8 to account for the full non-linear effects which are present within the system and the impacts of the varying experimental designs as done in previous work [25, 79, 80].

Average parameter influence

The sensitivity vectors derived in section Sensitivity Analysis only display the effects of an input parameter against a specific measurement. In order to obtain an input parameter’s identifiability, we must assess an input parameter’s influence across our chosen set of outputs. Li et al., [81], derived such a metric based upon the FIM defined in eqn (10). To examine the global effect of input parameters through the FIM, we use principal component analysis (PCA) [82] where the principal components (PC) are the eigenvectors of the FIM.

Let be the matrix of the ordered PC (eigenvectors of ), in which the absolute value of each element Q_ij reflects the contribution of the parameter to the variance of the output. We follow Li et al. [81], who measure an overall effect for the parameter as:

(11)

where and represents the non-zero eigenvalues of . This measure reflects the difficulty in determining the parameter when only a single factor is estimated. Parameter identifiability is associated with E_i - the larger the value of E_i, the more identifiable the parameter is. We record the rank and the overall effect of every input parameter greater than 0.01, a number which has been discussed as the lowest possible value that may have the possibility of being identifiable in [25].

Sloppiness analysis

In section System Sloppiness, we explored the concept of sloppiness by examining the contour lines of the cost function. In order to examine sloppiness in an n-dimensional input parameter space, we examine the eigenvalues of FIM eqn 10. The eigenvalues of the FIM provide insight into the variation of parameters constrained by the available data. A model can be regarded as sloppy if the eigenvalues of the FIM have a uniform spacing on a log scale spread over many orders of magnitude [30, 63, 83, 84]. On the other hand, if the FIM eigenvalues have a non-uniform distribution, the model is regarded as stiff. We can then identify stiff directions in the input parameter space which corresponds to a set of input parameters where personalisation should take place.

One can interpret this analysis as follows, the eigenvalues of the FIM represents the variation that a model parameter contributes the model outputs. In the case where a model is stiff we have a select subset of model parameters with large eigenvalues compared to their complementary set. Thus when looking to calibrate model parameters this stiff subset of input parameters denote rapidly varying directions on the response surface, thus one can more easily obtain a personalised operating point of input parameters, due to the impact of varying input parameter sets been clear. Conversely in the case of sloppy models these properties are not clear in the response surface thus choosing the correct direction for personalisation more difficult.

Workflow

All the above sections define an iterative investigation in which we examine the effects of varying experimental designs on input parameter’s influence and sloppiness. This can be encapsulated in the following steps and in Fig 3.

1. Define an output set: As shown in section Clinical Measures, we define the various sets of discrete, continuous and mixed measurements, starting from the first and simplest output set.

2. Calculate S_T: Form the total order sensitivity matrix for the input parameters and chosen output set.

3. Form the FIM: Utilising eqn 10, the FIM represents all the information about the parameters constrained by the specific measurement set.

4. Analyse parameter’s influence and sloppiness: Using methods in section Average Parameter Influence and Sloppy Analysis, we record each input parameter’s rank and influence value being greater than 0.01 and the distribution of eigenvalues.

5. Add a new measurement: Move to the next measurement set and repeat stages 2-4.

Results

Sections Results Discrete, Continuos and Mixed detail the average input parameter influence across all outputs for varying experimental design, using the method presented in section Average Parameter Influence. The average input parameter rank and values are displayed as tables. The corresponding fig in each section displays the eigenvalues of the Fisher information matrix on a scale, for varying measurement sets (see sections Fisher Information and Sloppy Analysis). The sections below presents results for the varying discrete, continuous and mixed measurement sets, respectively, as described in section Clinical Measures.

Discrete measures

Table 5 displays the rank of each input parameter and their corresponding influence value calculated using eqn 11, using discrete measurements. We note, from set 1 to set 3F, the resistance of the systemic vascular bed and the systemic arterial compliance C_sa rank the highest, with the largest influence values, with the exception from set 2D in which E_shift,ra ranks the most influential. This can be explained by the experimental design. The newest measurement added for set 2D was the ejection fraction of the right atrium. Although in all the other cases, it appears and C_sa still dominate. As more measurements are added to the experimental design, we observe more input parameters record an influence score greater than 0.01. For case 3F where there are 11 outputs, 17 input parameters are recorded with an influence score larger than 0.01. As the measurement set increases, the largest influential value decreases. In addition, as more measurements are added, although more “influential" parameters are obtained, the majority have an influence score in the range of hundredths.

[Figure omitted. See PDF.]

The sloppiness analysis result in Fig 4 shows that the model cannot be regarded as sloppy, with a discrete output set. With an increasing output set, we observe more input parameters are regarded as stiff. Even with the largest output set, set 3F, the model still exhibits an eigenvalue spectrum of over 15 orders of magnitude.

[Figure omitted. See PDF.]

Continuous measures

With an increasing continuous measurement set, in Fig 6, a much higher number of influential input parameters are present, compared to the discrete case (Fig 5). Even with just a single continuous measurement of the systemic flow (Column 1A), 17 input parameters are regarded as influential. In set 4D, where there are 16 continuous output measurements, 20 input parameters are recorded as influential. Here, the ranking of influential input parameters shows much less consistency, compared to Table 5. The most influential parameters appear to loosely correlate with the latest output added to the measurement list. For example, in set 3B, the left ventricular pressure and systemic arterial pressure are new additions to the output set, and then the top ranking parameters are the minimal ventricular elastance and the end pulse time for the left ventricle . Where as in set 4C, pressures associated with the pulmonary system have just been added, then the top ranking parameters are the right atrial activation time E_shift,ra and the minimal elastance for the right ventricle . Despite the change in rankings, we note that a similar set of input parameters are recorded as influential input parameters, with just minor changes when new output measurements are added. As observed in the discrete measurement set, when more input parameters are regarded as influential, the concentration of influence decreases and is more evenly spread between the input parameters with an influence value greater than 0.01.

Fig 5 shows that for any continuous measurement set, the system can be regarded as sloppy. When increasing the output measurements, the set of sloppiness decreases, with the eigenvalue spectra decreasing from a range of 10⁻¹⁶ to 10⁻⁶. Compared to the discrete sloppy analysis (Fig 4), input parameters in the stiff direction exhibit larger values than observed previously.

[Figure omitted. See PDF.]

Mixed measures

When combing both discrete and continuous measurements, the results in Table 7 show a similar structure to that observed when only continuous measurements are utilised. When only BPN is the only output, all input parameters record an influence greater than 0.01, despite the noise, there are still clear influence parameters ( and ) which could be regarded as biomarkers. However, when we introduce the noise free BP, the influence values associated with the biomarkers increase largely. At set 4A, the first continuous measurement is introduced alongside the previous discrete ones, as a consequence we observe the number of input parameters with influence greater than 0.01 grows from 9 in set 3D to 17 in set 4A. In set 7D which contains all discrete and continuous measurements, the exact same ranking as set 4D in Table 6 displays, although the values of influence vary slightly. This indicates that continuous measurements dominate, when obtaining influential input parameters for a set of measurements. This pattern is also present in Fig 6, where once continuous measurements are introduced to the output list, sloppiness appears and the eigenvalues are greater than 10⁰ with an eigen-spectrum ranging from 10⁻¹² for set 4B decreasing down to 10⁻⁶ for set 7D. In sets 5A to 5D, the maximum flow in each chamber compartment was added to the optimisation process as a discrete measurement. Despite discrete measurements being more numerous than continuous ones in our analysis, we acknowledge that mathematically, a continuous measurement is, in principle, infinite-dimensional. While clinically these are often considered as independent objects which contain different information, we recognise that continuous functions typically require multi-dimensional representations (unless constant). The addition of these discrete values had no influence on the ranking or sensitivity of the most influential parameters. It is important to note, however, that the maximum value of flow is contained in the continuous measurement, so in effect no new information is provided to the solver; rather, a weighting on the maxima of the flow is applied.

[Figure omitted. See PDF.]

The delineation line between 3D and 4B denotes the stage at which continuous measurements are added to the system.

[Figure omitted. See PDF.]

[Figure omitted. See PDF.]

Discussion

Our study aims to assess the impact of experimental design on the input parameter influence and the system sloppiness. Overall, the results largely agree with previous work: continuous measurements lead to a larger selected subset of input parameters as prime candidates for personalisation in a cardiovascular DT [45, 55, 58]. When only discrete measurements are used, there is a smaller and more concentrated subset of identifiable input parameters. Perhaps surprising is the quantitative extent of this disparity. Only when the largest discrete measurement set, 3F, is examined, we obtain the same number of input parameters with a value greater than 0.01, compared to the first continuous measurement set 1A. It is important to note that these findings are specifically applicable to the LPM digital twin with Julia implementation in this work. In accordance with established verification and validation (V&V) frameworks, such as ASME V&V40 [85], we acknowledge that computational models inherently contain model uncertainty—uncertainty associated with the structure, assumptions, and limitations of the computational model itself. Without comprehensive code verification and validation procedures, parameter sensitivity results may be influenced by coding errors or flawed hypotheses rather than true physiological relationships. This limitation underscores the necessity of implementing rigorous V&V protocols in future work to distinguish between genuine parameter insensitivity and artefacts introduced by model implementation errors.

We also observe that as the size of the output set increases, the influence between input parameters appears to become more evenly distributed. For example, for the discrete measurements results shown in Table 5, the systemic vascular bed resistance and arterial compliance C_sa rank as the most influential parameters for all measurement sets except 2D. In set 1, C_sa and have influence values of 0.8 and 0.6 respectively. However, in set 3F, when the dimensionality of output space increases to 11, these two parameters’ influence values decrease to 0.11 and 0.13.

The first sloppy analysis of this cardiovascular model indicates that discrete measurements do not introduce sloppiness into the system, whereas for continuous measurements, the system begins to exhibit sloppy behaviour. Through the perspective of creating DTs, the stiff input parameters are clearly identified using discrete measurements which would lead to easier identification of a personalised global minimum parameter set. When using the combination of both the continuous and discrete measurement sets, as they increase in size, the number of stiff input parameters which can be considered as prime candidates for personalisation increases. Sloppiness provides a view into the structure of the input parameter space and an insight for why more “influential” input parameters appear when the dimensionality of the output grows. As more measurements are added, there is a noticeable change in the structure of the response surface, providing more guidance towards the personalisable global minimum. It should be noted that as more and more continuous signals are added, more outputs of the model are constrained. Therefore the increasing number of equally significant parameters is unsurprising. Whilst potential cross-correlation amongst measurements might seem a plausible explanation for this phenomenon, our observations suggest otherwise; parameter significance changes align with the physiological subsystems being measured rather than reflecting mere statistical redundancy, indicating genuine additional model constraint. For specific use cases, this suggests that, where multiple data are available, a weighting of the most important clinical features may be required in the optimisation function to identify the required biomarkers.

When creating a virtual representation of a patient, it is still an open question whether the DT should be personalised to a specific condition or encapsulate the full physiological envelope of a patient [2, 10, 23]. Our experiments and analysis provides an insight to this question: if one wishes to capture a full physiological envelope through the DT, a number of continuous measurements are essential. This is due to the larger number of influential input parameters, along with the higher values of stiff eigenvalues, when compared to the discrete measurement setting. This approach brings practical problems of course, because of the invasive nature associated with obtaining some continuous measurements (for example, ventricular pressure). A patient would have to be subject to a series of invasive tests with associated risks, in order to generate the data to for a personalised DT. Alongside this, continuous measurements have shown a higher set of sloppiness, indicating that a computationally expensive optimisation routine may have to be utilised to generate the virtual patient representation. Conversely, if the purpose of a DT is to target specific conditions, a set of non-invasive discrete measurements poses as an alternative. Although there is a smaller number of identifiable input parameters in this case, the influence is concentrated strongly around the biomarkers relevant to the discrete metrics. In addition, because the system does not exhibit sloppy behaviour, the personalisation process using discrete measurements may be more efficient than its continuous counterparts. Additionally, continuous measurements taken clinically are also susceptible to noise from several sources, including equipment accuracy and differences in method between operators. In the case of time series data, noise can be present at varying sets at each time step, and therefore it is expected that noise in continuous data will have an increased effect on model sloppiness and parameter identifiability.

One problem with sloppy analysis is the subjectivity in diagnosing whether sloppiness is present within a system. In this work, we have used the common definition of evenly spaced eigenvalues on a logarithmic scale, distributed over a minimum range of 6 orders of magnitude [61, 63]. The absence of sloppiness is evident in the discrete measurements setting (see Fig 4). In the case of continuous measurements, as the measurement set increases, the distribution of eigenvalues (Fig 5), whilst still evenly spaced, reduces from a spread of 10¹⁶ to just 10⁶ (i.e., more input parameters align with the stiff direction than before). But does this apparent reduction in sloppiness align with intuition? Given the increase of parameters in the stiff direction, one would expect more accurate optimisation of the input parameters when compared to set 1. However, this remains an open question and requires further study to investigate. This discrepancy highlights a fundamental challenge in sloppy analysis: the quantitative metrics we employ may not always capture the qualitative behaviour that practitioners intuitively expect from the system.

The study of sloppiness is common practice in most other areas of systems biology, however, this is not the case for cardiovascular models. The concept of sloppiness provides an important insight for examining the personalisability of cardiovascular models. By assessing the stiff and sloppy directions generated from the input parameters, sloppy analysis provides an alternative approach to identify optimal subsets for personalisation, compared to other methods such as profile likelihood and combining sensitivity and orthogonality [58, 79, 86]. This is an interesting area which should be explored in future research. When attempting to personalise a DT, there are several stages and sloppy analysis belongs to the vital off-line stage in which prime candidates for personalisation are identified. This off-line stage enables us to identify biomarkers for which can be personalised to produce the virtual representation of a patient. The off-line stage is vital because once patient data are introduced, any additional issues occurring during personalisation can then be attributed to issues within the clinical data. If we examine the parameters in Figs 4–6 we observe that the rankings obtained by the analysing their influence (Tables 5–7) are the same rankings obtained through the sloppy analysis. So in the discrete setting the systemic vascular bed resistance and the arterial compliance C_sa exhibit the largest eigenvalues making them the stiffest parameters in that setting. Thus these parameters are prime candidates for personalisation.

For the personalisation of cardiovascular DTs, the process in which this happens must operate on a multi-dimensional input parameter space in which some points give accurate representation of a patient’s physiological and pathophysiological state. Currently, much analysis on the input parameter space and the identification of the optimal parameter subset for personalisation are conducted on a local basis [25, 87, 88]. For example, it is still the norm to form the sensitivity matrices through local methods when analysing system sloppiness [35, 36]. If sloppy analysis is to be utilised more in the identification of biomarkers, local analysis should not be adopted for larger, more complex circulatory models. Personalisation is a global process, therefore it is vital to understand and quantify the global behaviour and the structure present within the input parameter space. This is why we have conducted our sloppiness analysis using the global sensitivity analysis outcomes in this work. Our workflow could be enhanced by incorporating alternative physiologically-informed priors. When certain parameter combinations produce unrealistic outputs (such as cardiac output or pressures outside clinically accepted ranges), constraints based on population data could be implemented. By leveraging known physiological distributions of outputs from clinical cohorts, we could work backwards to infer suitable distributions of input parameters that would generate these observed output patterns. This Bayesian approach would allow sampling from a more realistic parameter space, potentially improving the efficiency of the personalisation process. Such physiologically-constrained priors would be particularly valuable when working with continuous measurement sets that induce system sloppiness, as they would help narrow the solution space towards clinically relevant parameter combinations. This refinement represents a natural extension of our workflow that maintains the global sensitivity analysis framework while incorporating domain knowledge about cardiovascular physiology. In this work, we have avoided such a task due to the bias which would be introduced into the input space when trying to identify such physiological regions. This bias is likely to break the orthogonality conditions which are present within the calculation of Sobol indices [89, 90]. We will consider and investigate this in future work.

In order to perform such an extensive study, the associated computational expense is another important factor to consider. In total, we have tested 48 individual experimental designs, for each of which a sensitivity analysis has been performed with 75,000 samples to ensure convergence [75]. As our chosen global sensitivity method is Sobol indices [25, 53, 79], this means for each experimental design, model evaluations are required. Thus for 48 independent experimental designs, we have solved the lumped parameter 4-chamber model 136.8 million times. This study has only been made feasible due to the superior computational speed exhibited by DifferentialEquations.jl within Julia in solving the dynamical system for a single model run including 30 cycles took 0.060246(s). Another approach would be to utilise emulation which is also likely to significantly improve the computational time [46]. However, when emulating such a model, a smooth assumption on the input space is applied which can make identifying unphysiological or unstable regions challenging. When personalising DTs, computationally efficient and accurate tools should be utilised where possible, for the most effective allocation of computational resources for all stages of DT development. Of further interest in the development of LPMs in clinical use is the ability to validate the results of the model. Validation must be suitable for the context of use [91], and therefore also plays a key factor in experimental design, defining the clinical measurements taken and the choice of model output. Experimental design is therefore constricted not only by model behaviour as analysed here, but by clinical requirements. The requirement for testing in the context of use is one of the limitations of general cardiovascular models, as each use case must be included in the validation.

Alongside LPMs, there is also extensive research in higher dimensional (e.g. 1D, 2D and 3D) cardiovascular models which can be utilised as DTs [9, 92, 93]. The set of physiological details in these models is usually far superior to what can be established in LPMs. The main drawback or compromise, is the lack of ability to simulate global haemodynamics because of the astronomical computational cost. If we were able to create a full cardiovascular circulation representation, which could adapt to pathophysiological states, we would then be able to observe and predict other circulatory diseases, on top of the one in which the current condition occurs. Physiologically detailed models of a single piece of vessel or a compartment are of course of great importance, to further biological understanding where invasive clinical assessments are inappropriate or unethical. One promising area of the cardiovascular digital twin development is in the creation of multi-scale, multi-modal models, combining both LPMs and physiologically detailed higher dimensional representations of specific vessels or valves [46, 94]. This approach combines the advantages of both modelling domains and forms an attractive avenue for future research in cardiovascular personalisation and building DTs.

Conclusion

Our study highlights the importance of the experimental design for the quantification of input parameter influence and the associated sloppiness, for a lumped parameter personalised cardiovascular digital twin. Using a realistic lumped 4-chamber 36-parameter LPM as a test bench, we investigated 48 independent experimental designs. The most significant findings, corresponding to the ones identified in Section Rationale and Contributions, are: (i) Input parameter identifiability is not consistent when subject to varied measurement data and depends on the chosen experimental design. (ii) Sloppiness is present in LPMs, when the chosen experimental design contains continuous measurements. (iii) The personalisation of a digital twin to encompass a person’s complete physiological envelope necessitates invasive tests to obtain continuous measurements. Although this approach offers an increased number of identifiable parameters with potentials to be biomarkers, it comes at the expense of a sloppy system which in turn increases the difficulty in parameter identification. Conversely, discrete metrics may provide a simpler personalisation approach, yielding less identifiable but more targeted biomarkers, due to the absence of sloppiness in the system.

References

1. 1. Committee ADEI, et al. Digital Twin: Definition & Value—An AIAA and AIA Position Paper. Reston, VA, USA: AIAA. 2020.

2. 2. Trayanova NA, Prakosa A. Up digital and personal: How heart digital twins can transform heart patient care. Heart Rhythm. 2024;21(1):89–99.

* View Article

* Google Scholar

3. 3. Gillette K, Gsell MAF, Prassl AJ, Karabelas E, Reiter U, Reiter G, et al. A Framework for the generation of digital twins of cardiac electrophysiology from clinical 12-leads ECGs. Med Image Anal. 2021;71:102080. pmid:33975097

* View Article

* PubMed/NCBI

* Google Scholar

4. 4. Coorey G, Figtree GA, Fletcher DF, Snelson VJ, Vernon ST, Winlaw D, et al. The health digital twin to tackle cardiovascular disease-a review of an emerging interdisciplinary field. NPJ Digit Med. 2022;5(1):126. pmid:36028526

* View Article

* PubMed/NCBI

* Google Scholar

5. 5. Viola F, Del Corso G, De Paulis R, Verzicco R. GPU accelerated digital twins of the human heart open new routes for cardiovascular research. Sci Rep. 2023;13(1):8230. pmid:37217483

* View Article

* PubMed/NCBI

* Google Scholar

6. 6. Balogh EP, Miller BT, Ball JR. Technology and tools in the diagnostic process. 2015.

7. 7. Tlałka K, Saxton H, Halliday I, Xu X, Taylor D, Narracott A. In: International Conference on Computational Science, 2024. 173–87.

8. 8. Tlałka K, Saxton H, Halliday I, Xu X, Narracott A, Taylor D, et al. Sensitivity analysis of closed-loop one-chamber and four-chamber models with baroreflex. PLoS Comput Biol. 2024;20(12):e1012377. pmid:39715272

* View Article

* PubMed/NCBI

* Google Scholar

9. 9. Niederer SA, Lumens J, Trayanova NA. Computational models in cardiology. Nat Rev Cardiol. 2019;16(2):100–11. pmid:30361497

* View Article

* PubMed/NCBI

* Google Scholar

10. 10. Hose DR, Lawford PV, Huberts W, Hellevik LR, Omholt SW, van de Vosse FN. Cardiovascular models for personalised medicine: Where now and where next?. Med Eng Phys. 2019;72:38–48. pmid:31554575

* View Article

* PubMed/NCBI

* Google Scholar

11. 11. Shi Y, Lawford P, Hose R. Review of zero-D and 1-D models of blood flow in the cardiovascular system. Biomed Eng Online. 2011;10:33. pmid:21521508

* View Article

* PubMed/NCBI

* Google Scholar

12. 12. Westerhof N, Lankhaar JW, Westerhof BE. The arterial windkessel. Medical & Biological Engineering & Computing. 2009;47(2):131–41.

* View Article

* Google Scholar

13. 13. Westerhof N, Stergiopulos N, Noble MI, Westerhof BE. Snapshots of hemodynamics: an aid for clinical research and graduate education. Springer. 2018.

14. 14. Corral-Acero J, Margara F, Marciniak M, Rodero C, Loncaric F, Feng Y. The ‘Digital Twin’ to enable the vision of precision cardiology. European Heart Journal. 2020;41(48):4556–64.

* View Article

* Google Scholar

15. 15. Murphy SP, Ibrahim NE, Januzzi JL Jr. Heart Failure With Reduced Ejection Fraction: A Review. JAMA. 2020;324(5):488–504. pmid:32749493

* View Article

* PubMed/NCBI

* Google Scholar

16. 16. Dart AM, Kingwell BA. Pulse pressure--a review of mechanisms and clinical relevance. J Am Coll Cardiol. 2001;37(4):975–84. pmid:11263624

* View Article

* PubMed/NCBI

* Google Scholar

17. 17. Gatehouse PD, Keegan J, Crowe LA, Masood S, Mohiaddin RH, Kreitner K-F, et al. Applications of phase-contrast flow and velocity imaging in cardiovascular MRI. Eur Radiol. 2005;15(10):2172–84. pmid:16003509

* View Article

* PubMed/NCBI

* Google Scholar

18. 18. Alhashemi JA, Cecconi M, Hofer CK. Cardiac output monitoring: an integrative perspective. Crit Care. 2011;15(2):214. pmid:21457508

* View Article

* PubMed/NCBI

* Google Scholar

19. 19. Sagawa K. The ventricular pressure-volume diagram revisited. Circulation Research. 1978;43(5):677–87.

* View Article

* Google Scholar

20. 20. Kirk RE. Experimental design. Sage handbook of quantitative methods in psychology. 2009. p. 23–45.

21. 21. Tarantola A. Inverse problem theory and methods for model parameter estimation. SIAM. 2005.

22. 22. Strimbu K, Tavel JA. What are biomarkers?. Current Opinion in HIV and AIDS. 2010;5(6):463.

* View Article

* Google Scholar

23. 23. Huberts W, Heinen SG, Zonnebeld N, van den Heuvel DA, de Vries JPP, Tordoir JH. What is needed to make cardiovascular models suitable for clinical decision support? A viewpoint paper. Journal of Computational Science. 2018;24:68–84.

* View Article

* Google Scholar

24. 24. Saltelli A, Tarantola S, Campolongo F, Ratto M, et al. Sensitivity analysis in practice: a guide to assessing scientific models. Wiley Online Library. 2004.

25. 25. Olsen CH, Ottesen JT, Smith RC, Olufsen MS. Parameter subset selection techniques for problems in mathematical biology. Biol Cybern. 2019;113(1–2):121–38. pmid:30377765

* View Article

* PubMed/NCBI

* Google Scholar

26. 26. Saltelli A, Ratto M, Andres T, Campolongo F, Cariboni J, Gatelli D. Global sensitivity analysis: the primer. John Wiley & Sons. 2008.

27. 27. Bellman R, Åström KJ. On structural identifiability. Mathematical biosciences. 1970;7(3–4):329–39.

* View Article

* Google Scholar

28. 28. Iliadis A. Structural identifiability and sensitivity. J Pharmacokinet Pharmacodyn. 2019;46(2):127–35. pmid:30895420

* View Article

* PubMed/NCBI

* Google Scholar

29. 29. Wieland FG, Hauber AL, Rosenblatt M, Tönsing C, Timmer J. On structural and practical identifiability. Current Opinion in Systems Biology. 2021;25:60–9.

* View Article

* Google Scholar

30. 30. Daniels BC, Chen Y-J, Sethna JP, Gutenkunst RN, Myers CR. Sloppiness, robustness, and evolvability in systems biology. Curr Opin Biotechnol. 2008;19(4):389–95. pmid:18620054

* View Article

* PubMed/NCBI

* Google Scholar

31. 31. Itu LM, Sharma P, Suciu C. Patient-specific hemodynamic computations: application to personalized diagnosis of cardiovascular pathologies. Springer. 2017.

32. 32. Kochenderfer MJ, Wheeler TA. Algorithms for optimization. Mit Press. 2019.

33. 33. Saxton H, Schenkel T, Halliday I, Xu X. Personalised parameter estimation of the cardiovascular system: leveraging data assimilation and sensitivity analysis. Journal of Computational Science. 2023;74:102158.

* View Article

* Google Scholar

34. 34. Chis O-T, Villaverde AF, Banga JR, Balsa-Canto E. On the relationship between sloppiness and identifiability. Math Biosci. 2016;282:147–61. pmid:27789352

* View Article

* PubMed/NCBI

* Google Scholar

35. 35. Gutenkunst RN, Waterfall JJ, Casey FP, Brown KS, Myers CR, Sethna JP. Universally sloppy parameter sensitivities in systems biology models. PLoS Comput Biol. 2007;3(10):1871–8. pmid:17922568

* View Article

* PubMed/NCBI

* Google Scholar

36. 36. Monsalve-Bravo GM, Lawson BAJ, Drovandi C, Burrage K, Brown KS, Baker CM, et al. Analysis of sloppiness in model simulations: Unveiling parameter uncertainty when mathematical models are fitted to data. Sci Adv. 2022;8(38):eabm5952. pmid:36129974

* View Article

* PubMed/NCBI

* Google Scholar

37. 37. Carson J, Warrander L, Johnstone E, van Loon R. Personalising cardiovascular network models in pregnancy: A two-tiered parameter estimation approach. Int J Numer Method Biomed Eng. 2021;37(11):e3267. pmid:31799783

* View Article

* PubMed/NCBI

* Google Scholar

38. 38. Salvador M, Kong F, Peirlinck M, Parker D, Chubb H, Dubin A. Digital twinning of cardiac electrophysiology for congenital heart disease. Biorxiv. 2023;:2023–11.

39. 39. Gerach T, Schuler S, Fröhlich J, Lindner L, Kovacheva E, Moss R. Electro-mechanical whole-heart digital twins: a fully coupled multi-physics approach. Mathematics. 2021;9(11):1247.

* View Article

* Google Scholar

40. 40. Myers TG, Ribas Ripoll V, Sáez de Tejada Cuenca A, Mitchell SL, McGuinness MJ. Modelling the cardiovascular system for assessing the blood pressure curve. Math Ind Case Stud. 2017;8(1):2. pmid:28680510

* View Article

* PubMed/NCBI

* Google Scholar

41. 41. Duanmu Z, Yin M, Fan X, Yang X, Luo X. A patient-specific lumped-parameter model of coronary circulation. Sci Rep. 2018;8(1):874. pmid:29343785

* View Article

* PubMed/NCBI

* Google Scholar

42. 42. Chakshu NK, Sazonov I, Nithiarasu P. Towards enabling a cardiovascular digital twin for human systemic circulation using inverse analysis. Biomech Model Mechanobiol. 2021;20(2):449–65. pmid:33064221

* View Article

* PubMed/NCBI

* Google Scholar

43. 43. Canuto D, Pantoja JL, Han J, Dutson EP, Eldredge JD. An ensemble Kalman filter approach to parameter estimation for patient-specific cardiovascular flow modeling. Theoretical and Computational Fluid Dynamics. 2020;34:521–44.

* View Article

* Google Scholar

44. 44. Pant S, Corsini C, Baker C, Hsia T-Y, Pennati G, Vignon-Clementel IE, et al. Data assimilation and modelling of patient-specific single-ventricle physiology with and without valve regurgitation. J Biomech. 2016;49(11):2162–73. pmid:26708918

* View Article

* PubMed/NCBI

* Google Scholar

45. 45. Colunga AL, Colebank MJ, Program R, Olufsen MS. Parameter inference in a computational model of haemodynamics in pulmonary hypertension. Journal of the Royal Society Interface. 2023;20(200):20220735.

* View Article

* Google Scholar

46. 46. Strocchi M, Longobardi S, Augustin CM, Gsell MAF, Petras A, Rinaldi CA, et al. Cell to whole organ global sensitivity analysis on a four-chamber heart electromechanics model using Gaussian processes emulators. PLoS Comput Biol. 2023;19(6):e1011257. pmid:37363928

* View Article

* PubMed/NCBI

* Google Scholar

47. 47. Schäfer F, Sturdy J, Hellevik LR. Age and sex-dependent sensitivity analysis of a common carotid artery model. Biomechanics and Modeling in Mechanobiology. 2024;:1–19.

48. 48. Casas B, Lantz J, Viola F, Cedersund G, Bolger AF, Carlhäll C-J, et al. Bridging the gap between measurements and modelling: a cardiovascular functional avatar. Sci Rep. 2017;7(1):6214. pmid:28740184

* View Article

* PubMed/NCBI

* Google Scholar

49. 49. Pironet A, Dauby PC, Chase JG, Docherty PD, Revie JA, Desaive T. Structural identifiability analysis of a cardiovascular system model. Med Eng Phys. 2016;38(5):433–41. pmid:26970891

* View Article

* PubMed/NCBI

* Google Scholar

50. 50. Pironet A, Docherty PD, Dauby PC, Chase JG, Desaive T. Practical identifiability analysis of a minimal cardiovascular system model. Comput Methods Programs Biomed. 2019;171:53–65. pmid:28153466

* View Article

* PubMed/NCBI

* Google Scholar

51. 51. Marquis AD, Arnold A, Dean-Bernhoft C, Carlson BE, Olufsen MS. Practical identifiability and uncertainty quantification of a pulsatile cardiovascular model. Math Biosci. 2018;304:9–24. pmid:30017910

* View Article

* PubMed/NCBI

* Google Scholar

52. 52. de Bournonville S, Pironet A, Pretty C, Chase JG, Desaive T. Parameter estimation in a minimal model of cardio-pulmonary interactions. Mathematical Biosciences. 2019;313:81–94.

* View Article

* Google Scholar

53. 53. Sala L, Golse N, Joosten A, Vibert E, Vignon-Clementel I. Sensitivity Analysis of a Mathematical Model Simulating the Post-Hepatectomy Hemodynamics Response. Ann Biomed Eng. 2023;51(1):270–89. pmid:36326994

* View Article

* PubMed/NCBI

* Google Scholar

54. 54. Huang F, Ying S. On-line parameter identification of the lumped arterial system model: A simulation study. Plos one. 2020;15(7):e0236012.

* View Article

* Google Scholar

55. 55. Bjørdalsbakke NL, Sturdy JT, Hose DR, Hellevik LR. Parameter estimation for closed-loop lumped parameter models of the systemic circulation using synthetic data. Math Biosci. 2022;343:108731. pmid:34758345

* View Article

* PubMed/NCBI

* Google Scholar

56. 56. Hann CE, Chase JG, Desaive T, Froissart CB, Revie J, Stevenson D, et al. Unique parameter identification for cardiac diagnosis in critical care using minimal data sets. Comput Methods Programs Biomed. 2010;99(1):75–87. pmid:20097440

* View Article

* PubMed/NCBI

* Google Scholar

57. 57. Eck VG, Sturdy J, Hellevik LR. Effects of arterial wall models and measurement uncertainties on cardiovascular model predictions. J Biomech. 2017;50:188–94. pmid:27890534

* View Article

* PubMed/NCBI

* Google Scholar

58. 58. Colebank MJ, Chesler NC. An in-silico analysis of experimental designs to study ventricular function: A focus on the right ventricle. PLoS Comput Biol. 2022;18(9):e1010017. pmid:36126091

* View Article

* PubMed/NCBI

* Google Scholar

59. 59. Hettling H, van Beek JHGM. Analyzing the functional properties of the creatine kinase system with multiscale “sloppy” modeling. PLoS Comput Biol. 2011;7(8):e1002130. pmid:21912519

* View Article

* PubMed/NCBI

* Google Scholar

60. 60. Joubert D, Stigter JD, Molenaar J. Determining minimal output sets that ensure structural identifiability. PLoS One. 2018;13(11):e0207334. pmid:30419074

* View Article

* PubMed/NCBI

* Google Scholar

61. 61. Jagadeesan P, Raman K, Tangirala AK. Sloppiness: Fundamental study, new formalism and its application in model assessment. PLoS One. 2023;18(3):e0282609. pmid:36888634

* View Article

* PubMed/NCBI

* Google Scholar

62. 62. Silk D, Kirk PDW, Barnes CP, Toni T, Stumpf MPH. Model selection in systems biology depends on experimental design. PLoS Comput Biol. 2014;10(6):e1003650. pmid:24922483

* View Article

* PubMed/NCBI

* Google Scholar

63. 63. White A, Tolman M, Thames HD, Withers HR, Mason KA, Transtrum MK. The limitations of model-based experimental design and parameter estimation in sloppy systems. PLoS Computational Biology. 2016;12(12):e1005227.

* View Article

* Google Scholar

64. 64. Apgar JF, Witmer DK, White FM, Tidor B. Sloppy models, parameter uncertainty, and the role of experimental design. Mol Biosyst. 2010;6(10):1890–900. pmid:20556289

* View Article

* PubMed/NCBI

* Google Scholar

65. 65. Whittaker DG, Wang J, Shuttleworth JG, Venkateshappa R, Kemp JM, Claydon TW, et al. Ion channel model reduction using manifold boundaries. J R Soc Interface. 2022;19(193):20220193. pmid:35946166

* View Article

* PubMed/NCBI

* Google Scholar

66. 66. van Beek JHGM, Hauschild A-C, Hettling H, Binsl TW. Robust modelling, measurement and analysis of human and animal metabolic systems. Philos Trans A Math Phys Eng Sci. 2009;367(1895):1971–92. pmid:19380321

* View Article

* PubMed/NCBI

* Google Scholar

67. 67. Whittaker DG, Clerx M, Lei CL, Christini DJ, Mirams GR. Calibration of ionic and cellular cardiac electrophysiology models. Wiley Interdiscip Rev Syst Biol Med. 2020;12(4):e1482. pmid:32084308

* View Article

* PubMed/NCBI

* Google Scholar

68. 68. Comunale G, Peruzzo P, Castaldi B, Razzolini R, Di Salvo G, Padalino MA, et al. Understanding and recognition of the right ventricular function and dysfunction via a numerical study. Sci Rep. 2021;11(1):3709. pmid:33580128

* View Article

* PubMed/NCBI

* Google Scholar

69. 69. Korakianitis T, Shi Y. Numerical simulation of cardiovascular dynamics with healthy and diseased heart valves. Journal of biomechanics. 2006;39(11):1964–82.

* View Article

* Google Scholar

70. 70. Suga H, Sagawa K. Instantaneous pressure-volume relationships and their ratio in the excised, supported canine left ventricle. Circ Res. 1974;35(1):117–26. pmid:4841253

* View Article

* PubMed/NCBI

* Google Scholar

71. 71. Schenkel T. Circulatory System Models. 2023. https://doi.org/10.5281/zenodo.10428848

72. 72. Wanner G, Hairer E. Solving ordinary differential equations II. Springer Berlin Heidelberg New York. 1996.

73. 73. Sobol IM. Global sensitivity indices for nonlinear mathematical models and their Monte Carlo estimates. Mathematics and Computers in Simulation. 2001;55(1–3):271–80.

* View Article

* Google Scholar

74. 74. Jansen MJ. Analysis of variance designs for model output. Computer Physics Communications. 1999;117(1–2):35–43.

* View Article

* Google Scholar

75. 75. Saxton H, Xu X, Schenkel T, Clayton RH, Halliday I. Convergence, sampling and total order estimator effects on parameter orthogonality in global sensitivity analysis. bioRxiv. 2024;:2024–02.

76. 76. Archer G, Saltelli A, Sobol IM. Sensitivity measures, ANOVA-like techniques and the use of bootstrap. Journal of Statistical Computation and Simulation. 1997;58(2):99–120.

* View Article

* Google Scholar

77. 77. On the mathematical foundations of theoretical statistics. Phil Trans R Soc Lond A. 1922;222(594–604):309–68.

* View Article

* Google Scholar

78. 78. Pantazis Y, Katsoulakis MA, Vlachos DG. Parametric sensitivity analysis for biochemical reaction networks based on pathwise information theory. BMC Bioinformatics. 2013;14:311. pmid:24148216

* View Article

* PubMed/NCBI

* Google Scholar

79. 79. Saxton H, Xu X, Schenkel T, Halliday I. Assessing input parameter hyperspace and parameter identifiability in a cardiovascular system model via sensitivity analysis. Journal of Computational Science. 2024;79:102287.

* View Article

* Google Scholar

80. 80. Yang J. Decision-oriented two-parameter Fisher information sensitivity using symplectic decomposition. Technometrics. 2024;66(1):28–39.

* View Article

* Google Scholar

81. 81. Li R, Henson MA, Kurtz MJ. Selection of Model Parameters for Off-Line Parameter Estimation. IEEE Trans Contr Syst Technol. 2004;12(3):402–12.

* View Article

* Google Scholar

82. 82. Abdi H, Williams LJ. Principal component analysis. Wiley Interdisciplinary Reviews: Computational Statistics. 2010;2(4):433–59.

* View Article

* Google Scholar

83. 83. Brown KS, Sethna JP. Statistical mechanical approaches to models with many poorly known parameters. Phys Rev E Stat Nonlin Soft Matter Phys. 2003;68(2 Pt 1):021904. pmid:14525003

* View Article

* PubMed/NCBI

* Google Scholar

84. 84. Waterfall JJ, Casey FP, Gutenkunst RN, Brown KS, Myers CR, Brouwer PW, et al. Sloppy-model universality class and the Vandermonde matrix. Phys Rev Lett. 2006;97(15):150601. pmid:17155311

* View Article

* PubMed/NCBI

* Google Scholar

85. 85. ASME. V&V440-assessing credibility of computational modeling through verification and validation: application to medical devices. 2018.

86. 86. Kreutz C, Raue A, Kaschek D, Timmer J. Profile likelihood in systems biology. FEBS J. 2013;280(11):2564–71. pmid:23581573

* View Article

* PubMed/NCBI

* Google Scholar

87. 87. Blanco PJ, Watanabe SM, Passos MARF, Lemos PA, Feijóo RA. An anatomically detailed arterial network model for one-dimensional computational hemodynamics. IEEE Trans Biomed Eng. 2015;62(2):736–53. pmid:25347874

* View Article

* PubMed/NCBI

* Google Scholar

88. 88. Müller LO, Watanabe SM, Toro EF, Feijóo RA, Blanco PJ. An anatomically detailed arterial-venous network model. Cerebral and coronary circulation. Front Physiol. 2023;14:1162391. pmid:37435309

* View Article

* PubMed/NCBI

* Google Scholar

89. 89. Kotidis P, Demis P, Goey CH, Correa E, McIntosh C, Trepekli S. Constrained global sensitivity analysis for bioprocess design space identification. Computers & Chemical Engineering. 2019;125:558–68.

* View Article

* Google Scholar

90. 90. Klymenko OV, Kucherenko S, Shah N. Computer Aided Chemical Engineering. Elsevier. 2017. p. 151–6.

91. 91. Viceconti M, Pappalardo F, Rodriguez B, Horner M, Bischoff J, Musuamba Tshinanu F. In silico trials: Verification, validation and uncertainty quantification of predictive models used in the regulatory evaluation of biomedical products. Methods. 2021;185:120–7. pmid:31991193

* View Article

* PubMed/NCBI

* Google Scholar

92. 92. Taylor DJ, Feher J, Czechowicz K, Halliday I, Hose DR, Gosling R, et al. Validation of a novel numerical model to predict regionalized blood flow in the coronary arteries. Eur Heart J Digit Health. 2023;4(2):81–9. pmid:36974271

* View Article

* PubMed/NCBI

* Google Scholar

93. 93. Taylor DJ, Saxton H, Halliday I, Newman T, Feher J, Gosling R, et al. Evaluation of models of sequestration flow in coronary arteries-Physiology versus anatomy?. Comput Biol Med. 2024;173:108299. pmid:38537564

* View Article

* PubMed/NCBI

* Google Scholar

94. 94. Salvador M, Strocchi M, Regazzoni F, Augustin CM, Dede’ L, Niederer SA, et al. Whole-heart electromechanical simulations using Latent Neural Ordinary Differential Equations. NPJ Digit Med. 2024;7(1):90. pmid:38605089

* View Article

* PubMed/NCBI

* Google Scholar

Citation: Saxton H, Taylor DJ, Faulkner G, Halliday I, Newman T, Schenkel T, et al. (2025) The impact of experimental designs & system sloppiness on the personalisation process: A cardiovascular perspective. PLoS One 20(6): e0326112. https://doi.org/10.1371/journal.pone.0326112

About the Authors:

Harry Saxton

Roles: Conceptualization, Formal analysis, Methodology, Visualization, Writing – original draft

E-mail: [email protected]

Affiliations: School of Computer Science, University of Sheffield, Sheffield, United Kingdom, Insigneo Institute for in-silico Medicine, University of Sheffield, Sheffield, United Kingdom

ORICD: https://orcid.org/0000-0001-7433-6154

Daniel J. Taylor

Roles: Conceptualization, Methodology, Writing – original draft

Affiliations: Insigneo Institute for in-silico Medicine, University of Sheffield, Sheffield, United Kingdom, Division of Clinical Medicine, School of Medicine and Population Health, University of Sheffield, Sheffield, United Kingdom, NIHR Sheffield Biomedical Research Centre, Sheffield Teaching Hospitals NHS Foundation Trust, Sheffield, United Kingdom

Grace Faulkner

Roles: Visualization, Writing – review & editing

Affiliations: Insigneo Institute for in-silico Medicine, University of Sheffield, Sheffield, United Kingdom, Division of Clinical Medicine, School of Medicine and Population Health, University of Sheffield, Sheffield, United Kingdom, NIHR Sheffield Biomedical Research Centre, Sheffield Teaching Hospitals NHS Foundation Trust, Sheffield, United Kingdom

ORICD: https://orcid.org/0009-0008-4155-9974

Ian Halliday

Roles: Conceptualization, Funding acquisition, Methodology, Supervision, Validation, Writing – original draft, Writing – review & editing

Affiliations: Insigneo Institute for in-silico Medicine, University of Sheffield, Sheffield, United Kingdom, Division of Clinical Medicine, School of Medicine and Population Health, University of Sheffield, Sheffield, United Kingdom

Tom Newman

Roles: Methodology

Affiliations: Insigneo Institute for in-silico Medicine, University of Sheffield, Sheffield, United Kingdom, Division of Clinical Medicine, School of Medicine and Population Health, University of Sheffield, Sheffield, United Kingdom, NIHR Sheffield Biomedical Research Centre, Sheffield Teaching Hospitals NHS Foundation Trust, Sheffield, United Kingdom, Department of Cardiology and Cardiothoracic Surgery, Sheffield Teaching Hospitals NHS Trust, Sheffield, United Kingdom

Torsten Schenkel

Roles: Funding acquisition, Software, Supervision, Writing – original draft

Affiliations: Department of Cardiology and Cardiothoracic Surgery, Sheffield Teaching Hospitals NHS Trust, Sheffield, United Kingdom, Department of Engineering and Mathematics, Sheffield Hallam University, Sheffield, United Kingdom

ORICD: https://orcid.org/0000-0001-5560-1872

Paul D. Morris

Roles: Writing – review & editing

Affiliations: Insigneo Institute for in-silico Medicine, University of Sheffield, Sheffield, United Kingdom, Division of Clinical Medicine, School of Medicine and Population Health, University of Sheffield, Sheffield, United Kingdom, NIHR Sheffield Biomedical Research Centre, Sheffield Teaching Hospitals NHS Foundation Trust, Sheffield, United Kingdom, Department of Cardiology and Cardiothoracic Surgery, Sheffield Teaching Hospitals NHS Trust, Sheffield, United Kingdom

ORICD: https://orcid.org/0000-0002-3965-121X

Richard H. Clayton

Roles: Supervision, Writing – review & editing

Affiliations: School of Computer Science, University of Sheffield, Sheffield, United Kingdom, Insigneo Institute for in-silico Medicine, University of Sheffield, Sheffield, United Kingdom

ORICD: https://orcid.org/0000-0002-8438-7518

Xu Xu

Roles: Conceptualization, Formal analysis, Funding acquisition, Methodology, Supervision, Validation, Writing – original draft, Writing – review & editing

Affiliations: School of Computer Science, University of Sheffield, Sheffield, United Kingdom, Insigneo Institute for in-silico Medicine, University of Sheffield, Sheffield, United Kingdom

ORICD: https://orcid.org/0000-0002-9721-9054

[/RAW_REF_TEXT]

[/RAW_REF_TEXT]

[/RAW_REF_TEXT]

[/RAW_REF_TEXT]

[/RAW_REF_TEXT]

[/RAW_REF_TEXT]

[/RAW_REF_TEXT]

[/RAW_REF_TEXT]

[/RAW_REF_TEXT]

[/RAW_REF_TEXT]

[/RAW_REF_TEXT]

[/RAW_REF_TEXT]

[/RAW_REF_TEXT]

[/RAW_REF_TEXT]

[/RAW_REF_TEXT]

[/RAW_REF_TEXT]

[/RAW_REF_TEXT]

References

1. Committee ADEI, et al. Digital Twin: Definition & Value—An AIAA and AIA Position Paper. Reston, VA, USA: AIAA. 2020.

2. Trayanova NA, Prakosa A. Up digital and personal: How heart digital twins can transform heart patient care. Heart Rhythm. 2024;21(1):89–99.

3. Gillette K, Gsell MAF, Prassl AJ, Karabelas E, Reiter U, Reiter G, et al. A Framework for the generation of digital twins of cardiac electrophysiology from clinical 12-leads ECGs. Med Image Anal. 2021;71:102080. pmid:33975097

4. Coorey G, Figtree GA, Fletcher DF, Snelson VJ, Vernon ST, Winlaw D, et al. The health digital twin to tackle cardiovascular disease-a review of an emerging interdisciplinary field. NPJ Digit Med. 2022;5(1):126. pmid:36028526

5. Viola F, Del Corso G, De Paulis R, Verzicco R. GPU accelerated digital twins of the human heart open new routes for cardiovascular research. Sci Rep. 2023;13(1):8230. pmid:37217483

6. Balogh EP, Miller BT, Ball JR. Technology and tools in the diagnostic process. 2015.

7. Tlałka K, Saxton H, Halliday I, Xu X, Taylor D, Narracott A. In: International Conference on Computational Science, 2024. 173–87.

8. Tlałka K, Saxton H, Halliday I, Xu X, Narracott A, Taylor D, et al. Sensitivity analysis of closed-loop one-chamber and four-chamber models with baroreflex. PLoS Comput Biol. 2024;20(12):e1012377. pmid:39715272

9. Niederer SA, Lumens J, Trayanova NA. Computational models in cardiology. Nat Rev Cardiol. 2019;16(2):100–11. pmid:30361497

10. Hose DR, Lawford PV, Huberts W, Hellevik LR, Omholt SW, van de Vosse FN. Cardiovascular models for personalised medicine: Where now and where next?. Med Eng Phys. 2019;72:38–48. pmid:31554575

11. Shi Y, Lawford P, Hose R. Review of zero-D and 1-D models of blood flow in the cardiovascular system. Biomed Eng Online. 2011;10:33. pmid:21521508

12. Westerhof N, Lankhaar JW, Westerhof BE. The arterial windkessel. Medical & Biological Engineering & Computing. 2009;47(2):131–41.

13. Westerhof N, Stergiopulos N, Noble MI, Westerhof BE. Snapshots of hemodynamics: an aid for clinical research and graduate education. Springer. 2018.

14. Corral-Acero J, Margara F, Marciniak M, Rodero C, Loncaric F, Feng Y. The ‘Digital Twin’ to enable the vision of precision cardiology. European Heart Journal. 2020;41(48):4556–64.

15. Murphy SP, Ibrahim NE, Januzzi JL Jr. Heart Failure With Reduced Ejection Fraction: A Review. JAMA. 2020;324(5):488–504. pmid:32749493

16. Dart AM, Kingwell BA. Pulse pressure--a review of mechanisms and clinical relevance. J Am Coll Cardiol. 2001;37(4):975–84. pmid:11263624

17. Gatehouse PD, Keegan J, Crowe LA, Masood S, Mohiaddin RH, Kreitner K-F, et al. Applications of phase-contrast flow and velocity imaging in cardiovascular MRI. Eur Radiol. 2005;15(10):2172–84. pmid:16003509

18. Alhashemi JA, Cecconi M, Hofer CK. Cardiac output monitoring: an integrative perspective. Crit Care. 2011;15(2):214. pmid:21457508

19. Sagawa K. The ventricular pressure-volume diagram revisited. Circulation Research. 1978;43(5):677–87.

20. Kirk RE. Experimental design. Sage handbook of quantitative methods in psychology. 2009. p. 23–45.

21. Tarantola A. Inverse problem theory and methods for model parameter estimation. SIAM. 2005.

22. Strimbu K, Tavel JA. What are biomarkers?. Current Opinion in HIV and AIDS. 2010;5(6):463.

23. Huberts W, Heinen SG, Zonnebeld N, van den Heuvel DA, de Vries JPP, Tordoir JH. What is needed to make cardiovascular models suitable for clinical decision support? A viewpoint paper. Journal of Computational Science. 2018;24:68–84.

24. Saltelli A, Tarantola S, Campolongo F, Ratto M, et al. Sensitivity analysis in practice: a guide to assessing scientific models. Wiley Online Library. 2004.

25. Olsen CH, Ottesen JT, Smith RC, Olufsen MS. Parameter subset selection techniques for problems in mathematical biology. Biol Cybern. 2019;113(1–2):121–38. pmid:30377765

26. Saltelli A, Ratto M, Andres T, Campolongo F, Cariboni J, Gatelli D. Global sensitivity analysis: the primer. John Wiley & Sons. 2008.

27. Bellman R, Åström KJ. On structural identifiability. Mathematical biosciences. 1970;7(3–4):329–39.

28. Iliadis A. Structural identifiability and sensitivity. J Pharmacokinet Pharmacodyn. 2019;46(2):127–35. pmid:30895420

29. Wieland FG, Hauber AL, Rosenblatt M, Tönsing C, Timmer J. On structural and practical identifiability. Current Opinion in Systems Biology. 2021;25:60–9.

30. Daniels BC, Chen Y-J, Sethna JP, Gutenkunst RN, Myers CR. Sloppiness, robustness, and evolvability in systems biology. Curr Opin Biotechnol. 2008;19(4):389–95. pmid:18620054

31. Itu LM, Sharma P, Suciu C. Patient-specific hemodynamic computations: application to personalized diagnosis of cardiovascular pathologies. Springer. 2017.

32. Kochenderfer MJ, Wheeler TA. Algorithms for optimization. Mit Press. 2019.

33. Saxton H, Schenkel T, Halliday I, Xu X. Personalised parameter estimation of the cardiovascular system: leveraging data assimilation and sensitivity analysis. Journal of Computational Science. 2023;74:102158.

34. Chis O-T, Villaverde AF, Banga JR, Balsa-Canto E. On the relationship between sloppiness and identifiability. Math Biosci. 2016;282:147–61. pmid:27789352

35. Gutenkunst RN, Waterfall JJ, Casey FP, Brown KS, Myers CR, Sethna JP. Universally sloppy parameter sensitivities in systems biology models. PLoS Comput Biol. 2007;3(10):1871–8. pmid:17922568

36. Monsalve-Bravo GM, Lawson BAJ, Drovandi C, Burrage K, Brown KS, Baker CM, et al. Analysis of sloppiness in model simulations: Unveiling parameter uncertainty when mathematical models are fitted to data. Sci Adv. 2022;8(38):eabm5952. pmid:36129974

37. Carson J, Warrander L, Johnstone E, van Loon R. Personalising cardiovascular network models in pregnancy: A two-tiered parameter estimation approach. Int J Numer Method Biomed Eng. 2021;37(11):e3267. pmid:31799783

38. Salvador M, Kong F, Peirlinck M, Parker D, Chubb H, Dubin A. Digital twinning of cardiac electrophysiology for congenital heart disease. Biorxiv. 2023;:2023–11.

39. Gerach T, Schuler S, Fröhlich J, Lindner L, Kovacheva E, Moss R. Electro-mechanical whole-heart digital twins: a fully coupled multi-physics approach. Mathematics. 2021;9(11):1247.

40. Myers TG, Ribas Ripoll V, Sáez de Tejada Cuenca A, Mitchell SL, McGuinness MJ. Modelling the cardiovascular system for assessing the blood pressure curve. Math Ind Case Stud. 2017;8(1):2. pmid:28680510

41. Duanmu Z, Yin M, Fan X, Yang X, Luo X. A patient-specific lumped-parameter model of coronary circulation. Sci Rep. 2018;8(1):874. pmid:29343785

42. Chakshu NK, Sazonov I, Nithiarasu P. Towards enabling a cardiovascular digital twin for human systemic circulation using inverse analysis. Biomech Model Mechanobiol. 2021;20(2):449–65. pmid:33064221

43. Canuto D, Pantoja JL, Han J, Dutson EP, Eldredge JD. An ensemble Kalman filter approach to parameter estimation for patient-specific cardiovascular flow modeling. Theoretical and Computational Fluid Dynamics. 2020;34:521–44.

44. Pant S, Corsini C, Baker C, Hsia T-Y, Pennati G, Vignon-Clementel IE, et al. Data assimilation and modelling of patient-specific single-ventricle physiology with and without valve regurgitation. J Biomech. 2016;49(11):2162–73. pmid:26708918

45. Colunga AL, Colebank MJ, Program R, Olufsen MS. Parameter inference in a computational model of haemodynamics in pulmonary hypertension. Journal of the Royal Society Interface. 2023;20(200):20220735.

46. Strocchi M, Longobardi S, Augustin CM, Gsell MAF, Petras A, Rinaldi CA, et al. Cell to whole organ global sensitivity analysis on a four-chamber heart electromechanics model using Gaussian processes emulators. PLoS Comput Biol. 2023;19(6):e1011257. pmid:37363928

47. Schäfer F, Sturdy J, Hellevik LR. Age and sex-dependent sensitivity analysis of a common carotid artery model. Biomechanics and Modeling in Mechanobiology. 2024;:1–19.

48. Casas B, Lantz J, Viola F, Cedersund G, Bolger AF, Carlhäll C-J, et al. Bridging the gap between measurements and modelling: a cardiovascular functional avatar. Sci Rep. 2017;7(1):6214. pmid:28740184

49. Pironet A, Dauby PC, Chase JG, Docherty PD, Revie JA, Desaive T. Structural identifiability analysis of a cardiovascular system model. Med Eng Phys. 2016;38(5):433–41. pmid:26970891

50. Pironet A, Docherty PD, Dauby PC, Chase JG, Desaive T. Practical identifiability analysis of a minimal cardiovascular system model. Comput Methods Programs Biomed. 2019;171:53–65. pmid:28153466

51. Marquis AD, Arnold A, Dean-Bernhoft C, Carlson BE, Olufsen MS. Practical identifiability and uncertainty quantification of a pulsatile cardiovascular model. Math Biosci. 2018;304:9–24. pmid:30017910

52. de Bournonville S, Pironet A, Pretty C, Chase JG, Desaive T. Parameter estimation in a minimal model of cardio-pulmonary interactions. Mathematical Biosciences. 2019;313:81–94.

53. Sala L, Golse N, Joosten A, Vibert E, Vignon-Clementel I. Sensitivity Analysis of a Mathematical Model Simulating the Post-Hepatectomy Hemodynamics Response. Ann Biomed Eng. 2023;51(1):270–89. pmid:36326994

54. Huang F, Ying S. On-line parameter identification of the lumped arterial system model: A simulation study. Plos one. 2020;15(7):e0236012.

55. Bjørdalsbakke NL, Sturdy JT, Hose DR, Hellevik LR. Parameter estimation for closed-loop lumped parameter models of the systemic circulation using synthetic data. Math Biosci. 2022;343:108731. pmid:34758345

56. Hann CE, Chase JG, Desaive T, Froissart CB, Revie J, Stevenson D, et al. Unique parameter identification for cardiac diagnosis in critical care using minimal data sets. Comput Methods Programs Biomed. 2010;99(1):75–87. pmid:20097440

57. Eck VG, Sturdy J, Hellevik LR. Effects of arterial wall models and measurement uncertainties on cardiovascular model predictions. J Biomech. 2017;50:188–94. pmid:27890534

58. Colebank MJ, Chesler NC. An in-silico analysis of experimental designs to study ventricular function: A focus on the right ventricle. PLoS Comput Biol. 2022;18(9):e1010017. pmid:36126091

59. Hettling H, van Beek JHGM. Analyzing the functional properties of the creatine kinase system with multiscale “sloppy” modeling. PLoS Comput Biol. 2011;7(8):e1002130. pmid:21912519

60. Joubert D, Stigter JD, Molenaar J. Determining minimal output sets that ensure structural identifiability. PLoS One. 2018;13(11):e0207334. pmid:30419074

61. Jagadeesan P, Raman K, Tangirala AK. Sloppiness: Fundamental study, new formalism and its application in model assessment. PLoS One. 2023;18(3):e0282609. pmid:36888634

62. Silk D, Kirk PDW, Barnes CP, Toni T, Stumpf MPH. Model selection in systems biology depends on experimental design. PLoS Comput Biol. 2014;10(6):e1003650. pmid:24922483

63. White A, Tolman M, Thames HD, Withers HR, Mason KA, Transtrum MK. The limitations of model-based experimental design and parameter estimation in sloppy systems. PLoS Computational Biology. 2016;12(12):e1005227.

64. Apgar JF, Witmer DK, White FM, Tidor B. Sloppy models, parameter uncertainty, and the role of experimental design. Mol Biosyst. 2010;6(10):1890–900. pmid:20556289

65. Whittaker DG, Wang J, Shuttleworth JG, Venkateshappa R, Kemp JM, Claydon TW, et al. Ion channel model reduction using manifold boundaries. J R Soc Interface. 2022;19(193):20220193. pmid:35946166

66. van Beek JHGM, Hauschild A-C, Hettling H, Binsl TW. Robust modelling, measurement and analysis of human and animal metabolic systems. Philos Trans A Math Phys Eng Sci. 2009;367(1895):1971–92. pmid:19380321

67. Whittaker DG, Clerx M, Lei CL, Christini DJ, Mirams GR. Calibration of ionic and cellular cardiac electrophysiology models. Wiley Interdiscip Rev Syst Biol Med. 2020;12(4):e1482. pmid:32084308

68. Comunale G, Peruzzo P, Castaldi B, Razzolini R, Di Salvo G, Padalino MA, et al. Understanding and recognition of the right ventricular function and dysfunction via a numerical study. Sci Rep. 2021;11(1):3709. pmid:33580128

69. Korakianitis T, Shi Y. Numerical simulation of cardiovascular dynamics with healthy and diseased heart valves. Journal of biomechanics. 2006;39(11):1964–82.

70. Suga H, Sagawa K. Instantaneous pressure-volume relationships and their ratio in the excised, supported canine left ventricle. Circ Res. 1974;35(1):117–26. pmid:4841253

71. Schenkel T. Circulatory System Models. 2023. https://doi.org/10.5281/zenodo.10428848

72. Wanner G, Hairer E. Solving ordinary differential equations II. Springer Berlin Heidelberg New York. 1996.

73. Sobol IM. Global sensitivity indices for nonlinear mathematical models and their Monte Carlo estimates. Mathematics and Computers in Simulation. 2001;55(1–3):271–80.

74. Jansen MJ. Analysis of variance designs for model output. Computer Physics Communications. 1999;117(1–2):35–43.

75. Saxton H, Xu X, Schenkel T, Clayton RH, Halliday I. Convergence, sampling and total order estimator effects on parameter orthogonality in global sensitivity analysis. bioRxiv. 2024;:2024–02.

76. Archer G, Saltelli A, Sobol IM. Sensitivity measures, ANOVA-like techniques and the use of bootstrap. Journal of Statistical Computation and Simulation. 1997;58(2):99–120.

77. On the mathematical foundations of theoretical statistics. Phil Trans R Soc Lond A. 1922;222(594–604):309–68.

78. Pantazis Y, Katsoulakis MA, Vlachos DG. Parametric sensitivity analysis for biochemical reaction networks based on pathwise information theory. BMC Bioinformatics. 2013;14:311. pmid:24148216

79. Saxton H, Xu X, Schenkel T, Halliday I. Assessing input parameter hyperspace and parameter identifiability in a cardiovascular system model via sensitivity analysis. Journal of Computational Science. 2024;79:102287.

80. Yang J. Decision-oriented two-parameter Fisher information sensitivity using symplectic decomposition. Technometrics. 2024;66(1):28–39.

81. Li R, Henson MA, Kurtz MJ. Selection of Model Parameters for Off-Line Parameter Estimation. IEEE Trans Contr Syst Technol. 2004;12(3):402–12.

82. Abdi H, Williams LJ. Principal component analysis. Wiley Interdisciplinary Reviews: Computational Statistics. 2010;2(4):433–59.

83. Brown KS, Sethna JP. Statistical mechanical approaches to models with many poorly known parameters. Phys Rev E Stat Nonlin Soft Matter Phys. 2003;68(2 Pt 1):021904. pmid:14525003

84. Waterfall JJ, Casey FP, Gutenkunst RN, Brown KS, Myers CR, Brouwer PW, et al. Sloppy-model universality class and the Vandermonde matrix. Phys Rev Lett. 2006;97(15):150601. pmid:17155311

85. ASME. V&V440-assessing credibility of computational modeling through verification and validation: application to medical devices. 2018.

86. Kreutz C, Raue A, Kaschek D, Timmer J. Profile likelihood in systems biology. FEBS J. 2013;280(11):2564–71. pmid:23581573

87. Blanco PJ, Watanabe SM, Passos MARF, Lemos PA, Feijóo RA. An anatomically detailed arterial network model for one-dimensional computational hemodynamics. IEEE Trans Biomed Eng. 2015;62(2):736–53. pmid:25347874

88. Müller LO, Watanabe SM, Toro EF, Feijóo RA, Blanco PJ. An anatomically detailed arterial-venous network model. Cerebral and coronary circulation. Front Physiol. 2023;14:1162391. pmid:37435309

89. Kotidis P, Demis P, Goey CH, Correa E, McIntosh C, Trepekli S. Constrained global sensitivity analysis for bioprocess design space identification. Computers & Chemical Engineering. 2019;125:558–68.

90. Klymenko OV, Kucherenko S, Shah N. Computer Aided Chemical Engineering. Elsevier. 2017. p. 151–6.

91. Viceconti M, Pappalardo F, Rodriguez B, Horner M, Bischoff J, Musuamba Tshinanu F. In silico trials: Verification, validation and uncertainty quantification of predictive models used in the regulatory evaluation of biomedical products. Methods. 2021;185:120–7. pmid:31991193

92. Taylor DJ, Feher J, Czechowicz K, Halliday I, Hose DR, Gosling R, et al. Validation of a novel numerical model to predict regionalized blood flow in the coronary arteries. Eur Heart J Digit Health. 2023;4(2):81–9. pmid:36974271

93. Taylor DJ, Saxton H, Halliday I, Newman T, Feher J, Gosling R, et al. Evaluation of models of sequestration flow in coronary arteries-Physiology versus anatomy?. Comput Biol Med. 2024;173:108299. pmid:38537564

94. Salvador M, Strocchi M, Regazzoni F, Augustin CM, Dede’ L, Niederer SA, et al. Whole-heart electromechanical simulations using Latent Neural Ordinary Differential Equations. NPJ Digit Med. 2024;7(1):90. pmid:38605089