1. Introduction
With the substantial progress in medical imaging and the boosting of multimodalities, it becomes increasingly challenging to obtain a comprehensive and integrated 3D view of a given heart. Unequivocally, most of these are consequences of larger size of acquired data (often in multiple centres), complex fusion of the imaging modalities, and variability in cardiac function.
MUSIC (Multimodality Platform for Specific Imaging in Cardiology) is a software platform developed by the Multimodal Data Science team (
The MUSIC project is led by a multidisciplinary team including researchers, developers, clinicians and medical imaging operators.
MUSIC is composed of a desktop application, a web portal, and an academic consortium. The desktop application is called MUSICardio and is built on the open-source software medInria (
Combined analysis of imaging (CT-scan, MRI, PET) and electrophysiology (invasive, non-invasive mapping or image-based simulation) data to improve knowledge regarding the structural substrate generating dangerous arrhythmia;
The development of AI-based image processing tools for the automatic segmentation of cardiac structures and quantification of robust markers associated with the risk of arrhythmia;
The development of customised image-based modelling methods to simulate electrophysiological tests with computation times compatible with clinical practice;
The development of cardiac models for the optimal navigation of instruments in the virtual patient heart during interventional procedures;
The clinical validation of these different tools for diagnosis, prognosis or real-time guided therapy in patients with cardiac electrical disorders.
Treatment improvement: MUSIC offers image-based algorithms for diagnosis and prognosis as well as pipelines dedicated to the guidance of either atrial or ventricular interventions through imaging. For instance, this application has been used to guide scar-related ventricular tachycardia (VT) ablation in over 500 consecutive procedures in Bordeaux University Hospital.
A booster for research: MUSICardio is an application allowing the analysis, in one environment, of multi-parametric data sets from different Liryc IHU teams (structural, mechanics, hemodynamics, electrical, etc.) and to interface with simulation platforms such as SOFA (
A VT MUSIC consortium has been created including more than 30 international hospitals with expertise in ventricular tachycardia from Europe, USA and Australia. Participating centres anonymise and upload their data to our MUSIC web portal, then our expert operators process the data in the MUSICardio software to obtain customised models of the heart. These new data are then sent back to the hospitals, which can immediately include them during operations to visualise catheters in real time in patient-specific heart modelling.
Some imaging platforms similar on some points to MUSIC allow us to situate ourselves: OsiriX (
2. Architecture
MUSIC is a software platform allowing in a unique environment the analysis of multi-parametric data sets (e.g., structural, mechanic, hemodynamic, electrical, etc.) and to interface such data with simulation platforms in order to develop patient-specific modelling and simulation strategies. The MUSIC project is composed of the MUSICardio application and a data web portal, along with the VT MUSIC consortium.
2.1. Consortium
The VT MUSIC consortium brings together international centres with expertise in ventricular tachycardia and our institute. The data acquired in the hospitals are sent to us through a web portal and processed rapidly by our expert operators in our MUSICardio application. Thus, personalised models of the patient heart can be sent to the centres to be included in cardiac operations. Figure 1 shows the process of the consortium for the particular case of radiotherapy.
VT centres interested in joining the consortium are invited to contact our team. Access to the consortium is free.
2.2. Web Portal
A web portal has been developed in Java through the Play framework (
2.3. MUSICardio
The MUSICardio application within the MUSIC project is based on the medInria open-source software developed by Inria. Our team participates in the improvement of medInria as the core of MUSICardio. We are part of the medInria consortium, created by Inria, which leads the development of the software. The modularity of the plugin system of the software allows easily adding new plugins dedicated to functionalities needed in the cardiac field.
2.3.1. Usage
There are three current ways of using MUSICardio. The application can be directly used by our researchers for instance to use algorithmic tools developed by our team. Researchers, post-doctoral fellows, PhD students or other members of our laboratory can also write Python scripts, with the help of our team, to create pipelines to process and analyse their data in the application. Python is an understandable language for non-developers, and it can be learnt quickly and easily. Pipelines allow complex suites of algorithms and tools to be processed in simple graphical steps in MUSICardio, making it easier for users, especially if they have to perform the tasks many times. Finally, our expert operators (who are trained medical imagery staff, and formed to use our VT pipeline script in MUSICardio) can process our pipeline in MUSICardio, using as input the data sent by partner hospitals in the VT MUSIC consortium, and they can send back the output heart models to these hospitals.
2.3.2. Libraries
The core of the application (i.e., medInria) is written in C++ and Qt, uses external libraries and includes Python wrappers. The main external libraries used are shown in the Table 1.
Our application gives access to a wide range of data tools, workspaces or toolboxes through a system of plugins that can be easily activated or deactivated, allowing a modularity of the application binaries. We use some additional external libraries in the MUSICardio application for our plugins (Table 2):
These external libraries API can be accessed in plugins, allowing developers to access their functionalities and to develop more tools in the application.
2.3.3. Components
Figure 2 presents the architecture of the MUSICardio application through its medInria core, its import/export formats and functionalities plugins, and its external libraries.
2.3.4. Python
Python 3 is provided in the software. A Python console has been added, which allows researchers to work on their data directly in Python in the application and further perform parameter testing in the console. The interfacing between Python and the application in C++/Qt is done with the Swig (
2.3.5. Pipelines
Processing workflows can be executed through a pipeline system included in the MUSICardio application. This allows us to script complex process sequences in order to simplify the work of the operators, make the results more robust and obtain the output data fast. The pipeline scripts were developed in Python, allowing researchers and operators to create and adapt their own customised pipelines.
For instance, within the VT MUSIC consortium, clinicians and expert operators at Liryc IHU use the pipeline system to segment CT and MR images, and to generate a 3D structural model comprising a detailed anatomy of the whole heart, myocardial scar maps from CT-derived wall thickness or late-enhancement MRI analysis, structures at risk during SBRT (stereotactic body radiotherapy) procedures, or other structures such as coronaries, left phrenic nerve, atrioventricular node, ICD (implantable cardioverter defibrillator) lead tip, GI (gastrointestinal) tract, etc. The 3D models are used to treat refractory ventricular tachycardia. EP maps (electrophysiology data) are registered in the same geometrical coordinates to allow the user to perform the 3D rendering of combined structural and EP data.
Pipeline scripts can handle automatic segmentation using Deep Learning-based algorithms that have been developed by our team. This allows us to obtain accurate and quick automatic segmentations of the anatomical heart structures for a more precise diagnosis and treatment of cardiac diseases.
3. Data Management
3.1. Data Format
The data management system in the MUSIC project includes local databases per user managing the data imported or generated by the MUSICardio application, access from the application to several data sources including PACS systems, and our online web portal that allows hospitals in the VT MUSIC consortium to send their data to our institute and to receive output data from us.
Within the application, users can import and export data in various formats, including VTK, ITK, DICOM, Nnrd, GIS, Nifti, Gipl, OBJ, STL, Medit, etc. MUSICardio can export data to electrophysiology mapping systems such as BioSense Webster, Johnson & Johnson (New Brunswick, NJ, USA) “CARTO”, Boston Scientific (Natick, MA, USA) “RHYTHMIA”, and Saint-Jude Medical (Saint Paul, MN, USA) “EnSite NavX”. Within the VT MUSIC consortium, these compatible data are sent to VT centres around the world where clinicians can visualise their catheters in real time inside a virtual patient-specific heart created in the application through their navigation systems.
The application also allows us to export cardiac structures in DICOM RT-Struct format. These data are compatible with any radiation therapy (RT) planning software. Depending on the strategy and RT technology available onsite, these cardiac targets and associated segmentations may be registered onto a 4D planning CT and expanded to take into account the cardiac/respiratory motion and margins of errors for misregistration. Dose simulations are then performed to validate the final strategy, which is often the result of a trade-off between safety and efficacy of radiation delivery. Once validated, the actual SBRT (stereotactic body radiation therapy) treatment is delivered.
3.2. Data Sources
The MUSICardio application can connect to hospital PACS systems to access imaging data. This allows clinicians and hospital operators to easily work on data through a secure tool dedicated to hospitals. This avoids data import and export on physical media and travel across hospitals, and it allows for accurate data tracking. It also allows hospital users to filter data and easily find the information and data they need through metadata. Figure 3 shows the interface of the PACS access system.
4. Functionalities
4.1. Image and Mesh Workspaces
The MUSICardio application offers a wide range of tools and algorithms for segmenting, filtering, and reformatting, which are sorted into different workspaces to find them more easily.
The main image and mesh tools and algorithms in the application are outlined in the Table 3.
Figure 4 shows the interface of the thresholding process in the filtering workspace.
4.2. Segmentation
The Segmentation workspace handles multiple segmentation tools, one of the most important and used is the “Polygon ROI” toolbox. The “Polygon ROI” toolbox has been developed jointly by our team and radiologists. This tool allows us to manually segment images through polygon contours (ROI, region of interest), use a repulsor tool to refine these contours, automatically interpolate these segmentations between slices, or annotate them for AI. A focus in the development has been made on the production of contours with standardised labels. For now, eight different structures with different colours and names can be defined simultaneously. Figure 5 illustrates the ability to segment several anatomical zones and define a label name for each structure.
The results of segmentation can be saved as binary masks or internal contours structures. A contour structure is an internal format of MUSICardio and medInria. It allows us to save the work in progress and start again later or reuse the result of a segmentation as the model on another dataset. Other useful features are available:
Correction tools: each node of the polygon contour can be manipulated separately, or a repulsor tool is available allowing to push the segmentation boundary with a circle. See Figure 6a.
Multiple Views: segmentations can be done on different series simultaneously in multiple views to keep the same label name for series, and we can use a cursor to mark a position in series. The data have to be first reformatted. Figure 6b shows two MRIs at different dates. A lesion, in green, is visible only on one series (the right one). The evolution of another lesion is showed on the second series in blue. The green cursor allows us to find the way through the images.
4.3. Mesh Enhancement
The VTK library allows us to generate surface meshes from segmented masks, but these meshes are generally not of enough quality to be used for computer modelling and simulation. We decided to include the Mmg (
4.4. Electroanatomical Map Analysis
ElectroAnatomical Mapping (EAM) systems are used to create a 3D model of the heart and to record the cardiac electrical activity at the mapped points [7]. More than two decades of device improvements and technological refinements have made the clinical EAM systems routinely used during invasive radio-frequency (RF) ablation procedures for treating dangerous cardiac arrhythmia [8].
An electroanatomical map (EAM) is a combination of a 3D heart geometry (i.e., a mesh) and electrophysiological data in the form of 1D electrical signals (e.g., uni/bipolar maps, local activation times, intracardiac electrograms). After the EP procedure, the study containing the maps can be exported from navigation systems. Our application is capable of importing EAMs from navigation systems used in the clinics: CARTO (Biosense Webster, Johnson & Johnson) and RHYTHMIA (Boston Scientific), respectively. Once imported, these maps can be visualised and further processed using various toolboxes.
4.4.1. Review of Electrocardiograms
The recorded intracardiac waves and ECG signals can be reviewed. Users may create annotations to highlight important events. These annotations are given: name, colour, description. They can reference events that have a duration or just a specific time. Figure 7 illustrates the signal review and annotation window.
4.4.2. Radiotherapy Planning
Stereotactic body radiotherapy (SBRT) has recently been applied to treat refractory ventricular tachycardia [9,10]. MUSICardio provides an environment for an in-depth analysis and SBRT session planning.
CT and MR images are segmented, creating a 3D structural model comprising a detailed whole heart anatomy, myocardial scar maps generated from CT-derived wall thickness or late-enhancement MRI analysis, and the structures at risk during the interventions.
For patients who previously underwent catheter radio-frequency ablation, electroanatomical maps can be imported and co-registered with the CT and MR segmented meshes to allow the user to perform the 3D rendering of combined structural information and EP data.
The ablation targets and organs at risk can be delineated, creating contours to be exported in the DICOM RT-struct format. Figure 8 shows an example of such contours.
4.5. Electrocardiographic Imaging
In the Simulation workspace of MUSICardio, we developed different tools in order to enable the use of computational models for simulations along with their integration with multimodality imaging data. Of particular concern are the tools developed to improve the resolution of ElectroCardioGraphic Imaging (ECGI) inverse problems. Briefly, ECGI is a non-invasive technique that allows clinicians to construct electrical information on the heart surface or volume from electrical measurements on the body surface and patient-specific heart and torso geometries. The ECGI tools that are developed in MUSICardio are based on the formulation of Spach et al., who first related the epicardial and body surface electrical potentials by means of a transfer matrix that depends on the geometry of the heart and the torso of the patient [11]. Both surface and volume ECGI functionalities follow the following pipeline:
Load the 3D image data of the patient (i.e., CT image scan or MR images).
Segment body surface and heart surface or volume geometry and construct their corresponding masks. The user can also segment other organs such as lungs, bones and the liver. Figure 9a shows a 2D axial cross-section view of a segmented geometry showing the volume of the heart, lungs, liver and the body surface.
Generate computational meshes of the segmented geometries. Figure 9b shows an example of a generated 3D computational geometry.
Identify the position of electrodes in the body surface mesh (semi-automatic).
Construct a transfer matrix based on finite elements method (FEM) or on the method of fundamental solutions (MFS) [12]. Details about both methods can be found in [13].
Load the body surface electrical recordings.
Construct the electrical potential on the heart surface or the current density on the heart volume by solving the ECGI inverse problem. Several approaches have been implemented using different regularisation methods, see [13].
Visualise the electrical information constructed on the surface or the volume of the heart. Figure 10 illustrates an example of the electrical potential reconstruction using MFS (b) and FEM (c) as well as the ground truth of the corresponding solution (a).
For the post processing of the computed electrical potential, our ECGI pipeline script in Python allows one to compute activation maps.
Figure 10 shows an example of typical results that can be obtained using MUSICardio.
4.6. Artificial Intelligence
Deep Learning-based algorithms developed by our team have been recently included in Python script pipelines and further used in the MUSICardio application. These AI algorithms included in the scripts can improve the accuracy of cardiac segmentation and speed up the analysis and computations, resulting in a less invasive patient diagnostic, a more precise treatment, as well as a better therapy outcome.
The Python support in MUSICardio pipeline allows us to access an unlimited number of calculation libraries, including the AI libraries such as: TensorFlow, PyTorch, and scikit-learn. The pipeline script also has several extended front-end controls of the MUSICardio interface, which are able to facilitate the visualisation of processing steps undertaken during the AI project. The ability to display different type of images and resulted computations is crucial in building a coherent and easy-to-understand pipeline starting from the input image to the final Deep Learning (DL) model prediction. However, the majority of the AI project steps are already built in Python, and these can be easily integrated into the main pipeline.
One of the AI pipelines developed by our team automatically predicts the risk of ventricular arrhythmia (VA) from 3D CT image-based models, as proposed in [14]. This particular pipeline consists of several image processing steps including segmentation of CT images, visualisation of short-axis view orientation, wall thickness calculation, and 2D bullseye map flattening. The bullseye thickness map is then used as input to a VA classification network, after which the classification’s attention map is calculated from the prediction score using a gradient back propagation algorithm. The automatic image segmentation is performed using a pre-trained Dual-UNet segmentation network, while the VA classifier network consists of a conditional variational autoencoder and a fully connected classifier model. The two DL networks and all matrix calculations can be integrated directly in the pipeline Python script. The complete image processing pipeline is presented in Figure 11.
The pipeline ends with the VA classification from the 2D wall thickness map using a conditional variational autoencoder and a fully connected classifier model. A so-called “attention map” is also generated from the gradient back propagation of the classification score. The thickness map, attention map and classification score can be displayed at the end of the pipeline, as shown in Figure 12.
The pipeline that was implemented and designed to be used case by case was not adapted for the processing of multiple inputs/outputs required for the development stage of the AI models. Therefore, both AI models were built, trained and validated outside of MUSICardio using the Python environment and packages, including TensorFlow (Deep Learning algorithms) and SimpleITK, Numpy and VTK (image processing). The segmentation network was built using two consecutive 3D UNet models. The first model was used to coarsely segment the heart regions, which was used to crop the region of interest to be used as input in the second segmentation model for refine segmentation. The model achieved a Dice score of for left ventricle wall segmentation on the validation dataset. Similarly, the post-infarct VA prediction was trained using a retrospective dataset of 600 patients ( VA). The validation on the test dataset showed the prediction accuracy of , sensitivity of and specificity of . The architectures of the AI networks used in the example pipeline are shown in Figure 11.
Once a model such as this has been built, it can be implemented in MUSICardio pipelines for direct use: the compatibility of the application to run and install Python packages allows the direct integration of internal and external image processing tools, as well as the pre-trained AI models, into a single MUSICardio pipeline. Thus, MUSICardio proves to be a powerful tool to quickly prototype a fully functioning AI-based project.
4.7. MRI-Guided Radiofrequency Ablation of Cardiac Arrhythmias
Radiofrequency (RF) ablation procedures performed under MRI offer the advantage of a non-ionising imaging, providing that appropriate MR-compatible instrumentation (e.g., Vision-MR Ablation Catheter, Imricor Medical Systems, Burnsville, MN, USA) be used. MRI can provide 3D anatomical images of the heart at the beginning of the therapeutic session prior to real-time catheter navigation, online monitoring of therapy and immediate assessment of lesion transmurality. Therefore, such an integrated workflow does not require 3D registration between data acquired prior to and during the therapeutic procedure. Figure 13 shows an example of the use of MUSICardio in such a scenario on a preclinical study. Three-dimensional (3D) images (balanced ssfp sequence) acquired at the beginning of the session served as a road map and were loaded and processed in MUSICardio application (Figure 13a) in less than 5 min to segment (using the polygon ROI feature) right and left ventricle cavities. The resulting volumes were converted into meshes and then loaded in the prototype Monte Carlo platform provided by the MRI manufacturer (Siemens Healthcare, Erlangen, Germany). Real-time 3D catheter navigation was then performed using a dedicated tip-tracking MRI sequence (measuring X, Y and Z positions of MRI micro coils embedded into the catheter tip at an update rate of 40 ms) to precisely position the catheter at a desired location within the cavity (Figure 13b). Catheter position was displayed online in transparency over the original images and the previously computed 3D meshes. Then, RF energy was delivered under MRI thermometry ([15], data not shown) to monitor treatment progression on the fly. After several RF shots performed at different locations within the heart (positioned under 3D real-time tip tracking sequence), a 3D post-ablation T1-weighted MRI sequence (long TI without injection of contrast agent [16]) was run to visualise the lesion core (irreversible thermal coagulation) and surrounding oedema. Then, the wall thickness map of the left ventricle was computed from the epicardium and endocardium masks using a similar pipeline presented in Section 4.6. The segmentation was initially performed manually with MUSICardio, then a dual-UNet segmentation network was trained on the TI-weighted images to accelerate and automate the segmentation during the intervention (<1 min). An arbitrary threshold was applied on TI-weighted 3D images on the LV wall to automatically extract the mask of the RF lesions. This mask was then multiplied by the wall thickness map to create a measure of the lesion transmurality (expressed in percentage), which is a major parameter to assess the therapeutic success of RF ablation. The output result was finally displayed on the interface (Figure 13c), allowing the user to visualise native T1-weighted images and transmurality map within a single graphical interface.
5. Discussion
The MUSIC project brings together various partners in the cardiology world: hospitals, research institutes, and therefore clinicians, researchers, developers, modelling engineers, database specialists, etc. This mixture of professions and objectives requires communication, adaptation and imagination in order to develop computer tools as well as interfaces that correspond to the greatest number of people. In particular, the tools must be easy to use and adapted to the needs of the medical world, which prefers stable and intuitive tools. They must also be modular enough to be used by the researchers who are less interested in stability than in rapid prototyping of the tools and algorithms, and who may want to change the parameters and the tools employed more frequently. The plurality of our users also requires cross-platform applications, including developing and testing on macOS, Windows and Linux. Thus, a continuous integration process had to be created and improved in order to obtain stable and efficient software tools.
Our quality approach is necessary because of the plurality of our users but also because we can promote our code through spin-off companies. The inHEART (
An important issue in the development of clinical and research applications is the long-term funding of personnel and equipment not only for software development and continuous improvement but also for database management, beta testing, management, etc. This is not always present in classical academic funding.
For now, the web portal is accessible only by the members of the VT MUSIC consortium. Our imaging application MUSICardio is accessible by members of our laboratory: researchers and engineers for research purposes, and expert operators and clinicians for the VT MUSIC consortium. The core of MUSICardio (medInria) is open-source and can be tested by the public; however, a lot of the functionalities that we developed are private for now.
Our tools must also manage more and more data, some of which are very heavy. This impacts the technologies we can use but also the minimal hardware that users must have to run our software: processors, graphics card, hard disk, etc. The balance is complicated to find between the power needed to visualise and apply treatments on very large data and the power of the computers that users can have, especially in hospitals or research centres.
6. Perspectives
The MUSIC project continues to be developed and improved. We are currently working on a future version of the MUSICardio 4 software in close collaboration with the Inria team that handles medInria. This new version will include new algorithms, tools, and will be based on medInria 4 on which we work also. New features will include an improvement of the global GUI of the software, the data management system as well as the management of plugins.
We have started an exploratory work of real-time simulation of catheters during a cardiac intervention in MUSICardio. This could allow us to display in real time a catheter in the application and to superimpose it with patient images or meshes to study their relative positions.
We also want to develop augmented reality tools to display and manipulate meshes in MUSICardio. This will allow us to study mesh in more detail as well as communicate about our tools in scientific events, and it is a first step to study the use of augmented reality in diagnostic or operative cardiology.
It would also be interesting to certify our binaries by Microsoft and Apple software platforms to simplify their installation. This would allow an official installation in hospitals where computer security is omnipresent: clinicians often do not have the computer rights to install unofficial software, so it can take a long time or a complicated route to install new software on these computers.
In addition, an online mesh viewer developed in Three.js (
We will continue to develop the VT MUSIC consortium to welcome new hospital partners and to enable patients to receive the best possible cardiac treatments.
Our collaboration in the medInria consortium will allow us to organise the improvement of the core application used in MUSICardio and to be a driving force in the development of these innovative tools.
Finally, MUSIC is ideally suited to develop novel approaches combining artificial intelligence and biophysical modelling, being at the interface of these two scientific areas.
7. Conclusions
The exceptional improvement of the cardiac field in the past years has led to a critical need of better multimodal technologies. The MUSIC platform is an emerging and robust technology which allows the user to import, export and work on a large spectrum of imaging data from various modalities and modelling approaches used in the cardiac field, offering functionalities dedicated to cardiovascular diagnosis and therapy guidance. The VT MUSIC consortium brings international hospitals together around an innovative interface to perform patient-specific ablation procedures, helping doctors to improve the lives of many patients every day.
Software, M.M., F.C., J.C., P.M., M.J., B.L. and V.O.; Supervision, B.Q., N.Z., Y.C., P.J., H.C. and M.S.; Writing—original draft, M.M., F.C., J.C., P.M., M.J., B.L. and V.O.; Writing—review & editing, M.S. All authors have read and agreed to the published version of the manuscript.
The authors declare no conflict of interest.
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Enlarge this image.Figure 5. Contours segmented in polygon ROI toolbox. This example shows an ongoing segmentation of the left (green) and right (blue) ventricles of the heart.
Enlarge this image.Figure 6. Polygon ROI features. (a) Repulsor: the orange circle handled by the user allows to push the boundary of a previously segmented region of interest (b) Multiple views: a segmentation is done simultaneously in two views. The arrows show an ingoing segmentation of lesions in blue and green in two reformatted brain images, where the user can easily follow the differences.
Enlarge this image.Figure 8. Automatic contouring of targets and organs at risk. The contours can be exported to the standard DICOM Rt-Struct format. Each ablation target and organ at risk is segmented with a color code: the list of contour names is displayed in a table in the right part of the application, as shown in this screenshot.
Enlarge this image.Figure 9. Examples of ECGI-oriented segmentations and mesh generation. (a): A 2D cross-section view of a segmented geometry showing the heart (dark red), lungs (green), liver (yellow) and the remaining tissue (red). (b): A 3D mesh of a torso volume.
Enlarge this image.Figure 10. Distribution of the electrical potential on the heart surface. Simulated ground truth solution (a), MFS solution (b) and FEM solution (c).
Enlarge this image.Figure 11. Automatic pipeline for LV (left ventricular) wall thickness map extraction and VA prediction. The arrows illustrate the background calculations and AI predictions. (A) Input 3D CT image, (B) segmentation masks, (C) short-axis oriented masks, (D) 3D wall thickness map, (E) 2D bullseye thickness map.
Enlarge this image.Figure 12. The output results at the end of the pipeline. The VA classification result is shown in the bottom right corner at 98%. The two views show the mesh of the LV wall thickness and classification attention map. The views could be linked together, showing that higher attention (red on left mesh) was placed within the thinning region of the LV (yellow/green on right mesh). The interactive side-by-side display of the thickness and attention map helps to explain the model prediction and increases the user’s confidence in the AI tool.
Enlarge this image.Figure 13. Example of preclinical RF ablation procedure performed under MRI. Three-dimensional (3D) balanced SSFP sequence acquired at the beginning of the session was processed within MUSICardio (a) to segment the cavities of the RV and LV. The resulting 3D meshes were then exported to the Montecarlo platform (b), allowing real-time 3D catheter navigation and visualisation. After completion of the procedure, 3D T1-weighted images were acquired and processed in MUSICardio (c) to automatically compute the transmurality map for each location of RF delivery. Orange arrows show RF lesions with a central thermal necrosis in the hyposignal surrounded by a hypersignal corresponding to oedema.
External libraries used in medInria, core of MUSICardio.
| Library | Description | Origin |
|---|---|---|
| DCMTK | DICOM management | OFFIS e.V |
| dtk | Tools for modular software development | Inria |
| ITK [ |
Scientific imaging management | Kitware |
| LCC LogDemons [ |
LCC Log Demons algorithms | Inria |
| QtDCM | Qt widgets to handle DICOM images | Inria |
| RPI | Image registration algorithms | Inria |
| TTK | Tensor algorithms | Inria |
| VTK | Image processing, 3D graphics, volume rendering and visualisation | Kitware |
External libraries used in MUSICardio plugins.
| Library | Description | Origin |
|---|---|---|
| Eigen [ |
Linear algebra library | Inria |
| FFmpeg | Video tools | FFmpeg project |
| Mmg [ |
Remeshing tools | Inria |
| QuaZip | Qt zip/unzip tools | S.A. Tachenov |
| Qwt | Qt 2D plots library | U. Rathmann, J. Wilgen |
| TetGen [ |
Tetrahedral mesh generator, 3D Delaunay triangulator | Weierstrass Institute |
| zlib | Data compression library | J.-L. Gailly, M. Adler |
List of the main imaging and meshing tools in MUSICardio.
| Worspace | Description |
|---|---|
| Diffusion | Processing of diffusion and tractography. |
| EAM Analysis | Electroanatomical mapping analysis tools. |
| Filtering | Image binary operation (AND, OR, XOR, NOT), constant operation (addition, multiply, etc.), blurring, normalisation, thresholding, edge filters, mask application, morphological filters (dilate, erode, close, open), bias correction, etc. |
| Meshing | Convert mesh to and from mask, data array management, interpolation, projections, distance computations, mesh deformation, morphological mesh filters, binary operations on meshes, fiber computations, decimation, refine, etc. |
| Reformat | Crop data, extract 3D data from 4D ones, merge images and meshes, reslice (change orientation, pixel size, image dimensions), super resolution (compute accurate data from orthogonal ones), move an image to a new location. |
| Registration | Register two images with diffeomorphic demons [ |
| Radiotherapy | Radiotherapy ablation planning tools with DICOM-RT-STRUCT data. |
| Segmentation | Mapping of myocardial depth isthmuses, histogram analysis, level set segmentation, paint and polygon segmentations, VOI (volume of interest) segmentation, variational segmentation through landmarks, as well as vessel segmentation. |
| Simulation | Simulation processing tools, forward problem, inverse problem, FEM and MFS simulation tools (see |
List of tools dedicated to mesh enhancement.
| Tool | Description |
|---|---|
| Binary operations | Intersection, union and difference between two meshes. |
| Convert mesh to mask | Convert mesh into mask to work on it and convert back. |
| Mesh cleaning | Filling holes in meshes, remove unused vertices, merge very close points. |
| Mesh interactors | Selecting and deleting selected cells. |
| Mesh manipulation | Apply transformation to mesh: translation, rotation, scaling. |
| Mesh registration | Registration of a mesh to another mesh using landmarks or the iterative closest point method. Registration of a mesh to an image using landmarks. |
| Merge volumes | Concatenate meshes. |
| Projection mapping and interpolation | Projection of data on meshes, scalar or vector field, linear and nonlinear interpolation. |
| Remeshing | Decimate, refine, smooth meshes. |
References
1. McCormick, M.; Liu, X.; Ibanez, L.; Jomier, J.; Marion, C. ITK: Enabling reproducible research and open science. Front. Neuroinform.; 2014; 8, 13. [DOI: https://dx.doi.org/10.3389/fninf.2014.00013] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/24600387]
2. Lorenzi, M.; Ayache, N.; Frisoni, G.; Pennec, X. LCC-Demons: A robust and accurate symmetric diffeomorphic registration algorithm. NeuroImage; 2013; 81, pp. 470-483. [DOI: https://dx.doi.org/10.1016/j.neuroimage.2013.04.114] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/23685032]
3. Guennebaud, G.; Jacob, B.; Avery, P.; Bachrach, A.; Barthelemy, S. Eigen v3. Available online: http://eigen.tuxfamily.org (accessed on 1 June 2022).
4. Dapogny, C.; Dobrzynski, C.; Frey, P. Three-dimensional adaptive domain remeshing, implicit domain meshing, and applications to free and moving boundary problems. J. Comput. Phys.; 2014; 262, pp. 358-378. [DOI: https://dx.doi.org/10.1016/j.jcp.2014.01.005]
5. Si, H. TetGen, a Delaunay-Based Quality Tetrahedral Mesh Generator. ACM Trans. Math. Softw.; 2015; 41, 11. [DOI: https://dx.doi.org/10.1145/2629697]
6. Vercauteren, T.; Pennec, X.; Perchant, A.; Ayache, N. Diffeomorphic demons: Efficient non-parametric image registration. NeuroImage; 2009; 45, pp. S61-S72. [DOI: https://dx.doi.org/10.1016/j.neuroimage.2008.10.040] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/19041946]
7. Gepstein, L.; Hayam, G.; Ben-Haim, S.A. A Novel Method for Nonfluoroscopic Catheter-Based Electroanatomical Mapping of the Heart. Circulation; 1997; 95, pp. 1611-1622. [DOI: https://dx.doi.org/10.1161/01.CIR.95.6.1611]
8. Koutalas, E.; Rolf, S.; Dinov, B.; Richter, S.; Arya, A.; Bollmann, A.; Hindricks, G.; Sommer, P. Contemporary Mapping Techniques of Complex Cardiac Arrhythmias-Identifying and Modifying the Arrhythmogenic Substrate. Arrhythm Electrophysiol. Rev.; 2015; 4, 19. [DOI: https://dx.doi.org/10.15420/aer.2015.4.1.19]
9. Cuculich, P.S.; Schill, M.R.; Kashani, R.; Mutic, S.; Lang, A.; Cooper, D.; Faddis, M.; Gleva, M.; Noheria, A.; Smith, T.W. et al. Noninvasive Cardiac Radiation for Ablation of Ventricular Tachycardia. N. Engl. J. Med.; 2017; 377, pp. 2325-2336. [DOI: https://dx.doi.org/10.1056/NEJMoa1613773]
10. Gianni, C.; Rivera, D.; Burkhardt, J.D.; Pollard, B.; Gardner, E.; Maguire, P.; Zei, P.C.; Natale, A.; Al-Ahmad, A. Stereotactic arrhythmia radioablation for refractory scar-related ventricular tachycardia. Heart Rhythm; 2020; 17, pp. 1241-1248. [DOI: https://dx.doi.org/10.1016/j.hrthm.2020.02.036] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/32151737]
11. Barr, R.C.; Ramsey, M.; Spach, M.S. Relating epicardial to body surface potential distributions by means of transfer coefficients based on geometry measurements. IEEE Trans. Biomed. Eng.; 1977; pp. 1-11. [DOI: https://dx.doi.org/10.1109/TBME.1977.326201] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/832882]
12. Wang, Y.; Rudy, Y. Application of the method of fundamental solutions to potential-based inverse electrocardiography. Ann. Biomed. Eng.; 2006; 34, pp. 1272-1288. [DOI: https://dx.doi.org/10.1007/s10439-006-9131-7] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/16807788]
13. Karoui, A.; Bear, L.; Migerditichan, P.; Zemzemi, N. Evaluation of fifteen algorithms for the resolution of the electrocardiography imaging inverse problem using ex-vivo and in-silico data. Front. Physiol.; 2018; 1708. [DOI: https://dx.doi.org/10.3389/fphys.2018.01708] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/30555347]
14. Ly, B.; Finsterbach, S.; Nuñez-Garcia, M.; Cochet, H.; Sermesant, M. Scar-Related Ventricular Arrhythmia Prediction from Imaging Using Explainable Deep Learning. Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Springer: Berlin, Germany, 2021; pp. 461-470. [DOI: https://dx.doi.org/10.1007/978-3-030-78710-3_44]
15. Yon, M.; Delcey, M.; Bour, P.; Grissom, W.; Quesson, B.; Ozenne, V. Continuous cardiac thermometry via simultaneous catheter tracking and undersampled radial golden angle acquisition for radiofrequency ablation monitoring. Sci. Rep.; 2022; 12, 4006. [DOI: https://dx.doi.org/10.1038/s41598-022-06927-9] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/35256627]
16. Toupin, S.; Bour, P.; Lepetit-Coiffé, M.; Ozenne, V.; de Senneville, B.D.; Schneider, R.; Vaussy, A.; Chaumeil, A.; Cochet, H.; Sacher, F. et al. Feasibility of real-time MR thermal dose mapping for predicting radiofrequency ablation outcome in the myocardium in vivo. J. Cardiovasc. Magn. Reson.; 2017; 19, pp. 1-12. [DOI: https://dx.doi.org/10.1186/s12968-017-0323-0] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/28143574]
© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.
Details
; Quesson, Bruno 1
; Zemzemi, Nejib 4 ; Coudière, Yves 4 ; Jaïs, Pierre 1
; Cochet, Hubert 1 ; Sermesant, Maxime 5
1 IHU Liryc, 33600 Pessac, France;
2 Inria Bordeaux, 33000 Bordeaux, France;
3 Centre Inria d’Université Côte d’Azur, 06902 Sophia Antipolis, France;
4 IHU Liryc, 33600 Pessac, France;
5 IHU Liryc, 33600 Pessac, France;