A Hybrid Model for COVID-19 Monitoring and Prediction

Abstract

COVID-19 is caused by the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) and has a case-fatality rate of 2–3%, with higher rates among elderly patients and patients with comorbidities. Radiologically, COVID-19 is characterised by multifocal ground-glass opacities, even for patients with mild disease. Clinically, patients with COVID-19 present respiratory symptoms, which are very similar to other respiratory virus infections. Our knowledge regarding the SARS-CoV-2 virus is still very limited. These facts make it vitally important to establish mechanisms that allow to model and predict the evolution of the virus and to analyze the spread of cases under different circumstances. The objective of this article is to present a model developed for the evolution of COVID in the city of Manizales, capital of the Department of Caldas, Colombia, focusing on the methodology used to allow its application to other cases, as well as on the monitoring tools developed for this purpose. This methodology is based on a hybrid model which combines the population dynamics of the SIR model of differential equations with extrapolations based on recurrent neural networks. This combination provides self-explanatory results in terms of a coefficient that fluctuates with the restraint measures, which may be further refined by expert rules that capture the expected changes in such measures.

1. Introduction

A new virus called SARS-CoV-2 has caused a worldwide health crisis, which has been difficult to tackle as a result of the limited knowledge regarding the virus. The scientific community has only been able to learn about the virus as the pandemic progressed and more information was available. For example, it has been possible to determine morbidity and mortality rates in different population groups, as well as other environmental and social factors. However, there is still a lack of information regarding the SARS-CoV-2 virus. Moreover, it has not been possible to identify the factors that lead to serious illness in some Coronavirus patients.

The first cases of Coronavirus were reported in China, which became the epicenter of COVID-19 in January. Meanwhile, there was only a small amount of cases in other countries, which had been imported by travelers. By the end of the month, there were 10,000 cases in China and 129 abroad. However, the virus spread quickly and in February there were several outbreaks in South Korea, Italy, Germany and Spain. There have been more than 80,000 deaths and currently, more than 1.4 million cases have been confirmed in most of Central America, South America, North America, Asia, Australia and Europe, and the numbers continue to grow everyday. The spread rate is different in each country; however, the suppression and mitigation policies do not seem to be the only factor behind those variations. Japan, Hong Kong and Singapore have experienced a steady increase in cases since January. In Italy, the virus spread much faster, leading to a state of emergency. Soon, the virus spread to other European countries such as Spain, France or Germany.

The statistics gathered regarding this pandemic are far from representing the real situation and the level of contagion is much greater than reported by official authorities. The reason for this is that there are no adequate and reliable mechanisms that would be capable of diagnosing COVID-19 rapidly. This means that we are measuring two variables simultaneously: the case rate in each country and the capacity of the government to efficiently detect cases.

There have been considerable advances in the field of Precision Medicine (PM), where new techniques give us more opportunities to learn about COVID-19. If healthcare professionals had more knowledge regarding the virus, they would be able to establish an effective treatment for their patients, adapted to their genetic profile and predicted evolution. Until a suitable treatment is found or the mass vaccination campaign is completed, it is crucial to take the adequate measures that will help slow down the spread of the virus. To this end, the short, medium and long term evolution of the pandemic must be predicted, taking into account how it has evolved in a particular area and the changes in trends associated with mobility restrictions, confinement, etc. Therefore, a prediction system has been developed with detailed explanations of the prediction results, which are of help when making decisions. Auto-Explainable: Artificial Intelligence offers the opportunity to comprehend the results of AI systems and the system incorporates a XAI module for this purpose. Unlike in black box models, an intermediate representation is obtained with a direct epidemiological interpretation, which also allows the model to be adjusted by introducing additional hypotheses on the nature of the contention measures.

This paper presents a model developed to forecast the evolution of SARS-CoV-2; it is an artificial intelligence based model that combines several algorithms. The model is capable of assisting healthcare authorities in making decisions, for example, in selecting the most adequate spread suppression mechanisms for a region or in the adequate dimensioning and prioritization of the available medical resources. This model has been developed in the framework of the project Sistema De Inteligencia Epidemiológica Para El Apoyo En La Toma De Decisiones En El Control Del COVID-19 En LatinoAmérica, conv. no. 1015 de 2020, code 1127101576535. Sponsored by the Ministerio de Ciencia, Tecnología e Innovación de Colombia. The model is in use by the local medical authorities.

During the health emergency generated by the novel coronavirus SARS-CoV-2 and COVID-19 pandemic, in Caldas department, Colombia, the need of computer tools developed to measure and predict the socioeconomic impacts that can affect Caldas department from the beginning of the pandemic to an estimated time in the future.

Manizales, the capital city of Caldas department, showed a special requirement, due to a much higher population density in comparison with other municipalities of the department and in consequence with higher probability of new cases and fatalities caused by COVID-19. Health authorities need information in real time that supports the decision making related with the capacity and occupation of the available intensive care units (ICUs). The last, due to the remissions of patients arriving from different municipalities of the entire department in addition to the local demand. Considering the economic reactivation as well as relevant information needed to prioritize the opening of certain sectors that greatly boost the economic growth of the city and the region.

A review of Artificial Intelligence models for predicting the evolution of time series is presented. The mixture of experts developed for Manizales is then described and 83 evaluated. Finally some conclusions are presented.

2. Predicting the Pandemic with Artificial Intelligence Models

Numerous AI models are being used to predict the evolution of the pandemic. This article does not attempt to present an exhaustive review of the existing approaches. Rather, it attempts to show some varied examples of proposals which include intelligent algorithms, and which are representative of the state of the art. A list of some of the approaches used for the prediction of the evolution of COVID-19 infections is presented. Some are proposals based on local data while others are more ambitious, international studies. There are proposals that use deep learning [1,2,3,4,5,6,7], others use the Random Forest model [8,9], networks based on domain models [10] and SIR models [11]. The hybrid model proposed for the prediction in the present research includes an SIR model. Auto encoders [12,13] and binary classifiers [14] are also used in the prediction of COVID-19 evolution. Support Vector Regression and stacking-ensemble have also been proposed with some success [15], in addition to XGBoost classifier [16].

Artificial Intelligence has experienced significant growth over the last decades, giving rise to numerous technologies and approaches within the AI paradigm. However, separately, all of these approaches have some weaknesses which can be mitigated by combining the approaches, techniques or concepts from the different subfields of AI. This practice is called hybrid AI [17] and includes neural computing, machine learning, fuzzy logic and evolutionary algorithms [18,19,20].

For example, connectionist approaches in AI focus on the use of neural networks, while symbolic approaches involve the representation of problems in a way that can be understood by humans. Unfortunately, the results generated by neural networks [21] are opaque while symbolic approaches tend to generalize solutions. Thus, thanks to hybrid AI, both approaches are able to offer their benefits and to overcome their limitations.

Moreover, thanks to the variety of the technologies used in hybrid AI, solutions can be found to countless problems in a number of disciplines, including bioinformatics, control engineering, software verification, visual intelligence and ontological learning.

To illustrate this with a specific example, Deep Neuro Fuzzy algorithms [22] are implemented in smart systems; they jointly employ techniques based on fuzzy logic and deep neural networks to make the results scalable. The optimal performance of this approach has been demonstrated in practice in a series of case studies, offering advanced reasoning for the identification of intricate relations among data and the ability to build predictive algorithms (e.g., vehicle speed prediction [23] or overlapping networks).

Explainable AI (XAI) algorithms combine reasoning and learning. Separately, both have been the focus of countless researches over the years. Now, the interest of the research community lies in uniting these two approaches [24]. These algorithms join the conceptual approach of symbolic reasoning, which offers human readable results, with connectionist models so that links between the symbols may be created [25].Trustworthy AI [26] is the basis that underpins XAI, striving towards AI developments being ethical, constitutional and reliable.

This research makes progress in the field of XAI, which has attracted the attention of the scientific community because it is able to provide solutions that can be explained, for example, by transparently showing the process in machine learning when decisions are made [27].

Symbols have the capacity to adapt their knowledge to different contexts by transferring reward signals. Symbolic artificial intelligence prevailed in AI studies in the post-war era, until the late 1980s [28]. Symbolic developments include rule engines, expert systems or knowledge graphs (e.g., Google’s Knowledge Graph [29]). There are differences in the learning of Machine Learning and traditional symbolic reasoning. In Machine Learning algorithms, the input data is correlated with the outputs to created new rules. In turn, symbolic reasoning, systems are static, they require a human expert to intervene in the process by establishing and codifying the rules that link two phenomena. However, symbolic systems encounter some hindrances: (i) expert systems are monotonous [30], that is, as new rules are added, the system’s knowledge grows, however, he system is unable to unlearn a rule if the new one contradicts the prior. (ii) The computer does not understand the meaning of the symbols (contrary to neural networks, which associate symbols to vectorized representations of data).

To provide the system with explainable results, a model is proposed that combines the numerical solution of a system of differential equations, modeling the dynamics of the disease on the basis of coefficients which can be interpreted. This is combined with a deep learning system based on recurrent neural networks to predict the evolution over time of these coefficients. In addition, the system allows for the incorporation of additional hypotheses about the containment measures taken. In conclusion, the system allows us to interpret and adapt its predictions on the basis of these coefficients.

3. Proposed Model and Case Study: Caldas COVID-19 Evolution

To obtain predictions on the evolution of the disease in Caldas, a hybrid model has been proposed in which artificial intelligence and an expert system are incorporated into the SIR epidemiological model. A diagram of this model can be seen in Figure 1. The model is described in more detail below.

3.1. Extraction of Input Variables

Let $d.\in D$ be a tuple with information on patients who have been infected by COVID-19 in Caldas and whose components we denote as $d.\mathrm{diagnostic}\_\mathrm{date}$. A representation of the information for each patient is required by this model, as shown in Table 1.

From individual instances, the curves of each compartment of the SIR model are obtained using

$\left\{\begin{array}{cc}\hfill & S\left(t\right)=\#\{d\in D|d.\mathrm{infection}\_\mathrm{start}\_\mathrm{date}<t\},\hfill \\ \hfill & I\left(t\right)=\#\{d\in D|(d.\mathrm{infection}\_\mathrm{start}\_\mathrm{date}\ge t)\cap (d.\mathrm{recovery}\_\mathrm{date}<t\hfill \\ \hfill & \cup (d.\mathrm{death}\_\mathrm{date}<t)\},\hfill \\ \hfill & R\left(t\right)=\#\{d\in D|d.\mathrm{recovery}\_\mathrm{date}\ge t\cup d.\mathrm{death}\_\mathrm{date}\ge t\}.\hfill \end{array}\right.$

where the infection start date (infection_start_date) is identified with the date when symptoms began to appear, or with the date of the diagnosis in the case of asymptomatic individuals.

3.2. SIR Model

The SIR model is an epidemiological model that is given by the following system of differential equations:

(1)$\left\{\begin{array}{cc}\hfill & \frac{\mathrm{d}S}{\mathrm{d}t}=-\beta ·\frac{IS}{N}\hfill \\ \hfill & \frac{\mathrm{d}I}{\mathrm{d}t}=\beta ·\frac{IS}{N}-\gamma I\hfill \\ \hfill & \frac{\mathrm{d}R}{\mathrm{d}t}=\gamma I\hfill \end{array}\right.$

where S corresponds to the number of people without immunity to the infectious agent, i.e., susceptible population, I to the infected population and R to the individuals who are immune to the infection (recovered or deceased). All these variables have a time dependence that has been omitted from the notation to make it clearer. In addition, the model states that

(2)$N=S+I+R$

where N is a constant that refers to the total population. For the case we are dealing with, Caldas, takes the value of 993870 [31].

Therefore, the parameters of this model are $\beta$ and $\gamma$, variables that evolve over time. These are obtained by solving the system of differential Equation (1) with a sliding window adjustment. The Runge–Kutta 45 method is used to solve it, this means a Runge–Kutta whose error is controlled by assuming the accuracy of the fourth-order method but taking fifth-order steps in the accuracy formula.

3.3. Coefficient Extrapolation

The series $\beta \left(t\right)$ is extrapolated using a recurrent neural network called LSTM [32].

The series $\gamma \left(t\right)$ is extrapolated by taking the median of the series $\gamma \left(t\right)$ for $t<t_{max}$. This parameter is does not fluctuate much as it is the inverse of the time it would take for a person to recover from the disease, therefore, it is a constant.

LSTM is a type of neural network where each memory cell, $c_j$, is built around a central linear unit with a fixed self-connection (the CEC). In addition to the ${\mathrm{net}}_{c_j}$, $c_j$ obtains input from a multiplicative unit ${\mathrm{out}}_j$ (the “output gate”), and from another multiplicative unit ${\mathrm{in}}_j$ (the “input gate”), holding that

(3)$\begin{array}{cc}\hfill & y^{{\mathrm{out}}_j}\left(t\right)=f_{{\mathrm{out}}_j}\left(ne{t}_{{\mathrm{out}}_j}\left(t\right)\right)\hfill \\ \hfill & y^{{\mathrm{in}}_j}\left(t\right)=f_{{\mathrm{in}}_j}\left(ne{t}_{{\mathrm{in}}_j}\left(t\right)\right)\hfill \end{array}$

where

(4)$\begin{array}{cc}\hfill & {\mathrm{net}}_{{\mathrm{out}}_j}\left(t\right)=\sum _u{w}_{{\mathrm{out}}_{j}u}{y}^u(t-1)\hfill \\ \hfill & {\mathrm{net}}_{{\mathrm{in}}_j}\left(t\right)=\sum _u{w}_{{\mathrm{in}}_{j}u}{y}^u(t-1)\hfill \end{array}$

and

(5)${\mathrm{net}}_{c_j}\left(t\right)=\sum _u{w}_{c_{j}u}{y}^u(t-1).$

The index u of the summations sweeps through the different memory cells present in the neural network. Therefore, at time t the output of the cell $c_j$, which we denote by $y^{c_j}\left(t\right)$, is defined as

(6)$y^{c_j}\left(t\right)=y^{{\mathrm{out}}_j}\left(t\right)h\left(s_{c_j}\left(t\right)\right),$

where the internal state $s_{c_j}\left(t\right)$ is

(7)$\left\{\begin{array}{cc}\hfill & s_{c_j}\left(0\right)=0\hfill \\ \hfill & s_{c_j}\left(t\right)=s_{c_j}(t-1)+y^{{\mathrm{in}}_j}\left(t\right)g\left({\mathrm{net}}_{c_j}\left(t\right)\right)\mathrm{for}t>0.\hfill \end{array}\right.$

With this structure, LSTM is able to overcome the problems of error backpropagation by either blowing up or vanishing, due to the fact that the time evolution of the backpropagated error depends exponentially on the size of the weights. In fact, the multiplicative input gate unit protects the memory contents stored in j from perturbations by irrelevant inputs just as the multiplicative output gate unit protects the other units from perturbations by currently irrelevant memory contents stored in j.

In this work, a multi-step LSTM with tanh activation for the hidden state and the output hidden state, and a sigmoid activation for the input, forget, and output gates was used to perform predictions up to a 20-day horizon.

3.4. Expert System for Restraint Measure Modeling

The implementation of different measures that the government of Caldas is forced to impose in order to stop the expansion of the disease in critical moments, causes considerable changes in the $\beta \left(t\right)$ parameter of the SIR model. For this reason, it is necessary to incorporate an expert system that allows to make predictions on the basis of the possible exceptional measures that may be imposed in the future to counteract the pandemic.

The expert system’s key is, depending on the type of legal measure applied by the government (isolation, restriction of mobility,...), to modify the parameter $\beta \left(t\right)$ for $t>t_{\max }$, which was obtained with the LSTM neural network. To extract some general rules with which to modify these parameters, it is first necessary to classify the measures according to their degree of restriction or relaxation and observe the effect that each of the measures has had in the past (Figure 2).

Given that we have few measurements to be able to carry out a detailed statistical study, on the basis of the observations made in Figure 2 some general approximations have been extracted from the increase or decrease in spread rate after the implementation of each type of measure. These are summarized in Table 2.

Taking into consideration that the effect a measure has on the spread rate is not visible for a few days after the implementation, due to the incubation period of the disease (t time lag) and the adaptation period until the restriction becomes fully effective (k time lag), a sigmoidal function (Figure 3) is implemented to modulate these changes in $\beta$.

(8)$f_{t,k}\left(x\right)=\left\{\begin{array}{cc}1\hfill & \mathrm{if}x\le t,\hfill \\ \frac{\sigma \left(x\right)-\sigma \left(t\right)}{\sigma (t+k)-\sigma \left(t\right)}(\text{Target\_change}-1)+1\hfill & \mathrm{if}t<x\le t+k,\hfill \\ \text{Target\_change}\hfill & \mathrm{if}t>t+k,\hfill \end{array}\right.$

where

$\sigma \left(x\right)=\frac{1}{1+e^{-x}}.$

As a result of the decomposition carried out by the model and the ability to introduce these additional assumptions, the resulting system makes it possible to understand the causes of the prediction, adapting to the hypotheses to be made about them.

3.5. Platform Achitecture

To tackle this problem, it is proposed to use a microservices-oriented architecture that favours the growth of the system, providing it with flexibility, horizontal and vertical scalability (Figure 4). To achieve this, the following subsystems must be developed:

• Data extraction: periodically (once a day) extracts statistics on the impact of the COVID-19 pandemic from Colombia’s open portal (https://www.datos.gov.co/, accessed on 22 February 2021).

• Deep intelligence: This is a platform used to manage the different workflows in projects in the field of Big Data, Artificial Intelligence and Smart Cities. Specifically, it is used in the following ways:

  • -. Main data warehouse, as it stores both the data resulting from extraction and the data resulting from predictions.

  • -. Visualization engine: it is able to generate interactive visualizations with the stored data to show the result of the whole process in the most summarized way possible to the user.

• Data analysis: this is a system that periodically takes Deep Intelligence data on the indicators of COVID-19 impact in Colombia, in order to make the relevant predictions with the hybrid model explained above. Finally, it stores the results in Deep Intelligence.

As a result of the infrastructure built, a system for monitoring multiple variables is available, including those from which the variables necessary for the construction of the model are derived.

The origin of all the information recorded in the Epidemiological Intelligence System (Sistema de Inteligencia Epidemiológica (SIE)) comes from the information system that centralizes the data of the department of Caldas called registro COVID [33]. The information in this source is synchronized every hour.

Among the information included within the monitoring system, the following can be found:

• Personal data

  • -. ID: it is the patient identifier; the identification document is encrypted to preserve the privacy of the information.

  • -. Municipality: name of the municipality where the patient resides.

  • -. Department: name of the department where the patient resides.

  • -. Age Criteria: identifies whether the patient is an adult (A) or a minor (M)

  • -. Age: it is the numerical value of the patient’s age at the time of registration.

  • -. Gender: gender identifier of the patient, may be female (F) or male (M).

  • -. Neighborhood/Village: indicates the name of the neighborhood or village of the patient’s residence.

  • -. Address: the exact location of the patient.

  • -. Patient state: identifies if the patient has recovered or deceased.

  • -. Management: Describes whether the person was hospitalized (H) or stayed at home (C)

  • -. Type of contagion: states if the contagion has been within the community, from a relative or imported.

  • -. State of severity of the patient: a difference between mild, moderate or serious.

• General section:

  • -. Alert: allows to register the test status of the patient: in study, discarded, confirmed.

  • -. Record date: corresponds to the date on which the patient information was recorded.

  • -. Onset of symptoms date: corresponds to the date the patient started having symptoms.

  • -. Diagnosis date: date the patient’s infection was confirmed.

  • -. Recovered date: corresponds to the date on which the patient was recovered.

  • -. Date of death: date of the patient’s death.

Some of the steps in the construction of this model within the Deep Intelligence platform are shown in Figure 5.

4. Results

This section analyzes the results obtained from the conducted case study, as described in detail in the previous section. For this purpose, the chosen input sample is the data collected during 3 months, from mid-August to mid-November, taken from the web [33], where data is accessible under an open license. Using these data it is possible to compare the differences between what happened in reality and the different predictions that the model is able to make. Figure 6 shows the obtained predictions using different assumptions, starting from a randomly chosen date (4 September). That day was a weekday when no change in government measures was imposed, so the obtained results are in line with expectations. The predictions extend up to a 20-day horizon. Further details on the prediction error are given below.

On the other hand, Figure 7 shows the obtained results, assuming that no restrictive measures are applied, with an estimated 80% confidence interval.

To provide more insight on the error distribution, an example is shown in Figure 8a for the case of a one week prediction horizon, aggregated metrics can be provided using its absolute error to prevent sign compensation. The mean value of the absolute value of the relative error is thus depicted in Figure 8b, along with the sample standard deviation of its distribution.

Overall the relative error in the number of new cases is typically around 25%. The error in the number of active cases is usually below 10% for near-future predictions (below one week), and increase linearly once this limit is extended.

5. Conclusions and Future Work

Predicting the spread of COVID-19 is a complicated task, due to the impact of contingency measures, which continually change the population dynamics. In this work, a prediction system has been proposed that ensures that the evolution occurs within the framework of the dynamics of the differential equation systems of the disease. This makes it possible to interpret the results of the system from the point of view of the measures taken. In addition, a mixture of experts system for the correction of predictions has been designed to incorporate additional hypotheses on the contingency decisions taken. The system is built with an SIR model and an LSTM, and is capable of representing the state of the pandemic at a given time and projecting it into the future, so that any variations that appear model the system and it is fed back. The recurrent network adapts the coefficients of the SIR model and these explain the evolution of the system and possible changes. The model corrects the prediction once the SIR model identifies structural changes in the way the pandemic evolves and adapts its coefficients and results. In the same way, changes can be introduced into the model once changes in mobility constraints are known, for example. The decomposition made by the model facilitates the understanding of the results, as well as the incorporation of additional assumptions about the restrictions.

The mean squared error in terms of the number of positive cases has been estimated to be around 18% and 22% for the active and new cases respectively in the 2-week predictions. The proposed model has been extended to predict in the medium term 4-8 months and assess the evolution of the pandemic until remission by taking into account the different vaccination scenarios, possible restrictions, mutation effects and regulatory changes. The aim is to create a real decision-support system capable of providing support in making an effective response to the pandemic.

Author Contributions

Conceptualization, L.F.C.O.; data curation, F.P.-S.; formal analysis, G.H.; funding acquisition, A.C.-M.; investigation, P.C., J.A.-L., G.A.I., A.C.-M. and J.M.C.; methodology, J.M.C.; project administration, P.C. and J.M.C.; software, C.S.-C.-G. and G.H.; supervision, A.C.-M. and J.M.C.; writing—original draft, J.M.C.; writing—review and editing, C.S.-C.-G. and G.H. All authors have read and agreed to the published version of the manuscript.

Funding

This research was funded by the Ministerio de Ciencia, Tecnología e Innovación de Colombia grant number 1127101576535 (convocatoria Mincienciaton No. 1015 de 2020). The research was also partially supported by the project “Computación cuántica, virtualización de red, edge computing y registro distribuido para la inteligencia artificial del futuro”, Reference: CCTT3/20/SA/0001, financed by Institute for Business Competitiveness of Castilla y León, and the European Regional Development Fund (FEDER).

Conflicts of Interest

The authors declare no conflict of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript, or in the decision to publish the results

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Figures and Tables

Figure 1. A historical dataset is used to calculate how COVID-19 evolves over a certain period of time and the curves S (the time-dependent susceptible population), I (the time-dependent infected population) and R (the time-dependent removed (recovered, death) population) are extracted over this period of time. These variables are used to fit an SIR model using sliding windows. The differential equations are solved using a Runge–Kutta method and the fit is performed in the sense of mean squares, thus extracting the unknown parameters of the model: β and γ, as well as the basic reproductive number R0, all of them being functions of time. To extrapolate these parameters to higher time values (β′,γ′,R0′), an LSTM neural network is used, the results of which are further refined by using an expert system that takes into account possible future changes in the constraints imposed by the government (β″,γ″,R0″ in the diagram). Finally, by solving the SIR model again with these extrapolated coefficients, the predictions in the evolution of the S′, I′ and R′ curves are obtained.

Figure 2. Representation of the variation of the β parameter without applying the expert system (blue curve). The horizontal coordinate represents time, while the vertical coordinate represents this coefficient. The higher the value is, the more contacts have infected individuals. The solid vertical lines represent the dates on which the government imposed a new measure and the dotted vertical lines are the approximate dates on which the consequences of these measures are expected to become visible. This time lag, which has been considered to be 10 days, is due to the existence of an incubation period of the disease, where no symptoms are present even though contagion has occurred, and an adaptation period until the measure is fully established.

Figure 3. Representation of the sigmoidal function that has been used when applying a strong relaxation measure, assuming an incubation period t of 3 days and an adaptation period k to the new measure of 7 days. As a result, a smooth transition transition is obtained when a measure is considered in the expert model.

Figure 4. Architecture used to implement the proposed predictive model. The data is obtained from the CKAN server in Colombia’s open portal (upper left), using a Python-implemented API served with Flask (lower left), which also accesses additional information sources. The interaction with the Deep Intelligence platform (lower right) is performed by means of its API. Users (upper right) are able to access the platform using its graphical interface.

Figure 5. Some of the steps performed to build the system in Deep Intelligence [34]. (a) A data source in the platform, showing structured information. (b) Monitoring of additional indicators using the visualization tools provided by the platform. (c) Global curve visualization from an external source. (d) Visualization of the predictions obtained by the model.

Figure 6. Prediction of the evolution of the number of positive cases from 4 September. The series show the different predictions when various hypothesis on the restriction measures are used in the system. The prediction curves extend up to 20 days, which is the intended prediction range.

Figure 7. Prediction of the evolution of the number of positive cases from 4 September with a confidence interval of 80 %, assuming that no restrictive measures are applied. The prediction extends up to 20 days, which is the intended prediction range.

Figure 8. Analysis of the error distribution in the prediction of infected individuals. (a) Distribution of the relative error for a 7-day prediction horizon. (b) Error as a function of the prediction horizon. Solid lines depict the mean of the absolute value of the relative error, while the shaded regions depict its sample standard deviation.

Table 1

Input data.

Case ID	Symptoms Onset Date	Diagnostic Date	Recovery Date	Death Date
2592	2020-03-26	2020-04-11	2020-04-23	-
2593	2020-03-25	2020-04-11	2020-05-25	-
2594	2020-03-25	2020-04-11	2020-05-05	-
2595	2020-03-25	2020-04-11	2020-05-16	-
2596	2020-03-28	2020-04-11	-	2020-04-01
2597	2020-03-22	2020-04-11	2020-04-16	-
2699	2020-04-09	2020-04-11	2020-04-21	-

Table 2

Percentage of change according to the type of measure applied, based on the date from the measures taken in Caldas (Figure 2).

Government Measures	Percentage Change	Target Change
Strong restriction	−30%	0.7
Slight restriction	−10%	0.9
Slight relaxation	+10%	1.1
Strong relaxation	+30%	1.3