Digital Avatars: Promoting Independent Living for Older Adults

Abstract

Population ageing, together with the desire to maintain an autonomous lifestyle, poses today’s societies with a challenge that technological advances can help considerably to cope with. The widespread use of smartphones and their increasing computing power and storage capacity make them the ideal tool to achieve this goal. In this paper, we present Digital Avatars, a software framework adapted to the needs of older adults who wish to preserve their lifestyle, but who require assistance through technology. Building on previous work on the People as a Service model, Digital Avatars takes advantage of a smartphone’s capabilities and services to collect information about the people who own them. To do this, it applies Complex Event Processing techniques extended with uncertainty to infer the habits, preferences, and needs of the device owner to build with them an enhanced virtual profile of the user. These virtual profiles are the mechanism for monitoring the quality of life of older adults: analyzing their patterns of activity, reminding them of medication schedules, or detecting risky situations that generate alerts to relatives, caregivers, or the community health system.

This work is licensed under http://creativecommons.org/licenses/by/4.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.

1. Introduction

According to the World Health Organization [1], from 2000 to 2016, life expectancy has increased by 5 years (72.5 to 77.5) in Europe and by 5.5 years (66.5 to 72.0) globally. This progressive growth in life expectancy entails an increase in population ageing. The health of this segment of older adults needs continuous supervision and care: regular medical examinations and tests, specific pharmacology, and other therapies. Our ageing societies pose challenges to their public health systems, especially with respect to older adults living in relatively isolated and sparsely populated areas.

Gerontechnology [2] is a multidisciplinary field that brings together gerontology and technology to create technological environments for an inclusive, innovative, and independent life of older adults, improving their social participation. It focuses on the adaptation of technological environments to the health, housing, mobility, communication, leisure, and work of older people [3].

The aim of this work is to promote the well-being and quality of life of older adults by making intensive use of the capabilities offered by smartphones in terms of storage, computation, and sensors. Present day smartphones are able to run demanding applications, such as video games; manage all our social profiles; and work with office applications that required desktop computers not so long ago. This way, smartphones have become the most popular device for accessing computing resources and services.

Our proposal takes advantage of the capabilities currently offered by smartphones [4] to develop a framework based on the People as a Service (PeaaS) model [5] extended with Complex Event Processing (CEP) [6] for collecting and processing data.

PeaaS provides a conceptual framework for application development focused on the smartphone as a representative and interface to its owner. By analyzing the data received from the different smartphones’ sensors, together with those from personal health devices, we are able to infer the state, preferences, and routines of behavior of their owner and build with them her virtual profile. This information can then be offered to third parties to generate value-added services or to interact with the IoT environment in an automated way. All the information is stored locally in the smartphone, guaranteeing that its owner keeps full control over which data is being shared and with whom.

In our proposal, we extend the virtual profiles of the PeaaS model with behavioral rules that govern their inference, evolution, and use. This combination of rules and data related to a particular user and her environment is what we call her digital avatar.

For building the digital avatar, we apply CEP techniques [7] in order to process the information obtained from the smartphone and make complex decisions based on the perceived data. The processing of events takes place on the user’s own device, without sending personal data to any external server for data processing purposes. This avoids the need of continuous online communication and also reduces the amount of data transmitted over the Internet. It allows us to generate better and more appropriate responses to problems or adverse circumstances that may arise in the daily life of older people, increasing their safety and welfare by taking advantage of the interactivity and computing capabilities of their smartphones.

In this paper, we introduce our proposal for applying Digital Avatars to promote independent living for older adults, specially those inhabiting rural and low population areas. After this introduction, we present a motivating example for illustrating our proposal (Section 2), and we give an extensive description of the framework (Section 3), presenting its architecture, the digital avatar profiles, how we obtain and treat the necessary data, and how data uncertainty is addressed in our proposal. Some related works are presented in Section 4, while Section 5 discusses both the benefits and the risk of the approach. Finally, Section 6 concludes the paper.

This work is an extension of [8], presented at the Second International Workshop on Gerontechnology (http://4ie.spilab.es/workshop2019/). In this extended version, we present in more depth our proposal, providing many technical details about it. In particular, in Section 3.2, we describe the structure of the digital avatars containing the virtual profiles of the users, while Section 3.3 explains how the avatar in the smartphone interacts with IoT and personal healthcare devices for collecting information from them. Moreover, Section 3.4 presents the integration of CEP into smartphones and shows how to handle uncertainty in the data being processed.

2. Motivation: Older People in Rural Areas

In order to illustrate the application of Digital Avatars to promote independent living, in this section, we present a motivating scenario of an older person living in a relatively isolated rural environment.

María is an older person living in a small village in a rural and almost deserted area. After her husband died, and with her children living away, María started feeling lonely and considered moving to a nursing home. However, she finally decided to stay in her lifelong house while she was able to help herself properly. After all, she is fully autonomous and only needs her blood pressure and sugar levels to be checked regularly, as she now suffers from diabetes.

In order to avoid recurrent and cumbersome trips to the primary healthcare center, which is located in another town, the community health services have provided María with some smart gadgets that can be connected to the smartphone she uses for talking with her children and for texting with her grandson. A glucose meter and a blood pressure monitor send their data by Bluetooth Low Energy (BLE) to her phone. These measurements are recorded there, together with relevant information about her daily activities, such as patterns of movement inside her house and within the village, and usage of the smartphone itself (phone calls, execution of apps, etc.) Bluetooth is also employed to detect proximity to other smartphones, particularly to those of her contacts, which enables detecting patterns of visits and social relations between María and her neighbors.

This way, both the community healthcare center and María’s children are aware of her health condition, well-being, and even her mood. For example, abnormal sugar levels in her blood or the fact that she is not visiting her neighbor one afternoon, as she always does on Tuesdays, can be easily detected. In case that something seems to be wrong, the smartphone would request from her some interaction to check whether everything is ok, or it will raise an alert to the appropriate contact person or to emergency services.

In this scenario, the smartphone plays a central role in capturing, processing, and storing information about its owner. For that purpose, it monitors the phone’s sensors and also devices in María’s environment: movements detected by the phone accelerometer, GPS readings, use of mobile applications, and BLE signals from IOT health devices. The rules for processing all this information and detecting that any significant event has likely occurred are also stored in the phone and are handled by María’s digital avatar. Note the importance of dealing with confidence levels (i.e., aleatory uncertainty) when making decisions, given that all detected data is subject to deviations and potential measurement errors (e.g., a sensor may not function properly for a short period of time), as well as María’s variations in her regular habits due to environmental conditions (e.g., bad weather) or unexpected situations (e.g. a surprise visit from a relative) that may cause changes in her daily habits but do not represent a challenge to her health.

3. Framework Description

This section describes our proposal to apply Digital Avatars as a means of promoting independent living for older people. In particular, we discuss the functionality of the system, describe the framework architecture, and explain how a CEP engine is used to generate, store, and process virtual profiles of the users on their smartphones.

3.1. Architecture

Digital Avatars builds on previous work on the Internet of People [9] and PeaaS [5] models. In particular, the Digital Avatars framework is a realisation and extension of the PeaaS model, where the role of the inference engine is performed by a CEP engine, as it will be explained below. Not only is the engine responsible for building the profile and for aggregating and summarizing the information in it, but it is also capable of performing more complex and general tasks by means of CEP rules which endow PeaaS virtual profiles with “behavior,” allowing broader applications of the model, including the description of interactions among human users and/or IoT devices.

The Digital Avatars architecture is deployed at the application level due to the restrictions of the Android operating system. Thus, all the modules of the architecture rely on OS calls to access any particular functionality or resource of the smartphone. The architecture is illustrated in Figure 1. It represents the set of software elements required to build, populate, and share in a controlled way the user’s information stored in the smartphone.

[figure omitted; refer to PDF]

The digital avatar is the core element of the architecture. It consists of the user virtual profile and the privacy settings, which define the permissions to access that information. Virtual profiles are represented by JSON documents, stored in the phone with Couchbase Lite, which offers NoSQL database storage for smartphones. The profile keeps both historic records of events of interest, chronologically ordered, and higher-level profile information with meaningful knowledge about the user. In our case, this information is mainly related to health: blood pressure, glucose level, and other health indicators, activity and rest time, and even the places the user has visited or the people with whom she has interacted. The structure of the virtual profile of the digital avatar is presented in Section 3.2.

The digital avatar is built by an inference CEP engine capable of processing the raw data collected by the built-in sensors in the smartphone and through other smart devices connected to it. The CEP engine extracts meaningful information from the raw data coming from all these sources by applying a set of rules and patterns. One of the main advantages of CEP is that it works in real time, reducing latency in the decision-making process. Hence, it is very appropriate for asynchronous and real-time systems which must react quickly to changing or unusual situations. The use of CEP in our framework and how it deals with uncertainly are explained in Section 3.4.

The engine performs two main tasks: detection of alarm conditions and inference of higher-level knowledge. From the first task, we are able to detect particular situations and to derive some actions or changes in the system. For example, if the user suffers from an early stage of cognitive impairment and the smartphone detects that he has gotten lost, it will immediately alert a caregiver [10]. The inference task consists on the analysis of the information stored in the avatar for deriving added-value facts. For instance, based on the user’s daily activities and/or the places frequently visited, it is possible to predict where the user might be at a given time [11]. This high level contextual information of the user is stored back in the digital avatar.

The digital avatar is offered as a service, allowing both applications installed in the phone and external third parties to access it. Privacy concerns are addressed by the Privacy Settings, which define the access privileges to the user’s information. The Privacy Settings are a set of access policies for the digital avatar. The digital avatar is structured in sections or entities, and these policies allow to define the rules for granting read and write permissions for each section and third-party agent. For that, we have defined three privacy levels. The Private level lets no one but the user access that information. The Public level gives access to any certified agent. Finally, the Trusted level allows the user to specify a list of agents who will be able to fetch or update a particular section in the digital avatar.

The Digital Avatar Controller API provides the operations to deal with the information in the profile and making system calls. It prevents unauthorized access to the virtual profile and gives a unique and easy-to-use point of interaction with it. The API takes into account the Privacy Settings to allow or deny access to third-party applications. For instance, health professionals would be able to access only those parts of the profile concerning health issues, but not other private data.

The Mobile Device Resource layer is the element of the architecture that handles access to the sensors available in the smartphone. In addition, wearables and other devices connected to the smartphone are accessed through the External Service module. This module gives support to the interactions of the smartphone with external sources of information, like home-assisting devices or public web services offering contextual information like weather conditions, public transport, air quality, etc. Furthermore, this module manages the communications with third parties when a critical event is detected by the inference engine. For example, if a dangerous high blood pressure episode is detected, it will send an alarm to emergency services. Data collection and interaction with third-party smart devices is presented in Section 3.3.

Finally, the Execution Core module is where the front-end application is running. It is here where third-party applications run and make use of the CEP engine and the digital avatar through the Digital Avatar Controller. These applications offer specialized services to the user, for instance, adding new domain-specific information to the avatar.

All the components of the architecture reside in the smartphone. Third parties are able to access the digital avatar from the outside, provided they have the corresponding permissions, following the GDPR recommendations. The traditional client/server approach breaks down into a decentralized architecture in which each smartphone node acts as a server of its own produced data.

3.2. The Digital Avatar

The digital avatar is continuously evolving and growing from each interaction with the user, the sensors in the smartphone, and external IoT devices. Hence, it must be stored in an open and scalable format which permits to extend the syntax and includes mechanisms for considering privacy settings. We have designed a common ontology and notation based in YAML to create a standard and controlled environment. This notation is parsed from the JSON documents that compose the virtual profile database, providing a more readable and clearer representation of the data.

In order to enable developers to create applications compliant with the Digital Avatars framework, we provide a standard and easy-to-use structure for the digital avatar. This structure is based on an entity-driven metamodel. Entities are the basic information container, and they can be nested. The metamodel is designed to give a great level of freedom that allows the model to scale through the nesting of entities. We have defined five attributes present in every avatar entity. These attributes constitute keywords which represent the Name of the entity, its Type, its reading and writing Privacy levels, its Value, and a Timestamp indicating the last time the entity was updated. The Value of an entity may store either a simple value or a nested entity. According to this, the Type of the entity may be either the Entity or a basic data type. Additionally, the type Sensor allows storing information on the sensors in the smartphone.

Our YAML-based notation implements an indentation-structured language but with three parameters per line. As the Timestamp and the Type of the entity are already implicit in the implementation, we omit them. The Privacy levels are expressed by the third parameter of the representation using reserved words and symbols. We use Public and Private meaning that anybody or no one has a particular permission, respectively, and the symbols “?” and “!” express read and write permissions, respectively. If the privacy level is omitted, we assume that it is inherited from the parent entity.

Aside from the YAML notation, we have defined an ontology of entities for Digital Avatars. The ontology is a common standard structure based on root entities or sections inside which we organize the information of the digital avatar. These sections provide default spaces for information storage and offer a mechanism for secure and correct functioning of the framework. Each section implements a certain privacy level which is extended to all the information stored in it, guaranteeing that every piece of information in the digital avatar is protected.

We have defined a number of default and commonly used sections: Personal, Relations, Places, and External. The Personal section contains personal, health, and contact information about the user and about the smartphone itself, e.g., the sensors available in it, manufacturer, and precision. Relations is a collection or agenda containing information of each trusted interactor for the user. It may contain family member data, doctors, or even privacy groups with a valid certificate for an external application. The user can specify privacy groups like family or doctors for grant-specific permissions for sharing certain data. The Place section stores frequently visited places for the user along their daily routines. Last, the External section is used by the interactors or installed applications to store their data. A common structure is given to more quickly identify the sensors to which the interactor has permission to pool and for specific data storage rights. For example, an application can have access to the pulsometer and the step counter to monitor María’s physical activity and exercising patterns.

Code 1 shows an excerpt of María’s virtual profile, in particular, the Me and Health entities within her Personal section. As we can see, though the default privacy level for all Personal data is Private, some entities like Location or Steps declare Trusted privacy for reading purposes. This is done by enumerating the agents or privacy groups that have this permission and putting the reserved symbol “?” in front of them. In the Step entity, the privacy level is composed of three predefined privacy groups, Doctors, Family, and Friends, where the authenticated healthcare providers, family members, and friends of the user are stored with the hash fingerprints of their certificates. The writing permissions (“!”) are left as default.

Code 1: Excerpt of a digital avatar showing some personal data.

1 – Personal: ?!private

2  – Me:

3   – Name: "Maria Zambrano": ?public

4   – Phone: [phone]: ?Family, ?Friends

5   – Location: ?Family, ?Friends

6    – Latitude: 36.718967

7    – Longitude: –4.4337306

8   – Birth: 20/07/1939

9   – Certificate: "F0:89:B1:..": ?public

10

11  – Health: ?!Doctors

12   – Sleep: "0 7:32:16": ?Doctors, ?Family

13   – Steps: 2315: ?Doctors, ?Family, ?Friends

14   – Glucose: 183:

15   – BloodPressure:

16    – High: 132

17    – Low: 74

18   – HeartRate: 81

3.3. Data Collection

The raw data to feed the digital avatar comes from several sources. The most important of them is the smartphone itself, which provides basic data on the use of the device, such as the battery level, or whether it is connected for charging. Other sources of information can be wearables such as smartwatches or sport bands. By accessing the accelerometers and other sensors provided by these devices, it is possible to monitor the activity of the user (obviously, only if she carries them with her) and recognize different activities, such as typing, walking, and driving. It is also possible to detect how much time has passed since the last time the user handled the device.

The movements of the user can be monitored with the phone’s GPS sensor, in conjunction with network detection via Wi-Fi, inferring if the user leaves home and the distance travelled during walks or visits to other places or people. In this sense, it is possible to infer when the user interacts with other people (who use Digital Avatars on their phones) since her smartphone will read Bluetooth signals coming from other smartphones and detect the close presence of their owners. This way, we learn whether the user is accompanied or if she is related to other people in her everyday environment [12].

One step further is being able to detect the mood of the user, as discussed in [13, 14]. To do that, we observe the activity with the smartphone, analyzing the record of calls and their frequency, the use of various applications, programmed alarms, etc. This data is available on most smartphones.

Finally, other sources of information are health devices that communicate with the smartphone to send their data. Today, most home medical devices communicate via Bluetooth Low Energy (BLE) with their smartphone applications. Examples of such data can be provided by glucose meters or blood pressure monitors that are commonly and easily used in a domestic environment. In many cases, the communication protocol of the health devices is proprietary, so it falls onto the user to add the gathered information to their digital avatar manually. When the users take a measure, we can display a message asking them to introduce the value to add it to their profile. However, some devices offer the possibility of writing applications to gather and manage these data, which allows us to automatically record this information into the user’s digital avatar. The following subsection addresses this issue in more depth.

Accessing Third-Party Wearable Devices. As we have remarked, it is not easy to find wearable devices that measure health indicators and provide an open communication protocol or an API that can be used by third-party applications. In the search we have carried out, we have found a family of Samsung devices equipped with the Tizen operating system, which we have chosen for a first proof of concept of our proposal. Tizen is based on Linux and offers some SDKs for external developers to code their own applications and to have access to the data collected by the sensors embedded in these devices. In particular, for our tests, we have used the smartwatch Samsung Galaxy Watch equipped with a 3-axis accelerometer, a heart rate sensor, and a pedometer, among other sensors. We have not used the smartwatch’s GPS as we are using the GPS embedded in the smartphone, but it could be employed to improve the accuracy and reduce the uncertainty of the measurements.

In order to make use of the data provided by the Samsung Galaxy Watch, we have developed two paired applications, one running on the smartwatch and the other one running on an Android smartphone storing a digital avatar. Both ends interact with each other in a full duplex communication. The application running on the smartwatch collects data from the aforementioned sensors and sends that information to the application running on the smartphone. The latter processes the incoming information and updates the digital avatar according to its own rules.

Figure 2 shows the communication scheme of the two applications developed: the one that operates on the smartwatch and the one that runs on the smartphone where the digital avatar is installed. The smartwatch application has been developed using the .NET framework. This application runs on the Tizen Wearable layer of the smartwatch operating system, which uses a Linux kernel as the base layer. Both applications communicate in a bidirectional way. The smartwatch application collects data from its sensors and sends this information to the smartphone application, which processes this information and adds it to the digital avatar.

[figure omitted; refer to PDF]

Tizen allows three ways of programming applications. The first one is to code a native application on the C programming language. This option includes some interfaces to directly control the device hardware and offers better performance. However, it lacks a managed run time. The second option is to build a Web application, on top of HTML5, CSS, and JavaScript. This option has limited capacities and worse performance. The third option is to build the application on top of the .NET framework and the C# programming language. We have chosen this last option because it allows accessing all the device’s resources, the application is easier to develop, and it presents good performance. For the application that runs on the smartphone, we have used the Tizen library for Android.

To get both applications in touch, the first step is to create an agent in both ends. The agent acts as the connection and communication manager. After creating the agent, it looks for peers to connect to, and it pairs to the peers with the desired identifier. Once the connection is established, the devices stay connected in a way similar to Bluetooth connections. The app in the smartphone takes the active role. It polls periodically the smartwatch’s sensors to collect data from them and asks to send those data to the smartphone, where an Android service running in the background is waiting for them. Once the data have been received by the application, they are available for updating the digital avatar of the user.

As a proof of concept, we have successfully received data from the smartwatch’s accelerometer, heart rate sensor, and the number of steps the user has taken since the beginning of the day.

This simple data is the low-level events that feed the CEP engine. Based on them, CEP rules are executed to detect user-relevant states or to react to specific circumstances in real time.

3.4. Complex Event Processing with Uncertainty

The definition and detection of situations of interest from the analysis of low-level event notifications [15] is a form of information processing known as Complex Event Processing (CEP) [6, 7]. Allowing the analysis of large amounts of data from different sources, CEP detects significant information and reacts to a new situation in real time. The discovery of domain-specific critical situations via CEP engines is generally carried out on servers or desktop computers.

However, with the aim of building in the smartphone a digital avatar or complex virtual profile with high level information on the health and habits of the user, our proposal introduces CEP in the Digital Avatars architecture in the smartphone. Known as mobile CEP [16], this approach has useful inherent characteristics without the need to implement communication between the engine and the sensors, making the system information private and correlated and relying totally on the smartphone.

Typically, events received from physical systems, sensors, and networks, as described in our motivating scenario, are not free from uncertainty. Predictions of future events, estimations, physical measurements, or unknown properties of a system are a quality or state known as uncertainty, and they involve imperfect and/or unknown information.

In particular, measurement uncertainty is a type of uncertainty referring to the inability to know the value of a quantity, an intrinsic aspect of any physical setting with complete precision, such as unreliable data sources and communication networks, tolerance in the measurement of the physical elements values, estimates due to the lack of accurate knowledge about certain parameters, or the inability to determine whether a particular event has actually happened or not are the main causes of measurement uncertainty.

The uncertainty of sensor measurements is given by the accuracy and precision of these devices. The accuracy is usually provided by the sensor or smart device manufacturer in their product specifications, although this is not always the case. For example, for blood glucose meters, they must meet the ISO 15197:2013 [17] standard that establishes the accuracy for blood glucose measurements.

Regarding the verification of this uncertainty, each of these sensors or measuring devices is usually calibrated before they go on sale and should be recalibrated after a certain period indicated by the manufacturer. The main reason for carrying out this recalibration is that after that period, the measurements of the device become uncertain. Therefore, to learn, for example, the accuracy of devices that measure blood pressure, validation protocols are applied to sphygmomanometers [18].

A critical issue in any realistic model of a physical system is the explicit representation and management of measurement uncertainty. To do that, we have used a probabilistic approach, instead of employing fuzzy logic or possibility theory. We have worked on the representation of measurement uncertainty in software models [19, 20] and in CEP systems [21]. Our solution, presented in the form of a library that can be added to existing CEP engines, focuses on measurement information (also known as epistemic uncertainty), the confidence we have in the data we handle, and the rules determining the behavior of the digital avatars (aleatory uncertainty).

Incorporating the uncertainty that exists in the real world ensures models and applications that are more realistic. For this reason, we have detected and incorporated measurement uncertainty in both the events received by the different sensors and the information stored in the digital avatars, as well as in the system of rules managing the generation of new complex events and alarms.

As mentioned before, the Digital Avatars framework avoids using a centralized server and keeps data and computations on the smartphone of the user for which a virtual profile is being built. Therefore, a fast, lightweight CEP engine that processes events directly on mobile devices was needed. Esper-Android is the name of a mobile CEP prototype of the Esper CEP engine for Android on which Sebastian et al. [22] started working some years ago, but unfortunately, there is no updated version of Esper-Android compatible with current smartphones, given the dependence on third-party libraries no longer compatible with updated versions of Android.

As an alternative, we have decided to use Siddhi [23], a feature-rich stream-processing platform from WSO2, which has been successfully ported to both Android and Raspberry Pi devices. The SiddhiQL query language and Esper’s EPL (SQL-based) are very similar. SiddhiQL includes the main operators required to define the rules: select, filter, window, aggregations, group by, having, join, and pattern. As previously mentioned in Section 3.2, rules implementing the functional requirements that infer a person’s behavior and habits, or rules that trigger alarms when a person’s biometric values are outside the healthy range, can be built by combining these operators.

Consider for instance the selection rule shown in Code 2. By means of this rule, the Siddhi CEP engine monitors BloodPressure events received by Bluetooth in the smartphone from an associated blood pressure gadget, and it raises a HighBloodPressure event every time that at least one of the two constraints, systolic $BloodPressure\ge 140$ or diastolic $BloodPressure\ge 90$, is observed.

Code 2: Rule for detecting high blood pressure situations.

from every (bp = BloodPressure )[bp.high >= 140 or bp.low >= 90]

select eventTimestamp() as ts,

  bp.high as high,

  bp.low as low

insert into HighBloodPressure

Code 2 shows a standard Siddhi CEP rule, which is only executed when its Boolean condition holds. However, we are able to incorporate data uncertainty to the rule by using the uncertain data types (UBooleans, UReals, UIntegers, etc.) defined in the libraries we have implemented and presented in previous works [19, 21]. Thus, Code 3 presents the same high pressure detection rule but adding uncertainty to the data received from the blood pressure monitor. The operators (ge, gt, equal, add, mult, etc.) are also extended to handle and propagate this uncertainty in a transparent way. The resulting HighBloodPressure event also has an associated uncertainty, which is recorded as a probability (prob) and calculated in the rule itself. In this way, uncertainty is present in all calculations and process steps, and it spreads in a simple way without the need to incorporate complex probabilistic calculations.

Code 3: Rule for detecting high blood pressure adding uncertainty.

from every (bp = BloodPressure)[

  UBooleans.toBoolean(UReals.ge(bp.high, 140.0))

    or

  UBooleans.toBoolean(UReals.ge(bp.low, 90.0))]

select eventTimestamp() as ts,

    bp.high as high,

    bp.low as low,

    bp.prob$\ast$

    UBooleans.uor(UReals.ge(bp.high, 140.0), UReals.ge(

  ↪bp.low, 90.0)). getC()$\ast$

    P( HighBloodPressure) as prob

insert into HighBloodPressure

Similarly, a more complex rule with a combination of GlucoseLevel and WakeUp events detects a high glucose level according to the glucose value measured when the user has just woken up and starts their daily activities. The rule is shown in Code 4.

Code 4: Rule for detecting high glucose levels.

from every(e1 = WakeUp)-> e2 = GlucoseLevel[e1.id = e2.id &&

  ↪e2.level > 200]

select eventTimestamp() as ts,

    gl.id as id,

    gl.level as level

insert into HighGlucoseLevel

Again, Code 5 shows the rule for detecting high glucose levels, extended with data types and operations for incorporating uncertainty.

Code 5: Rule for detecting high glucose levels with uncertainty.

from every(e1 = WakeUp)-> e2 = GlucoseLevel[e1.id = e2.id

      &&

    UBooleans.toBoolean(UReals.gt(e2.level, 200.0))]

select eventTimestamp() as ts,

    gl.id as id,

    gl.level as level,

    e1.prob $\ast$ e2.prob $\ast$

    UReals.gt(e2.level, 200.0).getC() $\ast$

    P(HighGlucoseLevel) as prob

insert into HighGlucoseLevel

Additional rules would use these HighBloodPressure and HighGlucoseLevel events for purposes such as automatically sending a message alerting the user or even administering the appropriate dose of insulin based on their personal health records. The basics of these records with personal health data are stored in the user’s digital avatar too, which allows customizing the alert message and determining the appropriate dose.

4. Related Works

The large number of sensors integrated into smartphones, wearables, and other devices present in people’s lives generates a huge amount of information about the users themselves and their environment. Ambient Intelligence appears as a discipline whose goal is to make a person’s daily environment sensitive and responsive to their needs [24].

The main reason to gather information about the users of a system is to learn from them. With this knowledge, we can proactively meet their needs, minimizing manual intervention. Contextual data is used to infer virtual profiles with more specific information about the users [25, 26]. These profiles may be used to learn important aspects of the user’s habits and health condition such as diets, movements, exercise habits, and specific health information: heart rate, blood pressure, or glucose levels, among others. Currently, there are different approaches to create these virtual profiles [27–29].

Many existing solutions related to the monitoring of older people focus on geopositioning the user. Keruve and Neki (http://www.keruve.es/ and https://neki.es/) are two of the best-known enterprise solutions that allow the caregiver to locate the user in real time. However, the GPS devices that these and other companies sell are expensive, limiting their universal accessibility. In contrast, there are economic solutions based on mobile applications like CerQana and Tweri (https://cerqana.com/ and http://www.tweri.com/). However, none of them offer any further information other than GPS positioning.

A greater amount of information and consequently higher-quality knowledge can be apprehended by tapping into all the data that sensors in smart devices have to offer. This allows inferring user routines, movements, and health information. Several studies [13, 30] relate the performance of outdoor activities, smartphone usage, and sleep routines with the probability of suffering depression. All these indicators can be extracted from the data recollected by the sensors available in present day smartphones. Moreover, there is also a good number of studies about detecting the emotions of the users [31, 32]. In many of them, smartphones have a great influence, since the users interact quite frequently with them, and some capabilities of phones, such as the possibility to take pictures of the users, facilitate this task.

The idea of transferring CEP processing from a centralized server to the smartphone is proposed in [22], using the sensors embedded in the device as a source of simple events. A similar idea is proposed in [33], where smartphone resources are used as part of the processing of the events generated by their sensors. However, none of the proposals analyzed incorporate the treatment of uncertainty in the data collected or being processed.

5. Discussion

In this paper, we have presented some of our research work exploring alternative models for Social Computing, and we have shown how it is applied to the promotion of independent living for older adults. In this particular field, Digital Avatars offers benefits at three levels: for the older adults, for their caregivers, and for the community health systems. The most important of these benefits are discussed below.

First, we adopt a collaborative model with a peer-to-peer architecture built on smartphones, as opposed to the more common client-server architecture. This way, Digital Avatars becomes a tool where the users are able to decide with which other avatars or third parties to share their data and which external data they want to incorporate into their own avatar. This puts the exploitation of personal data in the hands of their owners, who are then able to control their information, alternatively to their data being processed and commercialized by third parties—without either transparency or clear benefits to the producers of the data.

Indeed, the framework is based on having one single virtual profile of the user, which can be shared to any third party with interests in offering some kind of personalized service to them. This discourages building multiple user profiles, one for each service provider, with redundant and inconsistent information about the user.

Moreover, our proposal takes into account the uncertainty that occurs in the real world, incorporating this uncertainty in the data obtained from sensors and external devices, in the rules of the CEP engine, and in the complex events that are generated from them.

In this work, we have applied the Digital Avatars framework to monitor user health, particularly for older adults living alone and independently. We have shown how we are able to monitor their health indicators, their habits, their social activities, and even their mood. The system reminds users of their health patterns through warnings, while it can also alert their caregivers or health professionals if the situation requires it.

This way, Digital Avatars facilitates continuous contact and monitoring of older people’s well-being by their family and caregivers. If authorized to do so, the caregiver can request health data from the user for monitoring it and, in any case, receives alerts as soon as they occur.

Finally, for the community health systems, our proposal allows health professionals to be informed of older people’s health conditions in sparsely populated environments, avoiding complicated, expensive, and time-consuming visits and consultations that can be scheduled less frequently. Digital Avatars immediately communicates any health alarm and regularly transmits the biometric parameters that the family doctor deems necessary.

On the other hand, the proposal also poses some challenges and potential risks. The most significant of them are the following. First of all, data collected in real time from different sources can generate a very large volume of information and its storage may exceed the capacity of a standard smartphone. The solution is to store raw data only for a limited time (which would depend on each data stream), aggregating the information at various levels, and storing only these aggregated data in the long term.

Furthermore, CEP engines require high processing power and are primarily oriented to run on servers or in the cloud. However, there are few alternatives for its execution on smartphones, as discussed in Section 3.4. Moreover, if the number of CEP rules and patterns to be processed is large, and the data collection is also performed at a high sampling rate, the smartphone may not be able to process all this information properly. Nevertheless, we can envision strategies for addressing this problem: data can be collected at a nonconstant rate and a first level of data filtering can be carried out where events of little significance or spurious or disabled events are discarded.

Moreover, one may fear that the CEP engine can cause a big penalty in energy consumption terms. However, in the proof of concept we have implemented, we have observed that the CEP engine worked well without incurring battery efficiency loss. The developed application, continuously running during a whole day, supposed an average of 7% of battery expense for a fully charged smartphone. This is a promising number, and more precise efficiency tests will be carried in the future.

As discussed in Section 3.3, most personal healthcare devices employ proprietary and secured protocols for communicating via Bluetooth with their associated smartphone applications. These applications are usually closed, not providing any kind of public API for accessing their data. However, some open solutions exist, from which proofs of concept can be built, and as in many other fields, open standards may appear in the future for making easier the interconnection of gadgets from different vendors.

An additional technology risk comes with the evolution of Android and other mobile operating systems. Indeed, the general trend in recent Android versions poses a challenge to our proposal, as it tends to close access to their internal data, sensors, and background applications. The framework presented in this work is based on these data sources that must be accessible in the smartphone. Increasing restrictions in their access would greatly hinder the implementation of Digital Avatars and the development of applications based on it.

Finally, a nontechnology risk is that the user may not be paying enough attention to the smartphone. For a number of reasons, the users may not always carry the smartphone with them, preventing the digital avatar to get important data about their activities and health indicators. Among these reasons, we can mention forgetful users, being uncomfortable with the use of a smartphone, a certain reluctance to technology, running out of battery, or many other possibilities.

6. Conclusions

In this work, we have shown the feasibility of using smartphones to promote independent living and quality of life of older adults. In particular, we aim to empower older people living alone in small villages and nearly uninhabited areas by reducing their dependence on caregivers and increasing their chances of remaining at their home places despite growing older.

The architecture of the Digital Avatars framework is related to what is known as Fog Computing. Indeed, the smartphone acts as a fog device, processing the raw data obtained from its built-in sensors and other personal healthcare devices connected to it, and generating more complex and added-value information, which in turn can be hosted locally in the smartphone, communicated to the user through notifications, or even shared with remote nodes hosted in the cloud.

In the healthcare monitoring scenario presented in this paper, these remote nodes—managed by healthcare professionals—can access the older person’s health indicators collected on personal care smart gadgets.

Going further into the case study used to illustrate our proposal, there are situations in which it would be interesting to directly connect two or more smartphones from different users to cooperate and reach common objectives or just to elaborate new and more complex information. In future iterations of the architecture, we aim to extend the communication capabilities of our framework, creating a distributed CEP ecosystem in which digital avatars are able to send out events to be processed by other smartphone nodes, which would be endowed with their own digital avatars, CEP patterns, and mobile apps.

Additionally, this research work discusses the technological limits of current smartphones and whether CEP engines can be run directly on these devices. In particular, our focus is put on issues such as data storage capacity, processing performance, accuracy of data collected by sensors and connected medical devices, and reliability of complex data generated by CEP tools. For addressing the two last issues, we integrate in our proposal the concept of uncertainty in the data collected and in CEP patterns and rules.

Our work represents a step forward towards making the smartphone become a personalized interface to its owner. Related research work by the authors of this paper considers further scenarios based on the Digital Avatars framework, in which fully programmable smartphone devices interact by executing scripts on-the-fly. This way, we are able to seamlessly configure smart things in the user environment based on the information contained in the digital avatar.

Acknowledgments

This work has been funded by projects PGC2018-094905-B-I00 (Spanish Government MINECO/FEDER) and UMA18-FEDERJA-180 (Junta de Andalucía/FEDER).

References

[1] World Health Organization, World Health Statistics 2019: monitoring health for the SDGs, sustainable development goals, 2019. https://apps.who.int/iris/handle/10665/324835

[2] J. A. Graafmans, T. Brouwers, "Gerontechnology™, the modelling of normal aging," Proceedings of the Human Factors Society Annual Meeting, vol. 33 no. 3, pp. 187-189, DOI: 10.1177/154193128903300308, 1989.

[3] A. Cruz-Jentoft, A. Franco, P. Sommer, J. P. Baeyens, E. Jankowska, A. Maggi, P. Ponikowski, A. Ryś, K. Szczerbińska, A. Milewicz, "European silver paper on the future of health promotion and preventive actions, basic research and clinical aspects of age related disease," Endokrynologia Polska, vol. 7 no. 4, pp. 408-414, DOI: 10.4017/gt.2008.07.04.001.00, 2008.

[4] R. Triggs, "How far we’ve come: a look at smartphone performance over the past 7 years. Available at," 2015. https://www.androidauthority.com/smartphone-performance-improvements-timeline-626109/

[5] J. Guillen, J. Miranda, J. Berrocal, J. Garcia-Alonso, J. M. Murillo, C. Canal, "People as a service: a mobile-centric model for providing collective sociological profiles," IEEE Software, vol. 31 no. 2, pp. 48-53, DOI: 10.1109/MS.2013.140, 2014.

[6] D. C. Luckham, "The power of events: an introduction to complex event processing in distributed enterprise systems," Rule Representation, Interchange and Reasoning on the Web, 2002.

[7] O. Etzion, P. Niblett, Event processing in action, 2010.

[8] M. F. Bertoa, N. Moreno, A. Perez-Vereda, D. Bandera, J. M. Álvarez-Palomo, C. Canal, "Digital avatars for older people’s care," Gerontechnology, pp. 59-70, DOI: 10.1007/978-3-030-41494-8_6, 2020.

[9] J. Miranda, N. Mäkitalo, J. Garcia-Alonso, J. Berrocal, T. Mikkonen, C. Canal, J. M. Murillo, "From the internet of things to the internet of people," IEEE Internet Computing, vol. 19 no. 2, pp. 40-47, DOI: 10.1109/MIC.2015.24, 2015.

[10] J. Berrocal, J. Garcia-Alonso, J. M. Murillo, C. Canal, "Rich contextual information for monitoring the elderly in an early stage of cognitive impairment," Pervasive and Mobile Computing, vol. 34, pp. 106-125, DOI: 10.1016/j.pmcj.2016.05.001, 2017.

[11] A. Pérez-Vereda, C. Canal, "A people-oriented paradigm for smart cities," International Conference on Web Engineering (ICWE 2017), LNCS, vol. 10360, pp. 584-591, 2017.

[12] J. Garcia-Alonso, J. Berrocal, A. Perez-Vereda, J. Galán-Jiménez, C. Canal, J. M. Murillo, "Using Bluetooth Low Energy advertisements for the detection of people temporal proximity patterns," Mobile Information Systems, vol. 2020,DOI: 10.1155/2020/8506323, 2020.

[13] S. Saeb, M. Zhang, C. J. Karr, S. M. Schueller, M. E. Corden, K. P. Kording, D. C. Mohr, "Mobile phone sensor correlates of depressive symptom severity in daily-life behavior: an exploratory study," Journal of Medical Internet Research, vol. 17 no. 7, article e175,DOI: 10.2196/jmir.4273, 2015.

[14] V. Osmani, "Smartphones in mental health: detecting depressive and manic episodes," IEEE Pervasive Computing, vol. 14 no. 3, pp. 10-13, DOI: 10.1109/MPRV.2015.54, 2015.

[15] G. Cugola, A. Margara, M. Pezzè, M. Pradella, "Efficient analysis of event processing applications," Proceedings of the 9th ACM International Conference on Distributed Event-Based Systems - DEBS '15, pp. 10-21, .

[16] J. Dunkel, R. Bruns, S. Stipkovic, "Event-based smartphone sensor processing for ambient assisted living," 2013 IEEE Eleventh international symposium on autonomous decentralized systems (ISADS), .

[17] ISO (International Standars Organization), ISO: in vitro diagnostic test systems — requirements for blood-glucose monitoring systems for self-testing in managing diabetes mellitus. Standard ISO 15197:2013, 2013.

[18] T. Stryker, M. Wilson, T. W. Wilson, "Accuracy of home blood pressure readings: monitors and operators," Blood Pressure Monitoring, vol. 9 no. 3, pp. 143-147, DOI: 10.1097/01.mbp.0000130224.18777.ce, 2004.

[19] M. F. Bertoa, L. Burgueño, N. Moreno, A. Vallecillo, "Incorporating measurement uncertainty into OCL/UML primitive datatypes," Software and Systems Modeling, vol. 19 no. 5, pp. 1163-1189, DOI: 10.1007/s10270-019-00741-0, 2020.

[20] L. Burgueño, M. F. Bertoa, N. Moreno, A. Vallecillo, "Expressing confidence in models and in model transformation elements," Proceedings of the 21th ACM/IEEE International Conference on Model Driven Engineering Languages and Systems - MODELS '18, pp. 57-66, DOI: 10.1145/3239372.3239394, .

[21] N. Moreno, M. F. Bertoa, L. Burgueño, A. Vallecillo, "Managing measurement and occurrence uncertainty in complex event processing systems," IEEE Access, vol. 7, pp. 88026-88048, DOI: 10.1109/access.2019.2923953, 2019.

[22] S. Stipkovic, R. Bruns, J. Dunkel, "Pervasive computing by mobile complex event processing," IEEE 10th International Conference on e-Business Engineering, pp. 318-323, .

[23] S. Suhothayan, K. Gajasinghe, I. Loku Narangoda, S. Chaturanga, S. Perera, V. Nanayakkara, "Siddhi: a second look at complex event processing architectures," Proceedings of the 2011 ACMWorkshop on Gateway Computing Environments, pp. 43-50, .

[24] E. Aarts, S. Marzano, The new everyday: views on ambient intelligence, 2003.

[25] P. Bellavista, A. Corradi, M. Fanelli, L. Foschini, "A survey of context data distribution for mobile ubiquitous systems," ACM Computing Surveys, vol. 44 no. 4, 2012.

[26] M. Raskino, J. Fenn, A. Linden, "Extracting value from the massively connected world of 2015," Gartner research, vol. G00125949 no. 2005, .

[27] T. M. Grønli, G. Ghinea, M. Younas, "Context-aware and automatic configuration of mobile devices in cloud-enabled ubiquitous computing," Personal and Ubiquitous Computing, vol. 18 no. 4, pp. 883-894, DOI: 10.1007/s00779-013-0698-3, 2014.

[28] P. Makris, D. N. Skoutas, C. Skianis, "A survey on context-aware mobile and wireless networking: on networking and computing environments’ integration," IEEE communications surveys & tutorials, vol. 15 no. 1, pp. 362-386, 2012.

[29] H. S. Park, K. Oh, S. B. Cho, "Bayesian network-based high-level context recognition for mobile context sharing in cyber-physical system," International Journal of Distributed Sensor Networks, vol. 7 no. 1,DOI: 10.1155/2011/650387, 2011.

[30] M. J. Murphy, M. J. Peterson, "Sleep disturbances in depression," Sleep medicine clinics, vol. 10 no. 1, pp. 17-23, 2015.

[31] S. Agarwal, B. Santra, D. P. Mukherjee, "Anubhav: recognizing emotions through facial expression," The Visual Computer, vol. 34 no. 2, pp. 177-191, DOI: 10.1007/s00371-016-1323-z, 2018.

[32] S. Bonilla, E. Moguel, J. Garcia-Alonso, "Facial recognition of emotions with smartphones to improve the elder quality of life," International Workshop on Gerontechnology, pp. 15-25, 2018.

[33] A. Dhillon, S. Majumdar, M. St-Hilaire, A. El-Haraki, "MCEP: a mobile device based complex event processing system for remote healthcare," 2018 IEEE International Conference on Internet of Things (iThings),DOI: 10.1109/cybermatics_2018.2018.00064, .