Over half of all adults older than 65 years report having some sort of sleep disturbance and many others live with undiagnosed sleep disorders.1 Poor sleep quality has health consequences over time. According to the National Sleep Foundation, "longer term, too little sleep may contribute to heart disease, high blood pressure, diabetes, obesity, and stroke."2
Aging has a direct association with an increased prevalence of factors which affect sleep, which can lead to increased sleep disturbances.1 Life changes can compound to affect further the quality and quantity of sleep. For example, across ages, there has been a significant increase in sleep disturbances reported globally during the COVID-19 pandemic.3
In recent years, there has been a surge in the use of mobile health (mHealth) technology, to self-manage health, including sleep.4 The personal sleep monitoring device (PSMD) market has seen a significant increase, with a predicted value to grow globally by 2024.4 Devices such as Apple watches (Apple Inc, Cupertino, CA, USA), Fitbits (Google LLC, San Francisco, CA, USA), and Misfits (Fossil Group Inc, Richardson, TX, USA) are already somewhat commonplace among those who can afford them, with most including sleep tracking capabilities. These devices provide information such as sleep duration and quality, through actigraphy, in order to better inform users about their sleep and guide sleep health behaviors. Having this information may help guide decision making and reinforce healthy sleeping habits. Despite this trend, experiences with using these devices often do not meet user expectations.5 It is important that PSMDs are easy to operate and that the user's overall feelings about the device are positive. If there is a lack of trust in the device or there is too much of a learning curve, then the technology will not match the expectations of the people using it.6 Understanding usability experience of the PSMD can influence sleep self-management outcomes by ongoing mHealth engagement.
BACKGROUND
In the world of health self-monitoring technology, usability evaluations determine whether or not people are able to operate the system.7,8 When looking at an application that stores, analyzes, or records relevant personal health information for user access, it is important for the user to understand and operate the system. Usability, historically, has been frequently measured by three key attributes: effectiveness, efficiency, and satisfaction.9 More recently, mHealth usability has expanded to also include learnability, operability, universality, and user interface esthetics.10,11 Proper navigation and usage of the system are key to enable users to reap benefit from the information provided. If they are not able to operate and understand it, the system becomes effectively useless or of limited use.
Qualitative analysis of the user experiences reveals individual perceptions on peoples' satisfaction as ways that developers can improve it. The primary aim of this study was to thematically expand the descriptions of user experience of self-management of sleep using a PSMD during a 4-week period among a sample of persons 65 years or older with self-reported sleep disturbances. The 32-item Consolidated Criteria for Reporting Qualitative Research checklist is applied in reporting these results.12
METHODS
The methodological approach was based on the process outlined by Fereday and Muir-Cochrane,13 integrated data-driven emergent codes with theory-driven ones based on the tenets of the Unified Theory of Acceptance and Use of Technology (UTAUT).14 This theory outlines the conditions under which someone is willing to accept new technology. It consists of four theoretical constructs: Performance Expectancy, Effort Expectancy, Social Influence, and Facilitating Conditions, which represent determinants of Use Behavior or the Intention to Use.14 This theory explains the barriers that people face when testing and accepting new technology. This theoretical model was chosen in extending the model to mHealth as evident in the contemporary literature.15-18
The research team was interdisciplinary and positioned in the fields of engineering and nursing. The research team (authors of this article) reflected a range of experiences in mHealth usability, engineering, community-based nursing, symptom management, and research expertise in qualitative methods. Members of the team represented early and advanced career investigators.
The study commenced after university institutional review board review was completed and reviewed as part of a National Institutes of Health (P20NR016599) pilot study testing of a PSMD. Participants 65 years or older with self-reported sleep disturbances from a local community on the Eastern Coast of the United States were recruited for a 4-week sleep self-monitoring study to improve sleep self-management.
Before beginning the study, participants were sent a copy of the informed consent. A face-to-face meeting was then scheduled and consent reviewed and at that time signatures obtained. A paper copy of the signed informed consent was provided to the participant and one kept in a separated and locked file by the principal investigator.
In this study participants wore an off the shelf sleep monitoring device the MisFit Shine (see Figure 1) that was paired with a mobile application to show sleep patterns, quality of sleep, and number of hours of sleep per day. Recruitment took place through an Aging Service Network Provider and local new paper advertising.
Inclusion criteria included age 65 years or older, willingness to wear the PSMD and learn to use the device, ability to communicate in the English language, ability to physically and cognitively function independently, and have cognitive ability based on scores of 5 on the Mini Cog. Participants were given basic consistent instruction on the use of the device and how to review their sleep data. Tablets were provided to those participants who needed them.
At the end of the 4 weeks, an open-ended face-to-face interview was conducted by one researcher from the team and focused on user experience (interview guide available upon request). The interviews consisted of 24 semistructured interview questions. These interviews were audio-recorded, transcribed by coauthor PC, checked against the recording, and then analyzed to better understand the experience using the PSMD to self-manage sleep. A written report of all sleep log data was provided to each participant, at the end of the study, documenting quantity and quality of sleep.
Data Analysis
We used a hybrid, comparative qualitative analysis technique to analyze data derived from these open-ended, semistructured interviews of the experience using a PSMD. These interviews are the source documents for this data analysis. The goal of this qualitative interview and analysis was to better understand the user experience. Three researchers used three independent methods to analyze the qualitative data that were integrated in the final analysis.
The primary method of analysis was deductive coding and analysis based on the UTAUT.14 The methodological approach integrated theory-driven codes based on the tenets from the UTAUT.14 An a priori template of codes was created.19 Second, an a priori deductive analysis of coded themes was based on the questions of the Systems Usability Scale (SUS).8 The System Usability Scale has questions such as "I found the system unnecessarily complex" and "I found the various functions in this system were well integrated."8 Interviews were analyzed for themes as associated with each of the 10 SUS questions.
The third researcher used a deeper emergent coding process of the transcripts from the open-ended interviews. Here, analysis was not based on any existing template of themes related to a conceptual model, theory, or measure, but rather derived from the transcribed data itself. After analysis, research team members peer-reviewed the coding methods and thematic analysis for reliability checking. The final steps in the analysis integrated the a priori and the emergent thematic analyses. Emergent themes were integrated based on dominant codes or identified as subcodes associated with themes of the UTAUT.14 Nvivo Version 12 (QSR International, Melbourne, Australia) was used for qualitative coding, organization, and analysis.
In summary, the 10 methodological steps of the hybrid thematic analysis process are listed in Table 1.
RESULTS
Twenty-six people, 12 males and 14 females, with a mean (SD) age of 72 (4.99) years participated in the study. One participant left the study early and thus is not included in this analysis. Of these participants, all but four had never used a personal sleep self-monitoring device or mHealth application. Listed in Table 2 is the integrated analysis that offers higher-level themes and subthemes from these qualitative interviews. Importantly, these themes expand the UTAUT concepts adding "Self-meaning." What we discover is organized in Table 2 as key themes and subthemes and the System Usability Scale6 items in connecting the three approaches in this hybrid analysis. The following is a description of the theme and its definition and interpretation within the context of these users experience.
Self-meaning
This theme is defined by how the person attributed the meaning of the PSMD to themselves and elaborated in the following subthemes of privacy, capability, age and technology. The subtheme of privacy is the meaning ascribed to the technology of self-monitoring and sharing of information that was about themselves. Participants described this as "being monitored." They commented on how pervasive these types of technologies are and expressed a range in degree of concern over their privacy. This idea that someone "else" may be watching or monitoring them was described in interacting with the device but also as a possibility in being able to easily share with a healthcare provider, for example, their sleep health information. Participants' intentional use was to self-monitor, to watch themselves, to know more about their own sleep patterns, as a way of understanding sleep disturbances as part of their health self-management. There was meaning generated in how the private self (self being monitored) interacted with the device.
The power of this technology and its ability to shape us in ways that we're not even aware of. (Participant 20)
The subtheme of capability reflects how mastering the use of the device made the participant feel more capable in managing his/her own health. This produced a related sense of being capable to use technologies, in general, that assist in managing health. Capability as meaningful to their user experience was a personal appraisal that was affirmed by interacting successfully with the device.
All participants in this study were 65 years or older, and most had never used a PSMD. Age as it relates to technology use can be biased against older users.20 Many applications focus on discrete health problems or are part of multilevel interventions for chronic diseases. Exclusion of older people in the design process may influence the adoption of technology.20 These open-ended interviews did suggest that age was not a barrier to technology use. Participants did not perceive their age as an influence on what it means personally to self-monitor and manage a symptom, health concern, or mHealth application.
Although the themes above were emergent, several themes aligned categorically as a priori codes based on the deductive analysis reflecting the questions of the SUS.8 Through analysis, The SUS8 items were integrated into the larger UTAUT14 as illustrated in Table 2 and described below.
Effort Expectancy
This theme is defined as the degree of the participants' ease in the use of their PSMD.14 Subthemes to better differentiate the types of effort emerged as engagement and device confusion. First was the engagement of learning to use the device. The participant users were strong in their ability to use the device based on learning. Although several participants were curious about the device, they felt they understood how it worked enough to monitor their sleep. Second, managing their mHealth sleep information was another feature of engagement. This aspect of self-monitoring use required synching of data from the device to the app, retrieval of data from the app, interpretation of the data, and then personal application, based on the information. In this analysis, engagement managing information was quantified as frequency in checking the data. As time went on across the 4 weeks, participants checked their information and the device less often.
I don't know. I would do it more of a percentage. Like maybe in the beginning, it was pretty much all the time, 90% of the time or 100%. And as time went on, it was just there. Part of the family. (Participant 19)
Checking their device was explained as a frequency that changed over time. This also included checking the device and the data about oneself and checking on the device for its function and accuracy against one's perceived sleep experience. This included the assessment of the outcomes of the sleep data monitoring. Device confusion is a subtheme of user experiences. The PSMD did have some issues with battery life and participants found this confusing. The PSMDs were not fully reliable in that way. There was confusion when the device had a power failure. Additionally, confusion with the device was described when the data the device was offering were not the same as their perception of their sleep duration and quality.21
Performance Expectancy
This theme is defined as the individual belief that the PSMD can be used and will help them to attain their sleep self-management goals.22 Performance expectancy can be on the spectrum of either a positive, supporting this belief, ambivalent, or negative in that they do not believe the PSMD will help them to attain their sleep self-management goals.
Positive performance expectancy was described as the expectation of the PSMD as being useful to meet sleep self-management goals and support behavioral change that increased intention to continue to use the device for sleep monitoring.
It means checking on my health. See how it's going, and I think that's important. (Participant 24)
Negative performance expectancy was related to the function of the device and perceived inaccuracies that mediated their trust in the device.
Trust was a subtheme that related to this perception of performance expectancy of the device. Participants referred to not "trusting" the information at times, and therefore, this distrust undermined the expectations of the device in offering reliable information that might improve their overall self-management and endpoint outcome of improved sleep. There were concerns that the device missed time when the person was asleep and more often when they were awake in the night. The reliability of the device data was very frequently checked against their recall of the night before in judging device functionality.
No, I don't think it had any bearing on how I slept or how well I slept. When I get up at night, I do not really fall right back to sleep, and it did not even pick that up. It did not really change, no. Because I didn't totally trust the data that was coming out of it. (Participant 18)
The subtheme of agency describes a sense of control in expecting the PSMD to support sleep self-management. The use of the PSMD served as a reminder in attending to fundamental sleep-health practices. Improving awareness of sleep patterns and associated behaviors was described as a positive performance expectation that included evaluation of the information from the device and how the participants used it to change sleep related behavior.
Yes, yes. It makes me more mindful of my rest. In other words, there were times I would go to bed real late at like 2 or 3. Wearing this monitor I am much more aware about when I am going to sleep. Which is a good thing. It focused my attention on my sleep quality. (Participant 1)
Social Influence
Social influence is the degree that the participant using the PSMD perceives those important others believe that they should use the PSMD or that important others would or do use a similar health monitoring device.22 Positive social influence was predominant and mentioned significantly as an aspect that participants felt could improve their self-management of sleep. There was the social influence of wearing the device that roused curiosity among close others. Additionally, the positive social influence of close others that also self-monitored was a type of positive peer support. There was a higher level of positive influence in considering sharing the information with a healthcare provider, with peers who were also concerned about sleep-health and/or family members.
Neutral social influence was also a more dominant theme than negative social influence. This subtheme emphasized that participants did not feel their social influences cared one way or another about their use of the PSMD, noticed it, or were aware of it. Negative social influence was described as close others ridiculing their use of the device as a negative appraisal. Additionally, when asked if they would be willing to share their sleep data with close others or monitor together, there was at times the negative social threat of compromising privacy that was a concern.
Although most people described themselves as "needing to learn" or "be taught," there was a perception of connection with the study team and friends who had previous experience with similar mHealth technologies. Sharing of information with others and learning how to use the device itself were viewed as a form of social support and the theme of "just a little support" was what was needed.
I try! Am I capable? You teach me, I'll learn. (Participant 4)
Facilitating Conditions
This theme is the degree to which the participants believe there are features that make the PSMD easy for them to use.14 The device selected for this study was the MisFit Shine (see Figure 1). We distinguished between the device features (accuracy, design of the watch-face, watchband, battery) and their experience using these features. Ergonomic fit of the device included the lightweight, plain design and comfort. It was largely thought to be quite easy to use. These were all positive facilitating conditions. Features of the watch face being difficult to use and needing to tap it for the time as well as the wrist clamp were negative design features. Most participants wanted the device to be used as a watch and look like a watch in addition to monitoring sleep.
Ease of use was identified as a facilitating condition. Battery life and needing to change the small battery frequently were issues that occurred commonly for study participants. This was described as a significant drawback within the subtheme of component difficulty. Although participants in this study were provided the device for free and had the option of keeping the PSMD after the study ended, the device was specifically selected because of previous reviews of its accuracy and its low cost, thus wider possible access.
Cost as a facilitating condition was discussed and most participants felt the cost was a big issue in considering using a PSMD and balancing that against accuracy and features. Most, but not all, participants reported that cost would be an issue. Some also mentioned the cost of time. The time it takes to use a device and pay attention to the aspects of personal sleep self-monitoring was recognized as a condition to continue use.
Concerns over the safety of their personal sleep data are described as conditions that would facilitate use or not and is related to the earlier subtheme of privacy. Over 12 participant interviews stated data protection as a concern, which was small but still present. In not having a concern, it was more because of the type of data that the device was collecting that seemed benign if someone else were to see it.
If someone from Facebook wants to look at my data, that's not fine with me. (Participant 9)
Intention to Use
Intention to use is the ultimate outcome of a usability assessment and analysis. Intention to use was the participants' opinions about continuing to use the PSMD, not using it, or using it for an alternative purpose. In this study, the monitoring of sleep was the parameter of attention related to people with existing self-reported sleep disturbances. However, many participants, despite all having reported sleep disturbances, were also interested in the alternative options of activity monitoring. For some, not having a smartphone/tablet was the obstacle to ongoing use of a PSMD. Most did report an intention to continue to use with a simple answer of "yes, probably," whereas fewer than half described no intention or less interest in ongoing use.
DISCUSSION
The use of PSMDs may be motivated by existing health conditions, such as sleep disturbances; these mHealth technologies may inspire the belief that one's health can be improved through understanding one's sleep patterns objectively and addressing sleep behaviors based on this information. This qualitative analysis aimed to understand the aspects of user experience as a way to inform usability after the 4-week study period and intention toward ongoing use of PSMDs in the future. This analysis reinforced many of the themes of the UTAUT14 while also offering specific nursing-based insights that broadened this theory within the details of user experience.
One of the interesting themes that emerged was that the usability experience included aspects of interaction between the participants (users), their perception of their sleep, and the external interaction with the data collected. This was a learned construction of self-management of health by way of its extension to the technology that informed their perceptions of their sleep health. This is described here in this analysis as user self-meaning in relation to PSMD technologies. This adds a specific extension to the UTAUT in health technologies that monitor specific aspects of health, quantify them, and are directly interacted with as information to change a health behavior and influence a specific outcome of improved sleep quantity and quality.
The UTAUT14 and SUS8 as the guiding theory and measure for deductive coding of the open-ended interviews may be overly complex models for seeking feedback on intention to use a PSMD yet are shown here to be helpful in evaluating the dimensions of usability based on the four key constructs that the UTAUT theorizes as explaining intention to use: (1) performance expectancy; (2) effort expectancy; (3) social influence; and (4) facilitating conditions.14 Showing how the UTAUT was applied to guide evaluation based on participant thoughts on the usability of this mHealth self-management system for sleep self-management is useful in understanding the specific aspects of technology that may influence intention to use and are important to people with self-identified health needs.
Limitations
There are limitations to acknowledge in this study. Within our sample, individuals who were more motivated to use mHealth as a response to an unmet health problem may have created a self-selection bias. In addition, the hybrid approach using both deductive and emergent coding methods rather than expanding understanding about usability still largely was predominantly deductive based on the UTAUT14 framework for this sample.
Ethical limitations are also important to acknowledge. Aging is associated with sleep changes that are often a combination of many factors, including changes in behavior brought on by a myriad of social, psychological, and physical symptoms and the interactions of these. Therefore, it is recommended that sleep interventions aimed at behavioral change clearly rule out pathological sleep disturbances and use a combination of interventions to improve the outcome of sleep health. Although unintended, self-monitoring can create health anxiety without associated support. Participants were highly encouraged to share their sleep data and follow up with their healthcare providers. Personal sleep monitoring devices use actigraphy, which has shown increasing validity for home-based monitoring. However, devices vary, and this study was limited to only one device and clinical practice guidelines recommend use of only Food and Drug Administration-approved devices.22
CONCLUSION
These results derived from a process of thematic coding that involved deductive coding and inductive coding. Results from this study provide the following recommendations for mHealth developers, engineers, designers, individuals, and nursing clinicians in recommending PSMD to persons 65 years or older to use to self-manage a specific health condition, such as sleep disturbances: (1) system usability is influenced by ease of learning to use the device, simplicity, trust in the device, accuracy, and privacy. (2) Capability is an important self-reference, facilitated through symbolic interaction (meaning) between the person and technology, that can influence user experience. This capability is influenced by having a person or people support learning to use the device. (3) Social influences impact usability experience and can be leveraged that include peer support and acceptance by other users that offer a potential for socially supported self-management and possible social support of conditions common to people as they age that are of concern such as sleep disturbances. (4) Perception of accuracy of the device is important. Although some gained insight into their sleep patterns, this was undermined by a perception of the inaccuracies of the device as compared with lived experience. This discordance between sleep perception and objective measurement has been found in other studies where sleep disturbances were present in older age.23,24 (5) Cost of the device and associated applications are important for access to mHealth and use.
In future studies, it is recommended that user experience be evaluated using qualitative approaches to extend theories of specific user experiences to inform a nursing midrange theoretical basis. Future studies can address important areas such as how user characteristics such as age and specific symptoms influence use and user perceptions of health and self-management outcomes in the application of mHealth technologies.25
References