Establishing quantitative criteria to distinguish individuals with and without insomnia is important for clinical and research applications, but consensus has not yet been reached for specific values. The purpose of this study was to identify the optimal quantitative thresholds for actigraphy and sleep diary that differentiate older adults (> 60 years) with insomnia from good sleeper controls.
A total of 119 participants (79 insomnia [35% male], 40 control [31.7% male]; mean age = 71.7 [7.2] years) completed at least 7 nights of sleep diary and actigraphy. Receiver operating characteristic curve analyses and the Youden index were used to identify optimal threshold values. Outcomes for each measurement method included sleep onset latency (SOL), wake time after sleep onset (WASO), sleep efficiency (SE), and total sleep time (TST).
Sleep diary measures produced areas under the curves (AUC) in the high range (0.84–0.97), whereas actigraphy performed poorly at discriminating the two groups (AUC 0.58–0.61). The Youden index identified SOL = 18 minutes, WASO = 21 minutes, SE = 92%, and TST = 388 minutes as the sleep diary measures that yielded the highest sensitivity and specificity values for insomnia-control discrimination. Accounting for hypnotic medication and sleep apnea use did not change the findings.
Sleep diary parameters discriminated individuals with insomnia from good sleepers more accurately than actigraphy. These quantitative criteria are similar to those reported by other investigators using different methods and samples, including younger adults. The results suggest that the sleep diary, an inexpensive self-report sleep measure, may be used in clinical and research settings to help distinguish older adults with and without insomnia.
A commentary on this article appears in this issue on page 135.
Levenson JC; Troxel WM; Begley A; Hall M; Germain A; Monk TH; Buysse DJ. A quantitative approach to distinguishing older adults with insomnia from good sleeper controls. J Clin Sleep Med 2013;9(2):125–131.
Insomnia is a disorder characterized by difficulty falling asleep, difficulty maintaining sleep, or non-restorative sleep, accompanied by significant distress and/or impairment in daytime functioning.1–3 Insomnia prevalence estimates range from 1% to 20% of the general population4,5 and 10% to 42% of individuals seen in a primary care setting,6–10 while recent results from the America Insomnia Survey suggest that prevalence estimates range from 3.9% to 23.2% among subscribers of a managed health care plan.11,12 Insomnia is accompanied by significant individual and societal cost13 and is associated with greater healthcare utilization, greater impairment,10,14 and an increased risk of comorbid medical and psychiatric conditions.14,15 Some investigators suggest that age is a consistent risk factor for insomnia, as up to 50% of older Americans report chronic sleep difficulties, with insomnia being the most commonly reported sleep disturbance.16 A recent meta-analysis of polysomnographic measures found that sleep onset latency (SOL) and percentage of stage 1 sleep increase with age, while percentage of REM sleep and sleep efficiency (SE) decrease after age 60.17 Even “good sleepers” among older adults experience sleep quality that is compromised as compared to younger adults.18 Still, other investigators argue that the increased risk of insomnia may be better accounted for by comorbid conditions.19 Nevertheless, the study of insomnia and its treatment across various age groups remains a high priority.
Current Knowledge/Study Rationale: Quantitative sleep criteria for insomnia may help to establish more precise clinical phenotypes. No consensus currently exists, particularly for older adults, who experience increased insomnia prevalence and burden. The aim of this study was to define empirically-derived quantitative sleep criteria for older adults with insomnia.
Study Impact: The sleep diary has high sensitivity and specificity for differentiating older adults with insomnia from good sleepers. Clinicians may use the cutoff values to identify patients with insomnia and to educate patients about normative sleep. In research settings the findings could help to establish a more reliable and uniform phenotype of older adults with insomnia.
Despite the prevalence and burden of insomnia and general agreement on its qualitative criteria,4,20,21 there is less consensus regarding quantitative criteria for the disorder with regard to frequency, duration, and severity of sleep symptoms.4 Though the ICD-10,3 DSM-IV,1 and ICSD-22 may be used to diagnose insomnia, the manuals differ slightly in their diagnostic criteria. Because insomnia may exist on a spectrum with normal sleep, some cutoff is required to determine pathological levels of disturbed sleep. Thus, there is a strong need for consensus in the quantitative criteria of insomnia, particularly with regard to the consideration of changes to the diagnostic criteria for insomnia disorder that are currently proposed for inclusion in DSM-V.22 Furthermore, quantitative criteria may need to be adjusted for age or specified for distinct age groups given the potential effect of age on sleep noted above.
The absence of standard quantitative criteria is problematic for a number of reasons. Since prevalence estimates depend on the criteria used to define the disorder, the lack of consensus may be one contributor to the broad variability of these estimates.14,20 The existence of several different nosologies may result in markedly different classification of results in both clinical and research environments,21 as well as semantic and definition confusion in epidemiological studies.4 Moreover, the broad variability in quantitative criteria makes it very challenging to compare the results of different research studies or to draw meaningful conclusions about the studies' findings,4,15 thus reducing the generalizability of the results. While heterogeneity in the presentation of any disorder is to be expected, the lack of severity criteria in some diagnostic manuals but not in others, coupled with inconsistency in quantitative criteria overall, may result in difficulty designing treatment recommendations that are applicable to the majority of patients. Finally, in populations in which sleep disturbances in general are more prevalent (e.g., older adults), precise quantitative criteria may improve the accuracy of study entry criteria for studies comparing individuals with insomnia from “healthy” controls.
Only a handful of studies have systematically evaluated quantitative criteria for defining insomnia. For instance, Lichstein and colleagues20 reviewed 61 clinical trials of insomnia, finding that the most common frequency and duration criteria were 3 nights of insomnia over 6 months of sleep complaints, respectively. However, they were unable to identify the modal severity criterion based on published studies. These investigators also used data from an epidemiological survey of 772 individuals with insomnia and normal controls to identify ≥ 31 minutes of SOL or WASO as the optimal severity criterion, based on 14 days of sleep diary data.
Lineberger and colleagues23 examined the criteria that best discriminated 160 age- and sex-matched men and women with insomnia from controls using 14 days of sleep diary data. The authors were unable to identify any single optimal combination of frequency and severity criteria, although the best frequency criterion value tended to decrease as the severity criterion value increased. They also found that sensitivity/specificity measures improved when terminal WASO (TWASO, wake time between sleep onset and time out of bed in the morning) was excluded from the severity criterion for those under the age of 50. This was not true, however, for those age 50 or older. Among this older age group, the optimal severity criterion was ≥ 20 min of SOL or MWASO (middle of the night WASO, wake time between sleep onset and awakening in the morning) when TWASO was excluded, but the optimal cutoff was ≥ 31 min of SOL, MWASO, or TWASO when it was included.
Most recently, Natale and colleagues24 studied 408 individuals with insomnia or normal controls to identify the most efficient actigraphic parameter in the assessment of insomnia. Using multiple linear discriminant analyses the authors reported that the combination of total sleep time (TST) = 440 min, SOL = 12 min, and number of awakenings (NA) lasting > 5 min = 1.8 best differentiated the groups.
Given the limited literature and that quantitative criteria for discriminating between individuals with versus without insomnia may differ in older adults, the aim of this report was to determine which criterion would best distinguish individuals with insomnia from good sleeper controls using sleep diary and actigraphy. A secondary aim was to compare the performance of actigraphic versus diary-based sleep parameters in their ability to distinguish individuals with insomnia from good sleeper controls.
We were able to address these questions because specific quantitative severity criteria for sleep measures were not used to define entry criteria for eligible participants. Moreover, the present focus on the optimal severity criterion in older adults is especially relevant given the increased insomnia burden experienced by this population discussed earlier,16–18 and given the evidence that age is relevant in the selection of particular criterion values.23 Finally, we were able to compare severity criteria among subgroups of insomnia sufferers based on their use of medications and the presence of sleep apnea.
Participants and Design Overview
The data used in the present analyses were drawn from a study of older adults with chronic insomnia who were treated with brief behavioral treatment of insomnia25 (AG 20677, D.J. Buysse, PI), which was a part of a broader program project examining behavioral intervention strategies for sleep problems of older adults (AG 20677, T.H. Monk, PI). Participants (n = 119) were recruited from a single primary care practice or from the community via advertisements. Eligibility criteria included age ≥ 60 years, absence of dementia, absence of substance use disorders, no recent hospitalizations, no ongoing chemotherapy or other cancer treatments, and psychiatric disorders must be treated. The insomnia group (n = 79) met criteria for primary insomnia according to the DSM-IV1 and for general insomnia according to the ICSD-22 based on self-report questionnaires and clinician interviews. The criteria for insomnia included a sleep complaint lasting ≥ 1 month, despite having adequate opportunity to sleep, associated with daytime impairment or significant distress. In order to increase the generalizability of the findings, the usual DSM-IV insomnia exclusion criterion for medical or psychiatric disorders was not applied in this study. As such, participants in this study were eligible if they met criteria for primary or comorbid insomnia (i.e., insomnia that is comorbid to another medical or psychiatric condition) according to strict DSM-IV criteria.
Insomnia participants completed PSG studies in order to assess sleep apnea and periodic limb movements. Individuals with apnea hypopnea index (AHI; number of breathing pauses or shallow breathing episodes per hour of sleep) > 20 or periodic limb movement arousal index (PLMAI; periodic limb movement per hour of sleep) > 20 (according to American Academy of Sleep Medicine Task Force standards) were excluded. While those with a previous diagnosis of sleep apnea or those using continuous positive airway pressure (CPAP) treatment were not excluded, the final sample of individuals with insomnia did not include any individuals with this history. Good sleeper control participants (n = 40) were individuals without clinically significant sleep complaints who completed self-report, interview, and actigraphy measures only, including a detailed sleep questionnaire and a structured sleep interview. Because control participants did not have PSG studies, we cannot exclude the possibility that some would have had asymptomatic sleep apnea or periodic limb movement disorder.
The study procedures have been described elsewhere in full.25 Briefly, after telephone screening interview, participants completed sleep diaries, structured interviews to evaluate sleep and psychiatric disorders, sleep, medical history and medication surveys, and sleep and psychiatric symptom questionnaires regarding depressive symptoms, subjective sleep quality, and level of sleepiness. In the pre-treatment assessment phase (and 4 weeks after the start of the intervention) insomnia participants completed the Pittsburgh Sleep Diary (PghSD)26 over the course of 2 weeks, during which time they also wore an actigraph. Insomnia participants also completed 2 consecutive nights of in-home PSG studies at these time points, the first of which served as a screening PSG to quantify apnea and periodic limb movements.
The study was approved by the University of Pittsburgh Biomedical institutional review board. All participants provided written informed consent and were financially compensated.
The Pittsburgh Sleep Diary26 is a daily self-report measure of bedtime and rise time, as well as a variety of other measures of sleep. Wrist actigraphy data (Minimitter Actiwatch 64; Minimitter, Bend, Oregon) were analyzed in 1-min epochs with Actiware version 5.04 software, using sleep diary data to identify bedtime and wake time. In cases where visual inspection showed an obvious discrepancy between sleep diary times and observed activity patterns, actigraphy bed and/or rise times were edited to reflect what was reported in the sleep diary. Sleep parameters calculated from the sleep diary and actigraphy data included SOL, WASO, TST, and SE. These variables were chosen as they have been validated actigraphy parameters or have been conventionally used in reports examining quantitative criteria for insomnia.20,23,24,27 Definitions provided by the Actiwatch software were used to determine these variables, based on bedtime and wake time reported in the sleep diary. In the present study, WASO refers to the amount of time awake from sleep onset to the final morning awakening (MWASO). We were able to examine two measures of SE from the sleep diary data: SE awake, which is based on the time between sleep onset and final morning awakening, and SE out of bed (SE OOB), which is based on the time between when participants got in bed and the time they got out of bed.
Subjective sleep quality and sleep disturbances during the month prior to study entry were measured with the Pittsburgh Sleep Quality Index28 (PSQI) and the Epworth Sleepiness Scale,29 both extensively validated self-report questionnaires. In addition to providing information regarding other demographic and clinical measures, information on participant medication use was also collected.
Prior to statistical testing, the data were examined for normality and transformations were used as necessary. Descriptive statistics were generated to characterize the entire study sample and insomnia and control subgroups on basic demographic and clinical variables, as well as 4 sleep measures (SOL, WASO, TST, SE) from sleep diary and actigraphy. T-tests were used to test for group differences on the continuous variables, and Fisher exact tests were used to test categorical variables.
Receiver operator characteristic (ROC) curves were generated for all sleep variables to examine sensitivity and specificity of the sleep measures and area under the ROC curve (AUC) values. Sensitivity was taken as the probability of classifying an insomnia subject correctly, and specificity the probability of classifying a control participant correctly, based on standard diagnostic criteria that were used to determine whether a participant was classified as having insomnia or as a good sleeper control. Larger AUC values indicate more accurate classification of participants. The Youden index was calculated to determine the optimal cutoff for each sleep diary measure for discriminating between the insomnia and control subjects. The Youden index reflects the intention of maximizing overall correct classification rates and minimizing misclassification rates, with values ranging from 0 to 1, where 1 indicates better fit. Statistics were run using SAS v 9.2. Variables were considered significant at p < 0.05 (2-tailed).
The demographic and clinical characteristics of participants in the study are presented in Table 1. The overall sample included 67.2% women, and had a mean age of 71.7 years. Participants with insomnia had a median AHI index of 8.6 (S.D = 5.3, range = 0.2–22.1) and a median PLMI+A score of 4.7 (SD = 4.8, range = 0.0–19.1). As expected, participants with insomnia reported poorer subjective sleep quality and more sleep disturbances, as measured by the PSQI. They were also more likely to be taking antihistamines (Fisher exact test p = 0.002), benzodiazepines (Fisher exact test p < 0.0001), and gastrointestinal medications (Fisher exact test p = 0.04) than good sleepers. Participants with insomnia reported statistically significantly longer SOL and WASO, lower SE awake and SE OOB, and shorter TST than the control group on the sleep diary. The groups differed on actigraphy-measured TST only, with those in the insomnia group sleeping less than those in the control group.
Descriptive statistics of individuals with insomnia and controls participating in the AW3 protocol
Descriptive statistics of individuals with insomnia and controls participating in the AW3 protocol
Sensitivity-Specificity and ROC Analyses (Table 2)
Area under the curve, cutoff values, and sensitivity and specificity characteristics for sleep diary and actigraphy
Area under the curve, cutoff values, and sensitivity and specificity characteristics for sleep diary and actigraphy
The sleep diary produced AUC measures in the high range, with all values ≥ 0.84.30 In contrast, actigraphy performed poorly at discriminating insomnia and control participants, with all AUC values in the low or moderate range (≤ 0.61).30 Sleep diary performed significantly better than actigraphy (p < 0.05) as the confidence interval for each actigraphy parameter did not overlap with the confidence interval of the corresponding sleep diary parameter. Moreover, none of the sleep diary confidence intervals contains the value 0.5, indicating that each parameter performed better than chance at discriminating the participants (p < 0.05). Using the Youden index, the following sleep diary measures best differentiated the groups: SOL = 17.7 min, WASO = 20.7 min, SE awake = 92.1%, SE OOB = 83.8%, and TST = 388.1 minutes. Sleep diary SE awake, the measure with the highest sensitivity and specificity values, correctly identified 76 of 78 participants with insomnia and 36 of 40 good sleeper controls. Some patients with insomnia were taking antihistamines (n = 22), benzodiazepines (n = 18), or both (n = 35), which may have had hypnotic effects. Sensitivity analyses excluding the medicated individuals produced nearly identical results (Table 3). Moreover, sensitivity analyses excluding participants with AHI > 15 also produced nearly identical results (Table 4).
Area under the curve for sleep diary and actigraphy by medication status
Area under the curve for sleep diary and actigraphy by medication status
Area under the curve, cutoff values, and sensitivity and specificity characteristics for sleep diary and actigraphy among subjects with AHI < 15
Area under the curve, cutoff values, and sensitivity and specificity characteristics for sleep diary and actigraphy among subjects with AHI
The present study evaluated quantitative sleep criteria to distinguish individuals with insomnia from good sleepers using two assessment methods. Sleep diary measures produced areas under the curves (AUC) in the high range (0.84–0.97), whereas actigraphy performed poorly at discriminating the two groups (AUC 0.58–0.61). The Youden Index identified SOL = 17.7 minutes, WASO = 20.7 minutes, SE awake = 92.1%, SE OOB = 83.8%, and TST = 388.1 minutes as the sleep diary measures that yielded the highest sensitivity and specificity values, with diary-measured SE awake performing best at differentiating the groups. Our study appears to be the first to examine quantitative sleep thresholds using sleep diary and actigraphy among a sample of older adults who were not required to meet specific quantitative severity thresholds for entry into the study. The absence of such entry criteria limits the circularity of the findings based on study entry criteria.
Given that actigraphy is more costly than diary measures, and given our limited support for the use of this sleep measure in distinguishing individuals with insomnia from good sleepers, our findings suggest that the sleep diary, a low-cost subjective report of sleep, is highly effective at differentiating these two groups. Indeed, Lichstein and colleagues20 found that the vast majority of the insomnia studies they reviewed already relied on self-report sleep diaries to determine study eligibility. Our findings provide support for the use of such low-cost methods for this purpose.
We identified SE awake = 92.1% as the sleep diary measure that best differentiated the groups, perhaps because SE takes into account TST, SOL, and WASO in its measurement. Sleep diary SE OOB = 83.8% differentiated the groups when using a definition of SE that includes time in bed at the beginning of the night and after the final awakening. This cutoff more closely approximates the SE value that we would expect of older adults, and the value that is often reported in the literature. For example, the three studies31–33 cited by Lichstein et al.20 using polysomnography to identify quantitative criteria for insomnia among adults reported SE < 85% as the cutoff value. We would expect this value to be even smaller in older adults, given previous research showing a decrease in SE after age 60.18 This SE finding indicates that older adults may be spending more time in bed, but not actually sleeping, at the beginning and the end of the night. Among previous reports, Lineberger et al.23 suggested that the optimal severity cutoff using SOL or MWASO was ≥ 20 minutes among older adults. Lichstein et al. identified ≥ 31 minutes of SOL or WASO as the optimal severity criterion among middle-aged and older adults, and Natale and colleagues24 reported that WASO = 25 minutes optimally differentiated the groups according to the Youden index in their study of individuals aged 16–71. While we identified 20.7 minutes as the optimal WASO value, the variability in all of these results may be related to differences in the specific sleep diaries that were used. This necessitates the validation and use of a standardized sleep diary that would be accepted widely, such as the Consensus Sleep Diary (CSD).34
One potential explanation for the poorer performance of actigraphy than sleep diary in the present report may be that actigraphy has much better sensitivity for identifying epochs of sleep than specificity for identifying epochs of wake. Indeed, a recent comparison of actigraphy and PSG in older adults with chronic primary insomnia found low specificity for detecting wakefulness (36.3%), with an underestimated total wake time and SOL.35 The potential for actigraphy to overestimate sleep is particularly problematic for individuals with insomnia who may have substantial amounts of sedentary wakefulness. Most published studies, including the current one, make an insomnia diagnosis and measure treatment outcomes based on patient history and self-report. Moreover, clinical history and subjective report of symptoms is recommended as standard research assessment of insomnia.36 Thus, it follows that the information reported in the sleep diary, a self-report measure, provided higher sensitivity and specificity values in discriminating the two participant groups.
These findings have implications in both clinical and research settings. First, clinicians can be confident in their reliance on the sleep diary to assist in the identification of older adults with insomnia. This method may reduce costs to patient and clinicians, and the optimal cutoff values provided here may be used as a guide for clinicians in their evaluations. While some individuals with insomnia may report values outside of our cutoff range, the values reported here may serve to raise the index of suspicion of insomnia among clinicians treating individuals with clinically significant symptoms of insomnia who do not meet the severity criteria reported here. Moreover, clinicians may use these values to educate patients about what is considered normative sleep versus sleep potentially indicative of a sleep disorder. These findings could also help to establish more accurate estimation of insomnia prevalence, and they may increase the reliability and uniformity of individuals entered into research studies of insomnia, allowing for the testing of treatments and the provision of treatment recommendations that may be applicable to more individuals suffering from the disorder.
Our findings should be interpreted in light of several strengths and limitations. First, a large portion of insomnia sufferers use pharmacotherapy to manage the sleep disturbance. Our ability to demonstrate that results of the ROC analyses were nearly identical when excluding participants using potentially sedating medications and those with AHI > 15, we believe, increases the generalizability of our results to the general population of older individuals with insomnia. The recruitment strategy of the participants is another strength, in that patients were recruited both from the community and from primary care practices, designed to capture a representative sample of community-dwelling adults. Last, participants in the insomnia group met diagnostic criteria for the disorder according to DSM-IV and ICSD-2 standards, but there were no quantitative severity criteria required for study entry, ensuring that our findings do not reflect a priori study inclusion criteria.
Future studies should aim to cross-validate our findings in a separate sample of older adults with insomnia and good sleeper controls in order to corroborate the reported cutoffs. Moreover, future work should replicate our findings using a sample of individuals with insomnia and good sleeper controls who also underwent PSG. Though participants with insomnia in this study completed in-home PSG, good sleeper controls did not, limiting our ability to examine the discriminant ability of this sleep measure. While PSG is considered the “gold standard” objective measure of sleep, it is also more costly and likely more impractical for patients than either sleep diary or actigraphy. Moreover, PSG is not clinically indicated for the routine assessment of insomnia by the American Academy of Sleep Medicine,37 though it may provide useful information about insomnia symptoms or other disorders commonly comorbid with insomnia. Still, a comparison of the discriminant abilities of PSG, sleep diary, and actigraphy would help to answer the question of whether the benefits of PSG outweigh the burdens. Another limitation is that we cannot exclude the possibility that good sleepers may have had asymptomatic sleep apnea or periodic limb movement disorder since they did not undergo PSG screening. However, they completed a detailed sleep questionnaire and had a structured sleep interview, which were used to identify and exclude good sleepers who reported sleep complaints. The presence of undiagnosed apnea or PLMD would be likely to attenuate, rather than enhance sensitivity and specificity findings.
Overall, these findings have important implications for the identification of individuals with insomnia and the uniformity of insomnia study entry criteria. The findings add to previous work20,23,24 in this area aiming to standardize frequency, duration, and severity criteria for insomnia and validate the utility of inexpensive prospective self-report measures of sleep.
This was not an industry supported study. Dr. Buysse serves as a paid consultant for Merck, Philips, and Transcept Pharmaceuticals, Inc., and he has been paid for lectures at international, non-CME educational meetings supported by Servier and Astellas. The other authors have indicated no financial conflicts of interest.