ADVERTISEMENT

Issue Navigator

Volume 10 No. 01
Earn CME
Accepted Papers
Classifieds







Scientific Investigations

The Effects of Poor Sleep Quality on Cognitive Function of Patients with Cirrhosis

http://dx.doi.org/10.5664/jcsm.3350

Charmaine A. Stewart, M.D.1; Robert Auger, M.D., F.A.A.S.M.3; Felicity T. B. Enders, Ph.D.2; Donna Felmlee-Devine, M.A.3; Glenn E. Smith, Ph.D.3
1Division of Gastroenterology and Hepatology, Department of Internal Medicine University of Illinois at Chicago, Chicago, IL; 2Division of Biostatistics, Department of Health Sciences Research; 3Department of Psychiatry and Psychology, Mayo Clinic, Rochester, MN

ABSTRACT

Objectives:

This study was conducted to assess the ill-defined relationship between sleep quality and multiple, specific domains of cognitive function in patients with cirrhosis.

Methods:

A comprehensive battery of neuropsychological tests (divided into six neurocognitive domains) and a standardized, validated measure of sleep quality (Pittsburgh Sleep Quality Index [PSQI]) were administered to patients with cirrhosis and without evidence of overt hepatic encephalopathy, recruited from liver transplant and advanced liver disease clinics (n = 34). An inflammatory bowel disease (IBD) control group (n = 23) was similarly recruited and evaluated to control for the secondary effect of a chronic illness on cognition. PSQI global and component scores were used to predict cognitive function in each neurocognitive domain, using linear regression

Results:

Global PSQI scores were significantly higher (indicating poorer sleep quality) in the cirrhosis group (median [range] = 10 [1-19]) than in IBD controls = 5 (1-14); p = 0.002). After controlling for age and education, short duration of sleep was associated with impaired memory for patients with cirrhosis; the use of soporific agents was associated with poor visual-perceptual function in patients with IBD.

Conclusions:

Poor sleep was associated with worsening of the already impaired cognitive function of patients with cirrhosis.

Citation:

Stewart CA; Auger R; Enders FTB; Felmlee-Devine D; Smith GE. The effects of poor sleep quality on cognitive function of patients with cirrhosis. J Clin Sleep Med 2014;10(1):21-26.


Up to 70% of individuals with cirrhosis (regardless of etiology) experience sleep disturbances.1,2 These disruptions commonly manifest as difficulty falling asleep and a shift in sleep schedule toward the latter part of the night, resulting in daytime sleepiness.2 Sleep deprivation has profoundly negative effects on cognitive function,310 which is already impaired in the majority of patients with cirrhosis.11,12 Hence, it is probable that sleep dysfunction from a circadian delay in patients with cirrhosis could have an additive effect on cognitive impairments.

In the general population, the psychomotor deficits associated with sleep deprivation are similar to those attributed to blood alcohol levels at or above the legal limit.10 Among patients with cirrhosis, deficits in sleep and cognition can have serious negative effects on their quality of life, safe driving, and workplace productivity.1,1315

In previous work, actigraphy and a validated sleep questionnaire were used to evaluate sleep disturbances in patients with cirrhosis compared to a healthy volunteer group.1 The cirrhosis group was found to have greater difficulties initiating sleep, significantly shorter total sleep time, more frequent awakenings, more difficulty awakening in the morning, and higher levels of daytime sleepiness. Underlying these findings was a circadian delay, as evidenced by the plasma melatonin peak in patients with cirrhosis and a healthy group.1 A correlation between severity of hepatic encephalopathy (using the Psychometric Hepatic Encephalopathy Score [PHES]) and sleep quality (as measured by Pittsburgh Sleep Quality Index [PSQI]) was not found; however, this study was limited in that individual cognitive domains were not examined.2

BRIEF SUMMARY

Current Knowledge/Study Rationale: This study was undertaken to compare the effects of poor sleep quality on cognitive function in patients with cirrhosis and patients with inflammatory bowel disease (IBD) to determine whether poor sleep in patients with cirrhosis affected their cognitive function.

Study Impact: The findings of worsening cognitive function of patients with cirrhosis who have poor sleep, indicates that larger studies are needed to further classify the sleep disorders and develop efficacious and safe treatment.

The objectives of our study were twofold: to evaluate differences in sleep quality between patients with cirrhosis and patients with another chronic disease (inflammatory bowel disease [IBD]); and in both groups, to test for an association between sleep quality and multiple specific domains of cognitive function. Sleep quality was assessed by a validated sleep questionnaire, the PSQI (global scores and individual component scores). Cognitive function was assessed by a comprehensive neuropsychological battery that evaluated specific neurocognitive domains, and therefore was expected to yield more accurate findings. We hypothesized that poorer sleep quality would be associated with more impaired cognitive function, and that this association would be stronger for patients with cirrhosis than for the IBD control group.

METHODS

Study Subjects

This study was approved by the Mayo Foundation Institutional Review Board. We recruited consecutive adult outpatients with a diagnosis of cirrhosis (based on histological, clinical, and radiological parameters) who were being treated in the liver transplant and advanced liver disease clinics at Mayo Clinic, Rochester, Minnesota, between May 2003 and June 2006. Patients were eligible for inclusion if they were English-speaking and abstinent from alcohol and other substances of abuse for at least 6 months. We excluded patients taking benzodiazepines or antipsychotic medicines, those with acute complications from liver disease, hepatic encephalopathy greater than grade 2, history of primary neurological disorders, and current uncontrolled psychiatric disorder.

Consecutive patients with inflammatory bowel disease (IBD) were recruited from the IBD clinics at Mayo Clinic, Rochester, Minnesota. The same inclusion and exclusion criteria used for patients with cirrhosis were employed, except patients with IBD who had liver disease or abnormal liver tests were excluded. All patients in the study were assessed with standardized neuropsychological tests and clinical evaluation during a single visit. The grade of hepatic encephalopathy for each patient was determined by a clinician on the basis of the West Haven Criteria within 24 h of neuropsychological testing. No patients were found to have overt hepatic encephalopathy.

Measures

Patients with cirrhosis and IBD control subjects completed the same set of neuropsychological and sleep measures, administered during a single clinic visit.

Neuropsychological Assessment

Participants were prospectively assessed with a standard pencil-and-paper neuropsychological battery as outlined in Table 1. Testing occurred under blinded conditions by the same trained neuropsychometrist. Results were interpreted by a single neuropsychologist.

Neurocognitive domains and associated neuropsychological tests

jcsm.10.1.21.t01.jpg

table icon
Table 1

Neurocognitive domains and associated neuropsychological tests

(more ...)

All scores were transformed to age-adjusted standard scores (placed on a common z-score metric), such that negative scores represented poorer performance. Age adjustment was based on appropriate normative samples (i.e., the standardization samples for WAIS/WMS-III,16,17 CVLT,16 CPT-II,17,18 Heaton,19 norms for Trail Making,20 Groove Pegboard,21 Meyers norms for Rey-Osterrieth22). These standard scores were then organized into pertinent cognitive domains as reflected in Table 1. All domains included ≥ 2 measures in order to enhance the reliability of the estimated z-score in each domain. The score for each domain is the average age-adjusted z-score for all subtests in that domain.

Although each domain examines a different class of behaviors, they work in concert and are interdependent. In this study, attention/concentration refers to the ability of an individual to focus on selected sensory stimuli and ideas without being distracted by other stimuli. Information processing speed refers to the ability to rapidly process and respond to stimuli. The learning domain evaluates the ability to acquire new information. The memory domain samples recall or recognition of words and stories. Visual perception refers to the understanding of figural and spatial relationships and construction of designs. The motor domain involves tests of manual dexterity over a period of time.16,22

Pittsburgh Sleep Quality Index

The PSQI is easy to administer and interpret, and is widely used in sleep investigations. The questionnaire has 24 items; 19 are self-rated, and 5 are rated by the bed partner of the participants. The self-rated items are combined to form 7 component scores of subjective sleep quality, sleep latency, sleep duration, habitual sleep efficiency, sleep disturbances, use of sleeping medications, and daytime dysfunction. Each of the 7 components is scored according to severity: a score of 0 = no difficulties for the component of interest; 1 = fairly good; 2 = bad; and 3 = very bad. These component scores are combined to generate a global PSQI score that reflects sleep quality and disturbances over a 1-month period. The global score ranges from 0 to 21, with higher scores indicative of poorer sleep quality.23

Using a cutoff of a global PSQI score > 5, the test demonstrated 89.6% sensitivity and 86.5% specificity in distinguishing between good and poor sleepers during an 18-month field trial. In our study, a PSQI score > 5 was considered consistent with the presence of poor sleep quality. We assessed both global PSQI and component scores, and determined their correlation with each of the neuropsychological domains.

Statistical Analysis

We compared cases and controls with respect to demographic predictors, using the Fisher exact test for binary variables and the Wilcoxon rank sum test for continuous variables. Unadjusted PSQI scores were compared between study groups with the Wilcoxon rank sum test for the global score and Fisher exact test for the subscores. Age was not tested as a confounder, because neuropsychological scores are already adjusted for age.

To assess the differential impact of PSQI scores on neuropsychological domain between cases and controls, we used the global PSQI score to predict neuropsychological domain scores using multiple linear regression. The outcome of the neuropsychological domain was predicted by global PSQI, together with case-control status, and the interaction of global PSQI and case-control status. The interaction term was included to test for any differences in association of global PSQI with each neurocognitive domain between cases and controls. These models were also adjusted for age and education, as these variables proved different between the patients with cirrhosis and IBD. As a secondary analysis, we repeated these models, replacing the global PSQI score with each PSQI subscore.

RESULTS

Participants

We enrolled 34 patients with cirrhosis and 23 IBD control subjects. The causes of cirrhosis included alcohol use (14/34, 41%) and cholestatic liver disease (4/34, 11.7%); the remainder (16/34, 47%) were uncharacterized. Median MELD score (range) was 11 (6-32). Gender distribution was similar in both groups (Table 2). Median age was greater for the cirrhosis group than the IBD control group (58 vs. 52 years; p = 0.028), whereas mean years of education was greater for IBD controls (15.5 vs. 14.3 years; p = 0.018).

Characteristics and sleep abnormalities in patients with cirrhosis vs IBD controls

jcsm.10.1.21.t02.jpg

table icon
Table 2

Characteristics and sleep abnormalities in patients with cirrhosis vs IBD controls

(more ...)

Group Differences in Global and Component PSQI Scores

Global PSQI scores were significantly higher in the cirrhosis group (mean ± SD = 9.8 ± 4.9) than in IBD controls (5.6 ± 4.2; p = 0.002), indicating poorer sleep quality in patients with cirrhosis (Table 2). Patients with cirrhosis also reported higher scores (poorer sleep quality) than did the IBD control group for the following 4 PSQI components: sleep disturbance (p = 0.034), daytime dysfunction (p = 0.019), habitual sleep efficiency (p = 0.001), and sleep latency (p = 0.031). The groups did not differ significantly in their reported sleep duration, frequency of using soporific medications, or subjective sleep quality.

There were differences in PSQI component scores for patients with cirrhosis compared with IBD controls. With regard to sleep disturbance component scores, all patients with cirrhosis reported sleep disturbances (34/34), with 23/34 (68%) reporting scores ≥ 2 (i.e., greater than once weekly). In comparison, 22/23 (96%) IBD controls reported sleep disturbances, but only 9/22 (41%) had scores ≥ 2 (p = 0.034). Using the same method of comparison, daytime dysfunction was reported in 29/34 (85%) patients with cirrhosis, of whom 11/29 (38%) had scores ≥ 2. For daytime dysfunction, in comparison, 12/23 (52%) IBD controls reported difficulties, 5 (22%) of whom had scores ≥ 2 (p = 0.019). Poor sleep efficiency was reported by 25/34 (74%) patients with cirrhosis, 15 (44%) of whom had scores > 2 (< 75% habitual sleep efficiency [HSE] versus 7/23 (30%) of IBD controls who reported impaired HSE, 4 (17%) of whom had scores ≥ 2 [p = 0.001]). Finally, prolonged sleep latency was reported in 29/34 (85%) patients with cirrhosis, 19 (66%) of whom had scores ≥ 2, compared with 5/23 (22%) of subjects with IBD (p = 0.03; Table 2).

In the cirrhosis group, 76% (26/34) of patients had global PSQI scores > 5. There were no significant differences in age, gender, MELD scores, or etiology of disease between patients with cirrhosis who had high (> 5) versus low (< 5) PSQI scores (data not shown). In the IBD control group, 39% (9/23) of patients had a global PSQI score > 5. There was a statistically significant difference between the patients with low PSQI compared with those with high PSQI with respect to age (p = 0.025), but no relationship was seen with gender (data not shown).

Interaction of Sleep Quality and Cognitive Function with Disease Type

We performed a set of linear regression models to compare cirrhotic cases with IBD controls with respect to the impact of PSQI score on neuropsychological function. The association of the global PSQI score with the neuropsychological domain score did not differ significantly between cases and controls for any of the 6 neuropsychological domains (Tables 3A and 3B). For only one neuropsychological domain did either of the 2 potential confounders prove statistically significant: higher education level was positively associated with attention/concentration after adjusting for age, global PSQI, and study group (p = 0.039).

jcsm.10.1.21.t03.jpg

table icon
Table 3

(more ...)

When this analysis was repeated for each PSQI subscore, a few more specific associations became apparent. There was no association between sleep disturbance and learning in the IBD controls; but for patients with cirrhosis (cases), a greater sleep disturbance subscore was associated with higher functional learning (p = 0.04; Table 3A). There was no association between duration of sleep and better memory in controls, but in cases, better sleep duration (i.e., lower PSQI sleep duration subscore) was associated with better memory (p = 0.05; Table 3B). In controls, more use of sleep medications was significantly associated with poorer visual perceptual function; this association was not observed in the case group (p = 0.04; Table 3B).

Another important statistical outcome of these models was to identify strong differences between cases and controls in the association between PSQI and neuropsychological domain. To do this, we relied upon the linear regression coefficient and the coefficient of determination. The coefficient for global PSQI is on the same scale for all the neuropsychological domains, as the domain scores are adjusted z-scores. Unfortunately, none of the global PSQI interaction terms has a strong association with the neuropsychological domain, as the difference between cases and controls represents at most 0.06 SD in neuropsychological function, suggesting that the stronger coefficients of determination are driven by other variables in these models. The interaction coefficients in Tables 3A and 3B are also on the same scale for all the PSQI subscales. There was a 0.2 SD difference in the association between sleep disturbance and learning for cases and IBD controls. There was a 0.3 SD difference in the association of daytime dysfunction and visual perception for cases and IBD controls (other large effects for the visual perception domain were driven by associations within the control group, which were not as strong for cases). There was a 0.23 SD difference in the association of HSE and motor function between cases and controls.

Linear regression analyses to assess the association between neurocognitive function and sleep quality (global PSQI scores and separate composite scores) with neurocognitive assessment as continuous variables for patients with cirrhosis versus IBD controls revealed that sleep latency and time to bed were significantly associated with poor attention/concentration. However, after these variables were adjusted for age and educational level, only time to bed component was significantly associated with poor attention/concentration (p = 0.008; R2 = 0.383).

DISCUSSION

In this study we sought to determine the difference in sleep quality between patients with cirrhosis and a chronic illness control group of patients with IBD, and to assess whether poor sleep was associated with worsened cognitive function among the patients with cirrhosis.

Although sleep disturbances and cognitive impairment are both common among patients with cirrhosis, a definitive link between these two complaints has not been documented. If a specific association were found, greater clinical attention might be paid to sleep disorder screening (which is not routinely done in patients with cirrhosis and without overt hepatic encephalopathy), and to the testing of treatment strategies to ameliorate both sleep and cognitive dysfunction in these patients.

We found that patients with cirrhosis had significantly worse sleep quality than the control IBD group. Moreover, there was an association between poor sleep and cognitive impairment in both groups. Although a relationship between memory impairment and poor sleep has been documented in the general population, this is a new finding for patients with cirrhosis who have poor sleep.24 In general, a direct and multi-faceted relationship exists between sleep and memory, such that sleep enhances recall performance on memory related tasks, and sleep is necessary for long-term memory consolidation. Sleep is important for newly acquired memory; declarative memory, a component of long-term memory, is dependent on sleep activity. The interdependency between sleep and memory has been substantiated by functional imaging of the brain, which has demonstrated decreased activity of the hippocampus, the area of the brain that is needed for memory functions, in individuals with poor sleep.24 This reduced hippocampus activity is associated with increased work of the individual to accomplish tasks. Overall, poor sleep leads to impaired learning and memory in the general population.25,26 Hence, the findings of this study are important for these sick patients, especially for patients with cirrhosis, most of whom are already cognitively impaired. The association between the use of soporific agents and visual perceptual impairment in patients with IBD is not surprising, since visual perceptual impairment is a known side effect of soporific agents.15,2729 However, the surprising finding of improved learning with sleep disturbances in cirrhosis is very probably due to a type I error, as we are not aware of a biological explanation for this finding.

The findings described herein are clinically significant, because they demonstrate that there is a high rate of self-reported difficulties with sleep among patients with cirrhosis, as has been shown by others.1,30,31 In addition, we found poor sleep quality is correlated with variable effects on cognitive function in this population in which diffuse neurocognitive deficits are pervasive, emphasizing the significant health burden that poor sleep has in the cirrhotic population. Sleep and circadian rhythm disorders are, however, treatable; therefore, there is a potentially reversible component of the cognitive deficits seen in patients with cirrhosis. Additionally, it is well known that the neurocognitive effects of these sleep disturbances have negative consequences on quality of life, workplace productivity, and safety,10 all of which could be addressed with accurate identification, characterization, and treatment of sleep disturbances in patients with cirrhosis. Hence, the identification of safe soporific agents could have a significant impact on the health of patients with cirrhosis, in whom metabolism of standard soporific agents is impaired by their diseased livers.3234

This study is one of the few comparing sleep in cirrhosis to a sick control group using a validated sleep questionnaire, and it illustrates the need for larger studies to evaluate the association between sleep and cirrhosis. The findings suggest that intervening with efficacious treatment will be important for patients with cirrhosis who have poor sleep quality.

DISCLOSURE STATEMENT

This was not an industry supported study. The authors have indicated no financial conflicts of interest. Financial Support: NIH NIDDK 60018 to Dr. Stewart.

ACKNOWLEDGMENTS

The authors thank Drs. Anne Marie Weber-Main and Michael Howell for their critical review and editing of manuscript drafts.

REFERENCES

1 

Montagnese S, Middleton B, Mani A, Skene D, Morgan M, authors. On the origin and the consequences of circadian abnormalities in patients with cirrhosis. Am J Gastroenterol. 2010;105:1773–81. [PubMed]

2 

Montagnese S, Middleton B, Skene D, Morgan MY, authors. Night-time sleep disturbance does not correlate with neuropsychiatric impairment in patients with cirrhosis. Liver Int. 2009;29:1372–81. [PubMed]

3 

Stickgold R, author. Dissecting sleep-dependent learning and memory consolidation. Sleep. 2004;27:1443–5. [PubMed]

4 

Stickgold R, author. Sleep-dependent memory consolidation. Nature. 2005;437:1272–8. [PubMed]

5 

Stickgold R, author. Neuroscience: a memory boost while you sleep. Nature. 2006;444:559–60. [PubMed]

6 

Stickgold R, author. Sleep: the ebb and flow of memory consolidation. Curr Biol. 2008;18:R423–5. [PubMed]

7 

Stickgold R, Fosse R, Walker MP, authors. Linking brain and behavior in sleep-dependent learning and memory consolidation. Proc Natl Acad Sci U S A. 2002;99:16519–21. [PubMed Central][PubMed]

8 

Stickgold R, James L, Hobson JA, authors. Visual discrimination learning requires sleep after training. Nat Neurosci. 2000;3:1237–8. [PubMed]

9 

Stickgold R, Hobson JA, Fosse R, Fosse M, authors. Sleep, learning, and dreams:off-line memory reprocessing. Science. 2001;294:1052–7. [PubMed]

10 

Durmer JS, Dinges DF, authors. Neurocognitive consequences of sleep deprivation. Semin Neurol. 2005;25:117–29. [PubMed]

11 

Stewart CA, Enders FT, Schneider N, Felmlee-Devine D, Kamath PS, Smith GE, authors. Development of a three-factor neuropsychological approach for detecting minimal hepatic encephalopathy. Liver Int. 2010;30:841–9. [PubMed]

12 

Ferenci P, Lockwood A, Mullen K, Tarter R, Weissenborn K, Blei AT, authors. Hepatic encephalopathy--definition, nomenclature, diagnosis, and quantification: final report of the working party at the 11th World Congresses of Gastroenterology, Vienna, 1998. Hepatology. 2002;35:716–21. [PubMed]

13 

Kanwal F, Gralnek I, Hays R, et al., authors. Health-Related quality of life predicts mortality in patients with advanced chronic liver disease. Clin Gastroenterol Hepatol. 2009;7:793–9. [PubMed]

14 

Wein C, Koch H, Popp B, Oehler G, Schauder P, authors. Minimal hepatic encephalopathy impairs fitness to drive. Hepatology. 2004;39:739–45. [PubMed]

15 

Bajaj JS, Hafeezullah M, Hoffmann RG, Saeian K, authors. Minimal hepatic encephalopathy: a vehicle for accidents and traffic violations. Am J Gastroenterol. 2007;102:1903–9. [PubMed]

16 

Wechsler DA, author. Wechsler Adult Intelligence Scale-III. 1997. San Antonio, TX: Psychological Corporation.

17 

Jastek S, Wilkinson G, authors. The Wide Range Achievement Test. 1984. Wilmington, DE: Jastak Associates.

18 

Conners K, author. Conners' Continuous Performance Test II. 1994. Toronto: Multi-Health Systems Inc.

19 

Heaton R, Grant I, Matthews C, authors. Comprehensive Norms for an Expanded Halsted-Reitan Battery: Demographic Corrrections, Research Findings, and Clinical Applications. 1991. Odessa, FL: Psychological Assessment Resources.

20 

Reitan RM, author. The relation of the Trail Making Test to organic brain damage. J Consulting Psych. 1955;19:393–9. [PubMed]

21 

Matthews C, Klove H, authors. Instruction Manual for the Adult Neuropsychology Test Battery. 1964. Madison, WI: University of Wisconsin Medical School.

22 

Osterrieth P, author. Le test de copie d'une figure complexe. Arch Psychol. 1944;30:206–356.

23 

Buysse DJ, Reynolds CF 3rd, Monk TH, Berman SR, Kupfer DJ, authors. The Pittsburgh Sleep Quality Index: a new instrument for psychiatric practice and research. Psychiatry Res. 1989;28:193–213. [PubMed]

24 

Jones K, Harrison Y, authors. Frontal lobe function, sleep loss and fragmented sleep. Sleep Med Rev. 2001;5:463–75. [PubMed]

25 

Meideros A, Mendes D, Lima P, Araujo J, authors. The relationship between sleep-wake cycle and academic performance in medical students. Biol Rhythm Res. 2001;32:263–70.

26 

Hill C, Hogan A, Karmiloff-Smith A, authors. To sleep perchance to enrich learning? Arch Dis Child. 2007;92:637–43. [PubMed]

27 

Giersch A, author. The effects of lorazepam on visual integration processes: How useful for neuroscientists? Vis Cogn. 2001;8:549–63.

28 

Martin E, Thiel T, Joeri P, et al., authors. Effect of pentobarbital on visual processing in man. Hum Brain Mapp. 2000;10:132–9. [PubMed]

29 

Klein B, Klein R, Knudtson MD, et al., authors. Associations of selected medications and visual function: the Beaver Dam Eye Study. Br J Ophthalmol. 2003;87:403–8. [PubMed]

30 

Cordoba J, Cabrera J, Lataif L, Penev P, Zee P, Blei A, authors. High prevalence of sleep disturbance in cirrhosis. Hepatology. 1998;27:339–45. [PubMed]

31 

Steindl P, Finn B, Bendok B, Rothke S, Zee P, Blei A, authors. Disruption of the diurinal rhythm of plasma melatonin in cirrhosis. Ann Intern Med. 1995;123:274–7. [PubMed]

32 

Elbekai RH, Korashy HM, El-Kadi AO, authors. The effect of liver cirrhosis on the regulation and expression of drug metabolizing enzymes. Curr Drug Metab. 2004;5:157–67. [PubMed]

33 

Delco F, Tchambaz L, Schlienger R, Drewe J, Krahenbuhl S, authors. Dose adjustment in patients with liver disease. Drug Saf. 2005;28:529–45. [PubMed]

34 

Verbeeck R, author. Pharmacokinetics and dosage adjustment in patients with hepatic dysfunction. Eur J Clin Pharmacol. 2008;64:1147–61. [PubMed]

35 

Wechsler D, author. Wechsler Memory Scale. 1997. 3rd ed. San Antonio, TX: Psychological Corporation.

36 

Delis D, Kramer J, Kaplan E, Ober B, authors. California Verbal Learning Test. 2000. San Antonio, TX: Psychological Corporation.