Comparison of Computerized and Standard Cognitive Test in Thai Memory Clinic

INTRODUCTION

There have been many computerized cognitive tests developed for the Western aging population or patients with cognitive impairment. However, there were few computerized cognitive tests in Asia. Several issues differed from the Western context as each country in Asia uses their own languages and has their own cultures. Most elderly people in suburban areas cannot speak English and are not able to use the computer although the increased use of smartphone and tablet technology has been noticed. Median education years of Asian aging populations are lower than the Western population. Thus, available computerized cognitive tests could not be applied to the Asian aging population.

Computer-based Thai Cognitive test or Computer-based Thai Mental State Examination (cTMSE) was developed aiming to help doctors to easily get the accurate results of TMSE[123] in a routine, busy outpatient clinics (OPD). cTMSE can assess memory, attention, language, executive, and visuospatial functions. However, cTMSE still needed to be assessed for validity. The purpose of this study was to compare the evaluation process; in terms of feasibility, duration of the test, participants/administrator preference, and the results of cognitive tests between cTMSE and the standard Thai Mental State Examination (sTMSE).

METHODS

There were two groups of participants: the elderly and dementia group. For the elderly group, participants with the following criteria were included; (1) age >60 years old, (2) not demented, and (3) no underlying structural brain disease or any diseases that would interfere with the cognitive evaluation. The TMSE was used to evaluate the cognitive function with the cutoff point of < 23 (from 30) to define cognitive impairment.[34] The diagnosis of dementia was based on a decline in cognitive function that affected the subject's activities of daily living. For the subgroup of patients with dementia in this study, only those with mild-to-moderate dementia (TMSE 11–23) were included in the study. All participants would be asked to have TMSE tests by standard method (sTMSE) and computer-based method (cTMSE), at least 2 weeks and up to 2 months apart. Standard TMSE was performed by a well-trained research assistant who was certified for doing Thai global cognitive tests. The duration of each session was recorded. Gross evaluation on hearing levels during standard test were recorded; 0 = normal hearing, 1 = mild hearing impairment, as the administrator had to speak louder while performing the test, and 2 = moderate-to-severe hearing impairment, as the administrator had to almost yell while performing the test. Patients’ and administrator preference was asked after the participants finished both methods of cognitive evaluation. Scores and duration of the tests were compared using dependent paired t-test. Agreement of the tests between two methods and Kappa statistics were analyzed. This research was approved by the Faculty of Medicine, Thammasat University Ethics committee: Protocol number MTU-EC-IM-2-206/58. Informed consents were performed per the study protocol.

RESULTS

There were 44 participants; 22 elderly, nondemented participants and another 22 participants with mild-to-moderate dementia. Baseline characteristics of participants were presented in Table 1. Participants with dementia were older and had lower education, more proportion of hearing impairment as compared to the nondementia group. However, the reliability of the test was assessed by comparing the same test by two methods in each participant. Thus, the difference in baseline characteristics between the control elderly and those with dementia would not affect the statistical analysis. Paired t-test showed no significant difference in scores between the two methods (mean sTMSE vs. cTMSE: 22.84 vs. 22.62, 95% confidence interval [CI]: [−0.465] to 0.987, P = 0.524). Percent of agreement between the two methods was 92.5%, with the Kappa of 0.85 (P < 0.001). As an additional test of agreement between sTMSE and cTMSE, we constructed a Bland–Altman plot, as shown in Figure 1. The Bland–Altman plot represented every difference between sTMSE and cTMSE scores against the average of the measurements and showed an estimated agreement interval. We found that the 95% limit for the range of possible error was −3.1, 3.6 and 5% outside the limits of agreement.

Table 1 Baseline characteristics of participants and patients

Close

Figure 1

Bland–Altman plot between computer-based Thai Mental State Examination and standard Thai Mental State Examination scores

Export to PPT

Duration of the test by sTMSE was slightly shorter than the cTMSE (7.31 min vs. 7.97 min, 95% CI: [−1.159] to [−0.175], P = 0.09). Overall, participants liked being tested by cTMSE more than sTMSE, especially in the subgroup of participants with moderate-to-severe hearing impairment, they liked cTMSE more than sTMSE as three of them (33%) responded that the two methods were an equal, whereas six (67%) preferred cTMSE. Administrators also liked to evaluate by cTMSE more than sTMSE.

DISCUSSION

There are a lot of computerized cognitive tests and batteries developed to use for different purposes. Some are used for the evaluation of an individual's cognitive status in patients with dementia. Others were developed for screening in older adults, looking for new cases of mild cognitive impairment and dementia. New generations of computerized cognitive tests have been introduced such as a short screening test and a computerized adaptive test which could adapt to the patient's level of functioning. This highlights the shift toward early screening/diagnosis and computerized testing will be implemented in health care.[5] Computerized cognitive tests have many advantages over traditional neuropsychological tests such as savings of costs and time, standardization of administration and stimulus presentation, accurate recording of responses, and the ability to automatically store and compare an individual's performance between testing sessions.[56] However, slow uptake to apply these new technologies in Asian older adults was noticed. One big barrier was the difference in languages and cultures, even among Asian countries. Each country has its own language and culture. Other constraints were the time needed to translate into other languages and evaluate for the accuracy of interpretation in use and the cost of copyright.

In Thailand, there are only a few Thai global cognitive tests. The most common cognitive test in Thai patients is Thai Mental State Examination (TMSE), which has been studied for the distribution of scores in a large aging population (4459 participants) across different age groups and educational levels and also in patients with dementia.[123] Due to the lack of well-trained health-care personnel, the larger number of aging population who require the cognitive test, and the busy and crowded situation of OPD, performing cognitive tests is difficult in routine practice. Computer-based Thai Cognitive test or cTMSE was developed aiming to help doctors to easily get the accurate results of TMSE in a routine, busy OPD. cTMSE can assess memory, attention, language, executive, and visuospatial functions. The computer is equipped with a headphone and built-in camera. cTMSE can be administered by personnel with limited training. After registration, the participants are asked to adjust the volume of the headphone to match their hearing level. The administrator begins the test by clicking the recorded questions. After the participants answer, copy the figure, and perform the task, the administrator scores by clicking and choosing the appropriate score on each question from a drop-down menu. All of the scores are automatically added up, and a report is created and printed. When the participants or patients had previous studies, the previous results could be chosen to compare.

This study showed the validity of the computer-based TMSE. The scores from sTMSE and cTMSE were well correlated and not significantly different. Although the duration used for sTMSE was shorter than cTMSE, the difference was minimal (1 min). This would be explained by the fixed period of time used to listen for the whole “recorded script” on each question before moving forward to the next questions. Duration for a cTMSE session was rather short; approximately 7 min for screening in aging adults and 9 min for cognitive evaluation in patients with mild-to-moderate dementia. Overall, the participants preferred to be evaluated by cTMSE, especially in the subgroup with moderate-to-severe hearing impairment.

CONCLUSION

Computer-based TMSE was feasible to use and accurate for screening in aging adults and for cognitive evaluation in patients with mild-to-moderate dementia.