Effectiveness of IMPUTE ADT-1 mobile application in children with autism spectrum disorder: An interim analysis of an ongoing randomized controlled trial

INTRODUCTION

Autism spectrum disorder (ASD) is a complex neurodevelopmental condition that presents unique challenges for children and their families.^[1] Addressing these challenges often requires specialized behavioral therapies with applied behavior analysis (ABA) and sensory integration therapy (SIT) being the cornerstone of treatment. The ABA involves systematically analyzing behavior, identifying environmental triggers, and implementing tailored interventions to promote positive change.^[2] The ABA employs techniques such as positive reinforcement, prompting, and shaping to teach new skills and reduce problematic behaviors.^[3] However, access to qualified therapists can be limited, particularly in developing countries.^[4] Even in developed nations, long waiting times before reaching a therapist can impede timely intervention.

Recent advancements in healthcare technology, particularly the integration of artificial intelligence (AI), have opened new avenues for addressing these challenges.^[5] Mobile medical applications, driven by AI algorithms, offer the potential to revolutionize the way that we approach ASD intervention.^[6]

Multiple investigators have explored the application of AI in autism intervention. Studies have investigated the effectiveness of various technological tools including augmented and alternative communication devices as well as computer-based interventions.^[7] These endeavors have shown promise indicating that technology-driven solutions hold significant potential to enhance outcomes for children and adults with ASD.

IMPUTE Inc., a software firm dedicated to healthcare technology, has developed a mobile medical application known as IMPUTE ADT-1. By leveraging AI-based algorithms, the application offers personalized programs that evolve in response to the user’s progress, and it has incorporated the principles of ABA into this mobile medical application. Previously, a phase II pilot study (unpublished data) exploring the feasibility and safety of this mobile medical application at the National Center of Child Health and Development in Tokyo, Japan, has shown favorable results.

This study assessed the functionality and usability of the ADT-1 app revealing minor glitches that were promptly addressed. Despite these initial challenges, participants expressed a willingness to continue using the app highlighting its potential impact on ASD intervention. Building on the promising results of the feasibility study, a randomized controlled trial (RCT) with an adequate sample size was planned to further evaluate the ADT-1 app’s efficacy, safety, and compliance.

MATERIALS AND METHODS

This open-label, parallel design, and two-arm RCT (1:1 allocation ratio) is being carried out in the Pediatric Neurology division of All India Institute of Medical Sciences (AIIMS), Rishikesh, India, from April 2023 onwards. The clinical trial started recruiting participants after obtaining approval from the Institute Ethics Committee (AIIMS/ IEC/22/658) and registering the clinical trial protocol on the clinical trial registry of India (CTRI/2023/02/050044). All participants were enrolled after obtaining written informed consent from the parents.

The primary objective of the study was to compare the efficacy of add-on treatment with IMPUTE ADT-1 in children with ASD aged two to six years as compared to standard care alone for 12 weeks (in terms of change in autism diagnostic observation schedule [ADOS-2] scores). The secondary objective of the study was to assess the compliance with IMPUTE ADT-1 among participants and also to evaluate the feedback of parents regarding IMPUTE ADT-1 at the end of 12 weeks.

Children with a definite diagnosis of ASD according to the Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition (DSM-V) criteria, aged two to six years were included in the study. Children with uncontrolled epilepsy (seizure within one month before the date for screening), significant hearing and/or vision impairment, which made it difficult for the proper use of ADT-1 by the child, and whose caregivers did not consent for enrolment in the trial were excluded from the study.

We assumed that the improvement in ADOS-2 comparison score at 12 weeks, as compared to baseline would be 50% higher, as compared to the control group (moderate effect size). With an accepted precision of 95%, power of 80%, alpha error of 0.05, and an allocation ratio of 1:1 between ADT-1 and control groups, we calculated the sample size to be 51 in each group. We assumed the attrition rate at 12 weeks to be around 10%. Hence, we decided to enroll 57 children in each group (a total of 114 children with ASD). However, we decided that we would perform a pre-specified interim analysis after the completion of the enrolment of a total of 70 participants.

Block randomization in a 1:1 ratio with variable block size was utilized in this RCT with the help of a computer-generated list of random numbers. It was performed by an independent person not involved in providing the intervention and assessing primary and secondary outcomes. Each random number was kept in an opaque sealed envelope and was opened only at the time of enrolment of the study subjects. Subsequently, the study investigator enrolled the participants and assigned them to either the ADT-1 group or the control group.

Each child underwent a detailed clinical history, physical examination, and neuropsychological assessment using a pre-specified case record form along with specific neuropsychological tests for the study purpose such as ADOS-2 and Autism Diagnostic Interview-Revised (ADI-R). These study-specific neuropsychological assessments were repeated after 12 weeks in each participant. Standard care for autism, as per recommendations and institutional protocol, was continued in both groups. This standard of care treatment was individualized and included ABA, SIT, structured teaching along with speech therapy and occupational therapy as determined by the treating pediatric neurologist, and developmental pediatrician.

Participants in the intervention group (ADT-1 group) were additionally advised to use the specific ABA-based mobile application (Impute ADT-1 application) for at least 60 min/ day continuously for 12 weeks. The parents were initially trained by the investigators on how to use the ADT-1 application with their children. They were also instructed to monitor for adverse effects on eyesight and other potential adverse effects such as increased addiction or preoccupation with smartphones, tablets or digital media using a pre-specified checklist provided to them. The instructions in the mobile application were in English language, and the use of the application by participants was supervised by parents.

This active session with the mobile application lasting at least 60 min/day (consecutively or inconsecutively based on the child’s needs, wants, and skillset), was supervised by the parents. However during this time, the child was allowed access to physical exercise, snacks, and playful physical contact with parents or caregivers. We recorded the total duration of mobile application use by the child. Our goal for the user was to develop a desire to open and use the application every day with the hope that learning itself becomes a conditioned reinforcer. However, the child was not allowed to use other mobile applications or browse the internet while using the mobile application, and parents were also advised about sleep hygiene. They were asked to avoid using this mobile application within two hours of planned bedtime.

The ADT-1 (Autism Therapy App, Developed by IMPUTE Inc. Japan) is a digital application intended to be a non-drug prescription treatment for ASD. Through this application, we planned to deliver evidence-based therapeutic interventions driven by a high-quality, patented software program to address a broad spectrum of learning challenges faced by children with ASD.

The application provides personalized programs tailored to each user’s needs, and the program evolves based on the user’s progress. We used data analytics to measure progress data for each child. The application also utilizes face tracking, eye tracking, and body tracking to gather behavior-related information for each child and apply it in reinforcement learning, employing AI-based algorithms. The contents, tasks, and principles of the mobile application were based on advice from experienced child psychologists, pediatric neurologists, and behavior analysts to ensure a high level of effectiveness and a high degree of engagement by the child.

The application begins with parent feedback. After a thorough assessment of the child, which is carried out on the application, it provides a comprehensive, personalized therapy program that matches the child’s needs, based on our adaptive learning algorithm [Supplementary Material 1]. The child is then guided through their sessions by an animated avatar of their choice. The software can be used on any mobile smartphone or tablet device.

The comprehensive assessment and parent feedback act as a guide to identify the progress path for the individual user. The comprehensive assessment program involves a series of activities that will help determine a clear understanding of the child’s current skill level. After the assessment, a personalized program becomes available for the user. The user then goes through a discrete trial training process. The program breaks down learning for users into small, discrete components and reinforces those points with continuous assessments.

Operability and continuity of this application such as ease of use were noted in the evaluation sheet of the application. Concomitant medications such as risperidone, aripiprazole, atomoxetine, and melatonin if advised by the treating clinician were continued.

Compliance and adherence to the use of the application were monitored in real time by a dedicated team of software personnel, who recorded the daily duration of use of the application, how the child performed, and the progress of the child, demonstrated by their scores. The parents provided feedback every week regarding the progress of their child through phone, and technical problems were addressed by the software team. Any child, who adhered to the daily minimum requirement of 60 min of use of the application ADT-1, was considered compliant. At each physical follow-up visit (at 4, 8, and 12 weeks), the child was evaluated for any adverse effects using a pre-specified checklist.

The primary outcome measure was the change in ADOS-2 comparison scores at 12 weeks as compared to baseline in both groups. The secondary outcome measures were adherence/compliance to the Intervention Program of each participant in the ADT-1 group expressed as a percentage of days the patient in the IMPUTE ADT-1 arm, out of the total study period, used at least 60 min of the application. The other secondary outcome measure was the proportion of parents providing a favorable response to each of the questions in the parent feedback questionnaire.

The ADOS-2 consists of four modules (Modules 1–4) tailored to individuals of different developmental levels. Each module assesses social interaction, communication, play, and imaginative use of materials. It also provides a comparison score, which allows clinicians to compare an individual’s performance to others of the same age and language level. This score aids in determining the severity of autism-related symptoms. In addition, ADOS-2 includes a calibrated severity score that provides a quantitative measure of autism symptomatology.^[8-10]

The ADI-R is a comprehensive parent/caregiver interview used to assess symptoms of ASD. It covers three main domains: social interaction, communication, and restricted and repetitive behaviors. The ADI-R utilizes specific cutoff scores within its domains to aid in the diagnosis of ASD.^[11]

The parent feedback questionnaire has ten questions pertaining to how much the parents feel the app ADT-1 helped their child and its user-friendliness. Parents needed to choose one option out of 1, 2, 3, 4 or 5 where 3 or more numbers indicated a favorable efficacy or usefulness of ADT-1. It has also one additional question in yes/no format, which asked about the worsening of addiction or attachment to smartphone and/or digital media after using the app ADT-1 by their child.

RESULTS

During the period between March 2023 and July 2023, a total of 70 children with ASD satisfying DSM-V criteria were enrolled after screening 97 children with ASD and only these children completed 12 weeks follow-up period on October 31, 2023, when the interim analysis was performed. Twenty-seven children with ASD were excluded (25 children older than six years of age and parents of two children expressed their inability to comply with the RCT protocol). A total of 37 children were recruited into the intervention arm and 33 into the control arm [Figure 1]. The RCT is still recruiting participants, but as prespecified in the study protocol, we performed the interim analysis, once the follow-up of 12 weeks was completed for the first 70 randomized patients.

Close

Figure 1:

Consort diagram of the study. n: Number of children

Export to PPT

The baseline sociodemographic and clinical variables were comparable between the two groups [Table 1]. In both groups, the majority of the participants were boys, belonged to middle or higher socioeconomic status, and were placed in functional level 2, according to DSM-V criteria. Hyperactivity, sleep disturbance, and aggressive behavior were the most predominant comorbidities in the participants in both groups, but their distribution was comparable between both groups. A total of nine and six participants were receiving oral melatonin for sleep disturbance in intervention and control groups, respectively (P = 0.53). Similarly, a total of 13 and 3 patients were receiving oral risperidone and aripiprazole in the intervention group, respectively, while a total of 12 and 1 participants were receiving risperidone and aripiprazole in the control group, respectively (P = 0.91 and 0.36, respectively). Only one participant in each group was receiving anti-seizure medication (ASM) during the study period (P = 0.93). Valproate was the ASM that was used in these two participants and none of them required polytherapy with ASM. None of the participants had breakthrough seizures during the study period. During the study period, the dose of any of the above medications was not changed in any of the participants.

All of the subjects were diagnosed with “Autism” according to ADOS-2 and ADI-R. At baseline, the score in the “Social affect” domain, “repetitive ritualistic behavior domain” total ADOS-2 score, and ADOS-2 comparison score was 8.43 ± 1.29, 4.31 ± 0.76, 12.73 ± 1.48, and 6.56 ± 1.06, respectively, in the intervention group. Similarly at baseline, the score in the “Social affect” domain, “repetitive ritualistic behavior domain” total ADOS-2 score, and ADOS-2 comparison score was 8.37 ± 1.41, 4.33 ± 0.72, 12.71 ± 1.62, and 6.45 ± 0.75, respectively, in the control group. At 12 weeks, as compared to baseline, change in the “social affect” domain, “repetitive ritualistic behavior domain,” total ADOS-2 score and ADOS-2 comparison score was better in the intervention group as compared to the control group (P < 0.001 for all).

At baseline, the level of ASD-related symptoms associated with ADOS-2 comparison score was high and moderate in 11 and 26 participants, respectively, in the intervention group and 7 and 26 participants in the control group, respectively (P = 0.41). At the 12-week follow-up, the level of ASD-related symptoms associated with ADOS-2 comparison score was moderate in all 37 participants in the intervention group, whereas in the control group, the severity was high and moderate in 4 and 29 participants, respectively (P = 0.04). While 11 participants in the intervention group shifted from high to moderate level at 12 weeks as compared to baseline, only three participants in the control group shifted from high to moderate level at 12 weeks as compared to baseline. The change in the severity of ASD-related symptoms in both groups at 12 weeks as compared to baseline was statistically significant (P < 0.001) suggesting the standard care treatment including ABA and SIT was effective in reducing the symptom severity of autism. However, in the intervention group with add-on IMPUTE ADT-1, the change in the severity of symptoms was more significant [Table 2].

No additional participant showed new-onset sleep disturbance in any of the groups. Moreover, five and three participants, who originally had sleep disturbance according to the BEARS questionnaire, showed improvement with sleep hygiene and oral melatonin advised to them at 12 weeks in the intervention and control group, respectively.

None of the caregivers were of the opinion that their child showed worsening of addiction or attachment to smartphone and/or digital media at 12 weeks as compared to baseline.

A total of 28 (75%) caregivers were willing to participate in future clinical trials again with IMPUTE ADT-1. A total of 29 (78%) caregivers provided an overall rating of 3 or more for the ADT-1 app. A total of 27 (73%) caregivers opined that their child benefited from the ADT-1 app. A total of 30 (81%) caregivers believed that the ADT-1 app improved their child’s verbal skills, and 28 (75%) caregivers believed that the ADT-1 app improved their child’s social skills. A total of 29 (78%) caregivers thought that the ADT-1app helped reduce your child’s repetitive behavior [Table 3]. None of the children showed or the caregiver reported any vision problems.

DISCUSSION

The present study showed that IMPUTE ADT-1 has the efficacy to further augment the benefit obtained from home-based ABA in developing countries. This application does not increase digital media/smartphone addiction and parents also feel that this is user friendly and therapeutically beneficial for their child.

Novack et al.^[12] conducted a study examining the effectiveness of a mobile application called Camp Discovery, which is based on ABA and aimed at teaching receptive language skills to children with ASD. The intervention group exhibited a more substantial improvement in receptive language skills. However, it is worth noting that standard psychometric scales were not employed to assess efficacy outcomes, and only approximately half of the enrolled children completed the RCT. This raises questions about the feasibility of this mobile application. In addition, the sample size of the RCT was deemed too small to draw any meaningful conclusions.

Rehman et al.^[13] investigated to identify features of highly-rated mobile applications (apps) designed to assist individuals with ASD, particularly those incorporating AI technologies. They meticulously screened up to 250 mobile applications ultimately identifying only 25 that were pertinent to children with ASD. These applications were centered on eye tracking, facial expression analysis, utilization of 3D cartoons, haptic feedback, engaging interfaces, text-to-speech functionality, implementation of ABA therapy, and augmentative and alternative communication techniques. The researchers recommended that future mobile applications emphasize progress tracking, personalized content delivery, automated reasoning, image recognition, and natural language processing. In the development of our mobile application, we made deliberate efforts to address these considerations and rectify the deficiencies observed in the previous iterations.

Martín^[14] also conducted an assessment of all existing mobile applications for ASD from a Spanish perspective. Their findings indicated that the majority of the applications studied primarily targeted communication, social behavior, and learning – an alignment with the trends observed in the digital market. However, a significant portion of these offerings lacked scientific validation. Moreover, research and development efforts in this domain, particularly from a Spanish-speaking approach, were found to be limited. Overall, the studies yielded positive results in terms of learning and the sustained adoption of behaviors and skills facilitated by these new technologies. The researchers identified a need for additional research and the further development of applications catering to leisure activities, resources for parents and professionals, and support for the specific needs of adults with autism.

Both of these comprehensive reviews highlighted a notable absence of formal evaluations through large-scale RCT with sufficient clinical follow-up durations for mobile applications targeting ASD.^[13,14] Hussain et al.,^[15] in another review, also analyzed 26 mobile applications for the treatment of autism and found that none of them have been explored properly in a research setting, like high-quality RCT. Furthermore, the studies that were conducted often utilized less specific and less validated measures to assess efficacy. These critical insights were taken into account during the development of our mobile application, and our RCT represents a significant advancement in this regard. We proposed a sample size of 114 participants in each arm, and we have already completed the follow-up for two-thirds of the enrolled participants.

There is no conclusive scientific evidence to suggest a direct link between smartphone use and the risk or severity of autism.^[16,17] Autism is a complex neurodevelopmental condition with multifaceted genetic and environmental factors at play.^[18] While some studies have explored potential associations between screen time and neurodevelopmental disorders including autism, results have been inconclusive and often subject to methodological limitations.^[19,20]

Hence, we decided to develop and validate this mobile application, though many clinicians advise complete avoidance of smartphone use in preschool children with autism. At the same time, we also advised parents and caregivers to approach screen time for children with balance and moderation and limit it to 60 min/day and also to ensure age-appropriate content and proper supervision.

Our RCT is not yet completed, and we have decided not to prematurely terminate the RCT, as already 114 participants (complete sample size) have been randomized, and although the intervention group showed better improvement in the severity of autism symptoms, we wanted to determine whether that benefit still remains statistically significant after completing follow-up of all patients enrolled.

There are some limitations of our RCT. The native language of most participants was Hindi and the written/auditory instruction in mobile application was in English. Although the application used simple English words, which can be understood by young children, still supervision and support by parents required for some participants to carry out the tasks. No sham mobile application was administered to the control group, which could have further ensured the blinding of intervention and validity of the results. The number of participants, who have completed the randomization period, is small, but it is the interim analysis of our RCT, and the final result will include all participants in both groups. Western children typically grow up as native English speakers whereas Indian children are not consistently exposed to English from a very young age. Disparities in access to smartphones may also exist between Indian and Western children. In addition, variations in the cultural upbringing of Indian and Western children can impact the acceptance of the mobile application by the former.

The ADT-1 application, driven by AI-based technology, represents a beacon of hope for individuals with ASD and their families. By addressing the scarcity of therapists and long waiting times, this groundbreaking tool has the potential to revolutionize the landscape of ASD intervention. Furthermore, the application is supposed to address the prohibitive cost of autism treatment and the severe shortage of trained therapists in developing countries. However, this RCT only explored the efficacy of ADT-1 application, as adjunctive future RCTs are needed to determine whether IMPUTE ADT-1 can replace behavioral therapy by trained psychologists.

CONCLUSION

The IMPUTE ADT-1 mobile application has the efficacy to improve the severity of autism symptoms in children. Parents of these children also feel that the application is beneficial for improving the socialization and verbal communication of their children.