Effectiveness of blended versus fully digital training in primary care psychiatry: A retrospective comparison from India

INTRODUCTION

Methods of training primary care doctors (PCDs) have evolved over the years.^[1-3] Technology-assisted teaching methods are gradually replacing traditional classroom teaching.^[1] These digital training methods come with lowered costs, increased accessibility, and ability to transcend the boundaries of time and space. They offer both the trainer and the learner convenience, adapt to individual needs, and offer self-paced learning.^[2] At the same time, the digital training methods are changing the role of trainers from content distributors to that of learning facilitators.^[4] Technology-assisted training broadly employs two learning methods:

1. Fully digital training or e-learning, where learners are connected exclusively using digital devices to access synchronous or asynchronous online learning packages through the internet.^[5]

2. Blended training or hybrid learning, which combines e-learning with traditional in-person training. The classroom training, in this case, is supplemented by synchronous or asynchronous e-learning.^[1,6-8] Synchronous learning involves live didactic or case discussions where the teacher and the participants join the class using video conference applications. Asynchronous learning involves providing online learning materials (teaching video, PowerPoint presentation, and academic articles) that the participant can utilize during training.

Technology-assisted teaching methods in health professionals (digital training) are slightly better or as effective as traditional teaching methods in terms of knowledge, satisfaction, and skills gained.^[2,3,9,10] The literature comparing blended learning with fully digital training is sparse. Most studies compare blended learning with traditional learning across diverse participant groups focusing on subjective training experience, knowledge, and skill acquisition.

The Chhattisgarh Community Mental Healthcare TeleMentoring Program (CHaMP) is a capacity-building program running since June 2019, through the collaboration of the state government of Chhattisgarh and the National Institute of Mental Health and Neurosciences (NIMHANS), Bengaluru, the apex mental health institution in India.^[11] Originally a blended training program, the CHaMP initiative aims to train PCDs from across the state of Chhattisgarh in identifying and treating persons with psychiatric disorders.^[12] The COVID-19 pandemic pushed us to favor fully digital training as classroom teaching came with the risk of exposure and infection. Following the nationwide lockdown in India and resulting travel restrictions, we had to shift to fully digital training from April 2020.

This made us question whether fully digital training of PCDs can be as effective or better when compared to the blended mode of training? To answer this, we retrospectively compared the above two methods of training in terms of engagement with the training program (inferred by attendance in the e-learning sessions and assignment completion), gains in knowledge, attitude, and practice related to primary care psychiatry and the number of patient consultations reported by the PCDs over 8 months after the onset of training.

MATERIALS AND METHODS

RESULTS

From June 2019 to March 2020, 546 PCDs attended training in blended mode and from April 2020 to November 2020, 395 PCDs attended training in fully digital mode.

[Table 1] summarizes the parameters used to infer engagement of the PCDs during the training. A significantly higher number of PCDs in the blended mode of training completed pre- and post-KAP forms and PSAs discussed CVCs more often and were certified and presented a greater number of cases. Post hoc analysis (z-test) for comparing column proportions showed that a higher proportion of PCDs in the fully digital mode did not present any CVC at all, while a higher proportion of blended training PCDs presented at least two CVCs.

In contrast, PCDs in fully digital training attended a higher number of e-LSD sessions. On post hoc analysis (z-test) for comparing column proportions, the proportion of participants attending no online sessions was significantly higher in the blended group, while the proportion of participants attending three e-LSD sessions was significantly higher in the fully digital training group. [Table 2] shows the comparison of pre- and post-KAP scores in blended and fully digital groups. While both blended and fully digital training PCDs scored higher in individual knowledge, attitude, and practice, as well as total post-KAP assessments, the gains in KAP scores were significantly higher in the blended group.

Three hundred and seventy-four PCDs out of the 941 PCDs enrolled identified one or more cases in the defined time frame. A significantly higher number of participants in blended training (49.3%) identified one or more cases compared to PCDs in fully digital training (26.6%) (Chi-square = 49.25, P < 0.001). The median number of patients identified by PCDs in blended training (median = 30, IQR = 63) was also significantly higher than the median number of patients identified by PCDs in fully digital training (median = 6, IQR = 12) (Mann–Whitney U = 20810, P < 0.001).

[Figure 1] shows the comparison of the number of patients reported over the 8 months from the start of training. PCDs in blended training, on average, identified a significantly higher number of patients throughout the follow-up. Repeated measures ANOVA for time and type of training effect revealed significant two-way interaction between type of training and time for the number of patients reported every month (Greenhouse–Geisser test for within-subjects effect: F = 6.21, P < 0.001).

Close

Figure 1:

Number of patients identified over 8 months by the PCDs. Repeated measures analysis of variance for time and type of training effect revealed significant two-way interaction between type of training and time for the number of patients reported every month. Greenhouse–Geisser test for within-subjects effect: F = 6.21, P < 0.001.

Export to PPT

Furthermore, over the 8 months following the start of treatment, PCDs in blended training on average identified a significantly higher number of patients with a common mental disorder (CMD) and substance use disorder (SUD) every month. RM ANOVA for time and type of training effect revealed significant two-way interaction between type of training and time for both the number of CMD patients (F = 3.381, P = 0.006) and the number of SUD patients (F = 4.66, P < 0.001). PCDs in blended training reported a higher mean number of patients with severe mental disorders. However, the RM ANOVA did not reveal significant time and type of training interaction for the identification of this group of patients [Figures 2 and 3].

Close

Figure 2:

Number of patients with common mental disorders identified over 8 months by the PCDs. Repeated measures analysis of variance for time and type of training effect revealed significant two-way interaction between type of training and time for the number of common mental disorder patients reported every month. (Greenhouse–Geisser test for within-subjects effect: F= 3.381, P = 0.006).

Export to PPT

Close

Figure 3:

Number of substance use disorder patients identified over 8 months by the PCDs. Repeated measures analysis of variance for time and type of training effect revealed significant two-way interaction between type of training and time for the number of substance use disorder patients reported every month. (Greenhouse– Geisser test for within-subjects effect: F = 4.66, P < 0.001).

Export to PPT

DISCUSSION

In our study, the blended mode of learning (with a component of in-person face to face interaction) was found to be a far better mode of training PCDs when compared to the fully digital mode across all levels of training evaluation.^[14] This superiority was noted in overall engagement throughout the training course, scores related to knowledge, attitude, and practice, and the progressive increase in the number of patients seen and treated. This superiority was maintained regarding identifying (and treating) CMDs and SUDs.

Individual learning styles, pre-existing familiarity with digital technology. and accessibility to uninterrupted internet connectivity or lack thereof may have contributed to these differences in outcomes. The previous studies comparing digital training modes have suggested limited direct interaction with trainers and peers, greater need for self-discipline, and lack of desire to compete with peers in fully digital mode as potential factors.^[15-17] In our study as well, we observed that trainees in the blended mode were more interactive and motivated to discuss the cases during e-LSD sessions as compared to the fully digital mode. The basic onsite training component established high-quality participant-trainer interaction, which may have played a significant role. One of the factors could be its potential to contain the essence of classic training and the added benefit of e-learning.^[18]

From the various focused group discussions held at regular intervals with participants during the program, we received inputs regarding subjective experiences of PCDs regarding the training. Many of the PCDs expressed that they felt more comfortable in traditional classroom programs due to preexisting familiarity. Some suggested that continuing e-LSD sessions along with regular OPDs posed a challenge. While PCDs in both training groups would get directives from the state for attending the sessions, PCDs attending onsite sessions as part of the blended mode would be relieved of their clinical duties for 2 days. This ensured a higher number of participants attending the orientation sessions and having a basic overall understanding of the training. Whereas in the fully digital mode, the participants carried out the entire training based on their availability and interest, which may explain relatively better retention of fully digital mode trained PCDs in online sessions.

However, the findings do not mean that fully digital mode is not efficient. Indeed, it is efficient on its own, going by the significant gains in all studied parameters. Leveraging technology in mental health capacity is a game-changer and has the potential to transcend many traditional barriers. Many such initiatives have seen the light of the day across the country and need to be continued and scaled up across the country.^[7,19-23]

Across the training modes, continuous handholding and mentoring by a specialist in the form of CVCs may have contributed to the progressive increase in patient detection and gains in mental health-related knowledge, attitude, and practice. This is another component that was missing in mental health capacity building.^[24]

The Lancet Global Mental Health Group suggested good quality mental health training of PCDs as a vital strategy to scale up mental health services.^[25] The blended, as well as fully digital training in primary care psychiatry, held their ground in translating knowledge/skills to patient identification and need more comprehensive implementation across the country to cover the burgeoning mental health treatment gap.^[12]

This is the first study to compare two different digital modes of training in primary care psychiatry to the best of our knowledge. The majority of existing literature focuses on subjective experience of training, knowledge, and skill acquisition, while we also evaluated objective measures of engagement, practice, and behavior change in participants over a long follow-up period, reflecting on the sustainability of intervention effects. This was a retrospective comparison and we were unable to collect and account for the sociodemographic variables of the participating doctors. There was no control group devoid of a training intervention. The follow-up period for both the groups was equal but at different points in time and the latter half of the study could have been impacted by multiple psychosocial factors in context the COVID-19 pandemic.

CONCLUSION

Pending prospective randomized comparisons, we would like to make the following conclusions: Blended mode of teaching and learning will yield better results in primary care psychiatry. This can be easily applied and sustained over long periods. Although continued engagement is possible in both modes, meaningful in-person interactions at some point during training seem to have an unmistakable imprint on the outcomes. While provided for a very brief proportion, these in-person interactions seem critical for better consolidation and assimilation of information, which translates into better practice.

An important implication from the public health perspective is that the inclusion of the in-person training component of non-specialist health professionals will be crucial for meaningful impact. The rapid expansion of digital technology does not seem to take away the importance of the former.