Effectiveness of blended versus fully digital training in primary care psychiatry: A retrospective comparison from India
How to cite this article: Gajera GV, Pandey P, Malathesh BC, Nirisha PL, Suchandra KH, Ibrahim FA, et al. Effectiveness of blended versus fully digital training in primary care psychiatry: A retrospective comparison from India. J Neurosci Rural Pract 2023;14:91-7.
We compared effectiveness of blended mode (consisting of traditional classroom teaching and e-learning sessions) and fully digital mode (e-learning sessions alone) of primary care psychiatry training for primary care doctors in Chhattisgarh.
Materials and Methods:
We retrospectively compared the engagement in training, knowledge (K), attitude (A), and practice (P) in primary care psychiatry, and patient identification by primary care doctors (n = 941) from Chhattisgarh region who underwent training through either blended training mode (n = 546) or fully digital training mode (n = 395) for 16 h each, using Clinical Schedules for Primary Care Psychiatry based modules between June 2019 and November 2020 with a tertiary care center (NIMHANS, Bengaluru) as hub.
The Statistical Package for the Social Sciences version 27 was used to analyze the data. Continuous variables were analyzed using the independent samples t-test, and discrete variables were analyzed using the Chi-square test. A repeated measures analysis of variance (ANOVA) (two-way mixed design) was used to see the interaction of training type and time of pre- and post-KAP measurement while controlling for years of experience. The number of patients identified by both training groups over 8 months was also compared using the repeated measures ANOVA (two-way mixed design).
Engagement inferred by the number of participants completing pre-KAP forms (75%), post-KAP forms (43%), post-session assessments (37– 47%), case presentation (33.9%), and certification (32.1%) was better in the blended group (P < 0.05). The mean gain in KAP scores was significantly higher in the blended group controlling for the years of experience as primary care doctor (PCD) (F = 30.36, P < 0.001). PCDs in the blended training group consistently identified a higher number of patients with mental illness over 8 months of follow-up (F = 6.21, P < 0.001).
The blended mode yielded better results in primary care psychiatry training compared to fully digital mode. In-person interactions while provided for a very brief proportion of the training seem to have an unmistakable imprint on the outcomes and seem critical for better consolidation and assimilation of information, which translates into better practice.
Community mental health services
Primary care psychiatry
Methods of training primary care doctors (PCDs) have evolved over the years.[1-3] Technology-assisted teaching methods are gradually replacing traditional classroom teaching. These digital training methods come with lowered costs, increased accessibility, and ability to transcend the boundaries of time and space. They offer both the trainer and the learner convenience, adapt to individual needs, and offer self-paced learning. At the same time, the digital training methods are changing the role of trainers from content distributors to that of learning facilitators. Technology-assisted training broadly employs two learning methods:
Fully digital training or e-learning, where learners are connected exclusively using digital devices to access synchronous or asynchronous online learning packages through the internet.
Blended training or hybrid learning, which combines e-learning with traditional in-person training. The classroom training, in this case, is supplemented by synchronous or asynchronous e-learning.[1,6-8] Synchronous learning involves live didactic or case discussions where the teacher and the participants join the class using video conference applications. Asynchronous learning involves providing online learning materials (teaching video, PowerPoint presentation, and academic articles) that the participant can utilize during training.
Technology-assisted teaching methods in health professionals (digital training) are slightly better or as effective as traditional teaching methods in terms of knowledge, satisfaction, and skills gained.[2,3,9,10] The literature comparing blended learning with fully digital training is sparse. Most studies compare blended learning with traditional learning across diverse participant groups focusing on subjective training experience, knowledge, and skill acquisition.
The Chhattisgarh Community Mental Healthcare TeleMentoring Program (CHaMP) is a capacity-building program running since June 2019, through the collaboration of the state government of Chhattisgarh and the National Institute of Mental Health and Neurosciences (NIMHANS), Bengaluru, the apex mental health institution in India. Originally a blended training program, the CHaMP initiative aims to train PCDs from across the state of Chhattisgarh in identifying and treating persons with psychiatric disorders. The COVID-19 pandemic pushed us to favor fully digital training as classroom teaching came with the risk of exposure and infection. Following the nationwide lockdown in India and resulting travel restrictions, we had to shift to fully digital training from April 2020.
This made us question whether fully digital training of PCDs can be as effective or better when compared to the blended mode of training? To answer this, we retrospectively compared the above two methods of training in terms of engagement with the training program (inferred by attendance in the e-learning sessions and assignment completion), gains in knowledge, attitude, and practice related to primary care psychiatry and the number of patient consultations reported by the PCDs over 8 months after the onset of training.
MATERIALS AND METHODS
Study design and setting
This was a retrospective study with a before-after design. Between June 2019 and March 2020, a blended learning mode was used to train the PCDs. This involved traditional face to face training and internet-based learning (both synchronous and asynchronous). From April 2020 to November 2020, a fully digital training mode was employed. The ethical approval was given by the Institutional Ethics Committee at NIMHANS, Bengaluru, with the reference number-NIMHANS/EC (BEH.SC.DIV.) 24th MEETING/2020–2021.
Tools and assessments
Clinical schedule for primary care psychiatry (CSP)
CSP is a training manual including a concise algorithm for rapid screening, first-line management, follow-up, and referral that PCDs can easily use in their routine outpatient departments (OPDs). CSP employs cluster-based diagnosis for six mental disorders (depression, anxiety, somatization disorder, psychosis, tobacco, and alcohol addiction).
Monthly report pro forma
A semi-structured tabular pro forma was made for the PCDs to record the numbers of the patients identified every month as per CSP.
Pre-KAP and Post-KAP forms
The knowledge, attitude, and practice (KAP) forms were developed by a team of experienced psychiatrists from NIMHANS, collaborating with the state nodal representative from Chhattisgarh to assess pre-training and post-training knowledge, attitude, and practices of primary care psychiatry of the participating PCDs. The self-rated KAP questionnaire has 30 questions based on knowledge regarding mental illness diagnosis and treatment (19 MCQs and 11 case-based short answer questions), 16 multiple-choice questions (0–4-point Likert scale) assessing the practice of PCDs, and 27 questions asking to rate the participants’ confidence for various psychiatric diagnoses and management on a 1–10 Likert scale. The total score for the KAP questionnaire was 364.
Post-session assessment (PSA)
Following each e-learning and skill development session, the participants filled a brief Google Forms-based assessment for the particular topic discussed in the session consisting of objective and short answer questions.
All of these online assessments were reviewed, and final scores were calculated by an experienced psychiatrist who was the active trainer in the CHaMP program.
Blended training mode
The original blended training program consisted of three modules:
Basic onsite training module
The 2-day basic onsite module formed the in-person component of the blended training. A psychiatrist from NIMHANS, Bengaluru, visited respective districts in Chhattisgarh and oriented the participants regarding the CHaMP project, use of digital technology (Zoom, Google Forms, and WhatsApp), maintenance of records using monthly report pro forma. Participants completed a Google Forms-based knowledge, attitude, and practice questionnaire (Pre-KAP Google Forms) on the same day. The onsite training engaged the participants for about 6 h every day with a total of 4 h spent on primary care psychiatry orientation using the CSP.
Six e-learning and skill development (e-LSD) sessions
The e-LSD sessions in the subsequent 6 weeks follow a peer learning approach. Each 2 h e-LSD session (one session per week) consisted of 30 min didactic presentation on one of the six mental illnesses discussed in the CSP by a psychiatrist, followed by 90 min of the case-based discussion presented by one of the participants. In each session, PCDs presented live cases through video conferencing (written informed consent was taken, and patients were deidentified), which are discussed with participants and moderated by the psychiatrist. At the end of each session, participants submitted a Google Forms-based PSA.
Collaborative video consultations (CVC)
The CVCs facilitated PCDs to screen, diagnose, treat, and, where required, refer patients with psychiatric illnesses in the primary care setting using individualized guidance and support from a psychiatrist at NIMHANS using video conferencing. One CVC case discussion during the training period was a mandatory requirement for the completion of training.
At the end of the 6 weeks of online e-LSD sessions, participants were made to fill the Google Forms-based knowledge, attitude, and practice questionnaire again (post-KAP Google Forms).
Fully digital mode
Due to COVID-related travel restrictions, the in-person component of the basic onsite module was omitted. We were restricted to “fully digital” mode and to achieve the same 16 h of training as in previously employed blended mode (4 h orientation as part of onsite training plus 12 h of e-LSD sessions), the onsite module was replaced by two additional e-learning sessions (2 h each) covering the psychiatric history taking, case record, reporting, and introduction to CSP. Organization of WhatsApp groups, registration, and completion of pre-training KAP assessments was done apart from the e-learning sessions, which accounted for the additional time spent during onsite sessions. The second and third modules remained unchanged.
Nine hundred and forty-one PCDs already enrolled in the CHaMP initiative formed the study sample. The participating PCDs in Chhattisgarh consisted of Medical officers (MOs, i.e., MBBS doctors), AYUSH Medical Officers (AMOs, i.e., BAMS doctors), and Rural Medical Assistants (RMAs). The RMAs are a special cadre of rural health-care practitioners in Chhattisgarh who have completed a 3-year Practitioner in Modern and Holistic Medicine course. The RMAs work under the supervision of medical officers.
Consent for the use of participant data for research was obtained as part of enrolment into the program. Over the follow-up period, not all the participants continued to engage with the training and 294 PCDs filled both pre- and post-KAP forms and these data alone were available for pre- and post-KAP score comparison.
Primary outcome measures
The primary outcome measures were:
PCDs’ engagement in either mode of training. The New Kirk Patrick Model of training assessment defines engagement as the degree to which participants are involved in the training and contributing to the learning experience. We inferred engagement from the number of e-LSD sessions attended, number of participants who filled Google Forms-based pre- and post-training KAP and PSA, number of CVC presentations, and completing all requirements for certification.
Change in pre- and post-training KAP scores was chosen to investigate knowledge, attitude, and practice related to primary care psychiatry. The total KAP scores range from 27 to 364.
The number of patients reported by PCDs every month in both training arms over 8 months from the start of the training (as we had a minimum of 8 months of monthly reports from all comparable participants). This outcome measure was chosen to reflect the change in behavior of the PCDs in terms of the detection of mentally ill patients.
The training material, Google Forms, and monthly report formats are available in the appendices.
The Statistical Package for the Social Sciences version 27 (license in the corresponding author’s name) was used to analyze the data. The data were normal across the pre- and post-KAP scores and years of experience and non-normal for the number of sessions attended, cases presented, certified participant numbers, and the number of patients reported per month for both training groups. Continuous variables were analyzed using the independent samples t-test, and discrete variables were analyzed using the Chi-square test. A repeated measures analysis of variance (ANOVA) (two-way mixed design) was used to see the interaction of training type and time of pre- and post-KAP measurement while controlling for years of experience. The number of patients identified by both training groups over 8 months was also compared using the repeated measures ANOVA (two-way mixed design).
From June 2019 to March 2020, 546 PCDs attended training in blended mode and from April 2020 to November 2020, 395 PCDs attended training in fully digital mode.
[Table 1] summarizes the parameters used to infer engagement of the PCDs during the training. A significantly higher number of PCDs in the blended mode of training completed pre- and post-KAP forms and PSAs discussed CVCs more often and were certified and presented a greater number of cases. Post hoc analysis (z-test) for comparing column proportions showed that a higher proportion of PCDs in the fully digital mode did not present any CVC at all, while a higher proportion of blended training PCDs presented at least two CVCs.
|S. No.||Activities done||Blended, n=546||Fully digital, n=395||χ2||P-value|
|2||Attended zero e-LSD sessions||152||27.9||55||13.9||31.40||<0.001|
|Attended one e-LSD session||65||11.9||48||12.2|
|Attended two e-LSD sessions||44||8.1||38||9.6|
|Attended three e-LSD sessions||33||6.1||43||10.9|
|Attended four e-LSD sessions||44||8.1||45||11.4|
|Attended five e-LSD sessions||99||18.2||75||19|
|Attended six e-LSD sessions||108||19.8||91||23|
|3||Filled first PSA||254||47||214||54||5.37||0.02|
|4||Filled second PSA||245||45||189||48||0.82||0.366|
|5||Filled third PSA||231||42||190||48||3.12||0.07|
|6||Filled fourth PSA||239||44||135||34||8.81||0.002|
|7||Filled fifth PSA||236||43||139||35||6.17||0.013|
|8||Filled sixth PSA||203||37||90||23||22.15||<0.001|
|10||Attempted CVC discussion||185||33.9||92||23.3||12.38||<0.001|
|12||Discussed no CVCs||361||68.8||303||77.09||9.56||0.008|
|Discussed one CVC||133||25.3||79||20.10|
|Discussed two CVCs||31||5.9||11||2.80|
In contrast, PCDs in fully digital training attended a higher number of e-LSD sessions. On post hoc analysis (z-test) for comparing column proportions, the proportion of participants attending no online sessions was significantly higher in the blended group, while the proportion of participants attending three e-LSD sessions was significantly higher in the fully digital training group. [Table 2] shows the comparison of pre- and post-KAP scores in blended and fully digital groups. While both blended and fully digital training PCDs scored higher in individual knowledge, attitude, and practice, as well as total post-KAP assessments, the gains in KAP scores were significantly higher in the blended group.
|S. No.||Variables||Blended training||Fully digital training||F||P-value|
|Pre-test estimated marginal mean||Post-test estimated marginal mean||Pre-test estimated marginal mean||Post-test estimated marginal mean|
|4||KAP – total||135.1||281.5||161.5||264.7||30.36||0.0001|
Three hundred and seventy-four PCDs out of the 941 PCDs enrolled identified one or more cases in the defined time frame. A significantly higher number of participants in blended training (49.3%) identified one or more cases compared to PCDs in fully digital training (26.6%) (Chi-square = 49.25, P < 0.001). The median number of patients identified by PCDs in blended training (median = 30, IQR = 63) was also significantly higher than the median number of patients identified by PCDs in fully digital training (median = 6, IQR = 12) (Mann–Whitney U = 20810, P < 0.001).
[Figure 1] shows the comparison of the number of patients reported over the 8 months from the start of training. PCDs in blended training, on average, identified a significantly higher number of patients throughout the follow-up. Repeated measures ANOVA for time and type of training effect revealed significant two-way interaction between type of training and time for the number of patients reported every month (Greenhouse–Geisser test for within-subjects effect: F = 6.21, P < 0.001).
Furthermore, over the 8 months following the start of treatment, PCDs in blended training on average identified a significantly higher number of patients with a common mental disorder (CMD) and substance use disorder (SUD) every month. RM ANOVA for time and type of training effect revealed significant two-way interaction between type of training and time for both the number of CMD patients (F = 3.381, P = 0.006) and the number of SUD patients (F = 4.66, P < 0.001). PCDs in blended training reported a higher mean number of patients with severe mental disorders. However, the RM ANOVA did not reveal significant time and type of training interaction for the identification of this group of patients [Figures 2 and 3].
In our study, the blended mode of learning (with a component of in-person face to face interaction) was found to be a far better mode of training PCDs when compared to the fully digital mode across all levels of training evaluation. This superiority was noted in overall engagement throughout the training course, scores related to knowledge, attitude, and practice, and the progressive increase in the number of patients seen and treated. This superiority was maintained regarding identifying (and treating) CMDs and SUDs.
Individual learning styles, pre-existing familiarity with digital technology. and accessibility to uninterrupted internet connectivity or lack thereof may have contributed to these differences in outcomes. The previous studies comparing digital training modes have suggested limited direct interaction with trainers and peers, greater need for self-discipline, and lack of desire to compete with peers in fully digital mode as potential factors.[15-17] In our study as well, we observed that trainees in the blended mode were more interactive and motivated to discuss the cases during e-LSD sessions as compared to the fully digital mode. The basic onsite training component established high-quality participant-trainer interaction, which may have played a significant role. One of the factors could be its potential to contain the essence of classic training and the added benefit of e-learning.
From the various focused group discussions held at regular intervals with participants during the program, we received inputs regarding subjective experiences of PCDs regarding the training. Many of the PCDs expressed that they felt more comfortable in traditional classroom programs due to preexisting familiarity. Some suggested that continuing e-LSD sessions along with regular OPDs posed a challenge. While PCDs in both training groups would get directives from the state for attending the sessions, PCDs attending onsite sessions as part of the blended mode would be relieved of their clinical duties for 2 days. This ensured a higher number of participants attending the orientation sessions and having a basic overall understanding of the training. Whereas in the fully digital mode, the participants carried out the entire training based on their availability and interest, which may explain relatively better retention of fully digital mode trained PCDs in online sessions.
However, the findings do not mean that fully digital mode is not efficient. Indeed, it is efficient on its own, going by the significant gains in all studied parameters. Leveraging technology in mental health capacity is a game-changer and has the potential to transcend many traditional barriers. Many such initiatives have seen the light of the day across the country and need to be continued and scaled up across the country.[7,19-23]
Across the training modes, continuous handholding and mentoring by a specialist in the form of CVCs may have contributed to the progressive increase in patient detection and gains in mental health-related knowledge, attitude, and practice. This is another component that was missing in mental health capacity building.
The Lancet Global Mental Health Group suggested good quality mental health training of PCDs as a vital strategy to scale up mental health services. The blended, as well as fully digital training in primary care psychiatry, held their ground in translating knowledge/skills to patient identification and need more comprehensive implementation across the country to cover the burgeoning mental health treatment gap.
This is the first study to compare two different digital modes of training in primary care psychiatry to the best of our knowledge. The majority of existing literature focuses on subjective experience of training, knowledge, and skill acquisition, while we also evaluated objective measures of engagement, practice, and behavior change in participants over a long follow-up period, reflecting on the sustainability of intervention effects. This was a retrospective comparison and we were unable to collect and account for the sociodemographic variables of the participating doctors. There was no control group devoid of a training intervention. The follow-up period for both the groups was equal but at different points in time and the latter half of the study could have been impacted by multiple psychosocial factors in context the COVID-19 pandemic.
Pending prospective randomized comparisons, we would like to make the following conclusions: Blended mode of teaching and learning will yield better results in primary care psychiatry. This can be easily applied and sustained over long periods. Although continued engagement is possible in both modes, meaningful in-person interactions at some point during training seem to have an unmistakable imprint on the outcomes. While provided for a very brief proportion, these in-person interactions seem critical for better consolidation and assimilation of information, which translates into better practice.
An important implication from the public health perspective is that the inclusion of the in-person training component of non-specialist health professionals will be crucial for meaningful impact. The rapid expansion of digital technology does not seem to take away the importance of the former.
Declaration of patient consent
The authors certify that they have obtained all appropriate consent.
Conflicts of interest
There are no conflicts of interest.
Financial support and sponsorship
- Champaign: Urbana-Champaign Univ Ill; 2000. p. :113.E-learning: A Review of Literature. Vol 8.
- [Google Scholar]
- Hoboken: John Wiley AND Sons; 2012.The Handbook of Blended Learning: Global Perspectives, Local Designs
- [Google Scholar]
- Publication Departament Psychiatry National Institute of Mental Health and Neurosciences (NIMHANS), Bengaluru, India.Clinical schedules for primary care psychiatry-version 2.1.
- [Google Scholar]
- Commuity Psychiatry During Pandemic Vol 182. (2nd ed). Bengaluru: NIMHANS Publication; 2020. p. :175-85.The Future of Community Psychiatry: The Role of Telemedicine. In:
- [Google Scholar]