Translate this page into:
Evaluation of learning management system-based diploma in primary care psychiatry program: Usefulness, satisfaction, and feedback from participants
, Priyanka Singhal3, Chidatma Narayana Arampady3, Radhakrishnan Govindan4, Narayana Manjunatha3,, Channaveerachari Naveenkumar3, Suresh Bada Math3
*Corresponding author: Narayana Manjunatha, Department of Psychiatry, National Institute of Mental Health and Neurosciences, Bengaluru, Karnataka, India. manjunatha.adc@gmail.com
-
Received: ,
Accepted: ,
How to cite this article: Balamurugan G, Jyothsna PM, Singhal P, Arampady CN, Govindan R, Manjunatha N, et al. Evaluation of learning management system-based diploma in primary care psychiatry program: Usefulness, satisfaction, and feedback from participants. J Neurosci Rural Pract. doi: 10.25259/JNRP_417_2024
Abstract
In India, about 80% of individuals with mental health disorders do not receive adequate treatment. To reduce the treatment gap, the National Institute of Mental Health and NeuroSciences developed various training programs for primary care doctors (PCDs), including the diploma in primary care psychiatry (DPCP) online program. The current DPCP program uses multiple technology platforms and has more manual work. Hence, the entire training process is planned to integrate into one learning management system (LMS). The objective of this study is to evaluate the usefulness, satisfaction, and feedback from participants of the LMS-based DPCP program. The study involved 12 PCDs from Bihar, India, who completed the LMS-based DPCP program. Feedback was collected using a self-report pro forma that assessed six domains related to LMS experience. The results indicated high satisfaction levels and ease of use among participants; most of them agreed that the LMS was user-friendly and also very satisfied with the LMS. However, one-fourth of participants reported taking time to familiarize themselves with the system, and half of them found LMS less helpful for formative assessments. The LMS-based DPCP course provides effective primary care psychiatric training, with participants reporting high satisfaction levels. However, there is a scope for improvement in formative assessments.
Keywords
Diploma in primary care psychiatry
Feedback
Learning management system
Primary care psychiatry
Training
INTRODUCTION
The National Mental Health Survey conducted in 2016 estimated that 10.6% of the population suffers from psychiatric disorders in India. The treatment gap for most psychiatric disorders was estimated to be around 80%, which is mainly due to the inadequate and inequitable distribution of mental health professionals.[1] Several novel initiatives were started in India to integrate psychiatric services within primary care settings to bridge this treatment gap. The National Institute of Mental Health and Neurosciences (NIMHANS), India, has been conducting short-term classroom training programs for primary care doctors (PCDs) at primary health centers since 1982. However, after being implemented, all these strategies have resulted in limited success.[2] Hence, digital technologies shall be harnessed to scale up the training of PCDs. Telemental health programs promise that there will be an incremental improvement in the number of health professionals who can provide primary psychiatry care to those in remote rural areas that were hard to reach earlier. Ultimately, this has the potential to decrease the growing treatment gap.
Diploma in Primary Care Psychiatry (DPCP) program
Hence, NIMHANS has developed an innovative DPCP program that utilizes digital platforms and distance education methods. This part-time, modular-based program focuses on imparting skills to PCDs to deliver initial psychiatric treatments.[3] Both the trainee PCDs and tele-psychiatrists who serve as trainers find technology-driven mental health training to be highly acceptable and feasible.[4] Several literatures explain the curriculum of this program in detail.[3] By considering the positive impact of this program,[4] the Uttarakhand state government in India recognized the medical officers with the DPCP program as psychiatrists till the time they served the government.[5]
In this training program, Zoom links have been created for virtual classroom (VCR), tele on-consultation training (OCT), and collaborative video consultation (CVC) sessions and shared on WhatsApp. Further, the attendance, tele OCT case records, and details were manually documented and maintained. Thus, the current training program has multiple technology platforms and manual work. Hence, the entire training process is planned to integrate into one platform, a learning management system (LMS).
LMS-based DPCP program
Selection of platform
There are two types of LMS, i.e., (i) cloud and (ii) installed LMS. The main difference between these two types is that the cloud LMS is hosted on a vendor’s server, whereas the installed LMS runs on the servers within the organization. Based on the available budget and technical support, it is decided to use the MoodleCloud platform.
Course customization
The curriculum module
The clinical schedules for primary care psychiatry (CSP) 2.4 is uploaded as a portable document format (PDF) in the curriculum module, and a YouTube link to a video presentation on CSP[6] is added to the module description.
Clinical module
(i) Onsite training: A blended learning method is adapted for the onsite training. The trainers from NIMHANS prepared eight training videos on the following topics: Depression, generalized anxiety disorder (GAD), Panic Disorder, Somatization disorder, Transdiagnostic management of common mental disorder (CMD), Psychotic disorder, Child and adolescent mental health, and Addiction. The “Page” tool is used in the LMS platform to add the pre-training video links. The “attendance” tool marks the PCDs’ onsite training attendance. (ii) VCR: The Big Blue Button video conferencing tool is used to conduct the weekly online virtual classes. The “attendance” tool marks the PCDs’ VCR attendance.
(iii) Point-of-care training modules: The tele OCT and CVC were conducted using the Big Blue Button video conferencing tool. “Feedback” tool used to collect participants’ feedback on each tele OCT.
Public health module
The PCDs designed a public education material and uploaded it to the LMS using the “Assignment tool.” Similarly, they uploaded the recorded videos of the public health talk.
Evaluation (quality control)
The formative assessment criteria were tracked through the “attendance” and “Checklist” tools. The on-camera exit examination has two parts, i.e., theory and clinical. The “quiz” tool is used in the theory examination to conduct multiple-choice questions. After completing the short essays in a white paper, the PCDs scanned the answer paper and uploaded it through the “assignment tool.” The “Big Blue Button” tool is used for clinical examinations.
Course delivery
This LMS-based primary care psychiatry program (PCPP) was delivered to 12 PCDs working in different taluks in Bihar state, India, from July 2023 to June 2024. The training team conducted the onsite training programs from July 3rd to 4th, 2023 at All India Institute of Medical Sciences, Patna, Bihar. The PCDs underwent the modules mentioned above from then to July 2024. The summative assessments were conducted in the past week of July 2024.
Aim of this paper
This paper aims to assess the uses and expectations of PCDs of the LMS-based DPCP program.
METHODS
Participants
The State Health Society of Bihar is in collaboration with NIMHANS to train their PCD in the DPCP program. Participants were approached through the state health department, which nominated PCDs to enroll in the program. All PCDs who underwent this LMS-based DPCP program were included.
Tools
The authors developed a self-report feedback pro forma based on reviews.[4-7] This pro forma has 20 items divided into six domains, i.e., Perceived ease of use, Perceived usefulness, Perceived satisfaction of LMS, LMS task-technology fit, Attitudes toward using LMS, and Perceived system usability of LMS. The participants were requested to share their responses on a five-point scale, from strongly agree to strongly disagree. Further, the usefulness of the tools used in LMS was rated on a five-point scale ranging from very helpful (5) to not at all helpful (1).
Procedure
The feedback pro forma was distributed online from 3rd to 5th July 2024 through Google Forms in email/WhatsApp. At the beginning of the survey, the purpose of the study was briefly explained. The data were analyzed using the International Business Machine Corporation IBM Statistical Package for the Social Sciences.
RESULTS
A total of 12 PCDS enrolled for this course. Out of these, nine PCDs have been working in primary care settings, and three PCDs have been in prison hospitals. The PCDs’ age ranges from 31.4 to 50.9 years, with a mean age of 41.2 years and a standard deviation of 5.428 years. Except for one prison PCD, all other participants were male.
Table 1 shows that 75–100% of the PCDs agreed/strongly agreed that the LMS is easy to use. However, 25% of them said that they spent more time to get familiar with LMS. Regarding the usefulness of LMS, they all agreed that it is extremely useful in getting information and updates about the program. However, half of them disagreed that the LMS could have been more helpful in formative assessments. All the participants were satisfied with using LMS.
| Sl. No. | Criteria | Strongly agree | Agree | Neutral | Disagree | Strongly disagree | |||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| f | % | f | % | f | % | f | % | f | % | ||
| Perceived ease of use of LMS | |||||||||||
| 1 | It only took me a little time to get familiar with LMS. | 5 | 41.7 | 4 | 33.3 | 0 | 0.0 | 2 | 16.7 | 1 | 8.3 |
| 2 | I could get quick support whenever I encounter any problem or issue with LMS. | 9 | 75.0 | 3 | 25.0 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 |
| 3 | I could easily use LMS even during my first time. | 7 | 58.3 | 5 | 41.7 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 |
| 4 | All guidelines are designed in an understandable and convenient structure. | 9 | 75.0 | 2 | 16.7 | 1 | 8.3 | 0 | 0.0 | 0 | 0.0 |
| 5 | All functions are running smoothly. | 11 | 91.6 | 1 | 8.3 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 |
| Perceived usefulness of LMS | |||||||||||
| 6 | LMS is extremely useful in the DPCP program | 10 | 83.3 | 2 | 16.7 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 |
| 7 | LMS helps me get informed of announcements and updates. | 8 | 66.6 | 4 | 33.3 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 |
| 8 | LMS effectively assists me in taking my formative assessments. | 3 | 25.0 | 3 | 25.0 | 0 | 0.0 | 0 | 0.0 | 6 | 50.0 |
| Perceived satisfaction of LMS | |||||||||||
| 9 | I feel that using LMS increases my learning. | 11 | 91.6 | 1 | 8.3 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 |
| 10 | I feel satisfied using LMS. | 10 | 83.3 | 2 | 16.7 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 |
| LMS task-technology fit | |||||||||||
| 11 | I believe that using LMS supports my learning well in the DPCP program | 11 | 91.6 | 1 | 8.3 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 |
| 12 | I believe that using LMS helps me complete my learning tasks. | 3 | 25.0 | 3 | 25.0 | 0 | 0.0 | 0 | 0.0 | 6 | 50.0 |
| Attitudes toward using LMS | |||||||||||
| 13 | The use of LMS is good for me. | 10 | 83.3 | 2 | 16.7 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 |
| 14 | To me, using LMS is important. | 10 | 83.3 | 2 | 16.7 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 |
| 15 | With the use of LMS, my skills and knowledge will improve. | 5 | 41.7 | 2 | 16.7 | 0 | 0.0 | 0 | 0.0 | 5 | 41.7 |
| Perceived system usability of LMS | |||||||||||
| 16 | I found LMS unnecessarily complex. | 0 | 0.0 | 0 | 0.0 | 1 | 8.3 | 6 | 50.0 | 5 | 41.7 |
| 17 | I thought LMS was easy to use. | 7 | 58.3 | 5 | 41.7 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 |
| 18 | I think I need the support of a technical person to be able to use LMS. | 0 | 0.0 | 2 | 16.7 | 3 | 25.0 | 2 | 16.7 | 5 | 41.7 |
| 19 | I found the various functions of LMS well integrated. | 7 | 58.3 | 5 | 41.7 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 |
| 20 | I felt confident using LMS. | 9 | 75.0 | 3 | 25.0 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 |
Primary care doctors, DPCP: Diploma in primary care psychiatry, LMS: Learning management system, f: Frequency.
Regarding task technology fit, all PCDs agreed that the LMS supported their learning, whereas half of them felt that the LMS was not very helpful in completing their learning tasks. Most of the PCDs had a positive attitude toward LMS. Most of the PCDs were confident in using LMS; however, two-thirds required technical support to use LMS. The overall feedback score ranges from 76 to 94, with a mean of 83 ± 5.878. The mean percentage score shows that the participants had positive feedback on all domains, with the “satisfaction” domain securing the highest ratings (97.5%), followed by “ease of use” (91%). Figure 1 depicts that all the PCDs agreed that the various tools used in LMS were extremely/very helpful.
- Primary care doctor feedback on tools used in a learning management system. VCR: Virtual class room, OCT: On consultation training, CVC: Collaborative video consultation, MCQ: Multiple choice questions.
Further, the correlation analysis shows that ease of use has a moderate positive relation with system usability (r = 0.610, P = 0.035); similarly, the task technology fit is positively related to the usefulness of LMS (r = 0.584, P = 0.046).
DISCUSSION
Perceived ease of use
The study revealed that most PCDs found the LMS easy to use, with 75–100% of PCDs agreeing/strongly agreeing that the LMS was user-friendly. This ease of use is critical, as previous research has shown that perceived ease of use significantly impacts the acceptance and utilization of technology in educational settings.[7] The positive correlation between ease of use and system usability aligns with the technology acceptance model, which proposes that ease of use directly influences perceived usefulness and actual system use.[8]
Perceived usefulness and satisfaction
While the LMS was deemed highly useful for accessing program updates and announcements, some participants expressed dissatisfaction with its role in formative assessments. The high level of satisfaction shows that, despite its limitations, the LMS was well adopted, which is supported by similar findings of studies in medical education where such LMS platforms have enhanced learning experiences.[9]
Task-technology fit and attitudes toward LMS
The study showed that DPCP trainees described a mean task-technology fit that best described their experiences with the LMS as moderate, and the task-technology fit was positively related to the usefulness of the LMS technology. These results suggest that while LMS generally supported the learning, there is scope for improvement of the DPCP program-specific-task alignment with technology. As one might have expected, this view is in line with the theory on task-technology fit, which posits that the effectiveness of any technology is enhanced when its capabilities align with the tasks meant to be supported by it.[8]
System usability and the need for technical support
PCDs have pointed out the need for technical support, demonstrating the weakest link in the implementation. This again reinforces findings from existing literature that wherever implementation of e-learning platforms has been successfully carried out, reasonable amounts of technical support have been provided.[10]
IMPLICATIONS FOR FUTURE TRAINING PROGRAMS
The potential for scaling up the mental health workforce training in India is bright with successful LMS integration of the DPCP program. The delivery of high-quality training to PCDs in geographically remote areas by digital platforms may contribute to narrowing the treatment gap for mental health services, especially in rural areas. Addressing the challenges highlighted by the study, especially in integrating the formative assessment and providing technical support, will further improve the usability of LMS.
Limitations
The study’s limitations include a small and geographically specific sample, limiting the generalizability of the results. The dependence on self-reported information brings in possible biases. The lack of comparison with other training methods, limited assessment of actual learning outcomes, and potential technical challenges needed to be fully addressed.
CONCLUSION
The introduction of the LMS-based DPCP program was promising for improvement in training and capacity building among PCDs in India. Overall responses from the PCDs articulated the general efficiency and usability of the LMS platform, indicating overall satisfaction with multiple dimensions, including ease of use, usefulness, and task-technology fit. However, the general results showed areas for improvement, especially regarding the role of LMS in formative assessment and the facilitation of task completion. Despite such difficulties, the LMS has thrived in helping along the process, particularly in reaching out to remote and under-resourced areas, and has helped bridge the gap in mental health services. This is indeed an innovative approach toward better psychiatric care delivery in primary health settings and can serve as a model for similar initiatives in other regions.
Ethical approval
Institutional review board approval is not required.
Declaration of patient consent
Patient’s consent is not required, as there are no patients in this study.
Conflicts of interest
There are no conflicts of interest.
Use of artificial intelligence (AI)-assisted technology for manuscript preparation
The authors confirm that there was no use of artificial intelligence (AI)-assisted technology for assisting in the writing or editing of the manuscript and no images were manipulated using AI.
Financial support and sponsorship: This project is funded by the Bihar Health Society, Patna, Bihar.
References
- National Mental Health Survey of India 2015-16 Prevalence, pattern and outcomes. 2016. Bengaluru: National Institute of Mental Health and Neuro Sciences, NIMHANS; Available from: https://indianmhs.nimhans.ac.in/phase1/Docs/Report2.pdf [Last accessed on 2024 Apr 25]
- [Google Scholar]
- Designing and implementing an innovative digitally driven primary care psychiatry program in India. Indian J Psychiatry. 2018;60:236-44.
- [CrossRef] [PubMed] [Google Scholar]
- Clinical schedules for primary care psychiatry: Version 2.4. 2023. Bengaluru: Department of Psychiatry, National Institute of Mental Health and Neurosciences; Available from: https://www.researchgate.net/publication/373805918_Clinical_Schedules_for_Primary_Care_Psychiatry_Version_24_September_2023
- [Google Scholar]
- Telepsychiatry-based on-consultation training: Acceptability and feasibility from primary care doctors and psychiatrists. J Psychiatry Spectrum. 2022;1:94.
- [CrossRef] [Google Scholar]
- The Uttarakhand mental health care (State Mental Health Authority) rules, 2023. 2023. Dehradun: Government of Uttarakhand; Available from: https://smha.uk.gov.in/department102/library_file/file-19-09-2023-07-31-30.pdf
- [Google Scholar]
- Clinical schedules for primary care psychiatry (CSP Ver 2.2) Available from: https://www.youtube.com/watch?v=ihzxi1dpOTc [Last accessed on 2024 Nov 05]
- [Google Scholar]
- Beliefs and expectations on digital learning and examinations among adult learners of a tertiary level health-care institution. J Educ Health Promot. 2022;11:349.
- [CrossRef] [PubMed] [Google Scholar]
- A Theoretical extension of the technology acceptance model: Four Longitudinal field studies. 2000 Available from: https://papers.ssrn.com/abstract=4062393 [Last accessed on 2024 Aug 14]
- [CrossRef] [Google Scholar]
- An exploration of student access to a learning management system-challenges and recommendations for educators and researchers. Eur J Dent Educ. 2021;25:846-55.
- [CrossRef] [PubMed] [Google Scholar]
- Online learning: Indian nurses' perception and a feasibility study. Indian J Psychiatric Nurs. 2021;18:139-43.
- [CrossRef] [Google Scholar]
