Flipping pharmacoepidemiology classes in a Saudi Doctor of Pharmacy program

Purpose: To examine student perceptions towards the flipped classroom approach and its impact on their learning and their course evaluation when compared to the traditional classroom method. Methods: Five classes of the pharmacoepidemiology course were delivered using the flipped classroom approach. Student perception towards the flipped teaching method was measured using a satisfaction survey. Measuring the impact of the flipped classroom on student learning and the student course evaluation was achieved by comparing the midterm grades and the results of the standard endof-course evaluations with the previous semester's cohort. Results: Students’ perceptions of the flipped classroom were mostly favourable. The course and its various components were viewed more favourably in the second semester than in the first semester. Statistically significant improvements were observed in the perception of the topics covered in the course (p = 0.045), fairness of the grade assessment (p = 0.004), and perception of course feedback (p = 0.021). No statistical difference was noted between the midterm examination scores of the first semester cohort (24.53 ± 3.80) and the second semester cohort (25.15 ± 3.00); [t (22.54) = 0.53, p = 0.3]. Conclusion: This study demonstrates that using the flipped classroom approach for teaching pharmacoepidemiology can improve student satisfaction, as well as maintain their academic performance.


INTRODUCTION
The traditional lecture method commonly allows students to receive information passively and hinders their active participation in the learning process [1,2]. This is attributed to the limited time of the class that is typically spent covering the content of the syllabus, rather than engaging students in interactive activities [3]. critical analysis [1,3,4]. The flipped classroom is one of the teaching approaches that focuses on student self-learning and in-class interactive participation [5]. In this teaching method, instructional materials are provided to the students to use outside of class, frequently with the aid of technology. Thus, the class time is freed up for interactive learning facilitated by the instructor [6][7][8][9].
The flipped classroom approach has been implemented and evaluated in many health disciplines, including medicine, pharmacy, nursing and public health [2,4,[10][11][12][13][14][15][16][17]. In a metaanalysis that examined the effect of the flipped classroom when compared to the traditional lecture format in health professions education, significant improvement was seen in student learning in the flipped classrooms [18].
In Saudi Arabia, the number of pharmacy colleges has grown from one to 27 in less than two decades [19]. Saudi universities offer two principal degrees to enter the pharmacy profession, namely the Bachelor of Pharmaceutical Science (B.Sc. Pharm) and the Doctor of Pharmacy (PharmD) [19]. The genders in both programs are taught separately.
English is the language of instruction and assessment in all pharmacy schools in Saudi Arabia. The curriculum is predominantly delivered through traditional didactic lectures and supplemented by diverse multimedia teaching methods, such as slide shows and video clips. Student learning and performance are assessed by written and oral examinations. Other means of assessment include presentations and short reports [20].
As noted in previous research, the use of teaching methods that promote self-directed learning and active student participation is limited across pharmacy curricula in Saudi Arabia. For that reason, students tend to be dependent learners with low learning autonomy [5,21]. Therefore, new teaching methods that focus on self-directed learning need to be introduced, since they have the potential to improve interaction between students and educators [5].
To the best of our knowledge, no study has assessed student attitude and perception towards the flipped classroom in undergraduate pharmacy education in Saudi Arabia. Therefore, this study was conducted to examine student opinion towards the flipped classroom approach and its impact on their learning and student course evaluation, when compared to the traditional classroom methods.

Design
Pharmacoepidemiology is a required 2-credit hour course for level 8 PharmD students enrolled in the College of Pharmacy at King Khalid University. This course consists of ten units, which are taught over eleven weeks. The class meets two hours per week, one hour for each class. Attending scheduled classes is mandatory, and a failing grade is given to students who miss more than 25 % of the classes without a valid excuse.
With the aid of PowerPoint slides, the content of the course is typically delivered using the traditional lecture method. Student learning outcomes are assessed by a quiz (10 %), two assignments (10 %), a midterm exam (30 %), and a comprehensive final exam (50 %). Twenty percent of the midterm questions are in a short answer format, and the remaining questions are multiple choice. However, all of the final exam questions are multiple choice. Student satisfaction with different aspects of their courses is typically measured using a standard end-ofcourse evaluation survey.
In the second semester of the academic year 2018/2019, five classes of the pharmacoepidemiology course were taught using the flipped classroom teaching method. The instructor was the same assistant professor who delivered the course the previous semester using the traditional lecture format. The flipped classes were delivered in two consecutive stages, initially involving the development of online lectures and, secondly, facilitating interactive sessions with the students. The online lectures were recorded in a voice-over slide show format using Blackboard ® . The length of each lecture ranged from 20 -30 min.
As this was the students' first exposure to the flipped classroom approach, the orientation session at the beginning of the term included a briefing on this teaching method. This involved a description of this innovative method, its benefits, and tips on the best ways of achieving the intended learning outcomes. Each online lecture was posted on Blackboard ® three days before the scheduled interactive session. This was accompanied by suggested references for further reading and written instructions for preparation. Thus, the students were asked to prepare for the interactive session by either viewing the online lecture or by completing the reading from the suggested references based on the learning objectives. A reminder email for the assigned preparation was sent to students one day before the scheduled class time.
Each interactive session was fifty minutes long. The first ten minutes of each session was spent on a brief recap of the key points of the independent learning materials. In addition, students were given the chance to ask questions and clarify any concepts. The remaining forty minutes were allocated to interactive tasks such as group discussions, educational games and calculations. The lecturer took a facilitative role by describing each activity, giving instructions, monitoring student progress, and giving support and guidance. The flipped sessions were designed according to previous research recommendations for effective flipped classroom implementation [22,23].

Evaluation and assessment
Student perception towards the flipped teaching method was measured using a satisfaction survey. Measuring the impact of the flipped classroom on student learning and student course evaluation was achieved by comparing the midterm scores and the results of the standard end-of-course evaluation with the previous semester's cohort.
A questionnaire was developed to collect data from the students who attended the flipped pharmacoepidemiology class sessions. The questionnaire included the following main sections: demographic information (age, first language, GPA, learning disability), engagement with the assigned pre-class independent learning, the preferred format for the assigned learning materials, and their perception of different aspects of the flipped classroom. Some of the questionnaire items were adapted from previous literature [2,4,10,[24][25][26][27][28], while others were based on a previous qualitative research study on the flipped classroom teaching approach [22]. An electronic copy of the flipped classroom satisfaction survey was administered to the students during the last 15 min of the fifth flipped session. The students were asked to read through the information sheet and the consent form, then complete the questionnaire if they agreed to participate in the study. Completion of the satisfaction questionnaire was encouraged but not mandated.
The standard end-of-course evaluation, however, is usually made available to students on the virtual learning platform, Blackboard ® , at the end of the course. This survey uses a 5-point Likert scale (ranging from strongly disagree to strongly agree) and consists of 8 items relating to the delivery of the course, e.g. clarity of the course objectives, fairness of grade assessment, satisfaction with the didactic lectures and class activities.
In order to determine whether or not the flipped classroom approach improved student satisfaction, a comparison of end-of-course evaluations was implemented between the first and second semester cohorts of the pharmacoepidemiology course. The direct impact of the flipped classroom on student learning was measured by comparing midterm exam results for the same materials taught using different instructional methods, the traditional lecture for the first semester cohort and the flipped classroom for the second semester group.

Data analysis
Descriptive statistics were used to summarise the characteristics of the participants, students' perception of the different aspects of the flipped teaching sessions, and students' responses on the standard end-of-course evaluation. The Mann-Whitney U test was used to compare students' responses on the individual Likert items from the end-of-course evaluation for the first and second semester cohorts. An independent t test was used to compare the mean scores for the midterm exam for both cohorts. The level of significance was established at an alpha equal to 0.05. All statistical analyses were conducted using SPSS version 20.0 for Mac.

Ethical approval
The study was approved by King Khalid University Research Ethics Committee (approval no. ECM# 2019-28), conducted as per Belmont Report: Ethical Principles and Guidelines for the Protection of Human Subjects of Research [29].

RESULTS
The courses enrolled 14 and 24 students in the first and second semesters, respectively. The response rate of the flipped classroom satisfaction survey was 100 %, as all students (n = 24) completed the survey and consented to take part in the study. The demographic data of the participants are listed in Table 1.
The survey participants were asked to indicate whether or not they had completed the assigned pre-work. The percentage of students who were prepared for the active session was 41.67 % (n = 10) in the first session; 54.67 % (n = 13) in the second and fifth sessions; and 62.5 % (n = 15) in the third and fourth sessions (Figure 1).  The students were asked in the survey to specify their preferred format for the assigned learning materials. Half of the participants (n = 12) reported that they preferred both text-based and audio-visual materials. Approximately 29 % of the students (n = 7) indicated a preference for audio-visual materials. The remaining 21 % of students (n = 5) prefer text-based materials.
A Chi-square test revealed no statistical significant differences in student responses on the preferred format for the pre-class learning materials, χ² (2) = 3.52, p = 0.197. Students' perceptions of the pre-class independent learning were mostly favourable ( Table 2).
Most of the 24 students (75 %) agreed or strongly agreed that learning the key foundational content prior to coming to the class greatly enhanced their learning of the course materials in class. The majority (87.5 %) also agreed or strongly agreed that the assigned prework was of a reasonable amount. Ninety-five percent agreed or strongly agreed that they were given sufficient time to complete the assigned independent learning and sufficient guidance on effective ways to benefit from the flipped classroom approach. Most students (70.8 %) agreed or strongly agreed that completing the pre-class learning materials enhanced their confidence during the contact time.
Students' perceptions of the in-class active learning were also mostly favourable (Table 3).
Most students (75 %) agreed or strongly agreed that the interactive in-class activities greatly enhanced their learning. About eighty-three percent agreed or strongly agreed that the flipped classroom allowed for more class engagement than a traditional lecture. Slightly more than 90% agreed or strongly agreed that the flipped classroom allowed for more peer and student-instructor interaction than a traditional lecture. The majority of students (87.5 %) agreed or strongly agreed that the flipped classroom would help them develop transferrable skills. Somewhat more than half (54.2 %) agreed or strongly agreed that the scheduled class time was structured in a way that enabled them to link the pre-class learning with the class activities, so they were confident of their ability to address the topics learned in the flipped classroom when taking the exam.
Statistical analyses revealed significant improvements in the perception of the topics covered in the course (p = 0.045), fairness of grade assessment (p = 0.004), perception of course feedback (p = 0.021).    Almost 83 % of students agreed or strongly agreed that they enjoyed learning pharmacoepidemiology in the flipped classroom format (Table 4). However, only 66% agreed or strongly agreed that they preferred the flipped classroom format over the traditional lecture model.
The response rate of the end-of-course evaluation was 71.43 % (n = 10) in the first semester and 75 % (n = 18) in the second semester. The course and its various components were viewed more favourably in the second semester than in the first semester (Table 5).

DISCUSSION
This research assessed student perception towards the implementation of flipped pharmacoepidemiology classes and its impact on their learning and student course evaluation. The current study reveals that students were divided regarding their preferred format for the learning materials. This supports previous research recommendations which imply that the pre-class learning resources need to be provided to students in various formats, including both recorded lectures and text-based reading materials, in order to satisfy student preferences [22].
Based on student responses to the flipped classroom perception survey and the standard end-of-course evaluation, flipping pharmacoepidemiology classes was successful and well received by the PharmD students. Similar to previous research, the students recognized that their pre-class independent materials enhanced their learning and confidence during the contact time [14,22]. The majority of students felt that the off-loaded content was of a reasonable amount, they were given sufficient time to complete the assigned learning, and offered sufficient guidance on effective learning with the flipped classroom approach.
One of the criticisms of the flipped classroom that has been highlighted in previous research was the overwhelming workload required to prepare for the active sessions. The authors stressed that the independent learning materials need to be of reasonable length [12,16,17,26]. In the current implementation of the flipped sessions, the length of the recorded lectures was between 20-30 minutes, which students found to be reasonable. The findings from this study mirror those observed in previous work in which students appreciated the active learning that takes place in the class and the significant amounts of interaction with other students and the instructor [11,30-32]. In the flipped classroom model, completing the assigned prework is the key to engaging with and benefiting from the in-class active learning. The findings of the current research indicate that, on average, almost half of the students turned up unprepared for the active sessions. This could be the reason that only half of the survey respondents felt positive about their ability to link the pre-class learning with the in-class activities, with the remaining majority being neutral on this topic.
It should also be noted that the number of students who completed the assigned independent learning increased after the first session, as students probably realized its importance, but this number dropped when approaching midterm exams as well as assignments deadlines. Hence, students should be encouraged to complete the assigned preclass learning and reminded of its importance for engaging with and benefiting from the active class learning. The present study revealed compatible results with those observed in the literature, where students exhibited a stronger preference for the flipped classroom teaching method over the traditional lecture [2,10,11,30,32].
The standard end-of-course evaluation showed an improvement in student satisfaction with the course. In particular, an improvement was seen in student perception of the topics covered in the course, the fairness of the grade assessment and the perception of course feedback. Additionally, as demonstrated by student responses in the flipped classroom survey, the students enjoyed learning pharmacoepidemiology in the flipped classroom format.
This study did not find a significant difference in student midterm exam results between the cohorts taught using the flipped classroom approach and the traditional lecture method. Findings from previously published studies suggest that the flipped classroom teaching method might improve student examination performance [2,4,13,14,16] or it might have no negative impact on summative exam results [11,15].
Implementing the flipped classroom approach may offer some valuable benefits for the future development of undergraduate pharmacy education in Saudi Arabia. However, unlike the traditional lecture, the flipped classroom method necessitates that students use a range of transferable skills, such as independent study, time management and teamwork.
In contexts similar to the one in the current study, the curriculum may be heavily dependent on the traditional lecture method or students may have poor skills and competence in mastering the flipped classroom approach. Thus, careful preparation needs to be made when implementing the flipped classroom. First, the instructor might focus on supporting students in building the skills that are essential for learning in the flipped classroom approach. Second, the complexity of the flipped classroom design needs to be appropriate to the level of student experience, particularly regarding the content and the non-traditional teaching methods. Additionally, introducing the flipped classroom in the early years of academic programs might support students in gaining transferable skills over the time of their program.
The major limitation of this study was that the current research only assessed the first three weeks of a flipped course. The student workload during this period is generally less than in subsequent weeks. Different outcomes could have been achieved if the implementation was conducted during busier periods of the semester. Another limitation was that the subject matter of the course studied is considered a pharmacy practice topic. Therefore, extrapolating the results from this study to science-based subjects, such as pharmacology and medicinal chemistry, is not possible.

CONCLUSION
The flipped classroom approach for a pharmacoepidemiology course for PharmD students is generally successful. This has been noted in both the positive student feedback from the flipped classroom perception survey and the end-of-course evaluation. However, successful implementation requires careful consideration, taking into account the level of the students and best practices for flipped classroom design. The flipped classroom has no negative impact on the academic achievement of the students.