Problem based learning: tutors’ views 5 years after implementation at a sub-Saharan University

Background: Curriculum evaluation is key to continuous assurance of quality of education. Objectives: To assess the teachers’ perceptions on how well student teaching and learning activities were conducted at an institution that had practiced PBL for five years. Methods: A cross sectional descriptive study. 150 teachers from 23 departments at the College of Health Sciences participated in the study. A 25 item self-administered questionnaire was used to collect data. Activities evaluated included; students assessment, self-directed learning, feedback and clinical exposure. Data were entered in epi data and analysed using SPSS. Ethical approval was obtained. Results: The participants’ average teaching experience was eight years. The PBL/COBES approach mostly achieved the aim of producing self-directed and lifelong learners. Half of the teaching staff actively provided regular feedback about the learning and teaching activities they were involved in. Early clinical exposure was widely accepted as a highly rewarding appropriate teaching and learning strategy. COBES activities were well organized although involvement of College staff was low. Conclusion: PBL/COBES program was successfully executed and had high acceptance among Faculty. The biggest chal-lenge was getting all staff to provide regular feedback. Self institutional curriculum review can be done cheaply and quickly to provide feedback for continual curriculum improvement.


Introduction
This curriculum evaluation exercise surveyed the teachers' opinions and perceptions of how a new curriculum implementation at the five year point was understood and implemented.Five years of implementing new innovative methods of teaching and learning at Health sciences at Makerere University (an over 90 year old institution) was a milestone in the institution's educational history.A milestone because change for the better was embraced.The premise for the change was anchored in an extensive stakeholders' feedback exercise that had been con-ducted seven years earlier1.The extensive stakeholders' survey had been prompted by not only an awareness of global trends but also a need to respond to calls for change from alumni, employees and others stakeholders.These stakeholders in one way or another were involved with the graduates of this College.Stakeholders' demands can be achieved by continuous selfreflection and reforms.
Community Based Education and Service (COBES) is a system where students on an annual rotation basis are placed in a preselected rural site under the supervision of a preceptor for a period of 4-6 weeks with a predetermined set of educational and health service delivery targets.This process is supported by teachers at the College.Support is mostly in form of traveling to supervise student and preceptors on site.These reforms are part of the edu cation evolution process at Makerere University .
At Makerere University, curriculum reforms occurred turning a predominately traditional teacher centred lecture based approach to a principally student centred, self-directed and small group learning described as a combination of Problem Based Learning and Community Based Education and Services (PBL/COBES) methodologies.The COBES component involved students spending part of the in-scheduled teaching time at a pre-selected prepared rural community site where the (students) engaged in Health Service Delivery activities 2 .This exercise accorded them the opportunity to practice and learn in the context of a rural community life where nearly 90% of the country's population lives 3 .These curriculum reforms were not fully accepted by some stakeholders (including students, alumni, and the teachers).Some students and teachers preferred the lecture methods and some alumni thought reforms meant less teacher supervision.
The purpose of this survey therefore was to assess the strengths and weaknesses of the curriculum design and its implementation.This would inform strategies for improving the teaching and learning environment and the overall quality of education for undergraduate education at Makerere College of Health Sciences.

Study area:
Makerere University College is a 90 year old institution.It runs both undergraduate and graduate and 250 faculty (full, part-time to adjunct) 4 .The current educational reforms that started in 2003 have included over 250 sensitization/training workshop days and eight members completing courses in Masters of Health Profession Education.A complete transformation from teacher centred, lecture based to PBL/COBES curriculum was achieved after three years of preparation and implementation .
A quality assurance unit incorporated during the implementation period, formulated a manual stipulating minimum standards of education.This was informed by the World Federation of Medical Education standards (WFME) frame work 5 .

Design:
A cross sectional descriptive study.

Study population:
All teaching staff from Makerere University College of Health Sciences (without the School of Public Health) and the National Referral hospital, Mulago and in 23 departments.The teachers involved in this survey had a mean of eight years of teaching experience.The majority were specialists in the Clinical and Basic Sciences areas.

Student learning:
The eight questions in this domain were about PBL as an appropriate strategy for achieving skills in problem identification, and problem-solving through team work and appropriate use of resources.The other questions were about tutor involvement in providing administrative feed-back (to student learning).In summary there were five questions about student effects, three about faculty involvement.

Data tool:
A pre-tested 25 item self-administered hard copy hand delivered questionnaire was used.It had both the 5point Likert and binary scales.Items in four domains were Stu dent Learning (7), Student Assessment (6), COBES (5) and Governance (6 items).The choice of the four domains was generated by consensus in a pre-planning meeting attended by the investigators and key representatives of the College educational committee drawing on WFME and other literature 1,5,6,7,8,9,10 .These domains were organisational culture, learning contexts and organizational values .Pretesting was done using 10 questionnaires given to 10 individuals.The appropriate adjustments were made to improve clarity of the items and questions

Data collection:
The questionnaires were sent to the 23 departments after prior communication inviting members with at least 5 years teaching experience to participate voluntarily.Trained research assistants followed up the questionnaires.150 questionnaires out of the 250 were returned and 107 were completely filled.

Data analysis:
Epi data program was used for the data entry.SPSS 12 version was used for analysis, standard descriptive statistics were performed for the different domains.

Ethical approval:
Ethical approval was obtained from Makerere College of Health Sciences, School of Medicine Research and Ethics Committee.

Results
Twenty one departments were represented as shown in table I, though 16 respondents did not indicate the departments they represented.In table 2, the views of the staff regarding PBL are shown: the majority 75% agreed that PBL curriculum empowered students to identify health problems and use available resources to solve them.Problem Based Learning has helped our students to appreciate the complexity of diseases and health problems, diagnostic technologies as well as treatment 67 (63) 10 ( 9) 30 (28) 107 regimens which require continued learning.

3.
Our new PBL approach to student learning has reinforced "team work" study in small groups focused on 93 (87) 5 ( 5

Number (%)
COBES performance was rated highly in all the five questions, 64-94% were in agreement of high scores (see table 3) that COBES gave the desirable rural community health exposure experience to students and, it was implemented effectively.It contributes to continuous professional development for the site preceptors.
The challenges faced during COBES were often well articulated.
The teachers evaluated the assessment methods, integration of teaching and early clinical exposure (see table 4).The majority 61% agreed that assessment methods employed under PBL were aligned with the teaching methods.Early student contact with real patient stim-ulated learning for 73% of students.Integration was perceived to be attained by 76% of the participants.Examination guidelines were not clear to at least 16% of the respondents.Only half of the staff that participated in this survey viewed the introduction of the PBL/COBES curriculum as positively transformative of the College teaching and learning landscape (see table 5).
More than a third (38%) viewed the use of available resources as optimally managed, however 75% viewed the College of Health Sciences as having the potential for greater administration, financial and management efficiency.

Discussion
In this review exercise we found that PBL/COBES program was successfully executed and had high acceptance by the teachers (tutors).

PBL success
Concerning student learning, seventy to eighty percent of the teachers judged that PBL had achieved the general intended aims of students attaining problem solving skills and working as a team .The teachers responded to a set of questions in order to judge whether PBL had achieved its intended purpose.These teachers had been involved since the inception of PBL and COBES, and had previously undergone over 250 workshop days of training in various aspects of teaching and learning.These included but not limited to exploiting knowledge of how people learn to promote effective learning and dealing with the challenges of running successful tutorials 11,12,13 .We therefore contend that collectively the respondents were all well grounded in the basic knowledge of the principles that underpin the practice of PBL/COBES and therefore qualify to pass judgment on its execution.
Problem solving is a basic tenet of the philosophy of teaching and learning in this regard.Faculty should be engaged in identifying better ways of implementing the problem solving aspect in a PBL curriculum.It is therefore important that there exists a mechanism that ensures Faculty frequently and regularly give feedback to the curriculum implementation committee or its equivalent.The four questions about COBES included the justification for it, running the COBES program and feedback about how it was run.

Feedback and assessment
Overall, fifty percent of the teachers in this study regularly provided feedback.Feedback concerning implementation challenges including both academic and administrative.This proportion was slightly different from those that were involved in planning learning activities; it is therefore possible though not necessarily so that those involved in planning are also involved in giving regular feedback.
Whereas fifty percent of the staff provided feedback and this is to be judged as good involvement, it is debateable as to what the benchmark involvement in giving feedback should be.Involvement would be defined as in numbers of teachers giving feedback the scope and detail of feedback.
Regular and timely feedback to students after assessment is crucial for learning and was an area for definite improvement in this evaluation.It may involve training staff on how to provide feedback and for students what feedback to expect.Students' polls to assess what percentage of them feel they get feedback and their level of satisfaction could be part of improving feedback management 14 .
Feedback should be immediate, inexpensive, it should require little preparation (but requires skill in giving it ) and should be highly desirable to the learner.Frequency coupled with a user friendly mechanism 15 ensures a steady flow of feedback, but it is also important that the administrative team responds to the feedback given 16 in an appropriate and timely manner.
Alignment of teaching methods and assessment refers to using assessment methods appropriate to the style of teaching.It is also assessing students in a way similar to how teaching and learning occurred.Alignment was judged as well done by nearly two thirds of the respondents.Assessment drives learning, students learn in the way they are would be assessed 17 .Alignment of these two is critical and they feed into each other in order to achieve learning.assessment should therefore consider not only learning the objectives and content of the subject in question but also the method of learning, for example if group work is employed in a significant way, group assignments should be a significant part of the assessment portfolio.
Examinations are a central part of curriculum implementation and therefore examiners and students should be absolutely clear about examination regulations.Exams are part of assessment, a lens through which an institution assesses itself through its students work.When assessment results do not match institutional or programmatic expectations, that is the time for the staff to collectively determine how to improve student performance.So it is critical that the rules by which these assessments run are clear and well understood 18 .In the evaluation a third of the teachers were not clear about exam regulations.All staff should be in the know of all the regulations and rules.An assessment blue print including examination regulations and in house trainings program may be the way forward in resolving this situation.

Community Based Education and Service (COBES)
Early clinical exposure as an essential strategy to teaching and learning was agreed on by the majority of the respondents.There was a significant buy-in of the concept.Early clinical exposure aids integration of basic and clinical studies.This integration aids better retention of knowledge and acquisition of skills.Better retention translates knowledge and acquisition of skills for competence development.
The teachers involved in COBES activities were 39% of the teaching staff.Where, there is no set target of how many teachers should be actively involved.It is desirable that as many as can do participate, perhaps a rotation policy would get more staff involved but it would sacrifice the advantage of experience and continuity for this smaller group (44%) that has already demonstrated exceptional interest and expertise.However CO-BES activity supervision is also taken on by site tutors or pre-receptors and staff from the school of Public Health who were not included in the 44%.COBES was effectively implemented as judged by 70% of the respondents.One of the purposes of COBES was to expose students to rural health, whether this exposure translates into students going back to work in these areas after training awaits to be seen 19,20 .
In a previous study by Mubuuke et al 20,21 ; that interviewed radiology COBES students at Makerere revealed that the students understood the importance and purposes of participating in learning activities in the community.

Governance
Issues of governance were explored through questions of education reform, resource utilization, professional development and program evaluation.The majority of teachers understood the justification of the transformation to the College status however, resource usage in general was judged as poor 20% and (50% were non-committal) and the reasons need to be ascertained.The word resource implied time, equipment, staff, infrastructure as well as money, though we urge that whenever a question regarding resources availability and usage is raised, a negative response is expected given the realities of general resource constraints in most Higher Education institutions in resource poor environment.
The creation of a College of Health Sciences (a higher functional status that carries with it semi autonomy) was a governance matter, that was conceived as reframing and questioning the previous vision and its interpretation.In addition it focused on both the internal and external governance models with the aim of drawing strategic direction.
It was anticipated that the College will realize more funding, more collaborations, a higher research output, more student enrolment and more staff recruitment and retention.More than it would have had it not transformed.Continuous professional development by way of regular refresher courses in select priority areas should be on going to help old and new faculty understand in depth how the new innovative methods of learning and teaching work as well as how they would make it work.Periodic self-institutional evaluations to track improvements and areas of noncompliance 22,23 , as indeed this evaluation exercise was done, and should continue and be regularized.
Among the strengths of this study; there were no significant differences in responses between those who agreed, disagreed and or had neutral responses.This implies general consensus in the opinions shared.
Eligibility to participate was based in part on longevity of service.A five year teaching experience was deemed the minimum appropriate for one to be able to pass judgement on the various aspects outlined in the study.
The average teaching experience for the participants was eight years.

Study limitations
This study was not without limitations, the unknown and therefore uncharacterized non-responders who could be the critiques of the changes implemented.The responses were self -reported and therefore may be subjective.However, over the implementation period and before numerous capacity building activities occurred.We believe this provided firm grounding in the understanding of PBL/COBES.This study meant to collect teachers' views only, so the students' views were missed.

Conclusion
Regular curriculum self evaluation is possible and may be adopted as part of a continuous curriculum improvement.PBL/COBES was over all well executed in a resource limited environment.

Table 4 :
Student assessment ratings by; assessment methods, integration and early clinical exposure (the assessment was by staff), College of Health Sciences, Makerere 2010

Table 5 : Key Governance aspects ratings as assessed by staff 264 265
For five years, feedback was gathered through regular weekly two hour long meetings at the College of Health Sciences.African Health Sciences Vol 15 Issue 1, March 2015 African Health Sciences Vol 15 Issue 1, March 2015