Health managers and community representatives' views of a system-wide intervention to strengthen public healthcare in the Free State, South Africa

Background A system-wide health system strengthening (HSS) initiative, the Health Systems Governance and Accountability (HSGA) intervention, was developed, translated to policy, and implemented in the Free State province. This study assessed health managers (HMs) and community representatives' (CRs) views of the intervention and whether it improved integration and performance. Method A questionnaire survey among 147 HMs and 78 CRs and 14 focus group discussions (FGDs) with a mean of 10.3 participants and a total of 102 HMs and 42 CRs, were conducted. The questionnaire and FGD data were descriptively and thematically analysed to triangulate findings. Results Many HMs (44%) mostly positioned at the operational levels indicated that implementation of the HSGA intervention did contribute to integration of health services. Most CRs (54%) believed that communities were actively involved in the intervention. However, both the self-administered questionnaire and the FGD data evidenced lack of policy awareness among, especially, operational-level HMs. Conclusion From the perspectives of HMs and CRs, the implementation of the intervention was viewed as a step forward in strengthening public healthcare to respond to system deficiencies in the Free State province. Earlier engagement of especially operational-level HMs during reforms may be beneficial in successfully implementing HSS interventions.


Introduction
Since 1994, several health policies that focus explicitly on alleviation of inequity in healthcare service provision have been implemented in South Africa 1-3 . These included the introduction of free-health policies, the district health system, and health sector reforms such as prioritisation of primary health care (PHC), in itself a social justice philosophy 4 . PHC re-engineering in South Africa aims to improve community health through ward-based PHC outreach teams (WBPHCOTs), school health services and district clinicals specialist teams (DCSTs) 5-6 .
Other relatively recent reforms are the establishment of 'Ideal clinics,' defined as clinics that open on time and have staff, infection control measures and security to protect the staff and patients 7 and a 'One patient-One File' system. The net effect of these reforms was more equal distribution of public health resources, as well as greater access to services for previously deprived groups 2 . However, stark policy-implementation gaps -fuelled by inefficient processes or systems such as limited financial resource allocations and delegations, top-down directives, and ineffective supervision and performance management systems -frequently occurred, and there may have been an overall lack of systems thinking [8][9] . Many international health policies recognise the World Health Organization's (2008) 10 vision that communities should be involved in shaping PHC services 11 . South Africa's National Health Act (No. 63 of 2003) 12 requires the establishment of clinic and community health cen-tre (CHC) committees with functions prescribed in the provincial legislation in question. A representative hospital board for each central hospital (such as Universitas Hospital in the Free State province) with functions prescribed by the national Minister of Health must be in existence. The relevant Member of the Executive Council (MEC) at the provincial level must appoint a representative board for each public health establishment classified as a hospital or for each group of such establishments and prescribe the functions and procedures for meetings of these boards. Although community participation also takes many other forms such as volunteerism by non-governmental organisations (NGOs) and faith-based organisations (FBOs) in clinic data capturing and lay counselling services, it is especially through their participation in clinic and CHC committees and advocacy in hospital boards that communities can contribute to identifying the health system's weaknesses and help to refine necessary changes in the organisational workings of public health systems at the grassroots level. The health and wellbeing of South Africans -the majority of whom are dependent on government services 13 -remain plagued by a relentless burden of infectious and noncommunicable diseases [14][15] , persisting social disparities, and inadequate human resources for health [1][2]17 to provide care for a growing population with a rising tide of refugees and economic migrants 18 . Despite the fact the public health system has been transformed into an integrated, comprehensive national service, failures in leadership, stewardship and management have led to inadequate implementation of good policies 19 . It is generally accepted that public health services in South Africa are not always of sufficient quality to be effective 20. External factors such as political preferences for the appointment of public health managers (HMs) have been reported to influence the employment of public health professionals 21 . Resulting management skill deficits have a detrimental effect on the quality of public health services. Shortages and high migration of health professionals 1-2;17;20;22 , as well as wide inequality [23][24][25] and unemployment [26][27][28] , further limit the capacity of the public health sector to meet the health needs of the nation. The international COVID-19 outbreak has emphasised the need to strengthen public health systems 29 and to adopt systems thinking in doing so 30 . The idea of adopting a system or 'whole-system' approach to understand and implement public health system strengthening (HSS) interventions has gained traction in modern times, espe-cially among health policy-makers [31][32][33] . 'Whole-system' designs and methodologies are also increasingly applied in health systems research [34][35][36][37][38] . A 'whole-system' approach inculcates a way of thinking that considers the importance of linkages and interdependencies between the components of a healthcare system [39][40] . Systems thinking is also useful to assess the effectiveness of public policy implementation and service delivery 41-42 . In the current setting, initial understanding of the deficiencies of the public health services provided by the Free State Department of Health (FSDoH) was based on a multi-method situation appraisal 19 . A system-wide HSS initiative, the Health Systems Governance and Accountability (HSGA) intervention, was developed, translated to policy, and implemented. The HSGA intervention and its formalisation into an official policy, the HSGA policy 43 , was designed to address fragmentation and improve public health service delivery in the Free State by the provincial health department in collaboration with its stakeholders. The considered intervention model was an integrated approach to health service delivery in the Free State province starting from the community up through all levels to the central health services. It was designed to improve health system performance by effectively supporting the interaction between disease-specific programmes, complemented by a precise routine assessment method, i.e., the BSC performance-monitoring tool [44][45] . This study aimed to assess HMs and CRs' views on whether public health service integration had indeed been enhanced and if the integration of the roles and responsibilities of HMs and CRs, as well as attempts to enhance the Health Information Management System, had improved 'whole-system' operations.

Design, population and sampling
A cross-sectional survey including, firstly, an anonymous self-administered questionnaire involving 147 HMs and 78 CRs and, secondly, 14 focus group discussions (FGDs) with 102 HMs and 42 CRs was conducted in the 2017/18 financial year. The mean number of participants per FGD was 10.3. In the case of both the HMs and the CRs convenience sampling applied in as far as only those respondents who responded to invitations to sessions to complete the self-administered questionnaire, and conduct the FGDs, were included. The participating HMs (n=102; 70.83%) mostly worked at the facility (clinic/ CHC or hospital) level where they were responsible for operational-level programmes. In the current study, members of the hospital boards and clinic and CHC committees whose views on an intervention to strengthen public healthcare in the Free State were sought, are referred to as community representatives (CRs). As governors of these facilities, they were well placed and sufficiently informed to report on different systems aspects. The CRs are entitled to access any public health information from clinic operational managers or hospital CEOs.

Data collection
Containing both structured and open-ended questions, the self-administered questionnaire elicited HMs and CRs' views on the effectiveness of the HSGA implementation in improving 'whole-system' performance in respect to seven goals of the health department, namely, to improve 1) leadership/governance, 2) financial management, 3) workforce management, 4) PHC re-engineering, 5) infrastructure management, 6) the Health Information Management System; and 7) referral and 'whole-system' interventions. The FGDs were conducted by research assistants experienced in this technique to supplement the quantitative data obtained through the self-administered questionnaire with a more in-depth qualitative understanding of whether and how public health service integration had been achieved and had, or had not, improved 'whole-system' operations. All sessions were audiotaped with the permission of the participants and transcribed verbatim. At the start of the group session, participants completed an attendance register and an informed consent form. A protocol and focus group guide were used to introduce the study and the focus group questions. The research assistant facilitated the group discussions to keep the participants focused on the topics of interest. The average length of the group discussions was 60 minutes. During the FGDs the participants could also revert to their home languages, i.e., mostly Sesotho and Afrikaans, to better express their views.

Data analysis
The self-administered questionnaire data were descriptively analysed using STATA 12 46 . The FGD data were transcribed and analysed for emerging issues using NVivo 9 47 . However, the FGD data analysis, interpretation and reflection were continuous and already commenced during discussions and transcribing.

Ethical considerations
Both the self-administered questionnaires and the FGDs were completed in private conference rooms or halls at the district hospitals. Participation in the study was voluntary. As the questionnaire was completed anonymously and numbers were allocated as identifiers during the FGDs, little or no risk were associated with participation in the study. The study participants were not paid for and did not have to pay to participate in the study. All participants signed an informed consent form. The study was approved by the Health Sciences Research Ethics Committee of the University of the Free State (Referral number HSREC 11/2016).

Findings
The findings are presented according to the aforementioned departmental goals. Table 1 presents the self-administered questionnaire findings on HMs and CRs' views on achieving the leadership/ governance goal. In respect to whether health sector reforms were implemented, a large proportion of the HMs (n=70; 47.62%) stated that they 'did not know.' Contrarily, most CRs (n=46; 58.97%) opined that health sector reforms were indeed taking place. Regarding whether district health plans were being implemented, most HMs (n=87; 59.18%) agreed that these plans were implemented, while almost a third (n=43; 29.25%) 'did not know.' Whether hospital service delivery plans were implemented, the largest proportion of the HMs (n=72; 48.98%) 'did not know,' while about four in every 10 (n=61; 41.50%) indicated that the plans were indeed being implemented. Whether clinic management structures were in place and functional, the largest proportion of more than four in every 10 HMs (n=65; 44.22%) affirmed the existence and functionality of these structures. However, just more than a third (n=50; 34.01%) 'did not know.' Regarding their views of the corporate office's influence on district health plans (n=107; 72.79) and hospital health plans (n=111; 75.51), about three-quarters of the HMs 'did not know.' The self-administered questionnaire findings corroborated the views expressed by the HMs in the FGDs, during which a recurring emerging theme was that plans from higher levels were not carried through:

Leadership/governance
"Things are not seen to the end. There are good things that were started like I think it was after 2015/2016, principals from the province came down to us wanting to know what we needed, and we developed action plans. We were promised things that were never done. It was a good initiative, but it was never seen through" [Facility Operational Manager].
The CRs expressed similar views during the FGDs: "The problem is that the Department would come with programmes but will not sustain them." "Get the resources in place and then come back to tick the box to say why is there no improvement or why is there only small improvement, while you have all your equipment, all your resources, the ambulance system in place, the forms are there." "The policies are very good on paper, we read them, we hear about them, but when it comes to the implementation part of it, it just doesn't happen." Table 2 depicts the HMs and CRs' views on achieving the financial management goal as discerned through the self-administered questionnaire. The largest proportions of both the HMs (n=57; 38.7%) and CRs (n=22; 28.21%) agreed that all hospitals charged fees for services as required. However, most of the HMs indicated that they 'did not know' whether records of accounting procedures were in place (n=82; 55.78%), periodic audits were conducted (n=91; 61.90%), monthly financial reports were delivered (n=85; 57.82%), and whether expenditure reports (n=70; 47.62%), revenue reports (n=71; 48.30%), records of accounting procedures (n=77; 52.38%), peri-odic audit reports (n=84; 57.14%) and monthly financial reports (n=75; 51.02%) were in use. A prominent theme emerging from the FGDs with both the HMs and CRs was constant resource scarcity: "We do not have enough resources for the implementation. If the resources were there, it would be easier for the model to work. One may say that we should push ourselves for the model to work, but you cannot push yourself if you do not have resources" [Unit leader]. "Our clinic is in town and our building belongs to the municipality. There is a very small space for consultation. When you go for medication at the pharmacy it is far. You have to walk a distance and there are stairs which is a problem for old people and the disabled"

PHC re-engineering
defined as a clinic that opens on time and has staff, infection control measures and security to protect the staff and patients 7 . Most CRs also agreed that communities were actively involved in implementing the 'One Patient-One file' system, i.e., a single patient identifier and a single file containing all the medical history of an individual patient (n=51; 65.38%). They also mostly agreed that communities were 'actively involved' in the implementation of the HSGA intervention (n=42; 53.85%). Just less than half of the CRs (n=37; 47.44%) believed that communities were 'actively involved' in implementing the Balanced Scorecard (BSC) performance-monitoring tool.
African Health Sciences, Vol 23 Issue 1, March, 2023    Table 5 depicts the HMs and CRs' views on, firstly, infrastructure management and availability of equipment and, secondly, the Health Information Management System. Regarding infrastructure management, the largest proportion of HMs (n=68; 46.58%) indicated that they did not participate in meetings on strategic infrastructure planning. Most HMs also stated that availability (n=68; 53.54%) and maintenance (n=85; 58.22%) of equipment at health facilities did not meet expectations. Nearly six in every 10 HMs thought that the lack of infrastructure maintenance (n=87; 59.59%) and unavailability of equipment at health facilities (n=86; 58.90%) had a 'negative' influence on public health system performance. HMs frequently raised the lack of maintenance during the FGDs: "There is absolutely no maintenance. There was this buzz word that was called 'maintenance hub,' and we bought into it saying at least if my door is broken, assistance will just be a call away. Or maybe one of the clinics would be having a hub where I could call to say my stuff is broken. But now you have to go on trying to fix stuff like your light bulbs. You are frustrated because you want to use something, but you can't" [facility Operational Manager]. Equal proportions (n=25; 32.05%) of the CRs agreed or disagreed that they participated in meetings on strategic infrastructure planning. As with most HMs, most CRs also indicated that infrastructure availability (n=45; 57.69%) and maintenance (n=41; 52.56%) at health facilities did not meet expectations. Large proportions of the CRs believed that the lack of infrastructure mainte-nance had a 'negative' (n=27; 34.62%) or a 'very negative' (n=17; 21.79%) influence on public health system performance. Even larger proportions thought that the unavailability of equipment at health facilities had a 'negative' (n=29; 37.18%) or 'very negative' (n=22; 28.20%) influence on the system's performance. In terms of the Health Information Management System, most HMs agreed that it was possible to indicate the five diseases with the highest consultation rates based on the District Health Information System (DHIS) in all districts (n=78; 53.06%), and in their own district (n=80; 54.79%). Almost a third (n=47; 32.41%) of the HMs believed there were 'seldom' and a further almost two in every 10 (n=26; 17.93%) that there were 'never' any shortages of health statistics forms at health facilities during the previous 12 months. Nearly two-thirds (n=96; 66.21%) of the HMs believed that health facilities always' submitted data to the DHIS. More than a quarter (n=39; 26.90%) stated that they 'did not know.' The largest proportion (n=42; 28.97%) of HMs also stated that they 'did not know' how often health facility staff analysed statistics for decision-making. However, according to many HMs, the statistics were 'always' (n=40; 27.59%) or 'mostly' (n=34; 23.45%) considered for decision-making. Many of the HMs believed that district-level staff 'always' (n=40; 27.59%) or 'mostly' (n=29; 20.00%) provided feedback to health facilities in response to submitted reports. Again, more than a quarter (n=42; 28.97%) of the HMs said they 'did not know.' The largest proportion (n=54; 37.24%) of the HMs also 'did not know' how of-African Health Sciences, Vol 23 Issue 1, March, 2023 Regarding the CRs' views, almost two-thirds (n=50; 64.10%) affirmed that it was possible -based on the DHIS -to indicate which five diseases had the highest consultation rates in their district. The largest proportion (n=25; 32.05%) of CRs thought that health facilities 'seldom' experienced shortages of statistics forms. However, the largest proportion of CRs (n=24; 30.77%) was also of the view that health facility staff 'seldom' analysed statistics for decision-making. Less than a third of the CRs (n=25; 32.05%) indicated that district-level managers 'always' provided feedback to health facilities in response to reports submitted.

Infrastructure management and the Health Information Management System
African Health Sciences, Vol 23 Issue 1, March, 2023

Discussion
The discussion is again structured according to the seven departmental goals.

Leadership/governance
In order to operate a provincial health system, district health plans are developed under the direction of the corporate office by the district managers. Their teams comprise of operational managers based in the facilities and the programme managers and coordinators that support them. Thwala et al. 48 posited that the positioning of the district managers as frontline service delivery 'stewards,' and provincial-level managers as 'overseers,' places the role of leadership/governance at the core of health service delivery to ensure that the district health system (DHS) works to achieve common goals. Contrary to Kemppainen's 49 observation that HMs should lead the implementation of plans within their organisations and influence their teams to respond to plans and reforms, including implementation of daily management processes, this was apparently not the case in the Free State. This could partially be attributed to employee attitudes as three of the most significant barriers to managing change include lack of management visibility and support, employee resistance to change and inadequate management skills 50 .
Effective management skills are key to helping organisations through times of change. Applebaum & Wohl 50 , p. 279 wrote that notwithstanding the best efforts of senior healthcare executives, major change initiatives often fail: "Change threatens the very stability and continuity that managers are attempting to control; therefore, change and managers are not natural partners." According to these authors, even those managers cognisant of the need for change resist parts that they see as "too major, too risky, or too different." However, successful initiatives to improve systems management and transformation have been reported. In 2019, a case study of a rural district in Mpumalanga province in South Africa described how declines in morbidity and mortality from severe acute malnutrition in young children were achieved following a district HSS approach embedded in supportive policy and processes at the national and provincial levels 38 . Centred on real-time death reporting, analysis and response, the HSS interventions produced three kinds of system-level change: 1) knowledge and use of evidence by providers and managers ('ways of thinking'), 2) leadership, participation and coordination ('ways of governing') and 3) in-puts and capacity ('ways of resourcing'). The study found that 'whole-system' approaches and coordinating action at multiple levels, were imperative in building enabling environments at the frontline.

Financial management
Public hospitals in South Africa are compelled to collect revenue as part of their budget and in line with their revenue collection strategy. While large proportions of both the HMs and CRs agreed that all hospitals charged fees for services, more than half of the HMs indicated that they 'did not know' whether financial management procedures and reports were in place. This is concerning since these are day-to-day financial management functions, and HMs are expected to know about them. Therefore, the observed spate of 'don't know' was an unexpected response from fiduciaries entrusted to know and enact their mandate and internal functional and financial responsibilities. Many HMs who lacked knowledge about basic managerial financial functions may have risked managerial abdication of responsibility or accountability as regulated by the Public Finance Management Act (PFMA) 51 and National Treasury Regulations and Instruction Notes 52-53 . Given the importance of financial resources as a driver of change 38;54 , these are concerning findings. However, the explanation for the wave of 'don't know' answers could again be that the survey may have been conducted too early in the HSGA intervention process. Indeed, implementation of the HSGA model may have contributed to improved financial management over the longer term as in 2016/17 and again in 2017/18 the FSDoH obtained unqualified audit opinions 55 .

Workforce management
Human resources (HR) have been described as one of the three most imperative health system inputs. The other two important inputs are physical capital and commodities 56 . Provision of quality healthcare services requires a committed and dedicated workforce that can work within an environment conducive to unleashing internal talents or skills. While service industries outside the health sector continuously demonstrate that transformational and service leadership styles are most efficacious in energising HR within organisations, healthcare organisations are often dominated by leaders who practice outmoded transactional types of leadership 57 . The 1997 White Paper for the Transformation of the Health System in South Africa 58 directed that management authority should be decentralised to the provincial and district levels to allow for a greater degree of autonomy and that HMs should be supported in acquiring the skills required to manage the HR of a decentralised health service.
In the current study, the largest proportion of HMs indicated that only 'some' staff in the districts and hospitals had job descriptions, training and career plans, and staff assessment and rotation systems. Large proportions of the HMs indicated that they 'did not know' whether staff roles in performance agreements were up-to-date and if staff morale was assessed in their institutions. Basic HR management capability challenges were thus evident among the HMs. A previous study 59 suggested that poor performance within organisations was directly related to weak or poor leadership, consequently yielding a negative impact on employee morale, performance and service delivery. A survey of American medical directors indicated that competencies related specifically to healthcare and clinical skills proficiencies, were more highly recognised relative to generic administrative or management skills 60 . This negated the fact that some background in management or some specialised training in healthcare management are essential skills for effective management of health services. Therefore, the importance of determining management capabilities and providing appropriate skills training was required as part of an overall management development process to improve policy and procedure implementation and health systems functioning 61 . This is corroborated by the National Department of Health which directs that HMs should be supported in acquiring the competencies required to manage a decentralised health service with context-sensitive workload indicators 62 . The current study's findings suggest that to deal with capacity challenges, HMs in the Free State would particularly benefit from training in leadership, HR and organisational behaviour.

PHC re-engineering
Former South African Health Minister Aaron Motsoaledi's turnaround strategy considered PHC re-engineering to be the main and foundational pillar of health system reform 63 . A well-functioning DHS is required for effective re-engineering of PHC. This strategy requires strong leadership and greater emphasis on health promotion, prevention, and community participation and empowerment 64 . The current study considered the HMs' views on the extent to which PHC re-engineering services, includ-ing school health, outreach, healthy lifestyle promotion (e.g., anti-tobacco, alcohol and substance abuse), family team (WBPHCOT), DCST, contracted general practitioner, and 'development partner' services were integrated into the Free State public health system. Large proportions of the HMs thought that such services were only 'partially' integrated. This is concerning because PHC is a vital pillar for fundamental health care at the community level and is also considered a cost-effective modality 65 .
On the positive side, most CRs affirmed that communities were involved in implementing interventions (e.g., Ideal clinic model, HSGA intervention, BSC performance-monitoring tool and One Patient-One file reform efforts). However, in September 2021, Ritshidze 66 , p. 39 reported that only six out of 22 PHC facilities it monitored in three of the five districts in the Free State had 'functional' clinic committees.
Adopting the view that the extensive revisions to the health system represent 'whole-system' (as opposed to piecemeal or programmatic) change, Gilson et al. (2017) 67 enumerated the lessons learnt in transforming the health system in the Western Cape and made recommendations for the successful implementation of such an approach. These included meeting the need for new PHC models oriented to the wider health and social challenges facing populations in the 21 st century; development of inter-sectoral partnerships and multiple forms of patient and community engagement; and innovative action to address the health challenges of particularly vulnerable groups and communities. However, recent evidence shows that health committees are not functioning effectively in South Africa 68 . Factors impacting negatively on health committee participation include health committees' unclear roles; committee members' skill deficits; facility managers and ward councillors' negative attitudes; and limited resources, support and recognition 69 .

Infrastructure management and the Health Information Management System
Infrastructure is considered an integral aspect of quality health service provision and a key pillar supporting the fundamental objective of promoting good standards of care for all patients 70 , p. 5: "This requires scrutiny of every element of the life cycle of a device from the specification of requirements, through evaluation of competing products, decontamination, procurement, introduction, maintenance and quality assurance to disposal and funded plans for equipment replacement." However, old and poorly maintained healthcare infrastructure is a pervasive problem affecting the delivery of quality healthcare in many parts of South Africa 71 and the Free State province 72 . In this study, HMs and CRs' views on whether infrastructure availability and maintenance met expectations and the effects thereof on public health system performance were assessed. Most HMs indicated that they did not participate in infrastructure planning meetings, that the infrastructure's availability and maintenance did not meet expectations, and affirmed the 'negative' or 'very negative' influence that unavailability of equipment and lack of maintenance had on the public health system's performance.
Most of the CRs stated that they at least 'partially' participated in infrastructure planning meetings. However, they also mostly agreed with the HMs that the availability and maintenance of infrastructure maintenance did not meet expectations. It is thus clear that the public health infrastructure in the Free State was not congruent with the view that "the quality of patient care and access to services is essentially determined by the quality of infrastructure, quality of training, competence of personnel and efficiency of operational systems" 73 . Ritshidze 66 , p. 16 recently reported that of the 22 clinics in the districts of the Free State it monitored, 27% were in poor condition, 95% needed additional space, 43% did not have enough room in the waiting area, 60% of toilets were in a 'bad condition,' and 27% of patients said facilities were 'dirty' or 'very dirty.' Since the early 1980s, the WHO has emphasised the importance of the Health Information Management System and skills training in implementing an integrated PHC approach 74 . Healthcare should be supported by systems to deliver care that is 'safe, effective, patient-centred, timely, efficient, and equitable,' and health information management has a critical role in designing such a system 75 . As more money is spent on health information management, the demand for the cost-effectiveness of healthcare creates new pressures to assess their impact and whether they are achieving "their putative benefits and justifying their costs" 76 , p. 549. Research in the Western Cape province emphasised the need for new forms of monitoring and evaluation that take a 'whole-system' perspective: "extending beyond services and programmes to system functions, drawing in a wider range of perspectives and knowledge, and considering not only what but also how health-system change is unfolding" 67 , p. 64. When the opinions of the HMs and CRs in the Free State were sought on whether it was possible to identify the five diseases with the highest consultation rate in their areas of jurisdiction, both groups mostly affirmed that this was indeed possible. Also positive was that most HMs thought there were 'never' any shortages of health statistics forms at health facilities and that health facilities 'always' submitted data to the DHIS. Worrying was that the largest proportion of HMs said they 'did not know' how often health facility staff analysed statistics for decision-making. According to about half of the HMs, statistics were 'always' or 'mostly' considered for decision-making. The Health Information Management System is a tool intended to enable HCWs at every level to use data for planning, implementation and evaluation. Notable in the current study was that many HMs lacked knowledge or awareness of health information management procedures. Subsequently, the aims of the Health Information Management System and the DHIS to contribute positively to managerial functions of planning, trend analysis, and ensuring quality and efficient healthcare, could not be achieved, and these systems did not serve as a driver of decentralised decision-making 77 .
The DHIS is a platform on which people, processes and technology interact to support the operations and general management to plan, decide, organise and manage the delivery of quality healthcare services 78 . Therefore, buy-in by both top and operational-level managers is essential for adopting and implementing the DHIS as a tool for management. In instances where the DHIS has not been adopted as a strategic imperative, the lack of commitment thereto by management at many levels manifests as an obstacle to the effective implementation of HSS interventions.
Despite wide training on the use of the DHIS, HMs and HCWs were essentially unable to put the data collected to best use for planning and decision-making. It seems that the training of HMs needs to include continuously supporting them and providing oversight on the importance, ability and use of health information 76 . Williamson et al. 74 recommended that monitoring of the progress of implementation of DHISs must be at three levels of the information system establishment: firstly, at the level of data collection, capturing, validation and reporting; secondly, at the level of data interpretation and presentation; and, thirdly, at the level of information use for decision-making and planning. Appropriate use of DHIS data can transcend good patient care and improve planning, administrative effort, clinical research and strategic information management to improve health system performance 77;79 . Research in several African countries have reported serious challenges in implementing DHIS2. A Malawian study aimed at evaluating how available interface elements influence usability in DHIS2, found that lack of relevant editing features and lack of conformity to the "Keep It Simple, Stupid (KISS) and minimalistic design principle" were challenges negatively affecting the usability of the system 80 . A study to evaluate the quality of reporting of key indicators of childhood malaria during the first four years of DHIS2 implementation in Senegal established that in contrast to public facilities of which 92.7% reported data in the DHIS2 system during the study period, only 15.3% of the private facilities used the reporting system. The quality of reporting for malaria indicators in the Senegal DHIS2 had improved over time and the system was found to be suitable for use to monitor progress in malaria programmes. However, the study recommended that Senegalese health authorities should maintain the focus on broader adoption of DHIS2 reporting by private facilities, the sustainability of district-level data quality reviews, facility-level supervision and feedback mechanisms at all levels of the health system 81 . A Nigerian case study demonstrated the high potential for effective monitoring of maternal and neonatal health using DHIS2 82 . However, coordinated action was needed at multiple levels of the health system to maximize reporting of existing data, rationalise data flow, routinise data quality review, feedback, and supervision, and ongoing maintenance.

Referral and 'whole-system' interventions
The WHO states that "[a]n effective referral system ensures a close relationship between all levels of the health system and helps to ensure people receive the best possible care closest to home" and can help to ensure cost-effective use of hospital facilities, while allowing for timeous access to specialist services 83 . Further to this source, in developing countries many clients seen at outpatient clinics at secondary facilities could be more appropriately looked after at PHC facilities at a lower overall cost to both the client and the health system. Assessing or monitoring referrals between healthcare services and workers can demonstrate the accomplishments of collective efforts, the balanced use of resources and capabilities through efficient use of network members, and the avoidance of duplication of efforts 84 . In the current study, most HMs stated that referral system processes were 'always' in place. A smaller number claimed they 'did not know.' Large proportions of HMs believed that that implementing the HSGA and the BSC performance-monitoring tool 'whole-system' interventions contributed to integrate health service delivery and improve health outcomes, while most CRs affirmed that the referral systems were 'always' or 'mostly' in place.
Relatively few studies on initiatives to bring about 'whole-system' change in health care in South Africa have been published. In 2014, a case study of factors facilitating early implementation of PHC reform in the Western Cape province, defined 'whole-system' interventions as "those that entail system wide changes in goals, service delivery arrangements and relationships between actors, requiring approaches to implementation that go beyond projects or programmes" 35 . The study found that successful implementation of the PHC outreach team strategy was characterised by factors such as a favourable provincial context of a well-established district and sub-district health system and longstanding values in support of PHC, a collective vision for the new strategy, distributed leadership and ownership of the new policy, an implementation strategy that ensured alignment of systems and appropriate sequencing of activities, privileging of community dialogues and local manager participation in the early phases, establishment of special implementation structures to enable feedback and ensure accountability, and an NGO partnership to support implementation.
A limitation of the current study is that its cross-sectional design precludes reporting of the longer-term impact of the concomitant implementation of HSGA intervention and BSC performance-monitoring tool. A strength of the study is that it is the first 'whole-system' assessment of a HSS intervention in the Free State province, using more than one technique, which allowed for triangulation of findings. The positionality of the first author as political executive driving the HSS intervention, may be interpreted as limitation as an element of bias may have been inevitable. However, the fact that the researcher was the executive authority of the FSDoH, could also be interpreted as a strength of the study. Indeed, the position held by the first author enabled the scale of the intervention and high level of participation by employees and senior managers. This provided for deeper insights into issues not readily observable to an external observer and conveyed a deeper awareness of multiple problems within the health system.

Conclusion
Policy-makers and managers of organisational transformation initiatives in healthcare can benefit from periodic assessments of the implementation of HSS interventions. Midcourse assessments of progress and response to evidence of unintended, emergent developments, provide decision-makers with opportunities to reassess and modify their plans. In the current study, HMs and CRs' views of a system-wide intervention to strengthen public healthcare in the Free State, the HSGA model, were studied. The great number of 'don't know' responses to questions about leadership/governance, financial management, workforce management, PHC re-engineering, infrastructure and information management, and referral and whole-system interventions, were likely indicators of serious policy-implementation gaps and did not portend well for the state of public healthcare in the Free State. The preponderance of "don't know" answers about issues that fall within the realm of their day-to-day activities -as also outlined in their job descriptions -requires a re-look at what the possible reasons for this apparent apathy and seeming policy-implementation gaps could be. On the positive side, the HMs and CRs mostly thought that the HSGA intervention and the BSC performance-monitoring tool were effective in integrating public health service delivery and improving performance and creating a good platform for achieving the desired outcomes. However, more research may be necessary to understand how earlier engagement with lower-level operational managers and functionaries can be used to facilitate the uptake and sustainability of HSS interventions.