Assessment of policy makers' individual and organizational capacity to acquire, assess, adapt and apply research evidence for maternal and child health policy making in Nigeria: a cross-sectional quantitative survey

Background Throughout the world, there is increasing awareness and acknowledgement of the value of research evidence in the development of effective health policy and in quality health care practice and administration. Among the major challenges associated with the lack of uptake of research evidence into policy and practice in Nigeria is the capacity constraints of policymakers to use research evidence in policy making. Objective To assess the capacity of maternal and child health policy makers to acquire, access, adapt and apply available research evidence. Methods This cross-sectional quantitative survey was conducted at a national maternal, newborn and child health (MNCH) stakeholders' engagement event. An evidence to policy self-assessment questionnaire was used to assess the capacity of forty MNCH policy makers to acquire, assess, adapt and apply research evidence for policy making. Results Low mean ratings were observed ranging from 2.68–3.53 on a scale of 5 for knowledge about initiating/conducting research and capacity to assess authenticity, validity, reliability, relevance and applicability of research evidence and for organizational capacity for promoting and using of research for policy making. Conclusion There is need to institute policy makers' capacity development programmes to improve evidence-informed policymaking.

of such barriers that have been consistently identified from some studies across sectors. These barriers include: the complexity of research evidence, organizational barriers, lack of available time, poor access to current literature, lack of timely research, lack of experience and skills for critical appraisal, unsupportive culture for research, lack of actionable messages in research reports, and limited resources for implementation [7][8][9][10][11] . In addition to these, some previous reports have noted that the use of research evidence is but one factor influencing all stages of what is in practice described as 'messy and political' policy cycle 12,13 The major challenges and barriers to the uptake of research evidence into policy and practice in low and middle income countries (LMICs) including Nigeria are associated with the capacity constraints of policy makers to acquire, access, adapt and apply available research evidence in policy making 2,14,15 . According to Dawad and Veenstra 16 , without adequate capacity, in knowledge translation/management and health policy research, policy makers will not have the capacity to access and synthesize sound information on which to base decisions and the potential for shared learning will be lost. Furthermore, Green and Bennett 17 , noted that knowledge and skill constraints associated with accessing evidence from various sources and competency in making use of the evidence appropriately are among the most important capacity needs of policy makers.
It is important to state that this capacity constraint regarding acquisition, assessment, adapting and applying research evidence in policy making is not only limited to the individual level but also at organizational level 2,14 . There is therefore an urgent need for the implementation of evidence-to-policy capacity enhancement mechanism that will target the improvement of both individual and organizational competencies.
However, in order to design an effective capacity enhancement strategy for evidence-to-policy process, there is need to assess the existing individual and organizational capacity to acquire, access, adapt and apply available research evidence in policy making. According to Deans and Ademokun 15 , before launching into capacity strengthening initiatives it is of paramount importance to understand what the existing capacities of policy mak-ers are and which need to be developed. This position is supported by the United Nations Development Programme 18 which states that capacity assessments are an essential step to developing rigorous and practical capacity development initiatives. This study was aimed at identifying the capacity gaps at individual and organizational levels regarding MNCH evidence informed policy making in Nigeria from which lessons can be drawn for other LMICs. The study was designed to obtain data from national MNCH stakeholders including tacit knowledge and knowledge relating to evidence transfer. This was to serve as a first step towards effort to build national MNCH policy makers' capacity on issues related to knowledge transfer and use of evidence in MNCH policy making.

Methods
The study design was a cross-sectional survey undertaken during a one-day national MNCH stakeholders' engagement event convened under the auspices of the West African Health Organization (WAHO) and the Federal Ministry of Health (FMOH) Nigeria, in October 2015 in Abuja, Nigeria. A cross-sectional quantitative survey was used because of the need to capture information based on data gathered for a specific point in time (ie., at the stakeholders' engagement event). The study design was also used to generate findings and outcomes that can be analysed in order to develop more in-depth research on policy makers' capacity constraints regarding evidence-informed policy making.
Among the objectives of the meeting was to assess the capacity of national policy makers and their organizations to acquire, assess, adapt and apply research evidence for MNCH policy making in Nigeria. Participants were officers of senior cadre drawn from the Federal Ministry of Health (FMOH) Abuja and its associated ministries, departments and agencies; others included directors from selected State Ministry of Health (SMOH); executive officers from development partner organizations (DPOs), civil society organizations (CSOs) and non-governmental organizations (NGOs). As part of the inclusion criterial, only individuals from the FMOH, SMOH, DPOs and CSOs/NGOs who were involved in MNCH programmes and policy making/implementation were invited. A questionnaire was administered to participants after African Health Sciences Vol 17 Issue 3, September, 2017 the completion of an informed consent form. The questionnaire and the informed consent form were approved by the University Research Ethics Committee of Ebonyi State University Nigeria (the institution of the principal author). The questionnaire was designed to assess participants' knowledge, capacity and organizational process of generation, synthesis and utilization of research evidence in policy making regarding MNCH. The questionnaire used for this assessment were modification of the self-assessment tool produced by the Canadian Health Services Research Foundation (CHSRF) available at the following websites: (i).
http://www.whatworksforchildren.org.uk/docs/ tools/SelfAssessmtWEB.pdf (ii). http://www.cfhi-fcass.ca/sf-docs/default-source/ documents/SAT-Self-Assessment-Tool.pdf We chose the CHSRF self-assessment tool as the basis from which we developed our data collection tool because a number of reports have demonstrated that it can help projects evaluate their capacity to use research evidence in the design and delivery of services [19][20][21][22] . The questionnaire contained core questions (hints), which assessed the following: • acquisition of research evidence; • assessing the validity, quality and applicability of research evidence; • adapting the format of the researcher results to provide information useful to decision makers; • application of evidence in decision making.

Analysis of the questionnaire
The data collected via the questionnaires was analyzed using the methods developed at McMaster University Canada by Johnson and Lavis 23 The main parameter measured was participants' perceptions of their own knowledge/understanding. The analysis is based on mean rating (MNR), median rating (MDR) and range. For instance, the figures represent Likert rating scale of 1-5 points, where 1 point = grossly inadequate; 2 points = inadequate; 3 points = fairly adequate; and 4 points = adequate; and 5 points=very adequate. The range was recorded as the range of values represented by lowest number chosen from the response scale and the highest (e.g., [2][3][4][5]. The mean was calculated as the numbers chosen from the response scale divided by the total number of responses to the question. While the median was determined by arranging the values chosen from the response scale in ascending order. In terms of analysis, values ranging from 1.00-3.49 points were considered low, whereas values ranging from 3.50-5.00 points considered high.

Profile and official designation attributes of participants
A total of 40 participants signed the informed consent form and completed the questionnaire (Table 1). A total of 16 (40.0%) respondents were males, and most of the respondents (63.9%) were more than 44 years old. Majority of the respondents (45%) were from the Federal Ministry of health and its associated ministries, departments and agencies. Most of the respondents (59%) were either directors or chairpersons in their organizations, have either spent <3years (37.5%) or 3-5years (45%) in their present designation.

Acquisition of research evidence relevant to MNCH
The outcome of the assessment of acquisition of research evidence relevant to maternal, newborn & child health among the stakeholders' participants is presented in Table 2. The mean ratings (MNRs) of the participants' knowledge about initiating/conducting research, the capacity of their organization to initiate research and source for research evidence were low ranging from 3.21-3.41 on the scale of 5. The MNR of the level of research incentives available in participants' organizations was very low at 2.68 on the scale of 5 (  Table 3. With reference to individual research skill, the mean ratings (MNRs) of the participants' capacity to assess the authenticity, validity, reliability & high quality of research evidence and capacity to assess the relevance and applicability of research evidence were low and ranged from 3.29-3.43 on the scale of 5. In terms of the organizational incentive for research, the mean ratings (MNRs) of the participants' organizations having incentives to encourage the application of research evidence in general and necessary incentives for assessment of the validity, quality and applicability of research evidence were very low ranging from 2.77-2.78 (Table 3).

Adapting the format of the research results to provide information useful to decision makers relevant to MNCH
The outcome of adapting the format of the research results to provide information useful to decision makers relevant to maternal, newborn & child health among the participants is presented in Table 4. With reference to individual research skill, the mean ratings (MNRs) of the participants' ability to summarize research results in a user-friendly way and ability to present results of research to decision makers ranged from 3.28-3.66 on the scale of 5. In terms of the organizational incentive for research, the mean ratings (MNRs) of the participants' organizations having incentives to encourage the provision of research evidence to decision makers was 3.0 on a scale of 5 (Table 4).

Application of evidence in decision making relevant to MNCH
The outcome of adapting the format of the research results to provide information useful to decision makers relevant to maternal, newborn & child health among the participants is presented in Table 5. In terms of the participants' organizations leading by example on how research use is valued in general and in maternal, newborn & child health specifically, the mean ratings (MNRs) were generally very low ranging from 2.85-3.18 on a scale of 5. The assessment of participants' organization's decision-making processes having a place for research indicated the mean ratings (MNRs) that are also generally very low ranging from 2.91-3.09 on a scale of 5 (Table 5). 2. Synthesize in one document relevant research as well as information and analysis from other sources.

3 36
3. Link the research results to key issues and provide recommendations.

4 35
(ii). Ability to present results of research to decision makers Use of charts, tables, graphs, pictogram, bullet/power point presentations, etc.

(b). Institutional/Organizational incentive for research
Organizational incentives to encourage the provision of research evidence to decision makers Summarized and easy-touse research evidence is routinely brought to the attention of relevant decision makers (such as through regular meetings or reports; or participation by researchers/analysts in management meetings to present/discuss evidence).

Discussion
The outcome of the assessment of individual capacity to acquire, assess and adapt research evidence relevant to MNCH among the participants generally indicated low MNRs. However, there are areas where the MNRs were very low suggesting areas of greatest capacity need.
In some previous studies, that made use of the CHSRF self-assessment tool, the areas of lowest ratings indicated areas of greatest capacity constraints 2,14,[20][21][22] . From the present study, any mean rating below 3.5 is considered poor and indicates weak capacity. In terms of capacity to acquire evidence, the mean rating was as low as 3.41 regarding knowledge about initiating/conducting research, which included the ability to effectively identify research problems; construct research questions; design research methodology; write research proposals/ protocols; analyze/interpret results and write research reports. In terms of individual skill to assess the authenticity, validity, reliability and high quality of research evidence regarding evaluating the reliability of specific research evidence and to compare research methods and results, the mean rating was low at 3.29. And in terms of the ability to adapt  evidence regarding summarizing research results in a user-friendly way as well as synthesizing in one document relevant research as well as information and analysis from other sources the mean rating was low at 3.29. These findings were not a surprise occurrence as they clearly confirmed earlier reports which indicated that many policy makers particularly the administrators lack research skill to acquire, assess and adapt evidence, which has led to a general lack of understanding about how evidence can be used properly [24][25] . In a recent systematic review by Humphries and co-workers 26 , it was observed from a number of studies reviewed that a deficit in the skills and experience of decision-makers in research literacy and research utilization, and a lack of formal management training were expressed as barriers to evidence-use in program management 6,7.27,28 . In an assessment of community capacity to acquire, assess, adapt, and apply research evidence relevant to HIV/AIDS, Wilson and colleagues 6 noted that capacity was lowest for the domains related to: acquiring research; assessing the reliability, quality, relevance, and applicability of research evidence; and summarizing results in a user-friendly way. This is clearly an area where there is a gap and will require some attention in any capacity enhancement strategy for policy makers. The mean ratings of the assessment of level of research incentives available in participants' organizations for acquiring, assessing, adapting and applying research evidence was generally very low. The worst outcome was reordered in terms of the organizational capacity for applying research evidence in which majority of the mean ratings ranged from 2.79-2.97. These findings clearly suggest that there is still a huge gap in terms of the organizational structure in place to support the use of research evidence in policy making in Nigeria. For instance, organizational incentive for research acquisition was as low as 2.68 regarding internet facilities; availability of research grants; in-service training, stipends, bonuses & per-diem for research works; provision of research facilities; sponsorship to conferences/workshops etc.
Similarly, organizational incentives for assessing the validity, quality and applicability of research evidence was as low as 2.77 especially concerning existence of quality control units; use of consultants/experts; application of computer software, statistical package; and promotion of ethical standards/practices. Also in terms of the orga-nizational capacity to adapt evidence regarding research evidence implementation committee and availability of administrative process for accepting/implementing research evidence the mean rating was very low at 2.78. These findings clearly indicate that there is a very poor research enabling environment in the participants' organizations. Similar organizational capacity constraints were identified in a number of previous studies as serious barriers to evidence informed policy making process. Wilson and co-workers 6 , in their study, found that overall organizational capacity to acquire, assess, adapt, and apply research evidence was low. They also noted that overall, the community based organizations studied were not only limited in incentive, and resources for assessing the quality and reliability of research, but they also have limited arrangements with external experts to help them. The factors related to organizational structure and process identified in a previous study, included lack of research structure, research, planning or decision-support positions; and under-resourcing described as an inability to allocate resources to research or evidence-related positions 28 .
Among the key organizational factors incapacitating evidence to policy process identified by some other studies was a lack creation of a learning culture and institution of good management -that promote better decision-making 29-32 . Bowen and colleagues 28 also identified the lack of information technology (IT) resources, including lack of databases or staff to support them and ensure data quality as key organizational constraint. In the further analysis of issues related to data constraints, four main components have been identified including: lack of data (availability and timeliness); lack of systems and resources for tracking, organizing and retrieving data; data overload; and lack of access to library resources, or capacity to conduct literature searches 28 . The greater capacity gap at organizational than at individual level is not a reporting bias. In fact, it would have been surprising if the result turned out otherwise. The improved awareness of the need for policy to be informed by the best available evidence has led many policy makers to initiate processes to improve their individual capacity for research. Unfortunately, there is yet to be a commensurate effort to improve the process at organizational level. Therefore, this study reflects the true picture of what is obtainable at organizational level in Nigeria policy making sector. This development is worrisome and poses a critical challenge to the evidence-informed policy-making process regarding MNCH in Nigeria, because inadequate organizational capacity and commitment towards evidence-informed policy making can incapacitate even the highly knowledgeable and skillful policy maker 1 . It is very obvious that the lack of enabling research environment and incentives in a policy making organization will further demoralize a policy maker in such organization who may want to make a deliberate effort to use evidence for policy making. It will therefore amount to a daunting task improving the poor maternal and child health outcomes in Nigeria with this high level of capacity constraints for evidence informed policymaking at individual and organizational levels. Nigeria is reportedly having more than 10% of all under-5 and maternal deaths world wide 3,4,33 . Strengthening the capacity of policymakers and their organizations for evidence informed policymaking holds great prospects for improved MNCH outcomes in Nigeria.

Conclusion
Although the emphasis that policy makers/decision takers ought to be skilled enough to find, appraise evidence or even conduct research may be based on a futile premise. However, it is our opinion that this capacity constraint may be playing contributory to weak health policy making and implementation in Nigeria. Irrespective of the busy schedule of policy makers, if they have better research skill they may find the evidence-informed policy making process a lot easier. This study is intended to promote the awareness of need for capacity enhancement at national level and the employment of people with research capacities in policy making positions. The study has provided base line scientific information that will be vital to the development of capacity enhancement training programmes on evidence-to-policy process targeted at policy makers at all levels of operation and governance in Nigeria.

Study limitation
One of the main limitations of this study was the rather relatively small study sample size of 40. A larger sample size may have provided better insights regarding the study outcome. However, our confidence and reliability of the study comes from the fact that we specifically targeted key national policy makers involved in MNCH programmes, and policy making/implementation. Individuals from the invited organizations who were not involved in MNCH were excluded from the study and that accounted for the limited number. Because there was substantial representation of the MNCH policymakers from the federal and states ministries of health, the findings can to a reasonable extent be generalised as far as FMOH and SMOH are concerned in Nigeria. We used the study to gain understanding at a small scale regarding the MNCH policy makers' capacity constraints on evidence-informed policy making in order to develop more in-depth research. A much more robust research methodology than a simple cross-sectional study and which involves expanded scope of MNCH stakeholders is required in future research on this important subject.