ORGANIZATIONAL CAPACITY AND UTILIZATION OF EVALUATION RESULTS
DOI:
https://doi.org/10.14738/assrj.89.10030Abstract
A major concern in programme management, given the potential of programme or project evaluations to inform decision-making, has been how to ensure the results or findings of an evaluation are used for the intended purpose. How information from an evaluation process and/or the products are used in practice and decision making is equally a concern for evaluators. This article presents a research that sought to explore the extent to which organizational capacity impacts the utilization of evaluation findings for programme improvement. Guided by the pragmatic paradigm and informed by the Utilization Focused Evaluation Model and Knowledge Use Theory, the research adopted a descriptive and correlational design using mixed methods. The sample size for this study was 232 project staff from Non-Governmental Organizations (NGOs) in Kisumu Central Sub-County, Kenya. To analyze qualitative data, the open ended responses from key informant interviews were recorded and coded appropriately for further analysis for themes through content analysis and comparative analysis. Frequencies and percentages were calculated to describe the basic characteristics of the quantitative data. To ensure validity and reliability of the research instruments, pilot testing was conducted. Cronbach’s alpha at α = 0.908 was attained as the reliability coefficient of the pre-test instruments. Tests of statistical assumptions were carried out before data analysis to avoid invalidation. A hypotheses H01: Organizational capacity does not significantly influence utilization of evaluation results was tested at α = .05 level of significance and was rejected since P = 0.000 < 0.05. The study findings demonstrate that there is a significant relationship between organizational capacity and the utilization of evaluation results. This research therefore reinforces literature and helps to understand the ways in which organizational capacity influences the utilization of evaluation results. It thus informs the evaluation field of study, fills gaps in the evaluation use literature and contributes to the appreciation of factors that predict and enhance the utilization of evaluation results.
References
African Water Facility. (2014). Project Appraisal Report, Innovative Sanitation Value Chain for the Un-sewered Urban Poor in Kisumu, Kenya. https://www.africanwaterfacility.org/fileadmin/uploads/awf/Projects/AWF-Project-Appraisal-Report-KENYA-KIWASCO.pdf. Accessed on 15 March, 2017
BaTall, O. 2009. The Role of National, Regional and International Evaluation Organizations in Strengthening Country-led Monitoring and Evaluation Systems. In Segone, M. 2009. (Ed.). Country-led monitoring and evaluation systems: Better evidence, better policies, better development results. New York: UNI CEF.
Baughman, S. (2010). Non-formal Educator Use of Evaluation Findings: Factors of Influence. PhD Dissertation. Virginia Polytechnic Institute and State University 103
Bourgeois, I., & Whynot, J. (2018). Strategic Evaluation Utilization in the Canadian Federal Government. Canadian Journal of Program Evaluation, 32.3, 327–346
Boyle, R. & Lemaire, D. (Eds.), Building effective evaluation capacity. New Brunswick, NJ: Transaction Publishing. Bryan, T.K. (2011). “Exploring the Dimensions of Organizational Capacity for Local Social Service Delivery Organizations Using a Multi-method Approach”. PhD Dissertation. Virginia Polytechnic Institute and State University
Burr, E. M. (2009). “Evaluation Use and Influence among Project Directors of State GEAR UP Grants”. PhD diss., University of Tennessee. http://trace.tennessee.edu/utk_graddiss/15
Christensen, R. K., & Gazley, B. (2008). Capacity for public administration. Analysis of meaning and measurement. Public Administration and Development, 28.
Christie, C. A. & Alkin, M. C. (2008). Evaluation theory tree re-examined. Studies in Educational Evaluation, 34, 131-135.
Concern Worldwide. (2014) SQUEAC REPORT, OTP PROGRAM Kisumu East District, Kisumu County, Kenya Samuel Kirichu, February 2013. http://www.coverage-monitoring.org/wp-content/uploads/2015/07/Kisumu-East-OTP_-Feb-2013_-Kenya_-SQUEAC.pdf
Cousins, J. B. and Leithwood, K. (1986). Current Empirical Research on Evaluation Utilization. Review of Educational Research 56(3): 331–364.
Cousins, J. B., Goh, S. C., Elliott, C. J., & Bourgeois, I. (2014). Framing the capacity to do and use evaluation. New Directions for Evaluation, 2014(141), 7–23.
Cronbach, L. J., & Azuma, H. (1962). Internal-consistency reliability formulas applied to randomly sampled single-factor tests: An empirical comparison. Educational and Psychological Measurement, volume 22
Eisinger, P. (2002). Organizational capacity and organizational effectiveness among street-level food assistance programmes. Non-profit and Voluntary Sector Quarterly, 31(1).
European Policy Evaluation Consortium (EPEC) (2005). Study on the Use of Evaluation Results in the Commission. Final Report, Dossier no.1: Synthesis Report and Annexes
Fleischer, D. N. & Christie, C. A. (2009). Evaluation use: Results from a survey of U.S. American Evaluation Association members. American Journal of Evaluation, 30(2).
Gary, T, H. & Melvin, M. (2003). Beyond Use: Understanding Evaluation’s Influence on Attitudes and Actions. American Journal of Evaluation, Vol. 24, No. 3
Goldman, I., Mathe, J.E., Jacob, C., Hercules, A., Amisi, M., Buthelezi, T. et al., 2015. Developing South Africa’s national evaluation policy and system: First lessons learned. African Evaluation Journal 3(1)
Gorgens, M., & Kusek, J.Z. (2010). Making Monitoring and Evaluation Systems Work: A Capacity Development Toolkit. Washington D.C, World Bank
Greene, J. C. (1988). Stakeholder participation and utilization in program evaluation. Evaluation Review, volume12.
Henry, G. (2003). Influential evaluations. American Journal of Evaluation, 24(4)
Hudson-Mabbs, S. (1993). Influences on the Use of Evaluation Information. Unpublished medical thesis. Murdoch University, Western Australia.
Ingraham, P. W., & Joyce, P. G. (2003). Governance performance: Why management matters. Baltimore, M.D.: The John Hopkins University Press.
Johnson, K., Greenseid, L. O., Toal, S. A., King, J.A., Lawrenz, F. &Volkov, B. (2009). Research on evaluation use: A review of the empirical literature from 1986 to 2005. American Journal of Evaluation, 30(30)
Khan, M. Adil. 1992. Monitoring and evaluation of development projects in South East Asia: the experience of Indonesia, Malaysia, the Philippines and Thailand (English). WBI working paper. Washington, DC: World Bank.
Kirori, G. N., (2015). Rural Development Policies in Kenya: A Descriptive Analysis Over 1970-2001 Period.
European Journal of Business, Economics and Accountancy Vol. 3, No. 1, 2015.
Krejcie, R. V., & Morgan, D. W. (1970). Determining sample size for research activities. Educational and Psychological Measurement, 30, 607-610.
Letts, C., Ryan, W. P., & Grossman, A. (1999). High performance nonprofit organizations. New York: John Wiley.
Lowden, C. (2005). Evaluating the impact of professional development. Journal of Research in Professional Learning
Magno, C., & Gonzales, R. D.L.C. (2011). Measurement and evaluation in the Philippine higher education: Trends and development. In E. Valenzuela (Ed) UNESCO Policy Studies: Trends and development in Philippine Education. Manila: UNESCO.
Mark, M. M., Henry, G. T. & Julnes, G. (2000). Evaluation: An integrated framework for understanding, guiding and improving policies and programmes. San Francisco: Jossey-Bass.
McCormick, E. R. (1997). Factors influencing the use of evaluation results. Dissertation Abstracts International: Section A: The Humanities and Social Sciences, 58.
Mouton, C. 2010, ‘The history of programme evaluation in South Africa’, MPhil thesis, Faculty of Arts and Social Sciences. Sociology and Social Anthropology Department, University of Stellenbosch.
Nyonje, R., Kyalo D. and Mulwa A. (2012) Monitoring and Evaluation of Projects and Programmes: A Hand Book for Students and Practitioners. Nairobi: Aura Books
Ochanda, J. (2005). A study of strategic management practices in the Kenya public sector. Nairobi: Unpublished MBA project, School of Business, University of Nairobi.
Patton, M. Q. (2008). Utilization Focused Evaluation (4 ed.). Thousand Oaks: Sage
Phillips, S., Goldman, I., Gasa, N., Akhalwaya, I. & Leon, B., (2014). A focus on M&E of results: An example from the Presidency, South Africa. Journal of Development Effectiveness 6(4)
Ramírez, R., Kora, G. and Brodhead, D. (2017). Translating Project Achievements into Strategic Plans: A Case Study in Utilization-Focused Evaluation. Journal of Multi-Disciplinary Evaluation Volume 13, Issue 28, 2017
Ramírez, R., Kora, G. and Shephard, D. (2015). Utilization Focused Developmental Evaluation: Learning Through Practice. Journal of Multi-Disciplinary Evaluation. Vol. 11, Issue 24
Republic of Kenya, (2007). Master plan for Implementation of NIMES, Nairobi: Government Printer.
Republic of Kenya. (2014). Second Annual Progress Report on the Implementation of the First Medium Term Plan (2008-2012), May 2011. Nairobi: Republic of Kenya.
Republic of Kenya. (2017). Performance Management Framework for County Governments. Council of Governors. Republic of Kenya. th
Sekaran, U. (2006). Research Methods for Business, Skills Building Approach. 4 edition. Heel print- N-Park, Noida.
Shadish, W. R., Cook, T. D., & Leviton, L. C. (1991). Foundations of program evaluation: Theories of practice. Newbury Park, CA: Sage.
Shulha, L. M., & Cousins, J. B. (1997). Evaluation use: Theory, research, and culture since 1986. American Journal of Evaluation, 18
Stolyarenko, K., (2014). National evaluation policy in Malaysia, Parliamentary Forum for Development Evaluation. Retrieved from: http://betterevaluation.org/node/add/resources [18 Aug. 2020]
Stufflebeam, D. L. & Shinkfield, A. J. (2007). Evaluation theory, models, & applications. San Francisco, CA: Jossey-Bass.
Stufflebeam, D.L., Madaus, G.F., & Kellaghan, T. (2000). Evaluation models: Viewpoints on educational and human services evaluation. (2nd eds.). Boston: Kluwer Academic Publishers. 105
Suarez-Balcazar, Y., Taylor-Ritzler, T., Garcia-Iriarte, E., Keys, C. B., Kinney, L., & Ruch-Ross, H., Curtin, G. (2010). Evaluation capacity building: A culturally- and contextually-grounded interactive framework and exemplar. In F.
Balcazar, Y. Suarez-Balcazar, T. Taylor-Ritzler, & C. B. Keys (Eds.), Race, culture and disability: Rehabilitation science and practice. Sudbury, MA: Jones and Bartlett. Torres, R. T., & Preskill, H. (2001). Evaluation and organizational learning: Past, present, and future. American Journal of Evaluation, volume, 22(3).
United Nations Development Program (UNDP). (2008). Capacity development—Practice note: 4.
United States Agency for International Development- USAID (2014) CAPACITY KENYA END OF PROJECT EVALUATION FINAL REPORT. http://pdf.usaid.gov/pdf_docs/pa00jrd2.pdf. Accessed on 15th March, 2017
Yu-Lee, R. T. (2002). Essentials of capacity management. New York: John Wiley & Sons.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2021 Edwin Ochieng Okul
This work is licensed under a Creative Commons Attribution 4.0 International License.
