The field trial was conducted to establish the reliability in producing similar results between evaluators of the proposed new tools for Monitoring and Evaluation (M&E) of the nutrition plans and programs in the Local Government Units (LGUs). To do this, orientation activities were conducted to familiarize the 46 M&E team (MET) members evaluating the provincial, municipal, city, and barangay levels in two regions with the proposed tools during the field trial. After the event, the perceptions of the MET members of the tools were gathered by asking them to rate the tools through a self-administered questionnaire, and by noting their written and verbal commentaries about the proposed system. During the field trial, each MET member, as well as the member of the Project Team (PT), individually evaluated the LGUs using
the tools. Secondary data on the LGUs performance scores using the old system were also gathered. The MET members’ perception was examined based on the median rank of their ratings and content analysis of their insights about the tools, whereas the reliability of the tools was assessed based on the interrater reliability of the MET members’ scores for the LGUs analyzed through paired samples t-Test, Pearson correlation coefficient, intraclass correlation coefficient, and technical error of measurement. The weighted scores of the MET and PT members were also compared. Moreover, the difference in the generated scores between the old and the new system was determined. The findings revealed that the MET members generally have a positive perception of the new system but raised some issues and concerns. Although the reliability of the tools was generally observed, actions are warranted for improvement. The tools generated statistically different scores when used by MET and PT members, and when compared to the existing system. Steps should be taken to improve the reliability of the proposed tools.
Burns MK. 2014. How to establish interrater reliability. Nursing2020 44(10):56‒58. https://doi.org/10.1097/01.NURSE.0000453705.41413.c6
Drummond KE, Murphy-Reyes A. 2018. Nutrition Research: Concepts & Applications. 1st Edition. Massachusetts (USA): Jones & Bartlett Learning.
Gawe HC. 2015. Updated Nutrition Monitoring and Evaluation Protocol – Local Government NME Guidelines. Presented at the 5th National Conference of the District/City Nutrition Program Coordinators, 3‒4th of November. Metro Manila, Philippines.
George HA, Davis SL, Mitchell CF, Moyer NM, Toner CG. 2013. Abstraction of core measure data: Creating a process for interrater reliability. J Nurs Care Qual 28(1):68‒75. https://doi.org/10.1097/NCQ.0b013e31826ee49e
Kang Y, Cho M, Rahman MM, Cho Y, Han S, Dutta ML. 2021. Design of a collaborative monitoring and evaluation system for a community-based nutrition project in rural Bangladesh. Eval Program Plann, 84:101892. https://doi.org/10.1016/j.evalprogplan.2020.101892
Jefferds MED, Flores-Ayala R. 2016. Chapter 5.6 What gets measured gets done: How nutrition monitoring, impact evaluation, and surveillance can support program improvement and policy development. In Eggersdorfer M, Kraemer K, Cordaro JB, Fanzo J, Gibney M, Kennedy E, Labrique A, Steffen J. Good Nutrition: Perspectives for the 21st Century. Basel, Karger (301–311). https://doi.org/10.1159/000452394
Lange RT. 2011. Inter-Rater Reliability in Encyclopedia of Clinical Neuropsychology. New York (USA): Springer New York.
Micah NJ, Luketero SW. 2017. Monitoring and evaluation systems and performance of non-governmental based maternal health projects in Bungoma South sub-county, Kenya. Eur Sci J 13(23):11‒38. https://doi.org/10.19044/esj.2017.v13n23p11
O'Sullivan RG. 2012. Collaborative evaluation within a framework of stakeholder-oriented evaluation approaches. Eval Program Plann 35(4):518‒522. https://doi.org/10.1016/j.evalprogplan.2011.12.005
Piedra-Fernández JM, Ganoza-Guerrero GE. 2016. Field trials and community trials: Importance in public health. Medwave 16(11):e6797. https://doi.org/10.5867/medwave.2016.11.6797
Sattler DN, McKnight PE, Naney L, Mathis R. 2015. Grant peer review: Improving inter-rater reliability with training. Plos One 10(6):e0130450. https://doi.org/10.1371/journal.pone.0130450
Smith PG, Morrow RH, Ross DA. 2015. Field Trials of Health Interventions: A toolbox. 3 ed. Oxford (UK): Oxford University Press.
Vidyarini A, Martianto D, Syarief H. 2021. Evaluation of food and nutrition security level at provincial level based on outcome indicators in Indonesia. J Gizi Pangan 16(1):1‒10. https://doi.org/10.25182/jgp.2021.16.1.1-10
Wandner SA. 2017. Lessons Learned from Public Workforce Program Experiments. Kalamazoo (USA): W.E. Upjohn Institute for Employment Research.