Explore the available tools used in system evaluation, focusing on the tools listed on the AHRQ website.

Assignment 3: Application: Evaluation Project
Part 5: Evaluation Tool

Proper measurement requires identifying the correct measurement tool. Would you measure a patient’s temperature using a meat thermometer? Without the correct tool, any answer that you generate to a question of interest will have little or no basis in reality. Similarly, when seeking to answer a PICO question, you must ensure that you have the right evaluation tool for the situation. An evaluation tool should make reliable and valid measurements that are relevant to the question at hand. In this part of your Evaluation Project, you select an evaluation tool that you believe to be most appropriate for use in addressing your PICO question. You also develop a plan for utilizing the selected tool in your evaluation process.

 

To prepare:

 

• Review the concepts of reliability and validity as covered in this week’s Learning Resources.
• Review the research design of your Evaluation Project.
• Think about criteria you may use to define the success of your evaluation plan. What tool could measure these criteria?
• Explore the available tools used in system evaluation, focusing on the tools listed on the AHRQ website.
• Select an evaluation tool that you believe would be most appropriate for your Evaluation Project.

 

To complete Part 5 of this Evaluation Project:

By Day Thursday 1/18/17

In a 2- to 3-page paper,

1)      Describe the evaluation tool that you selected for your Evaluation Project.

2)      Provide a rationale for your selection, and summarize the criteria you will use to define the success of the evaluation.

3)      Develop a plan for utilizing the tool for your Evaluation Methodology Plan. If the tool that you selected is not on the AHRQ website, provide evidence that it is a valid and reliable tool.

 

Required Readings

 

Friedman, C. P., & Wyatt, J. C. (2010). Evaluation methods in biomedical informatics (2nd ed.). New York, NY: Springer Science+Business Media, Inc.

Chapter 5, “Measurement Fundamentals” (pp. 113–144)

This chapter details the importance of reliability and validity in measurement in informatics. It defines the different types of validity and introduces statistical equations for accurately determining the reliability and validity of a research finding.

 

Anderson, E. F. (2011). A case for measuring governance. Nursing Administration Quarterly, 35(3), 197–203.

Retrieved from the Walden Library databases.

The author of this article describes the Index of Professional Nursing Governance, which is a tool used to measure the degree and implementation of shared governance. The author presents a case in which the index was used in a hospital to assess the degree of shared governance over time.

 

Kaphingst, K. A., Kreuter, M. W., Casey, C., Leme, L., Thompson, T., Cheng, M. R., et al. (2012). Health literacy INDEX: Development, reliability, and validity of a new tool for evaluating the health literacy demands of health information materials. Journal of Health Communication, 17(Supp 3), 203–221.

Retrieved from the Walden Library databases.

 

In this article, the authors describe the development, refinement, and testing of Health Literacy INDEX, a tool designed to determine criteria for judging the quality of health information materials.

 

 

Penfold, R. B., Kullgren, J. T., Miroshnik, I., Galbraith, A. A., Hinrichsen, V. L., & Lieu, T. A. (2011). Reliability of a patient survey assessing cost-related changes in health care use among high deductible health plan enrollees. BMC Health Services Research, 11(1), 133–143.

Retrieved from the Walden Library databases.

This article describes a study that sought to determine the reliability of survey questions used to measure the behaviors and knowledge of high-deductible health plan beneficiaries. The authors of the article highlight their findings after conducting control and trial surveys among the beneficiaries.

 

 

Seto, I., Foisy, M., Arkison, B., Klassen, T., & Williams, K. (2012). The evaluation of an evidence-based clinical answer format for pediatricians. BMC Pediatrics, 12, 34–41.

Retrieved from the Walden Library databases.

The authors of this article describe a tool (Clinical Answers) that supplies the user with summaries of key medical evidence-based practices. They highlight a survey given to pediatricians to examine their use of Clinical Answers and to garner ways the product could be improved.

 

 

Agency for Healthcare Research and Quality. (n.d.). Health IT survey compendium.  Retrieved from http://healthit.ahrq.gov/portal/server.pt/community/health_it_tools_and_resources/919/health_it_survey_compendium/27874

 

Required Media

Laureate Education (Producer). (n.d.f). Reliability and validity. Retrieved from CDN database. (NURS 6431)

This video explores the concepts of reliability and validity in measurement tools. The video stresses the importance of reliability and validity when selecting an appropriate tool to evaluate a PICO question.