Section 1: What is Research & Service Evaluation?

This Research and Service Evaluation Framework is developed by TEC Cymru’s research and evaluation team and is based upon the team’s extensive knowledge and experience. The framework has six sections:

Section 1: What is Research & Service Evaluation?

Section 2: What is Quality Improvement?

Section 3: TEC Cymru’s Four-Step Phased Approach

Section 4: Using Mixed Methodologies

Section 5: Using Patient & Public Involvement (PPI)

Sections 6: Useful Links & Templates

The framework provides hyperlinks throughout for additional information and points of reference.

This framework has been created to inform anyone undertaking a digital transformation in how to use research and service evaluation methods. This will help decision making, justification, and to measure whether value has been achieved.

Undertaking a project without an approach to research and service evaluation can result in a lack of evidence, lessons learned, and documentation of the success (or failure) of the innovation - all of which are important to encourage future investment. 

This framework will be shared, tested and iterated over time with digital transformation teams – it is a work in progress!

The Health Research Authority in the UK has a useful online decision-making tool to help people determine if their work sits under a research or service evaluation umbrella—see here.

A helpful definition of research is: “Research involves the attempt to extend the available knowledge by means of a systematically defensible process of enquiry.” (Clamp et al., 2004).

A helpful definition of evaluation is: “Evaluation is a systematic assessment of the design, implementation and outcomes of an intervention” (Magenta Book, 2020).

Table 2: Research & Evaluation

Research and service evaluation are similar yet mutually independent. They share similar steps in their process and can complement each other well. As shown below in Diagram 1, the difference occurs at the start and finish of the process, whereas the similarities sit within the core (methods/analysis).

Diagram 1: Research & Evaluation Similarities and Differences

Research is focused on producing generalizable knowledge which is empirical, theoretical, and controlled by the researchers (ensuring non-bias in findings). Service Evaluation is generally focused on specific and applied knowledge, aiming to draw evaluative conclusions about quality or worth and is controlled by those funding or commissioning the evaluation (potentially enabling more bias in findings). Evaluation has two main uses – providing evidence of a project’s overall impact and cost effectiveness as well as identifying what can be improved, developing evidence for future projects.

To get the best out of a research and evaluation component of an intervention or programme using both approaches can have many advantages. As standalone activities they can have limitations, e.g., evaluation that is not research involves making judgements without systematic collection of data. Research that is not evaluation can take a lot of time and cost to design and prepare, often unable to present any outcomes until the end of the process which make improvements along the way impossible. An example of an overlap methodology/analysis which complement each other well is a Four-Phased Quality Improvement (QI) Approach. This is discussed in the next sections.

For more information on ‘what is evaluation’ and ‘what to consider when planning an evaluation’, watch these short videos:

What is Evaluation?

What to consider when planning an evaluation?