Frequently Asked Questions (FAQs)
Why do I need to do an evaluation?
Evaluation forms part of the evidence base that helps us to work towards our population receiving ‘the right care, in the right place at the right time’ and is an intrinsic part of evidence informed commissioning.
Evaluation can help commissioners to understand whether a service works, how it works, if not, why not, as well as identify intended or unintended outcomes and areas for improvement.
There is considerable evidence that the findings of evaluations often influence policy, and service planning and implementation (Rossi, Lipsey and Freeman 2004). This means evaluation is a very valuable tool. The knowledge and information gained can benefit your service, patients and service users, staff, the CCG and the NHS as a whole. The key benefits are that the evaluation informs service planning, provides evidence of success and gives feedback to all those involved (Meyrick and Sinker 1992).
An important question to ask yourself is: “Can I afford not to do an evaluation?” (Magenta Book, 2011)
For more information see the APCRC Evaluation toolkit in the resources section.
What resources do I need to do my evaluation? How much will it cost?
A common question is how much of the budget should be spent on the evaluation. This very much depends on the purpose and nature of the evaluation. However, we tend to recommend that for projects in the thousands allocate approximately 10% of the total value towards the evaluation, those in the millions tend to be more towards the region of 3-5%.
For example a small pilot service worth around £20,000 would allocate £2000 whereas a £10million pound service would allocate nearer £400,000. This will also impact on the type of evaluation that is conducted, with the small scale pilot project likely to conduct an internal evaluation utilising existing resources whereas the large scale service is more likely to conduct an independent evaluation using external resources.
This reflects similar recommendations from the World Health Organisation and Big Lottery Fund. For more information see the APCRC Evaluation toolkit in the resources section.
Who can help me with my evaluation?
There are a number of people in your own organisation who may be able to help you, these include your Research, Development and/or evaluation team (don’t forget if you work for Bristol, North Somerset or South Gloucestershire CCG to come and see the APCRC www.apcrc.nhs.uk), Quality Improvement leads or programme and project managers experienced in evaluation.
You may also want to seek external help from experts and there are a number of local networks and organisations that may be able to help you including the Collaboration for Applied Health Research and Care West (CLAHRCwest – http://clahrc-west.nihr.ac.uk/), West of England Academic Health Science Network (West of England AHSN – http://www.weahsn.net/), Bristol Health Partners (http://www.bristolhealthpartners.org.uk/) or your local University.
You may however which to procure an external independent evaluation, if so see FAQ 4.
I want to commission an independent evaluation - what do I do?
You will need to comply with your own organisations procurement processes and so we recommend that you contact your procurement lead/team in the first instance as they will be able to advise. We recommend that you also engage an evaluation expert as a part of the procurement process.
Our template for invitation to tenders (see the plan section of the toolkit) provides a clear brief outlining purpose and requirements. We recommend you are not prescriptive regarding the methodology so that those bidding to provide the evaluation can demonstrate their expertise and allows for appropriate innovation. However, use your organisations procurement team as they will ensure that you comply with all the legal and organisations requirements.
In terms of contracts, if it is for an NHS service you will generally use the NHS Terms and Conditions for the Provision of Services: https://www.gov.uk/government/publications/nhs-standard-terms-and-conditions-of-contract-for-the-purchase-of-goods-and-supply-of-services
Do I need permission from anyone to do my evaluation?
The short answer is yes. You will need to seek agreement from the organisations and people involved. This may be through formal governance systems or by seeking informed consent from those who are being asked to participate.
How can I share my findings from my evaluation?
See our Review and Act page as they provide useful tools to help you. Speak to your communications team and see how they can help you as well as use resources from other local initiatives, such as Knowledge Mobilisation, that is a partnership between the APCRC, University of Bristol and University of the West of England, Bristol. For more information please see our Evidence Culture page.