More Related Content
Similar to Economic evaluation: overview (20)
Economic evaluation: overview
- 1. Martin Knapp Personal Social Services Research Unit, London School of Economics & Political Science & NIHR School for Social Care Research Economic evaluation: overview Economic evaluation: overview What Works Centre for Wellbeing London, 4 October 2017
- 2. 2 Economic evaluation: basic building blocksEconomic evaluation: basic building blocks Outcomes? Before-after change? Comparative to what? Trade-offs? Costs?
- 3. Economic evaluation: overview (A-E) In context: other measures (F-G) Economic evaluation: overview (A-E) In context: other measures (F-G) A. Why economic evaluation? B. Cost measurement C. Outcome measurement D. Study design and methods E. Making trade-offs Later (…after Paul…): F. Example using a randomised trial G. Example using modelling Image from Flat Icon http://www.flaticon.com/
- 5. Why? oBecause resources are scarce. oSo we – society - cannot meet every need, or agree to every request, or accommodate every preference. oAnd so we – society - must choose how to get the best out of our available resources. Consequently … o… any new service or ‘intervention’ will be looked at very carefully: Is it effective? Is it affordable? Does it save money? And is it cost-effective? Decision-makers need economic evidenceDecision-makers need economic evidence Of course, there are other relevant criteria too, such as: •Respectful of individual rights, dignity, culture. •Equitable (fair) •Protects vulnerable groups •Encourages social or community cohesion •Long-term sustainability
- 6. 6 o Commissioning of services o Provision of services (in-house management) o Marketing by manufacturers o Comparison between localities or providers o Market management by public bodies o Policy development - national or local o Lobbying by interest groups o Guideline development – e.g. NICE o Regulation of service quality Uses of economic evaluative evidenceUses of economic evaluative evidence Different needs and uses could mean that different types of economic evaluative are required
- 8. 8 o Decide which costs are relevant o Collect data on service utilisation patterns and similar activity indicators o Attach suitable unit costs to those indicators Cost measurement: three stagesCost measurement: three stages
- 9. Genes Family Income Emply’t Resilience Trauma Phys env Events Chance Influences on individual needs & assetsInfluences on individual needs & assets People with needs & assets
- 10. Health care Social care Housing Education Crim justice Benefits Employment Social netw Income Mortality Genes Family Income Emply’t Resilience Trauma Phys env Events Chance Needs & assets in turn influence costsNeeds & assets in turn influence costs People with needs & assets
- 11. Health care Social care Housing Education Crim justice NHS LAs DCLG DfE MoJ Benefits Employment DWP Firms Social netw Income CVOs AllMortality Indiv Genes Family Income Emply’t Resilience Trauma Phys env Events Chance … impacting on potentially many budgets… impacting on potentially many budgets People with needs & assets
- 12. Total cost excluding benefits averaged £5,960 per child per year, at 2000/01 prices (benefits = £4307) Romeo, Knapp, Scott (2009) British J Psychiatry 2009 Example: costs of young children with persistent antisocial behaviour Example: costs of young children with persistent antisocial behaviour
- 13. o All 10-year olds in a London borough, 1970 (n=1689). Led by Michael Rutter at that time o Teacher ratings, child questionnaires o Intensively studied 50% of children with psychological problems and random 8% of others o At age 10: No problems at school, no clinical diagnosis (65) Antisocial behaviour at school, only (61) Conduct disorder (16) Emotional problems at school, only (32) Emotional disorder (8) o Followed up at age 26-28 … Research question: What services were used and what costs incurred between aged 10 and 28? Example: Longer-term economic impacts of child behavioural problems Example: Longer-term economic impacts of child behavioural problems
- 14. 0 20000 40000 60000 No problems Conduct problems Conduct disorder Criminal justice Benefits Relationships Social care Health Education Scott, Knapp, Henderson, Maughan British Medical Journal 2001 Costs (£) from ages 10 to 28 Service costs ages 10-28 by childhood antisocial behaviour at age 10 Service costs ages 10-28 by childhood antisocial behaviour at age 10
- 15. Health & social care system perspective oHome care oInpatient services oOutpatient, A&E oCommunity health oGP time oSocial work inputs oResidential care settings, etc. Public sector perspective oHealth & social care oEducation services oCriminal justice oWelfare benefits, etc. Examples of different study perspectives Cost breadth depends on study perspectiveCost breadth depends on study perspective Societal perspective oAll of the above … plus oUser & family out-of- pocket payments oLost productivity oCost of unpaid care
- 16. How many service ‘units’ does an individual utilise? E.g. how many day centre attendances, GP consultations? Where does this information come from? oUser recall: e.g. how many attendances in the past month? Face-to-face, telephone, postal, web-based oProxy recall: information from family members or service staff oCase files for individual service users oStaff files (consultation / visit records) oManagement information systems oBilling systems Utilisation patternsUtilisation patterns
- 17. o Prices or user charges if we think market forces reflect social opportunity costs o Expenditure by service providers, divided by volume of provision or number of users o Opportunity costs - the value of alternatives or opportunities missed (the benefit forgone by losing its best alternative use) … o … especially important for non-marketed inputs such as (unpaid) carer time or volunteer activities o Previously calculated ‘off-the-shelf’ unit costs – the annual PSSRU volume for health & social care is ‘priceless’ http://www.pssru.ac.uk/project-pages/unit- costs/2016/index.php Unit costs – different optionsUnit costs – different options
- 19. Ideally, they should: a.directly link to the service aims b.be influenced by involving service users c.capture impacts on everyone affected d.be quantitative … using robust measures (good psychometric properties) e.supported by qualitative evidence reflecting user experience f.assess change over time g.assess change in comparison to an alternative scenario h.allow comparison with other studies or settings Outcomes – what are they?Outcomes – what are they?
- 20. Time Measuring outcomes - 2Measuring outcomes - 2 Wellbeing
- 21. Symptoms of illness Extent of disability Needs (met, unmet) Social functioning Self-care abilities Employment & activities Behavioural characteristics Quality of life (illness- specific) Normalised lifestyle Choice & control Family well-being Carer ‘impact’ Societal perceptions Outcomes measured in terms of ‘effects’ (e.g. in health services research) Outcomes measured in terms of ‘effects’ (e.g. in health services research) Generic indicators: •Health-related quality of life (‘health’) •Quality-adjusted life years (QALYs) •Disability-adjusted life years (DALYs)
- 22. 22 These effectiveness measures are usually robust in psychometric terms, and often intuitive. But … for most of the important decisions faced by policy-makers or other decision-makers, comparisons need to be made with a common, generic numeraire. Options for evaluation: • Money • Utility (QALYs) • Wellbeing Moving towards generic outcome measuresMoving towards generic outcome measures
- 23. • Include expenditure saved … but we need to go much further • Stated preference – just ask people; but do people answer honestly or can they do so accurately? • Revealed preference – observe how people make decisions already, and infer the value they attach • Compensation settlements through litigation – unreliable • Market value of (some?) outcomes; e.g. productivity gains from higher employment rate • Social return on investment (SROI) activities try to find monetary values for wider range of outcomes Remember: Money is worth (in wellbeing terms) different amounts to different people (e.g. varies with income) Outcomes measured in terms of moneyOutcomes measured in terms of money
- 24. Utility - a generic measure combining quality and quantity of life; widely used in health services research Different dimensions of health-related QOL are combined using societal weights The QALY (quality-adjusted life year) is example of a utility measure – combines years of (extra) life with the quality of that life QALY range: 0 (death) to 1 (perfect health) Evaluation question: how many additional QALYs are generated by treatment (relative to a comparator) Most commonly used QALY-generating tool is EQ5D Outcomes measured in terms of utilityOutcomes measured in terms of utility
- 25. 25 Wellbeing as the common numeraire to reflect outcomes Practical challenges: •Which measure(s) to use? Need to be self-report •How are measures obtained in practice? •Can long-term projections of wellbeing be made? •Aggregation across individuals – possible need for equity weights and/or adjustments in analyses (e.g. subgroups) •Which decision rules given current absence of thresholds / reference points to guide decisions? Outcomes measured in terms of wellbeingOutcomes measured in terms of wellbeing
- 27. • Simple before-after calculations (with no ‘parallel’ comparison group) • Randomised trial – allocate people to interventions by chance • Quasi-experimental design – allocate people to interventions in some other way • Observational study – look at people in the groups to which they are ‘allocated’ by routine services • Mathematical modelling – simulate some parts of the evaluation using extant data Evaluation designs – to capture before & after, but also to allow comparisons Evaluation designs – to capture before & after, but also to allow comparisons Each study design has advantages & disadvantages; they cost different amounts; they take different time durations
- 29. If the policy/practice question is: ‘Does this intervention work?’ Then the economic question is: ‘Is it worth it?’ Often the decision-maker faces difficult (perhaps controversial?) trade-offs So … we must define what we mean by ‘work’ and by ‘worth’ – hence we must define outcomes and costs. The core economic questionThe core economic question
- 30. If an intervention is more effective and also more costly, then calculate the cost per unit gain in outcome (effectiveness). So … Is it worth it? Trade-offs: Is it worth it?Trade-offs: Is it worth it?
- 31. C2 - C1 New service less effective and more costly 0 E2 - E1 New service less effective but less costly New service more effective but also more costly New service more effective and also less costly C = costs E = effects 1 = old service 2 = new service Possible results from cost- effectiveness analysis Possible results from cost- effectiveness analysis A B Y Z
- 32. If an intervention is more effective and also more costly, then calculate the cost per unit gain in outcome (effectiveness). Is it worth it? • Show decision-makers the cost-effectiveness findings; ask them to choose their preferred option. (Health economists often show acceptability curves (CEACs) to highlight data uncertainty & willingness-to-pay impacts.) • Ask decision-makers to be explicit about willingness to pay. • Set a threshold, rigidly or as a guide. E.g. NICE in England & Wales uses cost per QALY to compare across disorders / diseases: current guide (whose relevance is however disputed) is £20,000 per QALY. Trade-offs: Is it worth it?Trade-offs: Is it worth it?
- 33. C2 - C1 0 E2 - E1 C = costs E = effects 1 = old service 2 = new service Using C-E thresholdsUsing C-E thresholds A B Y Z The red line represents one particular cost- effectiveness threshold: -Above the line is not cost-effective -Below the line is cost- effective The blue line represents a different cost- effectiveness threshold: -X was previously cost- effective, but is not any longer X
- 34. Various types of economic evaluation … and many different labels Various types of economic evaluation … and many different labels Cost-effectiveness analysis Cost-benefit analysis Cost-utility analysis Cost-consequences analysis Cost-minimisation analysis Cost-offset analysis Social return on investment Arguably, the label is not important so long as it is absolutely clear what is being measured and how
Editor's Notes
- #24: … adds to it with the only bit of algebra in the presentation. This ICER is the incremental cost-effectiveness ratio: it is the difference in costs between the two service options divided by the difference in costs. It is the amount that needs to be spent to achieve a 1-point improvement in the outcome, such as one additional life saved, or one additional person supported, or a 1-point improvement as measured on a quality of life scale.