Nick Axford - evidence-based prevention [March 7 ADEPIS seminar]
Download as PPTX, PDF2 likes716 views
The document discusses the concept of evidence-based programs (EBPs) for children and families, detailing their defined structures, effectiveness, and the challenges faced in implementation, including limited impact and issues with fidelity. It emphasizes the importance of engaging parents and providers for the success of interventions, and presents a framework of prevention strategies ranging from universal promotion to targeted treatment. Additionally, it critiques the current state of EBPs, advocating for a reinvention and improvement of these programs based on new insights and practices.
![Nick Axford - evidence-based prevention [March 7 ADEPIS seminar]](https://image.slidesharecdn.com/namentorukliverpool7thmarch2016-160316151401/85/Nick-Axford-evidence-based-prevention-March-7-ADEPIS-seminar-3-320.jpg)
![Nick Axford - evidence-based prevention [March 7 ADEPIS seminar]](https://image.slidesharecdn.com/namentorukliverpool7thmarch2016-160316151401/85/Nick-Axford-evidence-based-prevention-March-7-ADEPIS-seminar-11-320.jpg)
![Selected sources
Hanleybrown, F., Kania, J. and Kramer, M. (2012) Channelling Change: Making Collective Impact Work. Stanford Social Innovation Review.
http://ssir.org/articles/entry/channeling_change_making_collective_impact_work [Accessed 8th February 2016]
Kemp, F., Ohlson, C., Raja, A., Morpeth, L. & Axford, N. (2015) Fund-mapping: The Investment of Public Resources in the Wellbeing of Children
and Young People in Northern Ireland. Belfast: NICCY
Langford, R., Bonell, C.P., Jones, H.E., Pouliou, T., Murphy, S.M., Waters, E., Komro, K.A., Gibbs, L.F., Magnus, D., and Campbell, R. (2014), The
WHO Health Promoting School framework for improving the health and well-being of students and their academic achievement, Cochrane
Database of Systematic Reviews 2014, Issue 4.
Lipsey, M., Howell, J. C., Kelly, M. R., Chapman, G., & Carver, D. (2010). Improving the effective- ness of juvenile justice programs: A new
perspective on evidence-based practice. Center for Juvenile Justice Reform: Georgetown Public Policy Institute, Georgetown University.
O’Connell, M. E., Boat, T. and Warner, K. E. (Eds.) (2009) Preventing Mental, Emotional, and Behavioural Disorders Among Young People:
Progress and Possibilities. Washington DC: The National Academies Press.
Santucci, L. C., Thomassin, K., Petrovic, L. & Weisz, J. R. (2015) Building evidence-based interventions for the youth, providers, and contexts of
real-world mental-health care. Child Development Perspectives, 9 (2), 67-73.
Shonkoff, J. P. and Fisher, P. A. (2013) Rethinking evidence-based practice and two-generation programs to create the future of early
childhood policy. Development and Psychopathology, 25, 1635-1653.](https://image.slidesharecdn.com/namentorukliverpool7thmarch2016-160316151401/85/Nick-Axford-evidence-based-prevention-March-7-ADEPIS-seminar-21-320.jpg)
Nick Axford - evidence-based prevention [March 7 ADEPIS seminar]
- 1. Outline 1. Programmes and standards of evidence 2. Where do targeted interventions fit? 3. A key challenge for targeted provision 4. A brave new world?
- 2. Evidence-based programmes (EBPs) • A programme is a discrete, organised package of practices, spelled out in guidance (sometimes called a manual) that explains what should be delivered to whom, when, where and how. • A programme is ‘evidence-based’ when it is ‘tested and effective’: - Tested’ means that the programme has been put through its paces by a high-quality impact evaluation - ‘Effective’ means that there is strong evidence from that evaluation that the programme makes life better for children or families
- 4. Standards of evidence 1. Intervention specificity – what is it? 2. Impact – does it work? 3. Evaluation quality – can we be confident in the results? 4. System readiness – can it be replicated? www.investinginchildren.eu/standards-evidence
- 5. Continuum of prevention 1. Promotion – universal (promoting good) 2. Universal prevention – universal (preventing bad) 3. Selective prevention – elevated risk (group) 4. Indicated prevention – elevated risk (individual) / early problems 5. Treatment – established disorder 6. Maintenance – preventing relapse O’Connell et al. (2009)
- 6. Spend on children in Northern Ireland Executive Departments Levels 1-2 Levels 3-4 Levels 5-6 Not disaggregated £m £m £m £m Agriculture & Rural Development 1.34 Culture, Arts & Leisure 17.39 0.65 0.05 Education 1,193.99 212.41 0.97 225.24 Employment & Learning 48.95 54.41 0.03 Enterprise, Trade & Investment Finance & Personnel Health, Social Services, Safety 29.35 17.00 424.08 13.57 Environment 3.06 0.05 Justice 0.80 14.90 0.03 Regional Development 7.67 Social Development 7.71 0.77 First & deputy First Minister 0.25 1.57 Total 1,301.75 293.28 440.75 240.46 Percentage split 57% 13% 19% 11%
- 7. 4 circles Children in need Children in contact with specialist services Children with informal support
- 8. Provide more services 4 circles
- 12. KiVa – bullying prevention • “Bullying”: verbal/psychological/physical behaviour, designed to cause harm/distress, direct/indirect, repeated over time, power differential • Universal element: - Classroom lessons (10 over year) - Posters, playground monitors - Parent engagement • Targeted element: - Structured process to deal with bullying incidents
- 13. Hard-to-reach families, or hard-to-access services?
- 14. What we learnt (and maybe should have known before…) 1. If we don’t engage the right parents, the programme won’t work 2. Engage providers first if you want them to engage parents 3. Have a clear recruitment process and train everyone involved 4. Invest in and incentivise recruitment and retention 5. Get out there! Go to parents; don’t expect them to come to you 6. Build relationships: visit, call, then visit and call again 7. Be practical: make it attractive and easy to come along 8. Be creative! 9. Recruitment is nothing without retention 10. Aim high but be realistic: life gets in the way of the best intentions
- 15. What’s up with evidence-based programmes? 1. Limited impact: no “breakthrough outcomes”, poor transportability 2. “Pseudo-science”: cherry-picking, developer bias 3. RCTs are not the gold standard 4. Fiddling trumps fidelity 5. Poor fit with systems (therefore not “scalable”) 6. Automatons replacing autonomy 7. Limit innovation 8. “Privatised” solutions to “private” problems 9. Technical not relational 10. ‘Real’ children and ‘My children’
- 16. Poor tansportability? Programme Name Effects in the US Number of studies in the US Effects in Europe Number of studies in Europe Big Brothers Big Sister Positive effect 4 No effect 1 Functional Family Therapy Positive effect 8 Positive effect 3 Good Behaviour Game Positive effect 5 Positive Effect 4 Incredible Years Parent Training Programme Positive effect 26 Positive effect 18 Incredible Years Child Training Programme Positive effect 5 No effect 3 Multisystemic Therapy Positive effect 12 Mixed effects 5 Nurse Family Partnership Positive effect 3 Mixed effects 4 Promoting Alternative Thinking Strategies (PATHS) Positive effect 8 Mixed effects 6 Multidimensional Treatment Foster Care (Oregon) Positive effect 8 Mixed effects 4 Strengthening Families 10-14 Positive effect 2 No effect 3 Programme Effect Size in the US Effect Size in Europe Countries Included Functional Family Therapy -0.09 to -0.59 -0.96 Sweden Good Behaviour Game -0.37 -0.35 The Netherlands Incredible Years Parent Training Programme -0.02 to -1.18 -0.10 to -0.72 UK, Norway Multisystemic Therapy -0.13 to -1.74 --0.40 UK Multidimensional Treatment Foster Care (Oregon) -0.49 to -1.6 -0.92 UK (In Sweden, the effect on internalizing (ES = -0.39) and externalizing (ES = -0.58) behaviour was measured.) *Effect sizes based on effects on primary outcomes. Some effect sizes have been calculated by the Washington State Institute of Public Policy
- 17. Developer bias? Programme Findings in developer-led studies Findings in independent evaluations Reconnecting Youth (Drug prevention programme) Increased GPA; increased self- esteem; increased school bonding; decreased hard drug use; and decreased drug control problems (Eggert et al. 1994) Negative effects on most outcome measures, no positive effects. Negative effects the stronger the better implementation fidelity (Sanchez et al. 2007) Triple P Positive Parenting Programme Positive mean effect on child problem behavior of d=0.35 in 33 trials (Nowak and Heinrichs 2008) No positive effects on any aspect of problem behavior evaluated by teachers, parents, or child self-reports (Eisner et al. 2007) Olweus Bullying Prevention Programme Reductions of up to 50% in bullying in the original study (Olweus 1994) No overall effects on either attitudinal measures or victimization (Bauer et al. 2007) ALERT (Drug prevention programme) Reduction in cigarette, marijuana and alcohol use by 19–39% (Ellickson et al. 2003) No effects on mediators or substance abuse itself (St Pierre et al. 2006) Source: Eisner, M. (2009). No effects in independent prevention trials: can we reject the cynical view?. Journal of Experimental Criminology, 5(2), 163-183.
- 18. Towards EBP 2.0? 1. Do the same but better (e.g. progressive standards, implementation) 2. Re-invent EBPs (e.g. build on neuroscience, build in flex) 3. Develop and apply evidence-based “kernels” and “bundles” 4. Make “services as usual” more evidence-based (e.g. Lipsey, PDSA, rct) 5. Engage multiple systems (e.g. HPS, “collective impact”) 6. Explore common logic models / meta-theory 7. Segment, and then target better 8. Improve prevention training for practitioners 9. Empower civil society (e.g. community engagement, kernels) 10. Reform systems (e.g. de-commission, earmarking)
- 19. Evidence-based kernels A “fundamental unit of behavioural influence” that underlie effective prevention and treatment and a small and simple entity that holds the potential to transform into something much larger. They must: • be inexpensive; • show an immediate effect; • be easily useable; and • be flexible to simultaneously solving additional problems that might arise in the course of prevention or treatment • be empirically found to be effective • be indivisible (i.e. it would not work if altered).
- 20. Selected sources Axford, N., Lehtonen, M., Tobin, K., Kaoukji, D. & Berry, V. (2012) ‘Engaging parents in parenting programs: lessons from research and practice’, Children and Youth Services Review, 34 (10), 2061-2071. Axford, N. and Morpeth, L. (2013) ‘Evidence-based programs in children’s services: a critical appraisal’, Children and Youth Services Review 35 (1), 268-277. Davies, F. A., McDonald, L. & Axford, N. (2012) Technique is Not Enough: Making Evidence-based Programmes Socially Inclusive. Discussion Paper for the British Psychological Society Professional Practice Board’s Social Inclusion Group. Leicester, BPS. Durlak, J. A., & DuPre, E. P. (2008). Implementation matters: a review of research on the influence of implementation on program outcomes and the factors affecting implementation. American Journal of Community Psychology, 41, 327-350. Eisner, M. (2009). No effects in independent prevention trials: can we reject the cynical view? Journal of Experimental Criminology, 5(2), 163- 183. Embry, D. D., & Biglan, A. (2008). Evidence-based kernels: fundamental units of behavioral influence. Clinical Child and Family Psychology Review, 11(3), 75-113. Gottfredson, D. C., Cook, T. D., Gardner, F. E. M., Gorman-Smith, D., Howe, G. W., Sndler, I. W. & Zafft, K. M. (2015) Standards of evidence for efficacy, effectiveness, and scale-up research in prevention science: next generation. Prevention Science 16 (7) 893-926.
- 21. Selected sources Hanleybrown, F., Kania, J. and Kramer, M. (2012) Channelling Change: Making Collective Impact Work. Stanford Social Innovation Review. http://ssir.org/articles/entry/channeling_change_making_collective_impact_work [Accessed 8th February 2016] Kemp, F., Ohlson, C., Raja, A., Morpeth, L. & Axford, N. (2015) Fund-mapping: The Investment of Public Resources in the Wellbeing of Children and Young People in Northern Ireland. Belfast: NICCY Langford, R., Bonell, C.P., Jones, H.E., Pouliou, T., Murphy, S.M., Waters, E., Komro, K.A., Gibbs, L.F., Magnus, D., and Campbell, R. (2014), The WHO Health Promoting School framework for improving the health and well-being of students and their academic achievement, Cochrane Database of Systematic Reviews 2014, Issue 4. Lipsey, M., Howell, J. C., Kelly, M. R., Chapman, G., & Carver, D. (2010). Improving the effective- ness of juvenile justice programs: A new perspective on evidence-based practice. Center for Juvenile Justice Reform: Georgetown Public Policy Institute, Georgetown University. O’Connell, M. E., Boat, T. and Warner, K. E. (Eds.) (2009) Preventing Mental, Emotional, and Behavioural Disorders Among Young People: Progress and Possibilities. Washington DC: The National Academies Press. Santucci, L. C., Thomassin, K., Petrovic, L. & Weisz, J. R. (2015) Building evidence-based interventions for the youth, providers, and contexts of real-world mental-health care. Child Development Perspectives, 9 (2), 67-73. Shonkoff, J. P. and Fisher, P. A. (2013) Rethinking evidence-based practice and two-generation programs to create the future of early childhood policy. Development and Psychopathology, 25, 1635-1653.
Editor's Notes
- #7: Levels 1 & 2 – services for all v – universal promotion or universal prevention Levels 3 & 4 – targeted services – providing early intervention for either children with emerging needs in targeted areas or individual targeted children with specific emerging needs Levels 5 & 6 – services for children with established high levels of need – short term needing hihg levels of intervention or long term needing high levels of maintenance
- #17: When effective programmes developed in the US are transported to Europe, they are not always effective. However, when the programme works in a different context, the effect sizes are not necessarily reduced. Big Brothers Big Sisters – EU study included conducted in Ireland FFT: 2 Swedish studies had a positive impact, 1 study in Ireland also had a positive impact (but issues with analysis) GBG: Positive effects on children were found in 3 studies in the Netherlands, 1 in Belgium; however no impact on teacher outcomes in 2 of the 4 studies MST: 1 Norwegian study had mixed effects, while another Norwegian study had positive effects; 1 UK and 1 Norwegian study had positive effects. The Swedish study has no effects NFP: 1 study in the Netherlands had positive effects, 2 studies in Germany had mixed effects, UK study showed no effects IY Child: no effect in Norwegian study and UK study. Effect on some secondary outcomes in Irish study. PATHS: positive effect in 1 UK study, no effect in three UK studies, mixed effect in Switzerland, no effect in the Netherlands; other studies not in english. MTFC: effect in one UK study, no effect in another UK studies no effect in one Swedish study and mixed effects in another Swedish study Strengthening Families: no effect in UK, Ireland or Sweden For the effect sizes, only studies where a positive significant effect was found in the EU study are considered
- #21: Possibly still hold the potential to change the face of children's services, and to an extent have proved instrumental in encouraging efficiency and positive outcomes for children across the world. Why? They equip practitioners who work with children with clear and manualised approaches and instructions, EBPs set a standard for prevention and treatment not seen before in the 20th century.
- #22: Possibly still hold the potential to change the face of children's services, and to an extent have proved instrumental in encouraging efficiency and positive outcomes for children across the world. Why? They equip practitioners who work with children with clear and manualised approaches and instructions, EBPs set a standard for prevention and treatment not seen before in the 20th century.
- #23: Possibly still hold the potential to change the face of children's services, and to an extent have proved instrumental in encouraging efficiency and positive outcomes for children across the world. Why? They equip practitioners who work with children with clear and manualised approaches and instructions, EBPs set a standard for prevention and treatment not seen before in the 20th century.