Journals Library

Health Technology Assessment

Volume 25 • Issue 20 • March 2021
ISSN 1366-5278

Cognitive therapy compared with CBT

for social anxiety disorder in adolescents:
a feasibility study
Cathy Creswell, Eleanor Leigh, Michael Larkin, Gareth Stephens, Mara Violato,
Emma Brooks, Samantha Pearcey, Lucy Taylor, Paul Stallard, Polly Waite,
Shirley Reynolds, Gordon Taylor, Emma Warnock-Parkes and David M Clark

DOI 10.3310/hta25200
Cognitive therapy compared with CBT for
social anxiety disorder in adolescents:
a feasibility study

Cathy Creswell,1* Eleanor Leigh,2 Michael Larkin,3

Gareth Stephens,4 Mara Violato,5 Emma Brooks,1
Samantha Pearcey,1 Lucy Taylor,1 Paul Stallard,6
Polly Waite,1 Shirley Reynolds,7 Gordon Taylor,8
Emma Warnock-Parkes2 and David M Clark2
1School of Psychology and Clinical Language Sciences, University of Reading,
Reading, UK
2Department of Experimental Psychology, University of Oxford, Oxford, UK
3School of Life and Health Sciences, Aston University, Birmingham, UK
4PPI representative, London, UK
5Health Economics Research Centre, Nuffield Department of Population Health,

University of Oxford, Oxford, UK

6Child and Adolescent Mental Health Service, Oxford Health NHS Trust, Bristol, UK
7Charlie Waller Institute, University of Reading, Reading, UK
8College of Medicine and Health, University of Exeter, Exeter, UK

*Corresponding author

Declared competing interests of authors: David M Clark reports personal fees from the NHS outside
the submitted work; he is also one of the authors of the cognitive model of social anxiety that CT-SAD-A
(Cognitive Therapy for Social Anxiety Disorder in Adolescents) is based on, and he has also played a central
role in developing the treatment.

Disclaimer: This report contains transcripts of interviews conducted in the course of the research and
contains language that may offend some readers.

Published March 2021

DOI: 10.3310/hta25200
This report should be referenced as follows:

Creswell C, Leigh E, Larkin M, Stephens G, Violato M, Brooks E, et al. Cognitive therapy compared
with CBT for social anxiety disorder in adolescents: a feasibility study. Health Technol Assess
2021;25(20).

Health Technology Assessment is indexed and abstracted in Index Medicus/MEDLINE, Excerpta

Medica/EMBASE, Science Citation Index Expanded (SciSearch®) and Current Contents®/
Clinical Medicine.
Health Technology Assessment
ISSN 1366-5278 (Print)

ISSN 2046-4924 (Online)

Impact factor: 3.370

Health Technology Assessment is indexed in MEDLINE, CINAHL, EMBASE, the Cochrane Library and Clarivate Analytics Science Citation Index.

This journal is a member of and subscribes to the principles of the Committee on Publication Ethics (COPE) (www.publicationethics.org/).

Editorial contact: [email protected]

The full HTA archive is freely available to view online at www.journalslibrary.nihr.ac.uk/hta. Print-on-demand copies can be purchased from the
report pages of the NIHR Journals Library website: www.journalslibrary.nihr.ac.uk

Criteria for inclusion in the Health Technology Assessment journal

Reports are published in Health Technology Assessment (HTA) if (1) they have resulted from work for the HTA programme, and (2) they
are of a sufficiently high scientific quality as assessed by the reviewers and editors.

Reviews in Health Technology Assessment are termed ‘systematic’ when the account of the search appraisal and synthesis methods
(to minimise biases and random errors) would, in theory, permit the replication of the review by others.

HTA programme
Health Technology Assessment (HTA) research is undertaken where some evidence already exists to show that a technology can be effective
and this needs to be compared to the current standard intervention to see which works best. Research can evaluate any intervention used in
the treatment, prevention or diagnosis of disease, provided the study outcomes lead to findings that have the potential to be of direct benefit
to NHS patients. Technologies in this context mean any method used to promote health; prevent and treat disease; and improve rehabilitation
or long-term care. They are not confined to new drugs and include any intervention used in the treatment, prevention or diagnosis of disease.

The journal is indexed in NHS Evidence via its abstracts included in MEDLINE and its Technology Assessment Reports inform National Institute
for Health and Care Excellence (NICE) guidance. HTA research is also an important source of evidence for National Screening Committee (NSC)
policy decisions.

This report
The research reported in this issue of the journal was funded by the HTA programme as project number 14/170/01. The contractual start date
was in March 2016. The draft report began editorial review in November 2018 and was accepted for publication in April 2019. The authors
have been wholly responsible for all data collection, analysis and interpretation, and for writing up their work. The HTA editors and publisher
have tried to ensure the accuracy of the authors’ report and would like to thank the reviewers for their constructive comments on the draft
document. However, they do not accept liability for damages or losses arising from material published in this report.

This report presents independent research funded by the National Institute for Health Research (NIHR). The views and opinions expressed by
authors in this publication are those of the authors and do not necessarily reflect those of the NHS, the NIHR, NETSCC, the HTA programme
or the Department of Health and Social Care. If there are verbatim quotations included in this publication the views and opinions expressed
by the interviewees are those of the interviewees and do not necessarily reflect those of the authors, those of the NHS, the NIHR, NETSCC,
he HTA programme or the Department of Health and Social Care.

© Queen’s Printer and Controller of HMSO 2021. This work was produced by Creswell et al. under the terms of a commissioning
contract issued by the Secretary of State for Health and Social Care. This issue may be freely reproduced for the purposes of
private research and study and extracts (or indeed, the full report) may be included in professional journals provided that
suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for
commercial reproduction should be addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation,
Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science Park, Southampton SO16 7NS, UK.

Published by the NIHR Journals Library (www.journalslibrary.nihr.ac.uk), produced by Prepress Projects Ltd, Perth, Scotland
(www.prepress-projects.co.uk).
NIHR Journals Library Editor-in-Chief
Professor Ken Stein Professor of Public Health, University of Exeter Medical School, UK

NIHR Journals Library Editors

Professor John Powell Chair of HTA and EME Editorial Board and Editor-in-Chief of HTA and EME journals.
Consultant Clinical Adviser, National Institute for Health and Care Excellence (NICE), UK, and Professor of
Digital Health Care, Nuffield Department of Primary Care Health Sciences, University of Oxford, UK

Professor Andrée Le May Chair of NIHR Journals Library Editorial Group (HS&DR, PGfAR, PHR journals) and
Editor-in-Chief of HS&DR, PGfAR, PHR journals

Professor Matthias Beck Professor of Management, Cork University Business School, Department of Management
and Marketing, University College Cork, Ireland

Dr Tessa Crilly Director, Crystal Blue Consulting Ltd, UK

Dr Eugenia Cronin Senior Scientific Advisor, Wessex Institute, UK

Dr Peter Davidson Consultant Advisor, Wessex Institute, University of Southampton, UK

Ms Tara Lamont Senior Scientific Adviser (Evidence Use), Wessex Institute, University of Southampton, UK

Dr Catriona McDaid Senior Research Fellow, York Trials Unit, Department of Health Sciences,
University of York, UK

Professor William McGuire Professor of Child Health, Hull York Medical School, University of York, UK

Professor Geoffrey Meads Emeritus Professor of Wellbeing Research, University of Winchester, UK

Professor James Raftery Professor of Health Technology Assessment, Wessex Institute, Faculty of Medicine,
University of Southampton, UK

Dr Rob Riemsma Reviews Manager, Kleijnen Systematic Reviews Ltd, UK

Professor Helen Roberts Professor of Child Health Research, UCL Great Ormond Street Institute of Child Health, UK

Professor Jonathan Ross Professor of Sexual Health and HIV, University Hospital Birmingham, UK

Professor Helen Snooks Professor of Health Services Research, Institute of Life Science, College of Medicine,
Swansea University, UK

Professor Ken Stein Professor of Public Health, University of Exeter Medical School, UK

Professor Jim Thornton Professor of Obstetrics and Gynaecology, Faculty of Medicine and Health Sciences,
University of Nottingham, UK

Please visit the website for a list of editors: www.journalslibrary.nihr.ac.uk/about/editors

Editorial contact: [email protected]

NIHR Journals Library www.journalslibrary.nihr.ac.uk

 DOI: 10.3310/hta25200 HEALTH TECHNOLOGY ASSESSMENT 2021 VOL. 25 NO. 20

Abstract

Cognitive therapy compared with CBT for social anxiety

disorder in adolescents: a feasibility study

Cathy Creswell,1* Eleanor Leigh,2 Michael Larkin,3 Gareth Stephens,4

Mara Violato,5 Emma Brooks,1 Samantha Pearcey,1 Lucy Taylor,1
Paul Stallard,6 Polly Waite,1 Shirley Reynolds,7 Gordon Taylor,8
Emma Warnock-Parkes2 and David M Clark2
1School of Psychology and Clinical Language Sciences, University of Reading, Reading, UK
2Department of Experimental Psychology, University of Oxford, Oxford, UK
3School of Life and Health Sciences, Aston University, Birmingham, UK
4PPI representative, London, UK
5Health Economics Research Centre, Nuffield Department of Population Health, University of

Oxford, Oxford, UK
6Child and Adolescent Mental Health Service, Oxford Health NHS Trust, Bristol, UK
7Charlie Waller Institute, University of Reading, Reading, UK
8College of Medicine and Health, University of Exeter, Exeter, UK

*Corresponding author [email protected]

Background: Social anxiety disorder (SAD) is common, typically starts in adolescence and has a low
natural recovery rate. Existing psychological treatments for adolescent SAD are only moderately effective. It
is possible that recovery rates for adolescents could be substantially improved by adapting a psychological
therapy that is highly effective among adults with SAD.
Objectives: To train child and adolescent mental health services (CAMHS) therapists to deliver cognitive
therapy for SAD in adolescents (CT-SAD-A) and assess therapist competence. To estimate the costs to the
NHS of training therapists to deliver CT-SAD-A and the mean cost per adolescent treated. To examine
the feasibility of a randomised controlled trial (RCT) to compare CT-SAD-A with the general form of
cognitive–behavioural therapy that is more commonly used.
Design: During the training phase of the study, it became clear that the RCT would not be feasible
because of high staff turnover and unfilled posts within CAMHS and changes in the nature of referrals,
which meant that few young people with primary SAD were accessing some of the participating services.
The study design was altered to comprise the following: a training case series of CT-SAD-A delivered in
routine CAMHS, an estimate of the cost to the NHS of training therapists to deliver CT-SAD-A and of
the mean cost per adolescent treated, and qualitative interviews with participating young people, parents,
therapists and service managers/leads.
Setting: Five CAMHS teams within Berkshire Healthcare and Oxford Health NHS Foundation Trusts.
Participants: Eight therapists received training in CT-SAD-A. Twelve young people received CT-SAD-A,
delivered by six therapists. Six young people, six parents, seven therapists and three managers participated
in qualitative interviews.
Interventions: Cognitive therapy for social anxiety disorder in adolescents (CT-SAD-A).

© Queen’s Printer and Controller of HMSO 2021. This work was produced by Creswell et al. under the terms of a commissioning contract issued by the Secretary of State for
Health and Social Care. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional
journals provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should
vii
be addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science
Park, Southampton SO16 7NS, UK.
 ABSTRACT

Main outcome measures: Measured outcomes included social anxiety symptoms and diagnostic status,
comorbid symptoms of anxiety and depression, social and general functioning, concentration in class and
treatment acceptability. Patient level utilisation of the intervention was collected using clinicians’ logs.
Results: Nine out of 12 participants achieved good outcomes across measures (r ≥ 0.60 across social
anxiety measures). The estimated cost of delivering CT-SAD-A was £1861 (standard deviation £358) per
person. Qualitative interviews indicated that the treatment was acceptable to young people, parents and
therapists, but therapists and managers experienced challenges when implementing the training and
treatment within the current CAMHS context.
Limitations: Findings were based on a small, homogeneous sample and there was no comparison arm.
Conclusions: CT-SAD-A is a promising treatment for young people with SAD, but the current CAMHS
context presents challenges for its implementation.
Future work: Further work is needed to ensure that CAMHS can incorporate and test CT-SAD-A.
Alternatively, CT-SAD-A should be delivered and tested in other settings that are better configured to treat
young people whose lives are held back by SAD. The new schools Mental Health Support Teams envisaged
in the 2017 Children’s Mental Health Green Paper may provide such an opportunity.
Funding: The National Institute for Health Research (NIHR) Health Technology Assessment programme.
Individual funding was also provided for Cathy Creswell, David M Clark and Eleanor Leigh as follows:
NIHR Research Professorship (Cathy Creswell); Wellcome Senior Investigator Award (Anke Ehlers and
David M Clark); and the Wellcome Clinical Research Training Fellowship (Eleanor Leigh).

viii
NIHR Journals Library www.journalslibrary.nihr.ac.uk
 DOI: 10.3310/hta25200 HEALTH TECHNOLOGY ASSESSMENT 2021 VOL. 25 NO. 20

Contents

List of tables xiii

List of figures xv

Glossary xvii

List of abbreviations xix

Plain English summary xxi

Scientific summary xxiii

Chapter 1 Introduction 1
Objective 1 2
Objective 2 2

Chapter 2 Methods 3
Participants 3
Inclusion criteria 3
Exclusion criteria 3
Procedures 4
Ethics review 4
Therapist training 4
Intervention: cognitive therapy for social anxiety disorder in adolescents 4

Chapter 3 Patient and public involvement 9

Aim 9
Methods 10
Results 10
Discussion and conclusions 11
Reflections/critical perspective 12

Chapter 4 Results 1: feasibility in routine child and adolescent mental health

services – recruiting and retaining therapists and young people 13
Recruiting and retaining therapists 13
Recruiting and retaining young people with social anxiety disorder 13
Action taken 18

Chapter 5 Results 2: training case series – clinical outcomes 21

Patient and therapy characteristics 21
Patient outcomes 21
Social anxiety symptom scales 21
Diagnostic profile 26
Related clinical outcomes 26
Revised Child Anxiety and Depression Scale depression scores 26
Revised Child Anxiety and Depression Scale generalised anxiety disorder scores 26
Parent-reported Revised Child Anxiety and Depression Scale 26

© Queen’s Printer and Controller of HMSO 2021. This work was produced by Creswell et al. under the terms of a commissioning contract issued by the Secretary of State for
Health and Social Care. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional
journals provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should
ix
be addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science
Park, Southampton SO16 7NS, UK.
 CONTENTS

Social participation and satisfaction 29

Concentration in class 29
Outcome rating scales 29
Children and Young People’s Improving Access to Psychological Therapies programme
goal-rating 29
Psychological process measures 33
Treatment acceptability 35

Chapter 6 Results 3: costs of treatment delivery in routine child and adolescent

mental health services 37
Aim 37
Methods 37
Results 38
Data completeness 38
Qualified supervisors’ and therapists’ time and costs 38
Qualified clinician time and cost per adolescent treated 40

Chapter 7 Qualitative interviews: approach, design, sample and analysis 47

Approach 47
Design 47
Recruitment 47
Sample 47
Data collection and transformation 49
Qualitative data analysis 49

Chapter 8 Results 4: young people’s and parents’ experiences 51

Young people’s perspectives 51
Varied motives and experiences 51
A challenging, helpful and active experience 51
Using measures to decide where to focus and to track progress 52
Identifying active ingredients 52
Learning from and with the therapist 53
Parents’ perspectives 55
Keeping an open mind 55
Noticing functional improvements 55
Appreciating the opportunity to be involved and informed 56
Reflecting on relationships, roles and needs 57
Reflecting on the context of help-seeking 58
Identifying active ingredients 58
Accepting the need for measures 59

Chapter 9 Results 5: therapists’ and service manager/leads’ experiences 61

Child and adolescent mental health services therapists’ perspectives 61
Engaging and expert training 61
Challenging but supportive supervision 61
A useful and well-used manual 64
Developing competence 64
Identifying active ingredients 65
Affinity with the model 66
Worrying about the number of measures 68
Child and adolescent mental health services managers’/leads’ perspectives 69
Worrying about releasing staff time 70
Struggling to identify suitable cases 71

x
NIHR Journals Library www.journalslibrary.nihr.ac.uk
 DOI: 10.3310/hta25200 HEALTH TECHNOLOGY ASSESSMENT 2021 VOL. 25 NO. 20

Managing staff workloads 71

Problems with backfill 72

Chapter 10 Adverse events 75

Chapter 11 Discussion 77
Treatment of social anxiety disorder in adolescents within child and adolescent mental
health services 77
Costs of delivering cognitive therapy for social anxiety disorder in adolescents in child
and adolescent mental health services 79
Participant experiences 80

Chapter 12 Conclusions and recommendations 83

Implications for the treatment of social anxiety disorder in adolescents 83
Broader implications for treatment delivery within child and adolescent mental
health services 84
Implications for research in child and adolescent mental health services 84
Summary and recommendations 86

Acknowledgements 87

References 89

Appendix 1 The National Institute for Health and Care Excellence screen for
social anxiety 93

© Queen’s Printer and Controller of HMSO 2021. This work was produced by Creswell et al. under the terms of a commissioning contract issued by the Secretary of State for
Health and Social Care. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional
journals provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should
xi
be addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science
Park, Southampton SO16 7NS, UK.
 DOI: 10.3310/hta25200 HEALTH TECHNOLOGY ASSESSMENT 2021 VOL. 25 NO. 20

List of tables
TABLE 1 National standards for public involvement: examples of how the
standards were met 9

TABLE 2 Allocation of therapists by team at each location across treatment arms

and NHS trusts 16

TABLE 3 Core professional training of therapists available for study 16

TABLE 4 Patient and treatment characteristics 22

TABLE 5 Patient outcomes at pre treatment and follow-up available CYP IAPT
and results of non-parametric Wilcoxon tests 23

TABLE 6 The ADIS diagnostic profile 26

TABLE 7 Table of unit costs 37

TABLE 8 Supervisor time and cost for training workshops and case supervision 38

TABLE 9 Mean supervisor time per therapist 39

TABLE 10 Therapist’s total and per session time/cost of being supervised 40

TABLE 11 Qualified therapist’s time per patient by treatment session: sessions 1–8 41

TABLE 12 Qualified therapist’s time per patient by treatment and booster

sessions: sessions 9–14 and two boosters 42

TABLE 13 Cost of qualified therapist’s time per patient by treatment session:

sessions 1–8 43

TABLE 14 Cost of qualified therapist’s time per patient by treatment and booster
session: sessions 9–14 and two boosters 44

TABLE 15 Mean clinician’s time and cost of CT-SAD-A treatment per patient 45

TABLE 16 Young people who participated in the qualitative interviews 48

TABLE 17 Parents who participated in the qualitative interviews 48

TABLE 18 Therapists who participated in the qualitative interviews 48

TABLE 19 Managers/leads who participated in the qualitative interviews 49

TABLE 20 The NICE screening questions for social anxiety 93

© Queen’s Printer and Controller of HMSO 2021. This work was produced by Creswell et al. under the terms of a commissioning contract issued by the Secretary of State for
Health and Social Care. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional
journals provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should
xiii
be addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science
Park, Southampton SO16 7NS, UK.
 DOI: 10.3310/hta25200 HEALTH TECHNOLOGY ASSESSMENT 2021 VOL. 25 NO. 20

List of figures
FIGURE 1 Therapist recruitment and retention to the study (Berkshire Healthcare
NHS Foundation Trust) 14

FIGURE 2 Therapist recruitment and retention to the study (Oxford Health NHS
Foundation Trust) 15

FIGURE 3 Patient recruitment and retention to the CT-SAD-A training phase

(Berkshire Healthcare NHS Foundation Trust) 17

FIGURE 4 Patient recruitment and retention to the CT-SAD-A training phase

(Oxford Health NHS Foundation Trust) 18

FIGURE 5 Individual patient LSAS-CA-SR scores at the following measurement

points: pre assessment, mid-treatment, at the end of weekly sessions and at the
follow-up assessment 25

FIGURE 6 Individual patient RCADS social anxiety T-scores at the following

measurement points: pre assessment, mid-treatment, at the end of weekly
sessions and at the follow-up assessment 25

FIGURE 7 The ADIS CSR pre treatment and at follow-up scores 27

FIGURE 8 Individual patient RCADS depression T-scores at pre assessment and at

the follow-up assessment 28

FIGURE 9 Individual patient RCADS GAD T-scores at pre assessment and at the
follow-up assessment 28

FIGURE 10 Parent RCADS social anxiety subscale T-scores pre treatment and at
follow-up 29

FIGURE 11 Individual patient social participation scores at pre assessment and at

the follow-up assessments 30

FIGURE 12 Individual patient social satisfaction scores at pre assessment and at

the follow-up assessments 30

FIGURE 13 Self-reported concentration in class at pre assessment and at the

follow-up assessment 31

FIGURE 14 Individual patient ORS: overall subscale scores at pre assessment and
follow-up 31

FIGURE 15 Individual patient ORS: social subscale scores at pre assessment and
follow-up 32

FIGURE 16 Individual patient data for first and last available CYP IAPT
goal-ratings made by patients for their primary goal in therapy 32

© Queen’s Printer and Controller of HMSO 2021. This work was produced by Creswell et al. under the terms of a commissioning contract issued by the Secretary of State for
Health and Social Care. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional
journals provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should
xv
be addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science
Park, Southampton SO16 7NS, UK.
 LIST OF FIGURES

FIGURE 17 Individual patient SCQ mean belief scores at pre assessment and at
the follow-up assessment 33

FIGURE 18 Individual patient SBQ scores at pre assessment at the follow-up

assessment 34

FIGURE 19 Individual patient mean SAQ scores at pre assessment and at the
follow-up assessment 34

FIGURE 20 Treatment acceptability 35

xvi
NIHR Journals Library www.journalslibrary.nihr.ac.uk
 DOI: 10.3310/hta25200 HEALTH TECHNOLOGY ASSESSMENT 2021 VOL. 25 NO. 20

Glossary
C.A.T. Project The adolescent version of ‘Coping Cat’, a cognitive–behavioural therapy programme that
has been developed for a range of anxiety disorders.

© Queen’s Printer and Controller of HMSO 2021. This work was produced by Creswell et al. under the terms of a commissioning contract issued by the Secretary of State for
Health and Social Care. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional
journals provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should
xvii
be addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science
Park, Southampton SO16 7NS, UK.
 DOI: 10.3310/hta25200 HEALTH TECHNOLOGY ASSESSMENT 2021 VOL. 25 NO. 20

List of abbreviations

ADIS-C/P Anxiety Disorders Interview LSAS-CA-SR Liebowitz Social Anxiety Scale

Schedule for Diagnostic and for Children and Adolescents,
Statistical Manual of Mental self-report version
Disorders, Fourth Edition, child and
MDD major depressive disorder
parent version
NICE National Institute for Health and
AnDY RAG Anxiety and Depression in Young
Care Excellence
People Research Advisors Group
NIHR National Institute for Health
ASD autism spectrum disorder
Research
BHFT Berkshire Healthcare NHS
ORS Outcome Rating Scale
Foundation Trust
OxHealth Oxford Health NHS Foundation
CAMHS child and adolescent mental health
Trust
services
PHQ-9 Patient Health Questionnaire, 9 items
CBT cognitive–behavioural therapy
PPI patient and public involvement
CSR clinical severity rating
PTSD post-traumatic stress disorder
CT-SAD cognitive therapy for social anxiety
disorder in adults RCADS Revised Child Anxiety and Depression
Scale (child/parent report)
CT-SAD-A cognitive therapy for social anxiety
disorder in adolescents RCT randomised control trial
CYP IAPT Children and Young People’s RRBI repetitive, restrictive behaviours
Improving Access to Psychological and interests
Therapies programme SAD social anxiety disorder
DSM-5 Diagnostic and Statistical Manual SAQ Social Attitudes Questionnaire
of Mental Disorders, Fifth Edition
SBQ Social Behaviours Questionnaire
ETC excess treatment costs
SComQ Social Communication
GAD generalised anxiety disorder Questionnaire
IPA interpretative phenomenological SCQ Social Cognitions Questionnaire
analysis
SSC Study Steering Committee
LSAS Liebowitz Social Anxiety Scale

© Queen’s Printer and Controller of HMSO 2021. This work was produced by Creswell et al. under the terms of a commissioning contract issued by the Secretary of State for
Health and Social Care. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional
journals provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should
xix
be addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science
Park, Southampton SO16 7NS, UK.
 DOI: 10.3310/hta25200 HEALTH TECHNOLOGY ASSESSMENT 2021 VOL. 25 NO. 20

Plain English summary

Why did we do this study?

People with social anxiety disorder (SAD) are scared of social situations because they fear embarrassment
or humiliation. SAD usually starts at around 13 years of age, typically does not go away without treatment,
and leads to personal and social difficulties. Clark and Wells’ cognitive therapy for SAD in adults (CT-SAD)
is a talking therapy that produces excellent outcomes. CT-SAD has not previously been adapted for or
tested with adolescents.

What did we do?

We adapted CT-SAD so that it was suitable for adolescents (CT-SAD-A). We intended to compare this with
current practice in child and adolescent mental health services (CAMHS). However, we were not able to
complete the trial owing to a high staff turnover and a lack of young people with SAD coming into CAMHS.
Instead, we examined outcomes for young people who received CT-SAD-A during the therapist training
phase and explored the views of young people, their parents, the therapists and CAMHS managers about
CT-SAD-A and the study.

What did we find?

Young people’s outcomes were very promising; for example, 10 out of 12 participants reported a reliable
improvement in social anxiety. The young people and their parents were generally positive about the
treatment. Therapists were also positive about the treatment, but they and their managers found it difficult
to implement the treatment within their CAMHS teams. The cost to the NHS to treat young people with
SAD within this study compared favourably with the cost of treating adults.

What does this mean?

We need to be careful about drawing conclusions from a small sample size, but we suggest that further
work is needed to ensure that CT-SAD-A can be delivered and tested in CAMHS. Alternatively, CT-SAD-A
should be delivered and tested in community or school settings that can treat young people whose lives
are held back by SAD.

© Queen’s Printer and Controller of HMSO 2021. This work was produced by Creswell et al. under the terms of a commissioning contract issued by the Secretary of State for
Health and Social Care. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional
journals provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should
xxi
be addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science
Park, Southampton SO16 7NS, UK.
 DOI: 10.3310/hta25200 HEALTH TECHNOLOGY ASSESSMENT 2021 VOL. 25 NO. 20

Scientific summary

Background

Social anxiety disorder (SAD) is characterised by a persistent and disproportionate fear of social situations.
It is the third most frequent of all mental health disorders, with a lifetime prevalence of up to 13%. The
age at onset is typically during adolescence (median 13 years), with most people developing the condition
before they reach their twenties. Without treatment, SAD often runs a chronic course and is associated
with risks of other negative mental health and social outcomes.

Currently, the most commonly delivered treatment approach for adolescents with SAD is a generic
cognitive–behavioural therapy (CBT) anxiety programme that is used across a range of anxiety disorders.
Unfortunately, in recent studies, young people with SAD have had significantly poorer outcomes than those
with other anxiety disorders. However, adults who receive a focused psychological treatment for SAD
[cognitive therapy for SAD in adults (CT-SAD)], a therapy that targets key psychological mechanisms that are
known to maintain the disorder, achieve much higher recovery rates. CT-SAD has also been shown to be
superior to a number of other psychological and pharmacological treatments. It is possible that adapting
CT-SAD for adolescents (CT-SAD-A) may improve treatment outcomes for adolescents with SAD over the
generic forms of CBT that are often applied, bringing direct benefits to young people and their families,
the NHS, child and adolescent mental health services (CAMHS), as well as adult mental health services and
society more broadly, by disrupting the negative trajectory associated with adolescent SAD.

Objective 1

This study initially set out to determine the feasibility of a randomised control trial (RCT) in routine NHS
CAMHS to assess the use of CT-SAD-A compared with the C.A.T. Project (a CBT programme that has
been developed for adolescents with a range of anxiety disorders) for adolescents with SAD. Specifically,
our aims were to train NHS CAMHS therapists to deliver the adapted treatment and assess therapist
competency and young people’s outcomes; identify appropriate clinical outcomes and economic measures
for a subsequent definitive trial; and examine whether or not a definitive trial can be conducted on the
basis of a feasibility trial that would:

l explore the acceptability of the treatments and trial procedures

l establish likely recruitment rates
l establish the likely rate of treatment drop out
l establish likely retention to research assessments post treatment and (in a subset of participants) at
3-month follow-up
l establish if adapted cognitive therapy for social anxiety disorder in adolescents can be delivered so that
it is clearly distinct from the C.A.T Project, with high levels of fidelity by practitioners and credibility with
patients in both arms
l conduct exploratory analyses of possible outcomes for the two treatments, including changes in social
anxiety symptoms and diagnostic status, depression, social functioning, school attendance, concentration
in class, quality of life, health-care resource use and other outcomes identified through patient and
public involvement
l describe negative impacts of the treatments and the trial procedures (to patients, their parents
and therapists).

© Queen’s Printer and Controller of HMSO 2021. This work was produced by Creswell et al. under the terms of a commissioning contract issued by the Secretary of State for
Health and Social Care. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional
journals provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should
xxiii
be addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science
Park, Southampton SO16 7NS, UK.
 SCIENTIFIC SUMMARY

Objective 2

Unfortunately, during the training phase of the study, it became clear that the proposed trial would not be
feasible within routine CAMHS, and on the basis of liaison with the Study Steering Committee and the
National Institute for Health Research (NIHR), the study aims were adapted to examine the training in and
delivery of CT-SAD-A in routine NHS CAMHS, in terms of therapists’ abilities to deliver CT-SAD-A, young
people’s outcomes, and the experiences of participating families (young people and their parents/carers),
the therapists delivering the treatment, and their service managers within a CAMHS setting. Specifically,
our aims were to:

l train NHS CAMHS therapists to deliver the adapted treatment and assess therapist competency
(an independent rater is currently rating therapist competency and these outcomes will be reported in a
later publication) and young people’s outcomes
l estimate the cost of delivering CT-SAD-A within an NHS CAMHS setting
l understand the experience of receiving CT-SAD-A and participating in a research study within a CAMHS
setting among young people and their parents
l understand the experience of CAMHS therapists receiving training in and delivering CT-SAD-A and their
experience of being part of a research study
l understand the experiences of the CAMHS service managers in relation to supporting the training and
delivery of CT-SAD-A and the accompanying research procedures within their services.

Methods

l We produced a training manual and accompanying materials for the adapted form of CT-SAD-A
[PowerPoint® presentations (Microsoft Corporation, Redmond, WA, USA) and video clips].
l NHS CAMHS therapists attended training workshops and received weekly supervision on a series of
CT-SAD-A training cases.
l We assessed patient outcomes during the training phase using diagnostic interviews and young person
and parent report questionnaires relating to social anxiety, common comorbid difficulties and broader
functioning.
l We estimated the costs of delivering CT-SAD-A in CAMHS during the training phase.
l We interviewed young people, parents, therapists and their managers/service leads and used interpretative
phenomenological analysis to explore their experiences of CT-SAD-A and the research.

Results

Of the 19 therapists recruited to the training phase, 10 withdrew from the study owing to job changes or
other pressures. Participating therapists had a limited amount of time available to work with young people
recruited to the study. Some teams identified few young people with SAD because referrals increasingly
required a focus on risk management. As a result of the difficulties with therapist retention and participant
identification, we concluded that the proposed RCT would not be feasible in the current CAMHS context.

Twelve young people received CT-SAD-A as part of the training case series. Large reductions were reported
for symptoms of social anxiety, generalised anxiety, depression, and large increases were reported for social
satisfaction and participation. Seven young people no longer met criteria for a diagnosis of SAD at the
end of the treatment. Large reductions were also found in the measures of the psychological mechanisms
that are targeted in CT-SAD-A. Young people universally endorsed high acceptability of the treatment.
The full cost to the NHS would be £4047 [standard deviation (SD) £1003] per adolescent treated in the
training phase or £1861 (SD £358), excluding training costs.

xxiv
NIHR Journals Library www.journalslibrary.nihr.ac.uk
 DOI: 10.3310/hta25200 HEALTH TECHNOLOGY ASSESSMENT 2021 VOL. 25 NO. 20

On the basis of the qualitative interviews, young people appeared to find some aspects of the intervention
challenging, but they reflected on the benefits (especially the improvements in what they were able to do)
outstripping the costs. They found the approach taken by their therapists to be open and non-judgemental,
and the therapeutic process to be active. They identified the use of measures and behavioural experiments
to be particularly helpful.

Parents were aware that young people found some aspects of the intervention challenging, but they
noticed and valued the benefits (especially in what their young people were able to do). They found the
approach taken by their therapists to be open and inclusive, and they noticed and appreciated that the
therapeutic process was an active one. They accepted the need for measures. They reflected on some of
the relational impacts of treatment, and some felt that they would have liked to have received support for
themselves as well.

The therapists were very positive about the training and the treatment model. They found the process of
developing competence through delivery and supervision to be challenging but also constructive, acceptable
and achievable. They framed the therapy as an active form of intervention and singled out behavioural
experiments as a key component. Their confidence in other components of the therapy was more varied.
They expressed concerns about the volume of measures involved and about the viability of delivering the
therapy as part of routine CAMHS work. They identified features that they would incorporate into their
future work, including the use of measures within treatment and the use of video in supervision.

Managers were sceptical about the viability of both the research and the intervention. They were
concerned about the pressure on staff and used ‘time-as-a-limited-resource’ as the main means of framing
this problem.

Conclusions

It is not feasible to conduct a RCT to compare CT-SAD-A with a generic form of CBT within the current
CAMHS context. The findings of the training case series provide preliminary evidence that good outcomes
can be achieved from the delivery of CT-SAD-A in NHS CAMHS and that young people, parents and
therapists like and value the intervention. However, because of high service demands and a relatively low
capacity within CAMHS, therapists and managers struggled to carve out and protect the time needed to
participate in training and deliver the treatment. Furthermore, in some services, referral patterns had become
increasingly dominated by risk management needs, and young people with SAD were not reaching services.
Further work is needed to ensure that CAMHS workers are fully supported to learn and test CT-SAD-A within
a CAMHS setting. Alternatively, CT-SAD-A should be delivered and tested in a community or school setting
that is better configured to treat a representative population of young people whose lives are held back by
SAD. The new schools Mental Health Support Teams envisaged in the 2017 Children’s Mental Health Green
Paper may provide such an opportunity.

Funding

Funding for this study was provided by the Health Technology Assessment programme of the National
Institute for Health Research (NIHR). Individual funding was also provided for Cathy Creswell, David M Clark
and Eleanor Leigh as follows: NIHR Research Professorship (Cathy Creswell); Wellcome Senior Investigator
Award (Anke Ehlers and David M Clark); and the Wellcome Clinical Research Training Fellowship
(Eleanor Leigh).

© Queen’s Printer and Controller of HMSO 2021. This work was produced by Creswell et al. under the terms of a commissioning contract issued by the Secretary of State for
Health and Social Care. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional
journals provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should
xxv
be addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science
Park, Southampton SO16 7NS, UK.
 DOI: 10.3310/hta25200 HEALTH TECHNOLOGY ASSESSMENT 2021 VOL. 25 NO. 20

Chapter 1 Introduction
S ocial anxiety disorder (SAD) is characterised by a persistent and disproportionate fear of social situations.
It is the third most frequent of all mental health disorders, with a lifetime prevalence of up to 13%.1,2
The age at onset is typically during adolescence (median 13 years),3,4 with most people developing the
condition before they reach their twenties. Indeed, studies that have focused on young people between the
ages of 11 and 17 years have estimated that 0.7% and 10.9%5 of young people meet diagnostic criteria at
SAD at any one point in time, with European studies (in Germany and Finland) identifying prevalence rates
between 1.4% and 3.2%.6–8

Social anxiety disorder is differentiated from normal shyness by the marked disability and interference caused
in day-to-day life. For example, among adults it often has a negative impact on social relationships (e.g.
friendships,9 marriages and children8) and is associated with more days off work,10 receipt of state benefits11
and more outpatient medical visits than the general population.12 SAD is also associated with an increased
risk of other mental health problems, including other anxiety and mood disorders, substance abuse and
psychosis,2,13,14 and has the lowest natural recovery rate of all anxiety disorders.15 Several of the handicaps
that result from having the condition in adolescence (e.g. missed opportunities for social learning and poor
educational achievement)16 cannot be overcome by treatment when the person reaches adulthood. These
factors highlight the need for effective treatment for SAD in adolescence.

Currently, the most commonly delivered treatment approach for adolescents with SAD is a generic
cognitive–behavioural therapy (CBT) anxiety programme that is used across a range of anxiety disorders.
Unfortunately, in recent studies, young people with SAD have had significantly poorer outcomes than
those with other anxiety disorders (e.g. for 7- to 17-year-olds, remission rates of 40.6% for SAD vs. 72%
for other anxiety disorders).17 However, among adults, a focused form of cognitive therapy for SAD
(CT-SAD), which targets key psychological mechanisms that are known to maintain the disorder,18 achieves
much higher recovery rates of up to 84% in randomised controlled trials (RCTs).19–21 Among adults,
it has also been shown to be superior (in terms of clinical effectiveness and cost-effectiveness) to traditional
group CBT, guided CBT self-help, exposure therapy, interpersonal psychotherapy, psychodynamic
psychotherapy and medication (selective serotonin reuptake inhibitors),22,23 and is recommended by the
National Institute for Health and Care Excellence (NICE) as a first-line intervention for SAD in adults.24

There is recent evidence that the same psychological mechanisms maintain SAD in adults and adolescents.25–28
Therefore, adapting CT-SAD so that it is suitable for use with adolescents may significantly enhance treatment
efficacy. NICE guidance also highlights that treatment for SAD in adolescents should address the need to
create treatment supporting environments, by working with parents, teachers and/or peers as appropriate.24

A recent case series in which cognitive therapy for SAD was adapted for use with adolescents (CT-SAD-A)
reported excellent outcomes.29 All adolescents (n = 5; 11–17 years) had severe SAD and achieved excellent
outcomes after receiving CT-SAD-A, with overall reductions in social anxiety symptoms that were greater
than the average reductions achieved with adults. Of note, all the adolescents were free of their primary
diagnosis of SAD as well as comorbid mental health diagnoses post treatment. Promising findings have
also come from a RCT which compared individual cognitive therapy to group CBT and an attentional
control.30 The individual cognitive therapy condition was associated with significantly greater reductions
in symptoms and impairment than group CBT and the attentional control, which did not differ from each
other. Notably, the cognitive therapy programme applied within this trial included a number of important
departures from the Clark and Wells’ approach,31 including a focus on psychoeducation and developing a
generic formulation for anxiety maintenance over the first four sessions. Nonetheless, given this evidence,
it seems possible that CT-SAD-A may improve treatment outcomes for adolescents with SAD over the
generic forms of CBT that are often applied. As SAD presents a risk for ongoing mental health problems,
impaired educational performance, restricted employment and productivity, and increased medical needs,

© Queen’s Printer and Controller of HMSO 2021. This work was produced by Creswell et al. under the terms of a commissioning contract issued by the Secretary of State for
Health and Social Care. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional
journals provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should
1
be addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science
Park, Southampton SO16 7NS, UK.
 INTRODUCTION

the successful development of a highly effective treatment for adolescent SAD also has the potential to
bring direct benefits to the NHS and child and adolescent mental health services (CAMHS), as well as adult
mental health services and society more broadly, by disrupting this negative trajectory.

Objective 1

This study initially set out to determine the feasibility of a RCT in routine NHS CAMHS to compare CT-SAD-A
with the C.A.T. Project (a CBT programme that has been developed for adolescents with a range of anxiety
disorders and that is already in use in CAMHS) for adolescents with SAD. Specifically, our aims were to
train NHS CAMHS therapists to deliver the adapted treatment and assess therapist competency and young
people’s outcomes; identify appropriate clinical outcome and economic measures for a subsequent
definitive trial; and examine whether or not a definitive trial can be conducted on the basis of a feasibility
trial that would:

l explore the acceptability of the treatments and trial procedures

l establish likely recruitment rates
l establish the likely rate of treatment drop out
l establish likely retention to research assessments post treatment and (in a subset of participants) at
3-month follow-up
l establish if adapted CT-SAD-A can be delivered so that it is clearly distinct from the C.A.T. Project,
with high levels of fidelity by practitioners and credibility with patients in both arms;
l conduct exploratory analyses of possible outcomes for the two treatments, including changes in social
anxiety symptoms and diagnostic status, depression, social functioning, school attendance, concentration
in class, quality of life, health-care resource use and other outcomes identified through patient and public
involvement (PPI)
l describe negative impacts of the treatments and the trial procedures (to patients, their parents
and therapists).

Objective 2

Unfortunately, during the training phase of the study, it became clear that the proposed trial would not
be feasible within routine CAMHS (see Chapter 4) and, on the basis of liaison with the Study Steering
Committee (SSC) and the National Institute for Health Research (NIHR), the study aims were adapted to
examine the training in and delivery of CT-SAD-A in routine NHS CAMHS, in terms of therapists’ ability to
deliver CT-SAD-A, young people’s outcomes, and the experiences of both participating families (young
people and their parents/carers) and the therapists involved (both therapists delivering the treatment and
their service managers) within a CAMHS setting. Specifically, the aims were to:

l train NHS CAMHS therapists to deliver the adapted treatment and assess therapist competency
(an independent rater is currently rating therapist competency, and these outcomes will be reported
in a later publication) and young people’s outcomes
l estimate the cost of delivering CT-SAD-A within an NHS CAMHS setting
l understand the experience of receiving CT-SAD-A and participating in a research study within a CAMHS
setting among young people and their parents
l understand the experience of CAMHS therapists receiving training in and delivering CT-SAD-A and their
experience of being part of a research study
l understand the experiences of the CAMHS service managers in relation to supporting the training and
delivery of CT-SAD-A and the accompanying research procedures within their services.

2
NIHR Journals Library www.journalslibrary.nihr.ac.uk
 DOI: 10.3310/hta25200 HEALTH TECHNOLOGY ASSESSMENT 2021 VOL. 25 NO. 20

Chapter 2 Methods

Participants

Participating therapists were identified within the participating NHS trusts on the basis of meeting the
following inclusion criteria:

l currently in clinical practice within participating CAMHS

l willing to be randomly allocated to deliver either (1) the C.A.T. project, or (2) CT-SAD-A, and to receive
training and supervision for the allocated treatment approach
l have at least 2 years’ experience of using CBT as their main treatment approach, receiving regular CBT
supervision during this time
l a minimum of 2 years working clinically with children and young people
l have treated at least two adolescent/adult social anxiety disorder cases and 10 anxiety cases using CBT.

Young people were referred to CAMHS in the participating trusts through usual routes (e.g. by general
practitioners, school nurses) for treatment of anxiety or mood disorders. Adolescents and their parents
completed the NICE screening questions24 (see Appendix 1) and the Revised Children’s Anxiety and Depression
Scale (RCADS) to identify potential SAD within a routine CAMHS assessment. Where adolescents ‘screened
positive’, they were invited to take part in the research. An initial assessment included a structured diagnostic
interview with the young person and carer [Anxiety Disorders Interview Schedule for Diagnostic and Statistical
Manual of Mental Disorders, Fourth Edition, child and parent version (ADIS-C/P)]32 to establish whether or not
the young person met the following criteria:

Inclusion criteria

l Young people (aged 11–17.5 years at intake) whose primary presenting disorder was a Diagnostic and
Statistical Manual of Mental Disorders, Fifth Edition (DSM-5), diagnosis of SAD.
l If young people had been prescribed psychotropic medication, the dosage must have been stable for
2 months.

Exclusion criteria

l Young people who had previously received CBT for SAD were excluded if they had received more than
four sessions in which key procedures of CT-SAD-A had been used (i.e. video feedback, attention
training, memory work and multiple behavioural experiments).
l Young people with established autistic spectrum disorders [or suspected on the basis of Social
Cognitions Questionnaire (SCQ)18; see Measures], learning disabilities, suicidal intent or recurrent
self-harm (i.e. comorbid conditions that are likely to interfere with treatment delivery).
l Young people with a primary presenting disorder other than SAD.

The managers/clinical leads who were involved in managing the participating therapists on a day-to-day
basis were invited to take part in qualitative interviews.

© Queen’s Printer and Controller of HMSO 2021. This work was produced by Creswell et al. under the terms of a commissioning contract issued by the Secretary of State for
Health and Social Care. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional
journals provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should
3
be addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science
Park, Southampton SO16 7NS, UK.
 METHODS

Procedures

Ethics review
The study was approved by the NHS South Central – Oxford B Research Ethics Committee (REC reference:
16/SC/0315) and the University of Reading Research Ethics Committee (UREC 16/43).

Therapist training
CAMHS therapists were provided with a detailed treatment manual ahead of the training. They were also
given a memory stick loaded with short video clips demonstrating many of the key steps in CT-SAD-A.
Finally, they received a pack containing all worksheets needed to support CT-SAD-A. The training materials
that were used in this study are freely available here: https://oxcadatresources.com.

Participating therapists completed a training programme comprising an initial 2-day workshop, focusing
on the CT-SAD-A model and the key procedures involved in the first stage of therapy. Therapists were
offered 6 months of weekly case supervision via Skype™ (Microsoft Corporation, Redmond, WA, USA) or
telephone and a further 1-day workshop, which covered the key procedures used later in the therapy in
more detail.

Supervision sessions involved a review of questionnaire measures, a detailed discussion of the previous
treatment session, a discussion of any problems that had arisen and step-by-step planning for the next
session. Videotaped treatment sessions were discussed on a regular basis in the supervision sessions.

Intervention: cognitive therapy for social anxiety disorder in adolescents

All young people received CT-SAD-A. Treatment was delivered in the CAMHS clinic in which the therapists
usually worked, with behavioural experiments conducted in different environments, as required (e.g. in
a shopping centre or a café). This therapy is based on Clark and Wells’ model of the maintenance of
the disorder.31 The treatment programme consists of 14 weekly sessions, each lasting 90 minutes, plus
2-monthly booster sessions and focuses on changing social anxiety-related beliefs and behaviours, with a
particular emphasis on the four maintenance processes specified in the model.31 These are (1) increased
self-focused attention and observation (self-consciousness) and an associated decrease in observation of
other people and their responses, (2) use of internal information (feelings and images) to make excessively
negative inferences about how one appears to others, (3) use of overt and covert safety behaviours that
prevent patients from discovering that their fears are unrealistic and interfere with the initiation and flow of
social interactions, and (d) pre- and post-event processing (e.g. anticipatory worry and rumination). Given
this, CT-SAD-A involves a number of components:33 (1) the development of a personalised cognitive model
including the patient’s negative thoughts, self-images, focus of attention, safety behaviours and anxiety
symptoms; (2) an experiential exercise to demonstrate the adverse effects of self-focused attention and
safety behaviours; (3) video (and still photograph) feedback to correct negative self-imagery; (4) training
in externally focused attention; (5) behavioural experiments to test patients’ negative beliefs by dropping
safety behaviours and focusing attention externally on social situations and by also purposefully displaying
feared behaviours or signs of anxiety (decatastrophising); (5) surveys to discover other people’s views of
feared outcomes; (6) memory work (discrimination training and memory rescripting) to reduce the impact
of social trauma experiences. The components of CT-SAD-A were well received by the adolescents in the
case series.29

Key adaptations were made to CT-SAD for working specifically with adolescents on the basis of previous
experience from a development case series.29 These were as follows: (1) working with peer victimisation
(which is likely to involve school liaison) and (2) working with parents/carers. The degree of parental
involvement varied from case to case. Typically, parents are involved at the start of treatment to learn
about social anxiety and cognitive therapy and at the end of therapy to learn about their child’s relapse
prevention plan and their role in supporting their child to stay well. Some parents will bring their child to
sessions and will help to ensure that they complete their weekly measures. In some cases, therapists may
speak with parents individually to identify and work with parental social beliefs that may interfere with

4
NIHR Journals Library www.journalslibrary.nihr.ac.uk
 DOI: 10.3310/hta25200 HEALTH TECHNOLOGY ASSESSMENT 2021 VOL. 25 NO. 20

their child’s progress in therapy. As described in Chapter 3, our PPI co-applicant and young people with
relevant lived experience also gave feedback on their experiences of treatment, the treatment manual and
the reviewed treatment materials to ensure it was appropriate and acceptable for adolescents with SAD.

Measures
Young people and their primary caregivers completed measures (1) prior to treatment, (2) at each treatment
session, both to guide treatment and so that data from the last weekly session that was attended could be
used to assess outcomes at the end of treatment, and (3) after the last booster session (1.16–4.63 months
after the final weekly session). These time points will be referred to as ‘pre’, ‘sessional’ and ‘follow-up’,
respectively, in the descriptions below. Each participant (young people, parents, therapists and service
managers) who took part in the qualitative interviews was compensated for their time (£20 each).

Initial screening measures

Screen for social anxiety disorder The screening questions proposed in the NICE guideline on the
identification, assessment and treatment of SAD24 were used to screen participants for social anxiety
symptoms in the absence of any validated brief screening measures. The young person and their parents/
carers were asked whether or not the young person gets scared about doing things with other people,
whether or not they find it difficult to do things when others are watching and whether or not they ever
feel they cannot do these things or try to avoid them (see Appendix 1).

Screen for autism spectrum disorder The Social Communication Questionnaire (SComQ) was used to
identify young people with an undiagnosed autism spectrum disorder (ASD). This is a 40-item, parent-report
measure based on the Autism Diagnostic Interview Revised. It requires a ‘yes/no’ response to 19 items
assessing behaviours occurring at any time throughout the young person’s life and 21 items assessing
behaviours between the ages of 4 and 5 years. The questionnaire consists of three subscales: reciprocal
social interaction (13 items), communication (8 items) and repetitive, restrictive behaviours and interests (RRBI)
(6 items). A score above the cut-off point of 15 indicates a possible diagnosis of ASD. Therefore, young
people scoring ≥ 15 were excluded from the study.

Outcome measures
The young person self-report version of the Liebowitz Social Anxiety Scale for Children and Adolescents
(LSAS-CA-SR)34 was administered to assess the adolescents’ self-reported social anxiety symptom severity
(at all time points: pre, sessional and follow-up). The LSAS-CA includes 24 items, rated on a scale from
‘none’ (0) to ‘severe’ (3), to assess fear and avoidance of social interaction and performance (range 0–144).
The LSAS-CA has well-established psychometric properties when administered to children and young
people from 7 to 18 years of age.35 Reliable improvement in the LSAS-CA-SR is defined as a drop of
16.13 points, calculated using internal consistency data over the course of treatment.35

Symptoms of broader anxiety disorders and depression were assessed using the Revised Children’s Anxiety
and Depression Scale (RCADS),36 which is routinely collected within CAMHS as part of the Children and
Young People’s Improving Access to Psychological Therapies programme (CYP IAPT) initiative. The RCADS
is a 47-item child report scale that assesses symptoms of separation anxiety disorder, SAD, generalised
anxiety disorder (GAD), panic disorder, obsessive compulsive disorder and major depressive disorder (MDD).
Responders rate how often each item applies on a scale of 0 (‘never’) to 3 (‘always’). T-scores can be
calculated according to a child’s age and sex. The RCADS has been shown to have robust psychometric
properties in children and young people from 7 to 18 years of age.37 We focused on the social anxiety
subscale and the depression and GAD subscales, as these are the most common co-occurring difficulties.
The RCADS was administered at all time points (pre, sessional and follow-up).

The Anxiety Disorders Interview Schedule – child and parent report (ADIS-C/P)38 is a structured diagnostic
interview used to determine whether or not the young person meets diagnostic criteria for SAD and other
comorbid anxiety, mood and behavioural disorders and to establish clinical severity ratings (CSR) for each

© Queen’s Printer and Controller of HMSO 2021. This work was produced by Creswell et al. under the terms of a commissioning contract issued by the Secretary of State for
Health and Social Care. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional
journals provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should
5
be addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science
Park, Southampton SO16 7NS, UK.
 METHODS

disorder. The ADIS-C/P was administered to young people and their parents by research assistants (psychology
graduates) trained to a high level of inter-rater reliability through observation, role play and ongoing
supervision. All diagnostic interviews were discussed with an independent experienced therapist, who also
assigned diagnoses and ratings. Agreement (kappa/intraclass correlation) was > 0.85 for diagnoses and
CSRs across all anxiety disorders and for SAD specifically. The ADIS-C/P was used at two time points: pre
and follow-up.

Concentration in class scale – young person report was used to measure the young person’s self-reported
concentration in class, administered at all time points (pre, sessional and follow-up). This is a single-item
self-report scale on which the young person indicates from 0 (‘not at all’) to 100 (‘totally’) how well they
have been able to concentrate on what the teacher is saying and what they have been learning in class in
the preceding week. This scale has been used in the recent (2016) pilot work by DC/EL, and respondents
have found it to be a relevant indicator of change.29

The CYP IAPT goals progress scale is a simple 10-point rating scale on which a young person rates the
extent to which they have made progress towards their therapy goals. This scale is routinely used in both
participating NHS trusts as part of the CYP IAPT initiative, and it was also administered at every therapy
session (sessional time point).

Social functioning was examined using an 18 item self-report measure of social participation and social
satisfaction, developed by Alden and Taylor39 for use with adults with SAD. Respondents indicate on a
seven-point scale how often they have engaged in different social activities and how satisfied they have
felt with their relationships with different types of people. This scale has been used in the recent pilot work
by DC/EL, and young people have found it to be a relevant indicator of change.29 It was administered at
pre and follow-up and also as a mid-treatment measure at session 7.

The Outcome Rating Scale (ORS)40 comprises four simple rating scales on which the young person/parent
rates how they (or their son/daughter) have been feeling over the last week (individually, interpersonally,
socially and overall). This scale is routinely used in both participating NHS trusts, as part of the CYP IAPT
initiative, and was administered at all time points (pre, sessional and follow-up).

Measures of treatment mechanisms

Social cognitions and attitudes were measured using the SCQ18 and the Social Attitudes Questionnaire
(SAQ),18 both of which were adapted for children and adolescents. The SCQ is a 22-item questionnaire assessing
SAD-related negative automatic thoughts. Each thought is rated twice. First, the respondent rates the
frequency with which the thought occurred in the last week when he/she was ‘nervous or frightened’. The
frequency rating scale is as follows: 1 = ’thought never occurs’, 2 = ’thought rarely occurs’, 3 = ’thought occurs
during half of the times when I am anxious’, 4 = ’thought usually occurs’, 5 = ’thought always occurs when
I am anxious’. Second, the respondent rates the extent to which the thought was considered to be true.
The belief rating scale runs from 0 = ’I do not believe this thought’ to 100 = ’I am completely convinced this
thought is true’. The SCQ has high internal consistency and discriminant validity in adults.18 It was adapted
for use with children and adolescents on the basis of PPI consultation, in that the wording was amended
to be more developmentally appropriate in seven items, and a further seven items were added (specifically:
‘I will embarrass myself’; ‘People will be angry with me’; ‘I will wet myself/have diarrhoea’; ‘I will get picked
on/teased’; ‘I will look stupid’; ‘I will be forced to do things I don’t want to do’; ‘People will laugh at me’).
The SCQ was administered at all time points (pre, sessional and follow-up).

The SAQ is a 50-item scale that assesses beliefs that are thought to make an individual vulnerable to SAD.
The beliefs were intended to fit into three broad categories: ‘excessively high performance standards’,
‘conditional beliefs’ and ‘unconditional beliefs’. However, item selection was based more on clinical
experience than on an attempt to sample each category in a comprehensive manner. Each item is rated
on a scale that ranges from 1 to 7: 1 = ’totally agree’, 2 = ‘agree very much’, 3 = ’agree slightly’,
4 = ’neutral’, 5 = ’disagree slightly’, 6 = ’disagree very much’, 7 = ’totally disagree’. Respondents are asked

6
NIHR Journals Library www.journalslibrary.nihr.ac.uk
 DOI: 10.3310/hta25200 HEALTH TECHNOLOGY ASSESSMENT 2021 VOL. 25 NO. 20

to choose the rating that ‘Best describes how you think’. The SAQ has high internal consistency and
discriminant validity in adults.18 For children and adolescents, two further items were included (‘If I make a
mistake in a social situation people will laugh at me or be angry with me’; ‘If people see I am anxious I will
be forced to do things I don’t want to do’) and the language was changed to be more developmentally
appropriate on the basis of PPI consultation. The SAQ was administered at pre and follow-up and also as a
mid-treatment measure at session 7.

Safety behaviours associated with SAD were measured using the Social Behaviours Questionnaire (SBQ)18 –
adapted for children and adolescents. The SBQ is a 28-item scale assessing the use of social phobia-related
safety behaviours when respondents are anxious or in a social situation. Each behaviour is rated on a
four-point scale ranging from 0 to 3: 0 = ’never’, 1 = ’sometimes’, 2 = ’often’ and 3 = ’always’. The SBQ has
good psychometrics properties in adults.18 It was adapted for use with children and adolescents through
the addition of four further questions (‘Wear clothes so I blend in’; ‘Seek reassurance from my friends
and family’; Get other people to speak for me or do things for me’; ‘Have an excuse or ‘get out’ of
planned activities’) and through changes to the wording in four other items, to be more developmentally
appropriate on the basis of PPI consultation. The SBQ was administered at pre assessment and follow-up
and also as a mid-treatment measure at session 7.

The final versions of the SCQ, SAQ and SBQ that were adapted for use with adolescents can be found at
https://oxcadatresources.com (accessed 7 February 2019).

Participant acceptability rating

At the end of treatment (follow-up), participants rated how acceptable they found the treatment.41 Young
people were specifically asked to report on the extent to which they agreed with the two statements
‘I was satisfied with the treatment’ and ‘I would recommend the treatment to others’ on a five-point scale
from ‘completely agree’ to ‘completely disagree’. They also had the opportunity to complete a free-text
box to comment on the treatment.

Other adolescent self-report measures

Patient Health Questionnaire-9 (PHQ-9) is a nine-item measure assessing the presence of the main
symptoms of major depression on a four-point scale from 0 (not at all) to 3 (nearly every day) over the last
2 weeks. This is followed by one item asking young people to rate how difficult these problems have
made it for them to work, take care of things at home or get along with other people on a four-point
scale from ‘not at all difficult’ to ‘extremely difficult’. This measure is widely used and has robust
psychometric properties when used with young people.42 This measure was added (after the study had
started) to ensure that suicidal ideation was routinely monitored within sessions. Because the data were
incomplete and the measure was included (at pre and follow-up time points) for clinical management
purposes, we have not included this as an outcome measure.

Health economic records

Clinician logs (clinician includes both supervisors and therapists) Clinician’s logs were designed to
capture the amount of health-care resources (i.e. qualified staff time) necessary to implement the CT-SAD-A
treatment within a CAMHS setting. They were completed by both supervisors and therapists who recorded,
as applicable, the amount of time spent in activities related to the CT-SAD-A treatment, including training,
supervision, preparation and delivery (i.e. contact with client) of the CT-SAD-A treatment. Data recorded
in the clinician’s logs were used to calculate the total mean amount of qualified staff time used by the
NHS per adolescent treated. Results were stratified by type of staff time use (i.e. time spent in training,
supervision, preparation and delivery of the CT-SAD-A treatment).

© Queen’s Printer and Controller of HMSO 2021. This work was produced by Creswell et al. under the terms of a commissioning contract issued by the Secretary of State for
Health and Social Care. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional
journals provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should
7
be addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science
Park, Southampton SO16 7NS, UK.
 METHODS

Qualitative interviews
Interviews were conducted with young people, parents, therapists and their managers/clinical leads to
explore their experiences of the intervention and the research study. The approach and design of the
qualitative work is described in full in Chapter 7.

8
NIHR Journals Library www.journalslibrary.nihr.ac.uk
 DOI: 10.3310/hta25200 HEALTH TECHNOLOGY ASSESSMENT 2021 VOL. 25 NO. 20

Chapter 3 Patient and public involvement

W e have described PPI in line with the GRIPP2-SF reporting checklist.43 We have also provided specific
examples of PPI activity to illustrate our adherence with the NIHR INVOLVE National Standards for
Public Involvement (Table 1).44

Aim

To collaboratively involve patients as research partners at all stages of the project.

TABLE 1 National standards for public involvement: examples of how the standards were met

Standard number
and title Example of what we did to meet the standard

Standard 1: inclusive l We involved PPI representatives from a range of settings and worked with PPI
opportunities representatives individually and in groups
l PPI representatives were reimbursed for the time they gave to the study
l We advertised for a young person member of our SSC, providing a short description of
what would be involved

Standard 2: working l A PPI co-investigator was integral to the planning of patient involvement activities
together l The PPI co-investigator was able to clarify his role and communicate how he found his role,
adjusting it if need be, and facilitated discussions with other PPI representatives to ensure
that their roles worked well for them
l The Chairs of the Study Management Group and the SSC brought relevant experience and
skills (including active listening, encouragement to contribute) to genuinely involve patient
contributors in decision-making
l The PPI co-applicant is a named author on research outputs from the study in recognition
of his integral role in the study team and essential contribution to the study

Standard 3: support and l A PPI co-applicant was named and costed in the application
learning l Payment was made for PPI work undertaken as part of the research
l SSC meetings were timed to enable the PPI representative to attend after school hours
l Psychological/research terminology was explained wherever appropriate, and PPI members
were encouraged and felt able to request this as needed
l PPI representatives were able to contact the PI at any point during the study and were
guaranteed a swift response

Standard 4: l We actively asked PPI representatives about their experiences of working on the study to
communications make changes to practice
l PPI representatives were given regular feedback on the project and how their contribution
had made a difference

Standard 5: impact l We clearly stated the objectives of PPI at the outset; this was developed with the PPI
co-investigator
l Feedback from the PPI representatives was routinely recorded, with records showing how
specific feedback was acted on
l We have reported on PPI activities and our learning from them in the study final report and
will report on other publications

Standard 6: governance l A PPI co-investigator was a senior member of the research team and took part in all
decision-making groups from the outset of the study
l PPI views were reflected in the final decisions made
l The PPI co-investigator made decisions about the development, delivery, reviewing and
reporting of PPI throughout the research project

© Queen’s Printer and Controller of HMSO 2021. This work was produced by Creswell et al. under the terms of a commissioning contract issued by the Secretary of State for
Health and Social Care. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional
journals provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should
9
be addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science
Park, Southampton SO16 7NS, UK.
 PATIENT AND PUBLIC INVOLVEMENT

Methods
l An expert by experience (GS) was a co-investigator on the project and, as such, had a leadership role at
all stages of the project, including the following:

¢ Development of the research question – the study was a NIHR commissioned call, so the research
questions were prespecified; however, they were based on one of the research recommendations of
the 2013 NICE guideline on the identification, assessment and treatment of SAD.24 GS was a patient
representative on this NICE guideline and, as such, contributed to the development of research
recommendations.
¢ Development of the application – as an applicant, GS contributed to the development of all aspects
of the application, in particular how to use PPI effectively throughout the study. Examples of specific
proposals that came from his involvement include the following: (1) GS would jointly facilitate PPI
panel meetings, bringing benefits from his personal and professional experience; (2) online materials
would be produced for patients and the public about SAD in adolescence and its treatment (this
aspect of the work was discontinued, as we did not progress to the trial).
¢ Conduct of the study – as a co-applicant and member of the study management group, GS oversaw
all aspects of the study.

GS represented PPI views at the study management group to ensure that:

¢ the adapted treatment was appropriate and acceptable for adolescents with SAD
¢ the clinical outcome and health economic measures that are meaningful to patients and carers were
determined
¢ recruitment and retention of trial participants were maximised by keeping participants’ needs in mind
¢ patients’ experiences of the adapted treatment and the trial procedures were considered appropriately
¢ dissemination to potential patients was delivered effectively.

In addition, GS attended all SSC meetings to express his own views and to also ensure that the young
person member of the SSC was able to fully contribute. GS was involved in drafting the patient
information sheets, the qualitative topic guide and theme development within the qualitative analysis.
GS also co-wrote this PPI section of the final report and contributed to drafting all other sections.

l A young person who had received treatment for SAD within CAMHS was a member of the SSC.
She attended SSC meetings and, as such, contributed to decisions made about all aspects of the study.
She also consulted on the drafting of the qualitative topic guide.
l A group of four young people who had received CT-SAD-A within a pilot phase of work consulted on
their experience of treatment, the adaptation of measures, and the development of the therapist
manual and materials for young people. GS provided oversight of and input into this PPI work.
l The Anxiety and Depression in Young People Research Advisors Group (AnDY RAG) at the University of
Reading comprises young people who have had experience of anxiety and/or depression and parents/
carers. They consulted on the drafting of all patient information sheets and the qualitative topic guide.

Results
The broad approach to PPI meant that patients’ views and experiences were incorporated at all stages of
the study. Particular examples of how the study changed because of PPI are listed below; however, it is
important to emphasise that these are a set of discrete examples that do not do justice to the overall
influence of PPI input throughout the study.

l After the initial SSC meeting, GS proposed changes to the structure of the agenda to ensure that PPI
input was a standing item so that if the PPI member of the SSC had not felt able to participate at
particular points in the discussion she would always have sufficient opportunities to express her views.
We have maintained this agenda structure in all subsequent SSC meetings, for this study and others.

10
NIHR Journals Library www.journalslibrary.nihr.ac.uk
 DOI: 10.3310/hta25200 HEALTH TECHNOLOGY ASSESSMENT 2021 VOL. 25 NO. 20

l GS encouraged the inclusion of a general goals measure, with the aim of measuring the relevance of
treatment to patients’ lives in their own terms, rather than through the use of clinical measures focused
on specific elements of the cognitive model of social anxiety or specific symptoms.
l A number of changes were made to the questionnaires measuring psychological mechanisms targeted
in the treatment (the SCQ, SBQ and SAQ) on the basis of feedback from young people with SAD who
had experience of the measures as part of treatment, AnDY RAG and the community population.
Their feedback led to changes that made the language more developmentally appropriate (e.g. on the
cognition measure ‘I am inadequate’ changed to ‘I am not good enough’) and the addition of items
relevant to young people with SAD (e.g. on the measure of safety behaviours, ‘Seek reassurance from
my friends and family’).
l A number of changes were made to how CT-SAD-A treatment materials were formatted, worded and
administered, based on feedback from young people who had had experience of it. Young people also
reported generally preferring to complete worksheets by hand rather than using technology, as they
found this helped them remember key points. A number of edits were made to paper worksheets to
address issues raised by the group.
l Young people and parents/carers in the AnDY RAG provided feedback on all patient information sheets.
They made a large number of suggestions to change wording and formatting to clarify the aims and
procedures of the study. Parents particularly emphasised the importance of providing greater clarity
regarding the fact that the study was taking place alongside ‘usual NHS care’ and that participants
would continue to be CAMHS patients. This was a particular concern for many families who had had
experience of long waits and ‘battles’ to get seen within CAMHS. Young people also highlighted the
importance of the information sheets having an ‘NHS-look’ about them so that they did not raise
concerns that researchers were ‘experimenting on’ them. Changes were made to the wording and the
design of the leaflet to show that it was fully integrated within NHS CAMHS and to address the other
issues raised by the group.
l The qualitative topic guide was developed with PPI input to take into account particular difficulties that
young people with SAD may have interacting with an unfamiliar researcher in an interview setting.
Emphasis was put on the use of visual materials, to increase clarity and remove demands for eye
contact with the researcher, and on the importance of ensuring a clear message that there were no
‘wrong answers’ throughout. A number of specific items were also suggested, including questions
relating to the impact of the treatment on the young person’s relationship with their parent(s). Notably,
GS also highlighted areas in which further PPI input from young people was not needed, that is,
regarding whether or not young people should be consulted about key terminology used within
treatment, as this was not relevant to the experience or success of the treatment.
l PPI input was used in the qualitative analysis to ensure that patients’ feedback on their experience of
treatment was not inadvertently misunderstood.

A further outcome that is important to note is that the PPI representative on the SSC wrote to the principal
investigator (PI) at the end of the study to express her thanks for having had the opportunity to participate.
She stated not only that it had been an interesting experience that had helped her with her university
applications, but that she also felt that it had played an important role in her recovery from SAD. This was
in line with GS’s previous experience on the NICE Guideline Development Group in which he had the
opportunity to contribute to a daunting social situation (large group, academic demands), but in which
there was a supportive environment and his anxiety was acknowledged and accepted.

Discussion and conclusions

The PPI in this study was very effective and influenced important aspects of the study, as highlighted
above. This might have been related to several factors. First, the PPI co-investigator had received training
in relevant research methods and had extensive experience of representing patients’ views through his
previous experience on a NICE Guideline Development Group. Second, the PPI member of the SSC
also had experience of contributing to research through previous involvement in the AnDY RAG.

© Queen’s Printer and Controller of HMSO 2021. This work was produced by Creswell et al. under the terms of a commissioning contract issued by the Secretary of State for
Health and Social Care. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional
journals provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should
11
be addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science
Park, Southampton SO16 7NS, UK.
 PATIENT AND PUBLIC INVOLVEMENT

Third, the researchers were experienced at involving patient partners in their research, for example through
facilitation of the AnDY RAG for over 2 years as well as involving PPI representatives in specific studies.

GS was involved from the beginning of the study, allowing him to help shape the study from the start and
ensure that the right procedures for PPI were in place. Successful involvement of patients in this study was
also facilitated by pre-existing relationships between some of the PPI representatives and members of the
research team, the collaborative approach taken by the team, funding to finance PPI time, and a supportive
attitude of PPI involvement from the SSC.

Some limitations should also be noted. Because the PPI co-investigator role was a part-time (low % full
time-equivalent) role, GS had to accommodate the role around his main occupation. He managed this very
effectively, attending meetings in person on most occasions and, when necessary, remotely, for example
by telephone or Skype. However, it was not possible to time the AnDY RAG meetings and the meetings
with four young people who had experience of CT-SAD-A at times when the relevant researchers, young
people and GS were available within the time frame needed to keep the study on track. Going forward,
having more substantial PPI roles might help enable the flexible timing required to make full use of PPI
co-investigators. A further limitation was that, because of often tight time frames, on occasion papers for
meetings were sent to PPI members quite close to the time of the meetings. More time would have been
useful to allow PPI members to understand the materials and ask questions about relevant terminology,
which was necessary to be able to fully understand and digest the documents and prepare PPI feedback.
Finally, though it is not particular to this study, despite their sometimes extensive relevant experience (such
as GS’s involvement with the NICE Guideline Development), PPI participants may have only been involved
in PPI with certain teams, which means that they may not have always been able to identify and be
involved in how to use PPI input creatively and as far as possible. A specifically designed training course in
PPI input and research methodology may be a useful resource to help address this challenge in which PPI
representatives can become aware of a range of different examples of how successful PPI can be achieved.

Reflections/critical perspective

The PPI in the study was embedded as far as possible at every level.

A strength of the study was the combination of dedicated expert PPI, who knew the project well through
close involvement, and the observations and contributions from the young person PPI representative on
the SSC, the young people who had previously received CT-SAD-A, and the AnDY RAG who had a more
distanced take on the study and was thus able to make unique contributions.

Patient and public involvement members also fed back that they felt supported and listened to, and that
their feedback was treated as important and valuable at all times. This was especially important owing to
the nature of SAD, and it enabled them to contribute effectively.

Future studies would benefit from costing more substantial PPI time to allow greater flexibility, consistent
provision of papers, etc. at least 2 weeks ahead of all meetings and dedicated training to help PPI
co-applicants in leading the development of creative and effective mechanisms to get the most out of
PPI contributions.

12
NIHR Journals Library www.journalslibrary.nihr.ac.uk
 DOI: 10.3310/hta25200 HEALTH TECHNOLOGY ASSESSMENT 2021 VOL. 25 NO. 20

Chapter 4 Results 1: feasibility in routine child and

adolescent mental health services – recruiting and
retaining therapists and young people

Recruiting and retaining therapists

The initial protocol included a feasibility RCT to compare CT-SAD-A with a CBT programme that can be
applied across different types of anxiety disorders and is commonly used in child and adolescent mental
health settings (‘The C.A.T. Project’).45 To ensure that we had enough trained therapists to deliver treatment
within each arm (n = 3), we set out to identify 10–12 therapists to allow for a potential loss of trained
therapists. In line with our plans, we initially identified 12 CAMHS therapists within the participating services
who met our study inclusion criteria and who were willing to take part. Within the early stages of the study,
five therapists withdrew from the project (Figures 1 and 2) after being allocated to training but prior to
attending the workshop, the ‘not included in training’ box. This meant that we had to engage therapists
from additional CAMHS teams within the participating NHS trusts and ensure that there would be therapist
representation across both study arms in each of the five teams (as therapists were not able to work across
teams). In total, 20 therapists were recruited to the study and 19 were allocated to treatment arms. In total,
10 of those therapists withdrew from the study (including the five therapists mentioned above) and one
went on maternity leave towards the end of the training phase. In addition, one therapist resigned from her
NHS role but agreed to continue to participate in the study. Figures 1 and 2 show therapist recruitment and
retention to the study, and also reasons for therapist withdrawal. The final distribution of therapists across
clinical teams is shown in Table 2, with their core professional training detailed in Table 3. As can be seen,
in many locations there was only one therapist trained in each of the two therapies, which left the study
extremely vulnerable to any further therapist loss.

Owing to high service demand and staff vacancies in all participating CAMHS teams, all service managers
placed restrictions on the amount of time (and, hence, the number of trial cases) that the participating
therapists could spend on the study (meaning only one or two patient contacts per week were possible).
Although we secured excess treatment costs (ETC) to reimburse teams for the additional therapist time
taken by the trial, this was of limited help to them, as they all had unfilled vacancies and so there was no
scope to backfill existing staff (particularly as ETCs are paid in arrears). As a result, with the therapists
available, we would have been able to recruit a maximum of n = 26 (13 per arm), rather than n = 48 as
initially planned to test the feasibility of a RCT. This was likely to be an optimistic figure based on therapists
being able to seamlessly pick up new trial patients immediately after completing therapy with a previous
patient. Although there was one further therapist in Berkshire Healthcare NHS Foundation Trust (BHFT) who
was available and willing to take part in the study, none of the sites across Oxford Health NHS Foundation
Trust (OxHealth) was able to provide additional therapists due to the high demand on services and there being
a number of unfilled vacancies/staff on maternity leave across all sites. Given this, it was not possible to secure
sufficient CAMHS therapist time to be able to run the feasibility trial within the participating NHS trusts.

Recruiting and retaining young people with social anxiety disorder

We found that some of the participating CAMHS teams identified few young people with a primary SAD,
as activity was increasingly dominated by risk management. Given this, some of the participating sites
struggled to identify suitable participants for the trial, recruitment of training cases was slow, and
therapists were not always able to work at their full allocated capacity. For example, between July and

© Queen’s Printer and Controller of HMSO 2021. This work was produced by Creswell et al. under the terms of a commissioning contract issued by the Secretary of State for
Health and Social Care. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional
journals provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should
13
be addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science
Park, Southampton SO16 7NS, UK.
 RESULTS 1: FEASIBILITY IN ROUTINE CHILD AND ADOLESCENT MENTAL HEALTH SERVICES

Total therapists in service (BHFT)

(n = 16)

Met study inclusion criteria Did not meet study criteria

(n = 16) (n = 0)

Invited to take part in research Not invited to take part in research

(n = 10) (n = 6)
Reasons:
• Service leads, too heavily committed, n = 2
• Too part-time to take on additional work, n = 2
• CYP IAPT training, n = 1
• Temporary contract, n = 1

Agreed to take part Did not agree to take part

(n = 7) (n = 3)
Reason:
• Additional workload/scrutiny, n = 3

Allocated to training Not allocated to training

(n = 7) (n = 0)

Included in training (CT-SAD-A or C.A.T.) Not included in training

(n = 5) (n = 2)
• CT-SAD-A, n = 3 Reasons:
• C.A.T., n = 2 • Resigned from service (C.A.T.), n = 1
• Changed job role (C.A.T.), n = 1

Available for study (across both arms) Not available for study
(n = 4) (n = 1)
• CT-SAD-A (including one maternity Reason:
leave), n = 3 • Extra workload (C.A.T.), n = 1
• C.A.T., n = 1

FIGURE 1 Therapist recruitment and retention to the study (Berkshire Healthcare NHS Foundation Trust).
BHFT, Berkshire Healthcare NHS Foundation Trust.

14
NIHR Journals Library www.journalslibrary.nihr.ac.uk
 DOI: 10.3310/hta25200 HEALTH TECHNOLOGY ASSESSMENT 2021 VOL. 25 NO. 20

Total therapists in service (Ox Health)

n = not available

Met study inclusion criteria Did not meet study criteria

(n = 28) n = not available

Invited to take part in research Not invited to take part in research

(n = 28) (n = 0)

Agreed to take part Did not agree to take part

(n = 13) (n = 15)

Allocated to training Not allocated to training

(n = 12) (n = 1)
Reason:
• Change of job, n = 1

Included in training (CT-SAD-A or C.A.T.) Not included in training

(n = 9) (n = 3)
• CT-SAD-A, n = 5 Reasons:
• C.A.T., n = 4 • Change of job (1 CT-SAD-A, 1 C.A.T.),
n=2
• Unable to commit to time period
involved (C.A.T.), n = 1

Available for study (across both arms) Not available for study
(n = 5) (n = 4)
• CT-SAD-A, n = 3 Reasons:
• C.A.T., n = 2 • Extra workload (both CT-SAD-A), n = 2
• Unable to randomise in locality as
CT-SAD-A therapist withdrew and no
other therapist availability (C.A.T.), n = 1
• Not committed to treatment/concerns
about recruiting beyond team (C.A.T.),
n=1

FIGURE 2 Therapist recruitment and retention to the study (Oxford Health NHS Foundation Trust). Ox Health,
Oxford Health NHS Foundation Trust.

© Queen’s Printer and Controller of HMSO 2021. This work was produced by Creswell et al. under the terms of a commissioning contract issued by the Secretary of State for
Health and Social Care. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional
journals provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should
15
be addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science
Park, Southampton SO16 7NS, UK.
 RESULTS 1: FEASIBILITY IN ROUTINE CHILD AND ADOLESCENT MENTAL HEALTH SERVICES

TABLE 2 Allocation of therapists by team at each location across treatment arms and NHS trusts

Available for study (and completed

Attended training workshops training cases for CT-SAD-A)

NHS trust CT-SAD-A C.A.T. Project CT-SAD-A C.A.T. Project

Oxford Health NHS Foundation Trust

Team 1 2 1 2 1a

Team 2 1 1 1 1a

Team 3 1 1

Team 4 1 1
Berkshire Healthcare NHS Foundation Trust

Team 1 2 (+ 1 on maternity leave) 2 2 (+ 1) 1a

a Following discussion with the three therapists who were left in the C.A.T. arm when the changes to the study were
made, they all agreed that they would have no further involvement in the study.
Final number of therapists who completed study in bold.

TABLE 3 Core professional training of therapists available for study

Number of years since

Core professional training qualification (at consent)

CT-SAD-A therapists Clinical psychologist 18

Clinical psychologist 1

Clinical psychologist 5

Social worker 5

Social worker/CYP IAPT 4/same year

CBT therapist (via IAPT CBT 2

adult mental health training)

C.A.T. therapists Clinical psychologist 1

Clinical psychologist 21

Clinical psychologist 5

December 2017, only one training case was recruited on a site in which there were three therapists
participating in the study. To overcome this, we had to reach out beyond the services to identify suitable
training cases (e.g. to neighbouring services and to local schools’ hubs). However, this created challenges
for services who were already struggling to meet the existing demand.

Figures 3 and 4 show patient participant recruitment and retention to the study. Of those young people
identified as potentially eligible for the study, less than one-quarter ended up participating in the training
series for a broad range of reasons. Notably, when potential participants were assessed and confirmed as
eligible for the study, uptake and retention was high. In total, 26 out of 35 (74%) young people who were
contacted by the research team participated in the study, 12 out of 14 (86%) were retained throughout
treatment and 11 of these 12 (92%) completed the follow-up assessment.

16
NIHR Journals Library www.journalslibrary.nihr.ac.uk
 DOI: 10.3310/hta25200 HEALTH TECHNOLOGY ASSESSMENT 2021 VOL. 25 NO. 20

Identified as socially anxious in routine

CAMHS assessment (BHFT)
(n = 42)

Met study screening criteria Did not meet study screening criteria
(n = 23) (n = 19)
Reasons:
• Recent change in psychotropic
medication, n = 4
• Over 17.5 years, n = 3
• Risk criteria met, n = 3
• Low mood identified as primary at
clinical assessment , n = 2
• Learning difficulty, n = 1
• Previously received CT-SAD-A, n = 1
• Established ASD, n = 5

Contacted about research Not contacted about research

(n = 17) (n = 6)
Reasons:
• Did not want to be contacted
(1 anxious about additional
assessments/travel difficult), n = 2
• Could not be contacted, n = 4

ADIS assessment completed Did not complete ADIS

(n = 14) (n = 3)
Reasons:
• Did not want to be video recorded,
n=2
• Recruitment full, n = 1

Study inclusion criteria confirmed Did not meet inclusion criteria

(n = 9) (n = 5)
Reasons:
• Primary diagnosis of GAD, n = 1
• Primary diagnosis of depression, n = 3
• Started beta-blockers shortly before
assessment, n = 1

Included in training case series Not included in training case series

(n = 6) (n = 3)
Reasons:
• Clinician unable to take on case, n = 1
• Training postponed,a n = 2

FIGURE 3 Patient recruitment and retention to the CT-SAD-A training phase (Berkshire Healthcare NHS Foundation
Trust). a, Training was postponed from September 2016 to June 2017 while the primary supervisor was on
maternity leave.

© Queen’s Printer and Controller of HMSO 2021. This work was produced by Creswell et al. under the terms of a commissioning contract issued by the Secretary of State for
Health and Social Care. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional
journals provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should
17
be addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science
Park, Southampton SO16 7NS, UK.
 RESULTS 1: FEASIBILITY IN ROUTINE CHILD AND ADOLESCENT MENTAL HEALTH SERVICES

Identified as socially anxious in routine

CAMHS assessment (Ox Health)
(n = 26)

Met study screening criteria Did not meet study screening criteria
(n = 21) (n = 5)
Reasons:
• Risk criteria met, n = 1
• ASD, n = 1
• Prescribed medication, n = 1
• Reason not known, n = 2

Agreed to be contacted about research Did not agree to be contacted

(n = 18) about research
(n = 3)
Reason:
• Did not want to be video recorded,
n=3

ADIS assessment completed Did not complete ADIS

(n = 12) (n = 6)
Reasons:
• Recruitment was postponeda before
family contacted, n = 3
• Did not attend, n = 1
• Picked up by C.A.T. arm, n = 1
• No longer met criteria due to risk, n = 1

Study inclusion criteria confirmed Did not meet inclusion criteria

(n = 11) (n = 1)
Reason:
• Panic disorder primary, n = 1

Included in training case series Not included in training case series

(n = 6) (n = 5)
Reasons:
• Withdrew – preferred medication, n = 1
• Training postponed,a n = 2
• Withdrew – disengaged in early sessions,
n=1
• Moved overseas, n = 1

FIGURE 4 Patient recruitment and retention to the CT-SAD-A training phase (Oxford Health NHS Foundation Trust).
a, Training was postponed from September 2016 to June 2017 while the primary supervisor was on maternity leave.

Action taken

In response to the difficulties that we experienced with recruiting and retaining CAMHS therapists and
identifying eligible young people within CAMHS, we considered extending recruitment to other NHS trusts.
However, after extensive consultation with representatives from a range of CAMHS teams, it was clear that
this was not going to be possible, because (1) a substantial increase in costs would be required to be able
to conduct the required diagnostic assessments across a wider geographical area and/or (2) other NHS
CAMHS teams reported having similar problems with staffing (i.e. unfilled posts and a rapid turnover of
staff) and caseloads that were increasingly dominated by high levels of risk. As a result of the difficulties

18
NIHR Journals Library www.journalslibrary.nihr.ac.uk
 DOI: 10.3310/hta25200 HEALTH TECHNOLOGY ASSESSMENT 2021 VOL. 25 NO. 20

with therapist retention and participant identification, we concluded that the feasibility question that we
set out to address had been answered and that the proposed RCT would not be feasible in the current
CAMHS context. Given this, we felt that it was not a responsible use of NIHR funds to proceed to the RCT
phase of the study. To maximise the learning from the work and investment that had taken place, we
proposed the following next steps:

l We would complete CT-SAD-A treatment and post-treatment assessments with all the training cases to
report on an extended case series of application of the novel treatment (CT-SAD-A) in routine CAMHS
(including costs of treatment delivery).
l We would move forward qualitative work that had been planned alongside the RCT phase of the original
proposal to learn about young people and parents’ experiences of receiving this novel treatment within
a CAMHS setting to inform future work about how best to deliver specialist psychological treatments
in CAMHS.
l We would extend the proposed qualitative work to include therapists and service managers to learn
from their experiences of (1) delivering CT-SAD-A within routine CAMHS and (2) participating in
psychological therapies research in routine CAMHS, to learn both about the application of specialist
psychological treatments in CAMHS and about what needs to happen to enable CAMHS to participate
in research.

This proposal was agreed by the SSC and the NIHR HTA programme. The following chapters report on the
outcomes of the revised programme of work.

© Queen’s Printer and Controller of HMSO 2021. This work was produced by Creswell et al. under the terms of a commissioning contract issued by the Secretary of State for
Health and Social Care. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional
journals provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should
19
be addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science
Park, Southampton SO16 7NS, UK.
 DOI: 10.3310/hta25200 HEALTH TECHNOLOGY ASSESSMENT 2021 VOL. 25 NO. 20

Chapter 5 Results 2: training case series –

clinical outcomes

Patient and therapy characteristics

The 10 females and two males included in the case series had an average age of 15.17 years (SD 1.80 years;
range 12–17 years). The sample was predominantly white British (10/12), which was representative of the
participating trust areas. None of the patients reported being prescribed psychotropic medication.

As shown in Table 4, seven patients received the planned 14 weekly therapy sessions, two patients
received 10 weekly sessions and three patients received 11 sessions, 12 sessions, and 17 sessions,
respectively. Four patients received the planned two booster sessions, one received three boosters, two
received one booster and the remaining five did not receive a booster session. The follow-up assessment
was carried out, on average, 2.88 months (SD 1.15 months, range 1.16–4.63 months) after the final
weekly therapy session. Outcomes are presented graphically for each case below.

Patient outcomes

The mean and median pre-treatment and follow-up data are presented in Table 5. Given the small sample
size and the non-normal distribution of a number of the variables, pre-treatment and follow-up comparisons
were made with non-parametric Wilcoxon signed-rank tests. The z-statistic is presented in Table 5, with
associated p-values and effect size estimates (r).

Social anxiety symptom scales

Liebowitz Social Anxiety Scale for Children and Adolescents: self-report version
The Wilcoxon paired rank-sum test revealed a significant difference in LSAS-CA-SR scores from pre
treatment to follow-up, with a large effect size. Scores dropped on average 66.80% over time, ranging
from 3% to 100% (Figure 5). Ten participants, 83% of the sample, made a reliable improvement in their
LSAS-CA-SR scores. Nine participants made a reliable and clinically significant improvement, based on a
pre-treatment to follow-up change of at least 2 SDs from the original mean (a drop of 48.75 points).

Revised Child Anxiety and Depression Scale: social anxiety subscale

RCADS social anxiety T-scores showed a significant and large decrease over the course of treatment.
Before treatment, eight patients were on the borderline or in the clinical range based on their age and
sex (T-score of > 65), whereas only one patient remained in the clinical range at follow-up (Figure 6).

© Queen’s Printer and Controller of HMSO 2021. This work was produced by Creswell et al. under the terms of a commissioning contract issued by the Secretary of State for
Health and Social Care. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional
journals provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should
21
be addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science
Park, Southampton SO16 7NS, UK.
 22
NIHR Journals Library www.journalslibrary.nihr.ac.uk

RESULTS 2: TRAINING CASE SERIES – CLINICAL OUTCOMES

TABLE 4 Patient and treatment characteristics

Diagnostic profile of young person at initial assessment (diagnosis yes/no and CSR)
Young
person’s Panic
age (years) disorder
Young person’s at initial No. of weekly No. of booster Specific Specific without
ID sex assessment sessions sessions SAD phobia 1 phobia 2 agoraphobia GAD PTSD Dysthymia MDD

1 Female 16 14 2 Yes 6 No – No – No – Yes 4 No – No – No –

2 Male 17 14 2 Yes 6 Yes 4 No – Yes 5 Yes 4 No – No – No –

3 Female 12 14 3 Yes 7 No – No – No – No – No – No – Yes 6

4 Female 15 14 2 Yes 6 No – No – No – Yes 5 No – No – No –

5 Female 17 14 1 Yes 7 Yes 4 No – No – Yes 6 No – Yes 6 No –

6 Female 13 14 2 Yes 4 No – No – No – No – No – No – No –

7 Male 13 10 1 Yes 5 No – No – No – Yes 4 No – No – No –

8 Female 17 12 0 Yes 7 No – No – No – Yes 5 No – No – No –

9 Female 15 11 0 Yes 5 No – No – No – No – No – No – No –

10 Female 17 14 0 Yes 7 Yes 6 No – No – Yes 5 Yes 6 No – Yes 5

11 a
Female 14 10 0 Yes 7 No – No – No – Yes 5 No – No – No –

12 Female 16 17 0 Yes 7 Yes 5 Yes 5 No – No – No – No – No –

PTSD, post-traumatic stress disorder.
a This is the young person who did not complete the post-treatment assessment.
Park, Southampton SO16 7NS, UK.
be addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science
journals provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should
Health and Social Care. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional
© Queen’s Printer and Controller of HMSO 2021. This work was produced by Creswell et al. under the terms of a commissioning contract issued by the Secretary of State for

TABLE 5 Patient outcomes at pre treatment and follow-up available CYP IAPT and results of non-parametric Wilcoxon tests

DOI: 10.3310/hta25200
Time point

Pre treatment Follow-up Wilcoxon

Measure n Mean (SD) Median (Q1, Q3) Mean (SD) Median (Q1, Q3) Test statistic p ra

LSAS-CA-SR 12 97.41 100.09 36.18 24.50 z = –3.06 0.002* 0.62

(26.02) (79.50, 119.50) (38.31) (3.50, 75.25)

RCADS social anxiety T-score 12 74.26 70.29 44.00 41.89 z = –2.93 0.003* 0.60

(21.93) (59.46, 78.19) (15.58) (30.61, 55.13)

ADIS-C/P SAD CSR score 11 6.09 6.50 2.45 0.00 z = –2.21 0.021* 0.47

(1.04) (5.00, 7.00) (3.11) (0.00, 5.00)

RCADS depression T-score 11 68.46 67.17 47.35 44.70 z = –2.94 0.003* 0.63

(16.84) (55.93, 85.91) (15.18) (37.96, 62.67)

RCADS GAD T-score 11 57.46 56.77 38.04 37.56 z = –3.02 0.003* 0.64

HEALTH TECHNOLOGY ASSESSMENT 2021 VOL. 25 NO. 20

(9.18) (51.37, 64.49) (8.07) (29.81, 46.28)
Parent RCADS social anxiety T-score 11 76.57 81.16 58.00 52.97 z = –2.80 0.005* 0.60

(14.45) (60.23, 90.30) (15.37) (46.31, 72.03)

Parent RCADS depression T-score 11 75.34 81.08 60.21 59.32 z = –2.05 0.041* 0.44

(18.68) (54.15, 86.80) (18.23) (48.22, 69.21)

Parent RCADS GAD T-score 11 66.76 81.08 50.84 45.17 z = –2.49 0.013* 0.53

(13.62) (54.15, 86.80) (11.15) (42.28, 64.21)

Social participation 12 39.58 (15.46) 41.50 60.35 59.00 z = –2.90 0.004* 0.59

(24.00, 51.75) (19.38) (42.50, 80.75)

Social satisfaction 12 14.83 (4.97) 14.50 18.75 20.50 z = –1.96 0.050 0.40

(10.50, 19.75) (6.69) (10.75, 24.75)

continued
23
 24
NIHR Journals Library www.journalslibrary.nihr.ac.uk

RESULTS 2: TRAINING CASE SERIES – CLINICAL OUTCOMES

TABLE 5 Patient outcomes at pre treatment and follow-up available CYP IAPT and results of non-parametric Wilcoxon tests (continued )

Time point

Pre treatment Follow-up Wilcoxon

Measure n Mean (SD) Median (Q1, Q3) Mean (SD) Median (Q1, Q3) Test statistic p ra
Concentration 12 53.33 55.00 64.17 65.00 z = –1.49 0.137 0.30

(24.71) (35.00, 73.75) (28.73) (42.50, 91.25)

ORS – individual 12 39.05 32.50 49.33 46.50 z = –0.89 0.374 0.18

(36.91) (3.16, 76.00) (38.65) (7.00, 90.75)

ORS – interpersonal 12 40.61 22.15 55.55 70.00 z = –0.86 0.388 0.18

(38.75) (6.24, 84.50) (39.07) (8.13, 90.25)

ORS – social 12 24.55 10.65 46.83 48.00 z = –1.73 0.084 0.35

(28.37) (1.90, 45.75) (42.19) (3.15, 93.25)

ORS – overall 12 28.14 23.00 50.85 50.00 z = –1.96 0.050 0.40

(28.21) (3.98, 43.78) (39.43) (7.38, 94.50)

CYP IAPT goal 1 7 1.57 1 7.71 8 z = –2.38 0.018* 0.64

(7.71) (1, 2) (1.98) (7, 9)

SCQ – belief 12 61.91 68.69 13.61 3.55 z = –3.06 0.002* 0.62

(23.18) (46.29, 81.17) (20.32) (1.46, 26.93)

SBQ 12 1.63 1.71 0.64 0.45 z = –2.85 0.004* 0.58

(0.50) (1.29, 1.96) (0.59) (0.24, 0.79)

SAQ 12 4.46 4.84 2.86 2.89 z = –2.67 0.008* –0.55

(1.22) (3.13, 4.99) (0.93) (2.50, 3.69)

*p < 0.05.
Q1, quarter 1; Q3, quarter 3.
a Effect size r > = 0.1 is interpreted as small; r > = 0.3 is interpreted as medium; and r > = 0.5 is interpreted as large.46
 DOI: 10.3310/hta25200 HEALTH TECHNOLOGY ASSESSMENT 2021 VOL. 25 NO. 20

140

120

ID
LSAS-CA-SR score (0 – 144) 100 1
2
3
4
80 5
6
7
60 8
9
10
11
40 12

20

0
Pre Mid End Follow-up
Time point

FIGURE 5 Individual patient LSAS-CA-SR scores at the following measurement points: pre assessment, mid-treatment,
at the end of weekly sessions and at the follow-up assessment.

140

120

ID
RCADS-child social anxiety T scores

100 1
2
3
4
80 5
6
7
60 8
9
10
11
40 12

20

0
Pre Mid End Follow-up
Time point

FIGURE 6 Individual patient RCADS social anxiety T-scores at the following measurement points: pre assessment,
mid-treatment, at the end of weekly sessions and at the follow-up assessment.

© Queen’s Printer and Controller of HMSO 2021. This work was produced by Creswell et al. under the terms of a commissioning contract issued by the Secretary of State for
Health and Social Care. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional
journals provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should
25
be addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science
Park, Southampton SO16 7NS, UK.
 RESULTS 2: TRAINING CASE SERIES – CLINICAL OUTCOMES

Diagnostic profile
Table 6 shows the presence of diagnoses measured using the ADIS-C/P pre treatment and at follow-up
for each participant. CSRs for SAD diagnosis at both time points are given in Table 5. CSRs for all other
diagnoses are shown in Figure 7. Follow-up diagnostic data were missing for ID 11 and, therefore,
they are not included in the following discussion (i.e. N = 11 for below).

At pre treatment, all patients had a diagnosis of SAD. Only two patients had no comorbid diagnoses;
GAD was present in eight patients and depression (either MDD or dysthymia) was present in three young
people. There were five diagnoses of specific phobia pre treatment across four patients, with one patient
diagnosed with two specific phobias. One patient had a diagnosis of post-traumatic stress disorder (PTSD)
and one had a diagnosis of panic disorder. (This is detailed in Table 4.)

At follow-up, seven patients (63.64%) had lost their primary diagnosis of SAD. These same seven patients
are also those patients who fell below the LSAS-CA-SR clinical cut-off at follow-up. Six patients lost their
diagnosis of GAD, and three diagnoses of specific phobia were lost (one patient had two phobias at pre
treatment and lost one of these at follow-up).

The number of depressive disorders in the sample increased from three to four by follow-up. Of the
four patients who retained their diagnosis of SAD at follow-up, one had lost their diagnosis of depression,
two had retained their diagnosis of depression and one had acquired a new diagnosis of depression.
One patient lost their diagnosis of SAD, but had become depressed at follow-up.

Related clinical outcomes

Revised Child Anxiety and Depression Scale depression scores

There was a large and significant drop in RCADS depression subscale scores over time (Figure 8). For nine
patients, depression scores were below the suggested threshold (T < 65) at follow-up. ID 4 continued to
show elevated symptoms of depression at follow-up and also developed a new diagnosis of depression
over the course of treatment (see ADIS data in Table 6) while losing their SAD diagnosis.

Revised Child Anxiety and Depression Scale generalised anxiety disorder scores
There was a large and significant drop in GAD scores over the course of treatment, with all patients
scoring below threshold at follow-up (Figure 9).

Parent-reported Revised Child Anxiety and Depression Scale

For the group as a whole (n = 11), parent ratings on the social anxiety, depression and GAD subscales of
the parent RCADS were initially elevated (see Table 5) and reduced significantly by follow-up. Individual
data for RCADS social anxiety subscale T-scores are presented in Figure 10. This indicates that all parents
except ID 12 rated their child as making at least some improvement in their social anxiety symptoms.

TABLE 6 The ADIS diagnostic profile

Panic disorder without

Time point Social anxiety GAD Specific phobia agoraphobia Depression PTSD

Pre (n) 11 7 5 1 3 1
Follow-up (n) 4 1 2 0 4 1
PTSD, post-traumatic stress disorder.
Pre treatment and follow-up data on n = 11/12 (data on ID 11 missing as family declined follow-up assessment).
ADIS-C/P diagnosis is dichotomous (1 = with diagnosis; 0 = without).

26
NIHR Journals Library www.journalslibrary.nihr.ac.uk
 DOI: 10.3310/hta25200 HEALTH TECHNOLOGY ASSESSMENT 2021 VOL. 25 NO. 20
Pre-treatment CSR
Follow-up CSR
Time point

FIGURE 7 The ADIS CSR pre treatment and at follow-up scores. Scores of ≥ 4 indicate a positive diagnosis. a, Follow-up ADIS data missing for ID 11.

Depression
Specific phobia 2
12

Specific phobia 1
SAD
GAD
11a

SAD
Depression
ADIS diagnoses at pre treatment and follow-up for each participant

PTSD
10

GAD
Specific phobia
SAD
SAD
9
GAD
8
SAD
GAD

7
SAD
SAD

6
Depression
GAD

5
Specific phobia
SAD
Depression

4
GAD
SAD
Depression

3
SAD
GAD
Panic disorder without agoraphobia

2
Specific phobia
SAD
GAD

1
SAD
9
8
7
6
5
4
3
2
1
0
ADIS CSR (0 – 8)
© Queen’s Printer and Controller of HMSO 2021. This work was produced by Creswell et al. under the terms of a commissioning contract issued by the Secretary of State for
Health and Social Care. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional
journals provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should
27
be addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science
Park, Southampton SO16 7NS, UK.
 RESULTS 2: TRAINING CASE SERIES – CLINICAL OUTCOMES

100

90

80

ID

RCADS-child depression T scores

70 1
2
60 3
4
5
50 6
7
8
40
9
10
30 12

20

10

0
Pre Follow-up
Time point

FIGURE 8 Individual patient RCADS depression T-scores at pre assessment and at the follow-up assessment.

80

70

60
ID
1
RCADS-child GAD T scores

2
50 3
4
5
40 6
7
8
30 9
10
12
20

10

0
Pre Follow-up
Time point

FIGURE 9 Individual patient RCADS GAD T-scores at pre assessment and at the follow-up assessment.

28
NIHR Journals Library www.journalslibrary.nihr.ac.uk
 DOI: 10.3310/hta25200 HEALTH TECHNOLOGY ASSESSMENT 2021 VOL. 25 NO. 20

140

120

RCADS-parent social anxiety T-scores

ID
100
1
2
3
80 4
5
6
7
60 8
9
10
12
40

20

0
Pre Follow-up
Time point

FIGURE 10 Parent RCADS social anxiety subscale T-scores pre treatment and at follow-up.

Social participation and satisfaction

The group as a whole showed large and significant improvements in their social participation (Figure 11).
There was an improvement at trend level in ratings of social satisfaction for the group as a whole (Figure 12).

Concentration in class
The group as a whole showed a small to medium improvement in self-reported classroom concentration
from pre treatment to follow-up, although this was non-significant. As shown in Figure 13, four young
people showed either no change (ID 10) or a deterioration in their concentration (IDs 4, 11 and 12).
Patients ID 10, 11 and 12 are those young people who did not show a clinically significant improvement
on the LSAS-CA-SR and, while ID 4 lost their diagnosis of SAD, they developed a new diagnosis of
depression during treatment.

Outcome rating scales

Although no significant improvements were detected on the ORS, a small effect (at trend level) was
observed on the overall scale and social scale. The overall and social subscale ratings for each individual at
pre treatment and follow-up are shown in Figures 14 and 15.

Children and Young People’s Improving Access to Psychological

Therapies programme goal-rating

Therapists were expected to ask patients to identify their key goals and rate the extent to which they had
reached those goals from 0–10 every session. There was wide variability in the collection and recording of
these data, with data from two time points available for only 8 out of 12 participants. For one of these
participants, there were only two time points at weeks 6 and 7, so these data have not been presented,
as they do not accurately reflect initial and final progress. Data for the remaining seven patients’ primary goal
are presented in Figure 16. Initial goals data were collected between sessions 1 and 3 for these participants.

© Queen’s Printer and Controller of HMSO 2021. This work was produced by Creswell et al. under the terms of a commissioning contract issued by the Secretary of State for
Health and Social Care. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional
journals provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should
29
be addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science
Park, Southampton SO16 7NS, UK.
 RESULTS 2: TRAINING CASE SERIES – CLINICAL OUTCOMES

100

90

80
ID
70 1
2
Social participation 3
60 4
5
6
50
7
8
40 9
10
11
30 12

20

10

0
Pre Follow-up
Time point

FIGURE 11 Individual patient social participation scores at pre assessment and at the follow-up assessments.

30

25

ID
1
20 2
3
Social satisfaction

4
5
6
15
7
8
9
10
10 11
12

0
Pre Follow-up
Time point

FIGURE 12 Individual patient social satisfaction scores at pre assessment and at the follow-up assessments.

30
NIHR Journals Library www.journalslibrary.nihr.ac.uk
 DOI: 10.3310/hta25200 HEALTH TECHNOLOGY ASSESSMENT 2021 VOL. 25 NO. 20

100

90

80
ID
70 1
2
Concentration (0 – 100)

3
60 4
5
6
50
7
8
40 9
10
11
30 12

20

10

0
Pre Follow-up
Time point

FIGURE 13 Self-reported concentration in class at pre assessment and at the follow-up assessment.

100

90

80
ID
70 1
2
3
60 4
ORS – overall

5
6
50
7
8
40 9
10
11
30 12

20

10

0
Pre Follow-up
Time point

FIGURE 14 Individual patient ORS: overall subscale scores at pre assessment and follow-up.

© Queen’s Printer and Controller of HMSO 2021. This work was produced by Creswell et al. under the terms of a commissioning contract issued by the Secretary of State for
Health and Social Care. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional
journals provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should
31
be addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science
Park, Southampton SO16 7NS, UK.
 RESULTS 2: TRAINING CASE SERIES – CLINICAL OUTCOMES

100

90

80
ID
70 1
2
3
60 4
ORS – social

5
6
50
7
8
40 9
10
11
30 12

20

10

0
Pre Follow-up
Time point

FIGURE 15 Individual patient ORS: social subscale scores at pre assessment and follow-up.

10

7
ID
CYP IAPT goal-rating

6 1
2
3
5 4
5
10
4
11

0
First Last
Time point

FIGURE 16 Individual patient data for first and last available CYP IAPT goal-ratings made by patients for their
primary goal in therapy.

32
NIHR Journals Library www.journalslibrary.nihr.ac.uk
 DOI: 10.3310/hta25200 HEALTH TECHNOLOGY ASSESSMENT 2021 VOL. 25 NO. 20

The last time this was collected was between sessions 8 and 14. Six patients rated themselves as closer to
attaining their main goal at the last assessment point, while the remaining patient’s rating was the same at
both the initial and final assessment points, although it did increase at points during the treatment.

Psychological process measures

Social Cognitions Questionnaire

Social Cognitions Questionnaire – belief

All patients reported some reduction in ratings of belief in their social cognitions, and the group as a
whole showed a large and significant reduction in belief ratings on the SCQ (Figure 17). For all but three
patients, these reductions were substantial (belief rating reduced by 88–100%). The same nine patients
who showed these large reductions in the SCQ also showed a clinically significant improvement on the
LSAS-CA-SR, as indexed by a drop of 48.75 points on the scale.

Social Behaviour Questionnaire

All but three patients reported substantial reductions in their use of safety behaviours at follow-up, as
indexed by scores of 0.55 or below on the SBQ (range 0–3) (Figure 18). The same nine patients who
showed large reductions in the SBQ also showed a clinically significant improvement on the LSAS-CA-SR,
as indexed by a drop of 48.75 points on the scale.

Social Attitudes Questionnaire

For the group as a whole, there was a large and significant reduction in negative social attitudes (mean SAQ)
from pre treatment to follow-up (Figure 19). Only one patient (ID 10) showed a strengthening of their
negative social attitudes, and they also retained their SAD diagnosis at follow-up.

100

90

80
ID
Average SCQ belief rating (0 – 100)

70 1
2
3
60 4
5
6
50
7
8
40 9
10
11
30 12

20

10

0
Pre Follow-up
Time point

FIGURE 17 Individual patient SCQ mean belief scores at pre assessment and at the follow-up assessment.

© Queen’s Printer and Controller of HMSO 2021. This work was produced by Creswell et al. under the terms of a commissioning contract issued by the Secretary of State for
Health and Social Care. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional
journals provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should
33
be addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science
Park, Southampton SO16 7NS, UK.
 RESULTS 2: TRAINING CASE SERIES – CLINICAL OUTCOMES

2.5

ID
1
2 2
SBQ mean (0 – 3) 3
4
5
6
1.5
7
8
9
10
1 11
12

0.5

0
Pre Follow-up
Time point

FIGURE 18 Individual patient SBQ scores at pre assessment at the follow-up assessment.

ID
5 1
2
3
SAQ mean (1 – 7)

4
4 5
6
7
3 8
9
10
11
2 12

0
Pre Follow-up
Time point

FIGURE 19 Individual patient mean SAQ scores at pre assessment and at the follow-up assessment.

34
NIHR Journals Library www.journalslibrary.nihr.ac.uk
 DOI: 10.3310/hta25200 HEALTH TECHNOLOGY ASSESSMENT 2021 VOL. 25 NO. 20

Treatment acceptability

Eleven young people completed the treatment acceptability scale at the follow-up assessment. All of them
somewhat or completely agreed with the statements ‘I was satisfied with the treatment’ and ‘I would
recommend the treatment to others’ (Figure 20).

100
Responders agreed with statement (%)

90

80

70

60 I was satisfied with the

treatment (%)
50
I would recommend the
40 treatment to others (%)

30

20

10

0
Completely Somewhat Neither Somewhat Completely
agree agree agree/ disagree disagree
disagree

FIGURE 20 Treatment acceptability.

© Queen’s Printer and Controller of HMSO 2021. This work was produced by Creswell et al. under the terms of a commissioning contract issued by the Secretary of State for
Health and Social Care. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional
journals provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should
35
be addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science
Park, Southampton SO16 7NS, UK.
 DOI: 10.3310/hta25200 HEALTH TECHNOLOGY ASSESSMENT 2021 VOL. 25 NO. 20

Chapter 6 Results 3: costs of treatment delivery in

routine child and adolescent mental health services

Aim

The aim of the economic analysis was to estimate the cost of training therapists to deliver CT-SAD-A as
well as the cost per adolescent treated from the perspective of the NHS.

Methods

Clinician’s logs (see Chapter 2, Health economics records) were completed by both supervisors and
therapists. Data recorded in the logs were used to calculate the cost of training therapists to deliver
CT-SAD-A and the mean total amount and cost of qualified staff time used by the NHS per adolescent
treated. Results were stratified by type of staff (i.e. supervisor and therapist) time use (i.e. time spent in
training, supervision, preparation and delivery of CT-SAD-A). Time spent by supervisors training and
supervising therapists, as well as time spent by the therapists being trained and supervised, were attributed
pro-rata to each patient treated. For each study participant, all components of treatment costs, stratified by
category of resource use, were computed by multiplying units of resource use by their unit costs. Values
were expressed in 2016/17 UK GBP (£). Unit costs used and their sources are summarised in Table 7.
Results were reported in terms of total amount of qualified staff time and associated cost for training and
supervision as well as in terms of mean values of qualified staff time and associated cost per patient
treated, with variability around the mean measured by SDs. Percentage of missing values were reported.
As missing data occurred exclusively on the time spent on face-to-face therapist contact with the young
person and were highly deterministic (i.e. readily identifiable and standardised given observed practice),
a conditional imputation method was conducted whereby missing data were estimated as an average of
known durations for that patient. For patients for whom all information was missing, the average duration
across all patients was used. Analyses were conducted using Microsoft Excel® (Microsoft Corporation,
Redmond, WA, USA), and Stata® statistical software, release 15 (StataCorp LP, College Station, TX, USA).

TABLE 7 Table of unit costs

Item Cost Source Notes

Clinicians – therapists £42/hour Curtis and Burns 47

Table 12.647 – generic single-disciplinary
CAMHS team

Clinicians – supervisors £62/hour Curtis and Burns47 Table 947 – scientific and professional
staff

Cost per working hour for community-

based scientific and professional staff,
band 8a

© Queen’s Printer and Controller of HMSO 2021. This work was produced by Creswell et al. under the terms of a commissioning contract issued by the Secretary of State for
Health and Social Care. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional
journals provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should
37
be addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science
Park, Southampton SO16 7NS, UK.
 RESULTS 3: COSTS OF TREATMENT DELIVERY IN ROUTINE CHILD AND ADOLESCENT MENTAL HEALTH SERVICES

Results

Data completeness
Supervisor-completed economic logs, which included information on time spent by supervisors and
therapists in workshops and weekly case supervision practice, had a 100% completion rate. Therapist-
completed economic logs, which included time spent in delivering CT-SAD-A to patients and associated
activities (e.g. preparation time), had rates of missing values that ranged from 50% to 67% for various
treatment sessions across all patients. No data were recorded for any of the treatment sessions throughout
the whole treatment programme for 4 out of the 12 patients.

Qualified supervisors’ and therapists’ time and costs

Three supervisors provided training and supervision to therapists during the training phase of the study.
Day-long (7 hours) workshop training was usually delivered by two supervisors. Five day-long workshops
were conducted during the duration of the study, because new therapists had to be recruited partway
through the project. Weekly case supervision was generally led by only one supervisor who was occasionally
helped by a second supervisor for part of the session. Sixty-two weeks of active case supervision were
reported during the study. The time between case supervision weeks could be longer than a temporal week.
Case supervision sessions were intended to be delivered to groups of four therapists at a time but, because
new therapists were recruited at different times throughout the programme, multiple supervision sessions
were delivered to a less than optimal number of therapists, and, on several occasions, a one-to-one case
supervision session had to be offered. The mean number of therapists attending weekly supervision sessions
was 2.8 (SD 1.27).

Table 8 shows that, during the whole training phase of the study, supervisors spent overall 4740 minutes
(around 79 hours) on preparation and delivery of all of the training workshops, which would cost the NHS
£4898. The mean time and cost per workshop were 948 minutes (SD 50.20 minutes) and £980 (SD £51.87),
respectively. For the whole case supervision, supervisors spent overall 8621 minutes (around 144 hours) on
preparation and delivery of all of the case supervision. As supervisors reported overall preparation and delivery
time per week of active supervision (even if more than one supervision session may have occurred in a week),
the mean time and cost of qualified supervisor time per supervision week were 139 minutes (SD 71 minutes)
and £144 (SD £74), respectively.

TABLE 8 Supervisor time and cost for training workshops and case supervision

Activity Total timea (minutes) Total costa (£) Time (minutes), meanb (SD) Cost (£), mean a (SD)

Training workshops

Preparation 540 558 108 (50.20) 111.6 (51.87)

Delivery 4200 4340 840 (0.00) 868 (0.00)

Total 4740 4898 948 (50.20) 979.6 (51.87)

c c d
Total time (minutes) Total cost (£) Time (minutes), mean (SD) Cost (£), meand (SD)

Case supervision

Preparation 2350 2428 37.90 (20.60) 39.17 (21.28)

Delivery 6271 6480 101.15 (54.20) 104.52 (56.00)

Total 8621 8908 139.05 (71.25) 143.68 (73.63)

a Total for all workshops (there were five full-day workshops).
b Mean per training workshop.
c Total for all case supervision weeks (the number of weeks of active supervision was 62, and supervisors reported
preparation and delivery time per week).
d Mean per case supervision week.

38
NIHR Journals Library www.journalslibrary.nihr.ac.uk
 DOI: 10.3310/hta25200 HEALTH TECHNOLOGY ASSESSMENT 2021 VOL. 25 NO. 20

Table 9 reports the mean time and cost of supervisor input per therapist attending each workshop and case
supervision week. Results are stratified by time used for preparation and time used for delivery. Supervisors
devoted on average 278 minutes (SD 157 minutes) (around 4.6 hours) to each therapist during the workshop
training phase, which would cost the NHS £287 (SD £162) per therapist. The large cost was due to the long
duration of each workshop (7 hours per day) and the relatively small number of therapists attending each
workshop (two to six therapists per workshop day). The amount of qualified supervisor time per therapist was,
on average, 50 minutes (SD 14 minutes) throughout the duration of the case supervision of the programme.
The associated cost to the NHS would be £51.5 (SD £14.6) per therapist.

There were five full-day workshops; the mean number of therapists participating in each workshop was
4.4, the number of weeks of active supervision was 62, and the average number of therapists participating
in each supervision week was 2.8.

To have an estimate of the overall cost to the NHS of training therapists to deliver CT-SAD-A, the amount
of therapist time spent being trained and the associated cost needed to be estimated. Overall, 9240
minutes (around 154 hours) of therapists’ time was spent on workshop training (five full-day workshops to
train a total of eight therapists), which would equate to a total cost to the NHS of £6468. Each therapist
spent on average 420 minutes (SD 0 minutes) (7 hours) at each workshop for an NHS cost per workshop
attended by a therapist equal to £294 (SD £0).

Table 10 presents descriptive statistics referring to therapists’ time and cost of participating in the weekly
case supervision phase of their training. A total of 62 case supervision weeks were recorded, as often
weekly case supervision occurred with more than one therapist. On average, each therapist participated in
24 (SD 12) case supervision sessions.

Overall, 12,540 minutes (around 209 hours) of therapists’ time was spent on case supervision sessions,
which would equate to a total cost to the NHS of £8778. Each therapist spent in total 1791 minutes
(SD 1107 minutes) (around 30 hours) on the whole case supervision sessions, which would equate to a cost
to the NHS of £1254 (SD £775) per therapist. Each therapist spent on average 73 minutes (SD 40 minutes)
at each case supervision session; the average NHS cost for a therapist to attend a supervision session would
equate to £51 (SD £28).

TABLE 9 Mean supervisor time per therapist

Activity Time (minutes), meana (SD) Cost (£), meana (SD)

Training workshops

Preparation 26 (5.48) 26.87 (5.66)

Delivery 252 (153.36) 260.4 (158.47)

Total 278 (157.07) 287.27 (162.30)

b
Time (minutes), mean (SD) Cost (£), meanb (SD)

Case supervision

Preparation 13.7 (6.71) 14.18 (6.93)

Delivery 36.12 (9.91) 37.32 (10.24)

Total 49.84 (14.09) 51.50 (14.56)

a Per therapist per workshop.
b Per therapist per case supervision week.

© Queen’s Printer and Controller of HMSO 2021. This work was produced by Creswell et al. under the terms of a commissioning contract issued by the Secretary of State for
Health and Social Care. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional
journals provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should
39
be addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science
Park, Southampton SO16 7NS, UK.
 RESULTS 3: COSTS OF TREATMENT DELIVERY IN ROUTINE CHILD AND ADOLESCENT MENTAL HEALTH SERVICES

TABLE 10 Therapist’s total and per session time/cost of being supervised

Cost (£) per

Number of supervision
supervision Total time Cost (£) of total Time (minutes) per session
sessions (minutes) spent time spent being supervision session attended,
Therapists attended being supervised supervised attended, mean (SD) mean (SD)

Therapist 1 32 2680 1876 83.75 (37.48) 58.63 (26.24)

Therapist 2 38 2885 2019.50 75.92 (38.97) 53.14 (27.28)

Therapist 3 29 2405 1683.50 82.93 (38.53) 58.05 (26.97)

Therapist 4 34 2710 1897 79.71 (39.25) 55.79 (27.48)

Therapist 5 3 480 336 160.00 (34.64) 112.00 (24.25)

Therapist 6 19 750 525 39.47 (13.43) 27.63 (9.40)

Therapist 7 16 630 441 39.38 (13.28) 27.56 (9.29)

All – mean 24.43 1791.43 1254 73.33 51.33

(SD) (12.34) (1107.48) (775.23) (39.93) (27.95)

One of the eight therapists who attended the training workshops withdrew before taking any case and, therefore, was not
included in this table.
One therapist attended the initial training workshop and three supervision sessions, but then withdrew from the study
owing to a change in circumstances; therefore, n = 6 therapists for the report of clinical outcomes and n = 7 therapists for
cost outcomes.

Qualified clinician time and cost per adolescent treated

Tables 11 and 12 summarise the mean qualified therapists’ time per patient for each of the 14 treatment
sessions and the two booster sessions, stratified by type of activity associated with each session, as
reported by therapists. Data were reported for only half, or less than half, of the patients. Data on the
associated costs are reported in Tables 13 and 14.

Based on the available data, the main cost driver across all sessions was the therapists’ face-to-face contact
with the young person only, which amounted to a mean time of 81.02 minutes (SD 16.86 minutes) and
would cost the NHS £56.71 (SD £11.80) per patient per session. Therapists also spent a considerable
amount of time preparing for each treatment session; on average, 39 minutes (SD 21 minutes) per patient
per session, which would cost the NHS £28 (SD £15). Therapists’ time spent on administrative tasks
amounted on average across all treatment sessions to 33 minutes (SD 18 minutes), which would cost the
NHS £22.91 (SD £12) per patient per session. The overall amount of time spent by therapists on each
treatment session and associated activities was on average across all sessions equal to 177 minutes
(SD 32 minutes), which would cost the NHS £124 (SD £22) per patient per session.

Finally, Table 15 reports the mean qualified clinician time and cost to the NHS of CT-SAD-A per adolescent
treated, stratified by type of time use. Missing values in treatment sessions were handled as described in the
Methods section, and clinicians’ training costs (workshops and weekly case supervision) were attributed
pro-quota to each patient. Each adolescent treated required on average 5279 minutes (around 88 hours)
of qualified clinicians’ time and would cost the NHS £4047 (SD £1003).

It has to be noted that, as therapists’ logs were not complete, we did not have any record of potential
cancellation of sessions with the young person and, consequently, we could not account for the potential
associated costs. To the best of our knowledge, this was not a recurrent event, but it is certainly a limitation
of our results. However, on one occasion, one of the supervisors reported that the case supervision session
with therapists was cancelled at the last minute because both therapists were unable to attend the
supervision session. We did include the time that the supervisor had spent in preparing for that supervision
session in our costs.

40
NIHR Journals Library www.journalslibrary.nihr.ac.uk
Park, Southampton SO16 7NS, UK.
be addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science
journals provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should
Health and Social Care. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional
© Queen’s Printer and Controller of HMSO 2021. This work was produced by Creswell et al. under the terms of a commissioning contract issued by the Secretary of State for

DOI: 10.3310/hta25200
TABLE 11 Qualified therapist’s time per patient by treatment session: sessions 1–8

Mean duration (minutes) per therapy session

Session 1 Session 2 Session 3 Session 4 Session 5 Session 6 Session 7 Session 8

(n = 6) (n = 5) (n = 5) (n = 4) (n = 6) (n = 6) (n = 4) (n = 4)

Type of contact Mean SD Mean SD Mean SD Mean SD Mean SD Mean SD Mean SD Mean SD

CT-SAD-A: face to face 78.33 8.16 85.00 8.66 86.00 6.52 90.00 0.00 55.00 44.16 80.83 11.58 85.00 7.07 85.00 10.00
(with young person only)
CT-SAD-A: face to face 7.50 6.89 3.00 4.47 5.00 7.07 1.25 2.50 9.17 18.00 2.50 6.12 3.75 4.79 2.50 5.00
(with young person and parent)

HEALTH TECHNOLOGY ASSESSMENT 2021 VOL. 25 NO. 20

CT-SAD-A: face to face 12.50 10.84 1.00 2.24 0.00 N/A 4.75 9.50 15.00 36.74 0.00 N/A 1.25 2.50 2.50 5.00
(with parent only)

Associated administration 40.00 20.00 24.00 25.10 32.00 16.43 32.50 18.93 26.67 17.22 28.33 16.93 37.50 15.00 32.50 20.62
(e.g. writing notes/report
about client)

Time spent on individual 61.67 16.02 48.00 16.43 60.00 36.74 45.00 17.32 37.50 12.55 35.00 13.78 40.00 33.67 30.00 21.60
preparation for the session

Other 14.17 16.86 15.00 18.71 15.00 18.71 16.25 21.36 20.83 18.00 17.50 19.94 18.75 22.50 11.25 22.50

Total 214.17 24.17 176.00 18.51 198.00 21.39 189.75 17.90 164.17 33.08 164.17 20.84 186.25 33.01 163.75 35.91
N/A, not applicable.
41
 42
NIHR Journals Library www.journalslibrary.nihr.ac.uk

RESULTS 3: COSTS OF TREATMENT DELIVERY IN ROUTINE CHILD AND ADOLESCENT MENTAL HEALTH SERVICES
TABLE 12 Qualified therapist’s time per patient by treatment and booster sessions: sessions 9–14 and two boosters

Mean duration (minutes) per therapy session

Session 9 Session 10 Session 11 Session 12 Session 13 Session 14 Booster 1 Booster 2

(n = 4) (n = 4) (n = 4) (n = 4) (n = 4) (n = 4) (n = 2) (n = 1)

Type of contact Mean SD Mean SD Mean SD Mean SD Mean SD Mean SD Mean SD Mean SD

CT-SAD-A: face to face 83.75 7.50 86.25 7.50 83.75 7.50 83.75 7.50 82.50 9.57 81.25 10.31 82.50 10.61 90.00 N/A
(with young person only)
CT-SAD-A: face to face 3.75 4.79 3.75 7.50 6.25 7.50 6.25 7.50 6.25 7.50 7.50 8.66 7.50 10.61 0.00 N/A
(with young person and parent)

CT-SAD-A: face to face 2.50 5.00 0.00 N/A 0.00 N/A 1.25 2.50 15.00 30.00 0.00 N/A 0.00 0.00 N/A
(with parent only)

Associated administration 32.50 20.62 32.50 20.62 33.75 18.87 33.75 18.87 38.75 20.16 37.50 15.00 27.50 17.68 40.00 N/A
(e.g. writing notes/report
about client)

Time spent on individual 28.75 10.31 22.50 9.57 25.00 12.91 35.00 23.80 40.00 14.14 27.50 12.58 40.00 28.28 10.00 N/A
preparation for the session

Other 11.25 22.50 11.25 22.50 11.25 22.50 11.25 22.50 10.00 20 15.00 19.15 15.00 21.21 0.00 N/A

Total 162.50 34.76 156.25 34.49 160.00 31.89 171.25 27.80 192.50 51.72 168.75 30.65 172.50 67.18 140.00 N/A
N/A, not applicable.
Park, Southampton SO16 7NS, UK.
be addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science
journals provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should
Health and Social Care. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional
© Queen’s Printer and Controller of HMSO 2021. This work was produced by Creswell et al. under the terms of a commissioning contract issued by the Secretary of State for

DOI: 10.3310/hta25200
TABLE 13 Cost of qualified therapist’s time per patient by treatment session: sessions 1–8

Mean cost (£) per therapy session

Session 1 Session 2 Session 3 Session 4 Session 5 Session 6 Session 7 Session 8

(n = 6) (n = 5) (n = 5) (n = 4) (n = 6) (n = 6) (n = 4) (n = 4)

Type of contact Mean SD Mean SD Mean SD Mean SD Mean SD Mean SD Mean SD Mean SD

CT-SAD-A: face to face 54.83 5.72 59.50 6.06 60.20 4.56 63.00 0.00 38.50 30.91 56.58 8.11 59.50 4.95 59.50 7.00
(with young person only)
CT-SAD-A: face to face 5.25 4.82 2.10 3.13 3.50 4.95 0.88 1.75 6.42 12.60 1.75 4.29 2.63 3.35 1.75 3.50
(with young person and parent)

HEALTH TECHNOLOGY ASSESSMENT 2021 VOL. 25 NO. 20

CT-SAD-A: face to face 8.75 7.59 0.70 1.57 0.00 N/A 3.33 6.65 10.50 25.72 0.00 N/A 0.88 1.75 1.75 3.50
(with parent only)

Associated administration 28.00 14.00 16.80 17.57 22.40 11.50 22.75 13.25 18.67 12.06 19.83 11.85 26.25 10.50 22.75 14.43
(e.g. writing notes/report
about client)

Time spent on individual 43.17 11.21 33.60 11.50 42.00 25.72 31.50 12.12 26.25 8.78 24.50 9.65 28.00 23.57 21.00 15.12
preparation for the session

Other 9.92 11.80 10.50 13.10 10.50 13.10 11.38 14.95 14.58 12.60 12.25 13.96 13.13 15.75 7.88 15.75

Total 149.92 16.92 123.20 12.95 138.60 14.97 132.83 12.53 114.92 23.15 114.92 14.59 130.38 23.11 114.63 25.14
N/A, not applicable.
43
 44
NIHR Journals Library www.journalslibrary.nihr.ac.uk

RESULTS 3: COSTS OF TREATMENT DELIVERY IN ROUTINE CHILD AND ADOLESCENT MENTAL HEALTH SERVICES
TABLE 14 Cost of qualified therapist’s time per patient by treatment and booster session: sessions 9–14 and two boosters

Mean cost (£) per therapy session

Session 9 Session 10 Session 11 Session 12 Session 13 Session 14 Booster 1 Booster 2

(n = 4) (n = 4) (n = 4) (n = 4) (n = 4) (n = 4) (n = 2) (n = 1)

Type of contact Mean SD Mean SD Mean SD Mean SD Mean SD Mean SD Mean SD Mean SD

CT-SAD-A: face to face 58.63 5.25 60.38 5.25 58.63 5.25 58.63 5.25 57.75 6.70 56.88 7.22 57.75 7.42 63.00 N/A
(with young person only)

CT-SAD-A: face to face 2.63 3.35 2.63 5.25 4.38 5.25 4.38 5.25 4.38 5.25 5.25 6.06 5.25 7.42 0.00 N/A
(with young person and parent)

CT-SAD-A: face to face 1.75 3.50 0.00 N/A 0.00 N/A 0.88 1.75 10.50 21.00 0.00 N/A 0.00 N/A 0.00 N/A
(with parent only)

Associated administration 22.75 14.43 22.75 14.43 23.63 13.21 23.63 13.21 27.13 14.11 26.25 10.50 19.25 12.37 28.00 N/A
(e.g. writing notes/report
about client)

Time spent on individual 20.13 7.22 15.75 6.70 17.50 9.04 24.50 16.66 28.00 9.90 19.25 8.81 28.00 19.80 7.00 N/A
preparation for the session

Other 7.88 15.75 7.88 15.75 7.88 15.75 7.88 15.75 7.00 14.00 10.50 13.40 10.50 14.85 0.00 N/A

Total 113.75 24.33 109.38 24.14 112.00 22.32 119.88 19.46 134.75 36.20 118.13 21.46 120.75 47.02 98.00 N/A
N/A, not applicable.
 DOI: 10.3310/hta25200 HEALTH TECHNOLOGY ASSESSMENT 2021 VOL. 25 NO. 20

TABLE 15 Mean clinician’s time and cost of CT-SAD-A treatment per patient

Time (minutes) Cost (£)

Activity Mean SD Mean SD

CT-SAD-A treatment (face to face) 2658.15 511.98 1860.70 358.38

Supervision (therapist) 1005.00 496.12 703.50 347.28

a
Supervision (supervisor) 685.16 209.93 708.00 216.93

Workshops (therapist) 560.00 116.56 392.00 81.59

Workshops (supervisor) 370.67 77.15 383.02 79.72

Total 5278.98 1300.73 4047.22 1003.16

a Supervision sessions were usually attended by more than one therapist.

Finally, although in our analyses we focused on qualified clinician time and cost per adolescent treated, we
acknowledge that other costs were incurred during the implementation of the treatment, such as costs of
materials like memory sticks, binders, paper for information packs for therapists. All of those items were not
accurately recorded in the clinicians’ logs and, therefore, were omitted from the analysis. Although every
item of cost is important and should be included in any cost analysis and their omission is a limitation of our
work, it is unlikely that those items would have represented key cost drivers.

© Queen’s Printer and Controller of HMSO 2021. This work was produced by Creswell et al. under the terms of a commissioning contract issued by the Secretary of State for
Health and Social Care. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional
journals provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should
45
be addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science
Park, Southampton SO16 7NS, UK.
 DOI: 10.3310/hta25200 HEALTH TECHNOLOGY ASSESSMENT 2021 VOL. 25 NO. 20

Chapter 7 Qualitative interviews: approach,

design, sample and analysis

Approach

We adopted an interpretative phenomenological analysis (IPA)48 approach to the qualitative component of

the study. IPA is phenomenological and interpretive, as it is concerned with both understanding how
people make sense of their experiences and acknowledging the role of the researcher in identifying
patterns of meaning across experiential accounts. It has been used extensively to explore people’s
experiences of psychotherapy and mental health services, including adult SAD.49 It has also been used
widely to explore families’ experiences of mental health issues.

Design

The qualitative component adopted a multiple perspective design,50 which incorporated the perspectives of
young people, parents, therapists and managers. Each of these perspectives constitutes a homogeneous
subsample. IPA has a commitment to an idiographic analysis, which means that a focus on ‘the particular’ is
presented alongside and within the general themes that are typical of most qualitative methods. Constructing
each subsample around a homogeneity of perspective means that important variations within each sample can
be understood in terms of the participants’ differing perspectives on their experiences. In multiple perspective
studies, the analysis of each sample is then combined to provide a broader and more fully contextualised view.
This form of design is particularly effective for identifying conflicts and congruences between the experiences
of different groups that are engaged with a shared phenomenon. For example, in this case each sample
(young people, parents, professionals) had a perspective on the same form of treatment, but the perspective
varied depending on whether one was receiving, supporting or delivering the intervention. As with most
qualitative approaches, IPA does not aim to produce an objective, generalisable account; instead, the aim is to
develop an interpretation that is credible and contextualised. That is, we set out to produce a plausible and
transparent analysis of the patterns in our participants’ accounts.

Recruitment

The research team approached all families in which a young person had initiated CT-SAD (14 families),
all therapists who were trained in CT-SAD (eight individuals) and all service managers responsible for
a participating therapist (four individuals), to invite them to take part in the qualitative component of
the research.

Sample

All of the young people, their parents/carers, therapists and their managers/clinical leads who had
consented to the study were invited to participate. Six young people, six mothers, seven therapists and
three managers consented to take part in the interviews.

Eight young people and their parents did not participate (one was still in treatment at the time of
interviewing, one had moved abroad, two were uncontactable and four declined to take part). All the
participating parents/carers were mothers. One therapist did not participate – this therapist had attended
the training workshops but had not had any training cases and could not be contacted regarding

© Queen’s Printer and Controller of HMSO 2021. This work was produced by Creswell et al. under the terms of a commissioning contract issued by the Secretary of State for
Health and Social Care. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional
journals provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should
47
be addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science
Park, Southampton SO16 7NS, UK.
 QUALITATIVE INTERVIEWS: APPROACH, DESIGN, SAMPLE AND ANALYSIS

participation in the qualitative component. Service managers/clinical leads were invited on the basis that
they had service level responsibilities at their locations and were involved in the research at this level in some
capacity. One manager did not participate as a result of a long-term absence from work. (Information about
the participants are provided in Tables 16–19, with pseudonyms used as in the results section.)

TABLE 16 Young people who participated in the qualitative interviews

Patient number, Interview Months Age

as used in the duration Diagnosis at post-treatment since end of (years) at
Pseudonym case series (minutes) assessment treatment interview

Jane N/A 23 Post treatment not completed (young 21 15

person completed only two sessions
of cognitive therapy)

James 7 17 None 21 15

Julia 4 20 MDD 8 16
Jade 6 26 None 5 14

Jennifer 8 29 None 1 18

Jasmine 12 20 Social anxiety disorder, specific phobia 3 16

of spiders, depression
N/A, not applicable.

TABLE 17 Parents who participated in the qualitative interviews

Pseudonym Interview duration (minutes)

Polly 30

Pippa 13

Phoebe 29

Paula 36

Patricia 24
Pamela 44

TABLE 18 Therapists who participated in the qualitative interviews

Pseudonym Interview duration (minutes) Background Number of cases

Caroline 41 Clinical psychologist 2

Conner 53 Social worker 3 (2 full)

Christopher 21 Social worker 0

Claire 58 Clinical psychologist 2 (1 full)

Carol 55 Clinical psychologist 3 (2 full)

Camilla 45 Social worker 2

Charlotte 49 CBT therapist (via adult IAPT) 2

48
NIHR Journals Library www.journalslibrary.nihr.ac.uk
 DOI: 10.3310/hta25200 HEALTH TECHNOLOGY ASSESSMENT 2021 VOL. 25 NO. 20

TABLE 19 Managers/leads who participated in the qualitative interviews

Pseudonym Interview duration (minutes) Role

Margaret 24 Clinical team manager

Marian 23 Clinical lead

Monica 21 Head of psychological therapies

Data collection and transformation

Interviews were conducted with individual participants (rather than in groups) so as to make interviews less
daunting for young people with a history of SAD and for practical reasons (given the geographical spread of
participants and the busy schedules of participating therapists and service managers). Indicative topic guides
were used. Interviews were conducted by the trial assessors or a research fellow with qualitative expertise. They
were all trained and supervised by a specialist in qualitative research (ML). Interviews with young people were
devised to be engaging and sensitive to the interpersonal and communicative preferences of this population.

Interviews took place in person or over the telephone. All interviews were audio-recorded and transcribed
in full and anonymised at the point of transcription.

Qualitative data analysis

Analysis involved detailed coding (to identify the experiences, claims and concerns of each participant),
followed by the elaboration of emerging themes for each case. In keeping with a multiple perspective
design, these themes were brought together at the level of each subsample group, before any form
of between-sample synthesis was attempted. Coding for the three samples (young people, parents,
professionals) was led by three different members of the research team (EB, SP, LT) with supervision and
comparative coding provided by ML. The coding team worked together closely to compare both the
organisation and the format of their work, and also the focus and plausibility of the codes themselves.

Credibility of the qualitative analysis was checked via analytic triangulation on two levels. First, in-depth
triangulation was provided through supervision. This involved co-analysis of selected transcripts; use of reflexive
practices in supervisory discussions; and audit checking for plausibility, coherence and comprehensiveness of
analysis at each key stage of the process, with each subsample. The aim of this level was to maintain high
standards of analysis.

Second, a broader triangulation exercise was conducted to check the appropriateness of the theme titles,
the persuasiveness and plausibility of the overarching narrative, and the intelligibility of the overall structure.
This involved a presentation of the draft analysis by the researchers conducting the qualitative analysis to both
the SSC and the Study Management Group, which included the investigators, the PPI lead and the young
person representatives with relevant lived experience, as well as independent therapists and researchers with
relevant experience. These groups were invited to comment on the presentation and provide feedback on
the sensitivity to the context, the rigour, transparency, coherence and utility of the analysis (after Yardley51).
The aim of this level was to check that the analysis is meaningful, useful and accessible.

The final analytic structure is detailed and complex, and can be used as the basis for identifying cross-cutting
themes within and across samples. In subsequent reporting for publication, we will take this approach,
presenting our analyses in the thematic format that is typical of IPA work. For the purposes of this document,
what is presented in the results is a more ‘concrete’ precis that presents information extracted from the
case-level and sample-level summaries to address the stated research questions of the evaluation as directly
as possible. For the most part, we present this preliminary analysis by focusing on each separate subsample.
Subsequent reporting for publication will incorporate a more integrative analysis.

© Queen’s Printer and Controller of HMSO 2021. This work was produced by Creswell et al. under the terms of a commissioning contract issued by the Secretary of State for
Health and Social Care. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional
journals provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should
49
be addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science
Park, Southampton SO16 7NS, UK.
 DOI: 10.3310/hta25200 HEALTH TECHNOLOGY ASSESSMENT 2021 VOL. 25 NO. 20

Chapter 8 Results 4: young people’s and parents’

experiences

Young people’s perspectives

On the basis of their interviews, young people generally appeared to find the intervention acceptable. The
young people reported that the therapists were calm, friendly and understanding. For example, one young
person commented: ‘[H]e was open [. . .] I could just talk for quite a long time and he wouldn’t like judge
me or anything’ (James).

Varied motives and experiences

Before starting CT-SAD-A, the idea of therapy was framed as potentially difficult and challenging. Young
people described a range of expectations – for example, some were sceptical and ‘didn’t really think it was
going to work’ (Julia), or felt that it ‘wouldn’t be any different’ (Julia) from previous counselling experience;
some expected that they would be ‘sat in a room talking the whole time’ (Jennifer), whereas others thought
they were going ‘to talk about er, [their] problems’ and ‘learn some like, er, er, um techniques for them’
(James). One young person acknowledged that the only experience they had of treatment was ‘things on
TV . . . and those are quite a distorted image, so [they] had no clue [what to expect]’ (Jade), and another
expressed ‘hope’ that the therapy would bring about improvements to their independence and confidence.

In the pre-treatment assessments, young people described their desire to reduce their levels of social
anxiety. However, in the research interviews, many of the young people reflected on their motivations for
engaging in this treatment or via this context. For example, some young people commented that they felt
motivated by being part of a research trial in which they would be helping others, while other young
people were motivated by the idea of a more interactive therapy:

I felt good doing it, because I felt like I was helping people.
Jade

Um, yeah, i–it was kind of good to know that it was all going to research, it wasn’t just, like, going
towards me or anything, it was kind of going towards like a bigger thing.
James

Instead of just like, you know, saying ‘ahh I think I can get over it’, just actually doing them umm, that
umm, yeah and umm that seemed like quite a reasonable idea.
Julia

A challenging, helpful and active experience

The therapy itself was generally represented as challenging but helpful. Most young people were able to
identify and reflect on features that were helpful. The pacing was felt to be appropriate, and the repetition,
practice and ‘building on’ progress were valued. For example, here, Julia reflects on how the rehearsal of
strategies meant that they were subsequently easy to access:

It’s been good, because it was so repetitive in the treatment that I like can, it’s like, something that
just comes to me naturally now it’s not, it’s not such a hard thing to do.
Julia

© Queen’s Printer and Controller of HMSO 2021. This work was produced by Creswell et al. under the terms of a commissioning contract issued by the Secretary of State for
Health and Social Care. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional
journals provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should
51
be addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science
Park, Southampton SO16 7NS, UK.
 RESULTS 4: YOUNG PEOPLE’S AND PARENTS’ EXPERIENCES

Some therapists had involved parents more actively in some aspects of the therapy, and this was also
largely experienced as helpful and positive by the young people who were interviewed. For example, a
number of young people reported that their parents would attend parts of sessions, which ‘kind of help
[ed], like, in that respect, because it meant my mum could help carry it on with me after’ (Julia), or support
them outside of sessions:

She was very involved because she had to be the one to stop doing a lot of things for me.
Jennifer

Using measures to decide where to focus and to track progress

Many of the young people reported that they found the measures difficult and described them as ‘really
thick, dense and . . . seem[ed] really, really daunting’ (Jade). Some described them as ‘tedious’ (Julia), and
many reported that they took ‘quite a long time’ (Julia) to complete. Nonetheless, for many of the young
people, the use of measures during the therapy was experienced as helpful, and some described how the
measures ‘kind of just became part of the treatment’ (Jasmine). The measures were seen as helpful in
two ways. First, the measures provided a focus for treatment (e.g. choice of a behavioural experiment),
as illustrated here by Julia:

It meant that we could focus on specific things that were harder than others and I think that was
quite useful because I don’t think–, if–, I don’t think, without the questionnaire, it would have been so
obvious the things that needed to be worked on.
Julia

Second, the measures were perceived as helpful, because they allowed young people to see their progress
represented visually:

It was quite useful cos you could see, you could compare the different things, what I’d done each
week to see what I was progressing in and what I wasn’t.
Jasmine

Identifying active ingredients

In terms of therapeutic mechanisms, it was striking that the young people were consistently able to identify
distinct features of the interventions that they could discuss knowledgeably, and evaluate in terms of their
utility. This is not always the case in studies of therapeutic experience. Behavioural experiments were generally
reported as one of the most useful and important features of the therapy (e.g. ‘I definitely think the ‘training
my attention’ and the ‘behavioural experiments’ were, like, key for me’; Julia).

The young people recognised and appreciated the importance of the therapy as an active process of
doing, rather than as just a space for talking and reflection. This is concisely illustrated here by Jade:

I would go and interact with people outside, which I didn’t like doing, but it was really, really helpful
[pause] because [pause] just talking about it wouldn’t have been enough.
Jade

Note that Jade says, ‘which I didn’t like doing, but it was really, really helpful’. Behavioural experiments
were a key component of this activity, and, interestingly, they were also often represented as the most
challenging part of the therapy: they were seen as difficult to do but worth doing. We can see how the
balance skews towards the positive in Jade’s repetition of ‘really, really’. This was typical of how the young
people evaluated their involvement in behavioural experiments:

I hated doing it, I absolutely hated it . . . but that was probably my favourite bit because it did the
most for me.
Jennifer

52
NIHR Journals Library www.journalslibrary.nihr.ac.uk
 DOI: 10.3310/hta25200 HEALTH TECHNOLOGY ASSESSMENT 2021 VOL. 25 NO. 20

I was so scared to do that as well, but it was nice to know at the end, like, I could do it. So I would
say that was helpful, it was difficult at the time, but it was necessary I think.
Jennifer

I just didn’t like doing the things because I felt uncomfortable, but I see the, the whole point that we
did it was to challenge the way I thought, and it did.
Jade

The complex response here underlines the importance of including qualitative methods in studies of
acceptability and mechanism. Had our interviews been responding to a rating scale here, we might have
been detecting the affective content of the appraisal (‘I hate it’) and missing out on the evaluative
reflection, which recognises the benefits (‘it was my favourite bit because it did the most for me’).

Other memorable features were the attention-training exercise, and the self-focused attention and safety
behaviours experiential exercise (two conversations with different instructions for focus of attention and
safety behaviours) and related video feedback. Young people were less clear about how and why these
components had been helpful, in comparison with the strong endorsements around behavioural
experiments and measures. For example, the experience of the two conversations appeared to be mixed.
Some young people seemed to have engaged with the learning behind the experiment:

I was expecting there to be a big difference with the two videos, like umm you know look really
awkward in the first one and there wasn’t that much of a difference, so it made me realise I don’t
look as anxious to other people as I may think in my head.
Jennifer

Others seemed less clear about the learning behind the exercise:

I recognised the safety behaviours, it was easier to try and not do them and it was good to take them
out and focus on them.
Jasmine

Learning from and with the therapist

The role of the therapist in establishing a collaborative relationship around these components seemed to
be important to the young people. This is generally evident in the language used to report on these
activities, which tended to highlight the collective voice:

We looked at my umm, behaviours and stuff that linked to it and then the things that I found the
most umm, like, scary to do and then, after we went through them, we decided on things we could
do that could like, get over them, and then we cos we were it’s quite easy to go to like shops and
cafes and stuff and we just gradually built it up to things that I found the worst.
Julia

We’d figure out umm, what we could do umm, and then we can go and ask someone to if they know
what the time is in the street and then we’d go and test it out umm, and then go back and reflect on
it and see how it actually was how I felt.
Jennifer

It was also illustrated by examples of therapist modelling and engaging in activities:

[Therapist] would stutter, er, then we’d see their reaction, but erm, yeah, he, he showed me that there
wasn’t, people didn’t really care, to be honest with you if you stuttered.
James

© Queen’s Printer and Controller of HMSO 2021. This work was produced by Creswell et al. under the terms of a commissioning contract issued by the Secretary of State for
Health and Social Care. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional
journals provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should
53
be addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science
Park, Southampton SO16 7NS, UK.
 RESULTS 4: YOUNG PEOPLE’S AND PARENTS’ EXPERIENCES

One young person expressed concerns about being able to apply what she had learned during therapy.
She sometimes struggled to see the relevance of the activities to her wider experiences, for example, in
one behavioural experiment she ‘had to go around and ask people for the time which [she didn’t] think
[she’d] ever need to do again’. This may have been a consequence of not having fully engaged with the
model underlying the intervention. As a result, this young person struggled to generalise the things she’d
learned in these activities to situations that she found herself in outside treatment:

It was quite hard, I think. [Pause] because [pause] I do things there [pause] but I–, I had to know how
to adapt it [pause] which was hard, yeah.
Jade

However, this young person also experienced depression, and aspects of her problems with low mood
appeared to interfere with the focus on social anxiety. She was advised to apply things she had worked on
in the sessions to her experience with mood, but she struggled to do this:

I did ask during the treatment, um, do you know how ki–, how I could get help for depression and
they said oh, and [Therapist] said, um, ‘You can use the same techniques’ but I didn’t really
know how.
Jade

For her, it ‘felt like the ending [of the treatment] was very very quick’ and that she ‘didn’t get much
closure’ (Jade). For example, she described weekly sessions, followed by one 2-week gap and a final
session after a further month (close to that described in the protocol), and felt that these sessions ‘tapered
off really quickly’. There was a general sense that the young person had been left slightly ‘behind’ the
pace of the therapy or outside the understanding of the therapist.

The other young people were generally confident that they could identify what they learned and apply it.
The importance of practice (outside the sessions) seemed to underline this for most of the young people.
For example, one young person noted that ‘remember[ing] all of the things [they’d] learned and then
just keep going over it and over it’ helped after the sessions had ended, as ‘then it didn’t seem like such
a difficult thing not having a session every week.’ (Julia). Some young people mentioned that the
‘experiment sheets’ were really helpful for this:

Yeah I got given like experiment sheets to fill out and do the same things in between like say on my
lunch break, umm, go and buy something myself from the shops cos I would never do anything like
that umm and then I’d have to fill out these sheets and then bring them back to my next session.
Jennifer

Of course, we might reasonably expect the young people to vary in terms of their preferences and motives
for engaging in different aspects of the treatment, as they do here. The strong preferential consensus
about the active focus of treatment is all the more striking against the background of variation with regard
to other aspects of the experience. For the most part, the young people were also very positive about the
consequences. Some reflected on the ‘recalibration’ of their worries:

Some really big things I still find quite daunting. I don’t know if I’ll be able to do that, but I think,
I kind of have a lot of the tools to kind of try and build up my confidence and stuff, so I think that will
be useful.
Jasmine

54
NIHR Journals Library www.journalslibrary.nihr.ac.uk
 DOI: 10.3310/hta25200 HEALTH TECHNOLOGY ASSESSMENT 2021 VOL. 25 NO. 20

Young people were often able to identify areas of significant functional progress that were important to
them, too. For example, here, James describes several improvements since engaging in the therapy:

I used to go to the support area, [. . .] for about 3 hours a day every school day. I haven’t been in
about a year . . . I’d never put my hand up to answer questions, I wouldn’t really go to lessons that
much, er, but now I kinda answer questions all the time and talk to people and all that. [. . .]
Performing was a big problem as well, but I, I do music now.
James

To summarise, young people appeared to find some aspects of the intervention challenging, but they
reflected on the benefits (especially the functional improvements) outstripping the costs. They found the
approach taken by their therapists to be open and non-judgemental, and the therapeutic process to be
active. They identified the use of measures and behavioural experiments as being particularly helpful.

Parents’ perspectives

On the basis of their interviews, parents generally appeared to find the intervention acceptable.

Keeping an open mind

The parents often shared ‘diffuse’ expectations about the therapy. They were unsure about what to
expect, but they described their initial thoughts as hopeful or being ‘quite interested to see how it would
[. . .] benefit’ their child. Experience of the problem and/or other interventions was framed as a ‘why not
give it a try?’ opportunity by some parents:

No expectations, just; if it works, great, if it doesn’t, then, you know, at least we’re in the system and,
um, we can get some help.
Pamela

It didn’t work so well with [the young person], but every child’s different, every child’s different,
so yeah, give it a go, you’ve got nothing to lose.
Patricia

However, others were ‘really excited to be part of the study’, as they felt that there would be ‘a lot more
audit, maybe, on [. . .] how the sessions were going’ (Phoebe) in comparison with experiences of standard
services, prior to taking part in the study.

Noticing functional improvements

Parents were generally very positive about the impact of the intervention in terms of what the young
person was now able to do. Some reported that it was ‘really helpful [. . .] very accommodating and [they]
wish[ed] it was more available’ (Polly), or that the improvements that their child had made were ‘leaps and
bounds very positive’ (Pippa). Parents noticed improvements in the young persons’ activities – particularly
their engagement and involvement with friends, their autonomy and independence in communication,
and their confidence in initiating activities outside the house. For example, one young person who had
previously found it difficult to stay overnight at a youth event had, since the end of treatment, stayed at
the event for an entire week and had further been selected to attend a 3-week event abroad:

He’d always liked the idea of going away for a weekend [. . .] but could never stay overnight [. . .] that
was a big breakthrough for us, with him staying overnight [at event] [and now he’s] off [overseas] for
three weeks next July.
Paula

© Queen’s Printer and Controller of HMSO 2021. This work was produced by Creswell et al. under the terms of a commissioning contract issued by the Secretary of State for
Health and Social Care. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional
journals provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should
55
be addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science
Park, Southampton SO16 7NS, UK.
 RESULTS 4: YOUNG PEOPLE’S AND PARENTS’ EXPERIENCES

These were incremental changes, which one parent described as ‘a very gradual process of, sort of,
confidence’ (Pamela), and could not be pinpointed to a specific time in treatment. However, by the end of
treatment, these improvements stood out as markedly different from the pre-treatment phase:

I don’t even know how to quantify how much better she is.
Polly

Parents reflected on the psychological changes that appeared to underpin these functional improvements.
For example, one parent described their child’s ability to overcome difficult situations by shifting her focus
of attention and drawing on learning from previous experiences:

When she went in for her theory test, you know, she was like, um, like she said ‘I took a deep breath,
thought; right, I can do this, nothing is going to happen, it’s going to be fine’ and, you know, it’s
those sort of things, like, that she, um, looks around her and her environment and takes it all in and
knows that it’s, it’s OK.
Pippa

Appreciating the opportunity to be involved and informed

Parents appreciated the flexibility of the therapists. This was reflected in the flexibility of the session length,
the appointment times, the information about the young person’s progress and the opportunity for
involvement in the therapy. This last point was generally to do with understanding the therapy so that they
could support and extend the young person’s progress outside the sessions. Some parents had a positive
experience of feeling better equipped to support the young person outside sessions:

Lots of things that [. . .] she would tell me, I would um, the ideas of the stuff that I would try to, um,
every time we went out, I would push her a little bit more to do little, little things and anything to help
her along the way.
Pippa

Whereas, others felt more like they were ‘on the back burner’ and found it ‘a bit tough, not knowing
what’s going on and just hoping that she was OK’. For example, Patricia felt less able to provide input
between sessions because she was unaware of what was happening during sessions:

I think it would have been easier if maybe I kinda sat in on the first ones so I know what she was
going through and then, if she wanted to talk about how it went, then I could say, you know, I could
have had input because I knew, you know, that I had something to, to, yeah, base it on.
Patricia

There were varying degrees of involvement. For example, some parents felt that it was a ‘case of just
keeping [them] in the loop and giving [them] a bit of evaluation of what is going on’, while others felt well
informed but appreciated that the therapist was best placed to lead this:

We would have a little meeting for, sort of, 15–20 minutes and the things that [the young person]
was willing to share, um, I was told about . . . I think, obviously [therapist] must have encouraged or
made it part of the process to, to share those feelings with me, which of course is helpful.
Polly

Families varied in the extent to which they mobilised people around the young person. For example, Polly
commented on having ‘lots of help’ throughout the therapy process and reported that she’d ‘talked to
friends about it’ who had ‘listened, sympathetically’ (Polly). In contrast, Pamela chose not to involve others,
reporting that it was ‘not the sort of thing [she] would tell [her] friends’, and that she ‘[hadn’t] really told
[her] mum and dad the extent of it either’, because she thought ‘they’d be upset about it’, and the family
‘didn’t tell [the young person’s] little sister about it’, because they ‘didn’t feel she’d understand’.

56
NIHR Journals Library www.journalslibrary.nihr.ac.uk
 DOI: 10.3310/hta25200 HEALTH TECHNOLOGY ASSESSMENT 2021 VOL. 25 NO. 20

Reflecting on relationships, roles and needs

Parents generally appreciated that the therapist was able to support the young person in ways that were not
easily available to parents; they acknowledged that ‘it’s different when it comes from a parent somehow’
(Phoebe), but they wanted to be able to support the young person in ways that were consistent with what
the young person was learning. There was an appetite for parental training, to provide this knowledge and
to build confidence. For example, some parents found it ‘difficult to know what you should or shouldn’t be
doing at home to help her and, um, or maybe what you’re doing that isn’t helping . . .’ (Pamela). Sometimes
the therapist was appreciated, because they provided a new or different route for communicating with the
young person.

Parents reflected on changes in their relationship with the young person. The time travelling to and from
therapy was mentioned as a generally positive and shared experience by parents:

We would have that little bit of time in the car, just me and her, on the way, we’d chat about all sorts
of stuff and I do, as [the young person] is a teenager, I don’t really get that time with her anymore,
so actually it was quite nice, the sessions were quite nice in that respect . . .
Pamela

The sense that the problem was being addressed also provided relief from stress to the parents in some cases:

She’s not hitting rock bottom anymore . . . it does take the pressure off.
Patricia

However, others felt that they needed their own support:

I didn’t feel very supported as a parent, but I don’t know if that’s maybe something that I should have
sought help for separately.
Pamela

Some parents also reflected on their own relationship with the therapist. In most cases, this was in the
context of a contrast with the young people’s relationships with their therapist and the relative importance
of these different relationships. For example, one parent reported that ‘[I]t’s not about me, it’s about him
and, actually, he needs to be completely comfortable, um, with [the therapist] in order to maximise the
benefits’ (Paula). Another mentioned that they were ‘solely focused on [the young person] and [they]
didn’t really have the same relationship with [therapist] that she did’. (Pamela). However, another parent
placed more importance on her relationship with the therapist and described the importance of the
therapist giving the parent her own space to talk about her own concerns, reporting that ‘it kind of helped
[her] to take some of the load off’ (Patricia).

The age of the young people’s siblings was varied, and siblings were not always involved in or informed
about the problem. Some siblings were actively involved in supporting the young people; one parent
speculated that ‘because [the young person’s sister]’s sort of observed how poorly he has been, she would
just obviously recognise anything that would help him would be a good thing’ (Paula), and some siblings
were reported to have found the young person’s reliance on their support difficult. Siblings sometimes had
to ‘fill in the gaps’ in situations in which the young person’s social anxiety prevented them from doing
something seemingly simple for themselves, such as asking for a table in a restaurant or going to the
counter in a shop:

It’s been difficult throughout for her sister to understand . . . [the young person] couldn’t do it and her
sister found that extremely frustrating.
Phoebe

© Queen’s Printer and Controller of HMSO 2021. This work was produced by Creswell et al. under the terms of a commissioning contract issued by the Secretary of State for
Health and Social Care. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional
journals provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should
57
be addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science
Park, Southampton SO16 7NS, UK.
 RESULTS 4: YOUNG PEOPLE’S AND PARENTS’ EXPERIENCES

Reflecting on the context of help-seeking

Parents did not express very strong views about the CAMHS context of the therapy. Parents did provide
more discussion of this than the young people, but this was largely based on not knowing what to expect.
One parent said that ‘not having dealt with any mental health service before . . . it was it’s all new and
a bit scary’. (Pamela). The difficulty of accessing services was also a point of focus in some parents’
interviews. One parent described the referral process to CAMHS as ‘really lengthy’, and another felt that,
had they not taken part in the trial, they ‘would have had to wait another year and [mum did not] know
what would have happened’ (Polly). Others reported that their difficulty in accessing CAMHS lay in either
themselves, or the young person’s school or college not knowing where to go for help:

I tried to get through to them at college when she was starting her apprenticeship, but I wasn’t
getting much joy there.
Pippa

A number of the young people in the trial had a comorbid diagnosis of depression or experienced
subclinical low mood. For some parents, the comorbid depression represented a barrier to engaging in
the treatment:

Probably her underlying depression wasn’t you know, it wasn’t being treated at that time and I think
that restricted her access to the CBT.
Phoebe

Others felt that, although the focus on social anxiety was helpful, there was some low mood left at the
end of the treatment that was not then addressed:

I think, although it’s fixed the social anxiety, or helped with that, it hasn’t helped with her low
mood . . .
Pamela

However, low mood was not a barrier for all of the young people who had mood problems at the start of
therapy. For example, although one young person ‘was quite worried about it and thought, you know,
“how’s this going to help me?”, because her mood was quite low as well’ (Pippa), her parent reported
that this worry was not fulfilled, as she thought ‘it went very well . . . and, now, the end result, it was so
worth it’ (Pippa).

Identifying active ingredients

Parents were able to identify features of the intervention that had struck them as important. These were
generally framed in terms of their perceived helpfulness. As with the young people, the emphasis was
often placed on the active components of treatment:

It’s very difficult to sit in a room for an hour and talk about yourself, I think, you know, to be able to
go out and do those physical challenges, um, is probably a bit easier.
Phoebe

A number of parents commented on the fact that this approach was a better fit for the young person:

He felt that it was more around, it was more experiential and it was more, um, er, of a pragmatic,
practical, um, treatment programme, rather than sitting and just talking about what he knew were
issues . . . so, right from the outset, it felt different and, actually, never ever minded going.
Paula

58
NIHR Journals Library www.journalslibrary.nihr.ac.uk
 DOI: 10.3310/hta25200 HEALTH TECHNOLOGY ASSESSMENT 2021 VOL. 25 NO. 20

Parents were aware that young people found some of the active aspects of treatment difficult:

[Therapist] took her out and did practical things with her, which I thought was very good, much to
[the young person’s], didn’t like it, but it it helped her, you know. She probably thought that was the
worst things for her to do, but actually doing it, it proved to her that it worked.
Pippa

Others emphasised the importance of having someone to talk to in a non-judgemental environment:

I think just talking to someone, as I said, regardless really of what the therapy is, I think that she could,
well I, I can’t discount [the treatment] obviously, but I, I honestly think, just the process of talking to
somebody in a non-judgemental environment . . . was invaluable.
Polly

Accepting the need for measures

The parents sometimes found their own measures difficult to complete (e.g. Polly) because of not having
the information they needed to answer some of the questions about their child’s feelings. For example:

I would read them and think ‘how am I supposed to know the answer to any of this?’. So it’s all guess
work and that felt, felt like, a bit like cheating.
Polly

Especially, like, as a parent, whereas she might not have told me a lot of stuff, I found a lot of it quite
difficult to answer because I just didn’t know.
Pamela

However, they accepted them as important, describing that they felt like ‘if it helps with the research, it’s
got to be done’ (Pamela). Some parents found the measures to be unproblematic for the young people to
fill out. For example, one parent reported that, because it was something that could be done in the ‘home
environment’, it ‘didn’t involve much challenge, so that wasn’t a problem for [the young person]’ (Phoebe).
Others commented on how repetitive they were:

We were filling in the same sort of forms over and over and over, which was explained and, you
know, understandable, but I mean, it did seem to be fairly repetitive.
Polly

Others noted that the young people’s questionnaires were ‘helpful’, because ‘you actually get to see it,
of how she was feeling at that time and whether her mood was low or high’ (Patricia), even if they
sometimes also commented that the young people found them ‘very tedious’ or (in one case; Patricia)
overwhelming to complete. Some parents commented that an electronic means of capturing the data
would have been preferable for them and/or their children, as this would have been more convenient for
the young people to fill in and would have taken the pressure off busy family life, since they would not
have had to remember to take them to sessions. For example:

I hadn’t already got them in the car, or she’s forgotten to bring them with her, I can’t stop, it would
make me late . . . but you could e-mail them if you’ve still got your phone with you, which 9 times out
of 10, you’re gonna have.
Patricia

Measures were accepted as integral to participating in research, with one parent reporting that, ‘with that
being part of the trial, [the young person] could, you know, that to him was, made logical sense that every
so often you’d have to evaluate what he was doing’ (Paula).

© Queen’s Printer and Controller of HMSO 2021. This work was produced by Creswell et al. under the terms of a commissioning contract issued by the Secretary of State for
Health and Social Care. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional
journals provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should
59
be addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science
Park, Southampton SO16 7NS, UK.
 RESULTS 4: YOUNG PEOPLE’S AND PARENTS’ EXPERIENCES

Parents appeared to find participation in the research acceptable. For some, the research was a mechanism
for receiving an intervention that otherwise might have been unavailable or that would have required a
long wait:

We probably saw it as a way that we might get some help quicker.

Pamela

One of the parents was keen on the additional monitoring (of progress) and transparency (about progress)
that research involves, compared with treatment as usual:

Prior to that, we hadn’t really felt that there’d been, certainly not, not, er, fed back to us that there
hadn–, didn’t seem to have been that much reflection and, well – ‘are we actually progressing and has
there been any developmental progress?’, those kinds of things we thought would be, um, a lot more
part of the, the core of the study, really, so we were quite excited.
Phoebe

The motivations of parents also included the benefit of helping others (e.g. Paula, Pamela), which was
important when previous experiences had been unhelpful. For example, one parent reported that,
although their main motivation was that they ‘just wanted someone to help’ their child, ‘if it helps other
people in the, in the long term, then that’s great’ (Pamela). Another parent reported that their child was
initially reluctant to access treatment for their difficulties, stating ‘I don’t need to, I’m fine’, but that they
were able to ‘discuss it with him around it being a research trial’ and were able to ‘bag it as something
different, um, and he was more than happy to do that’ (Paula). Research was also framed as more
inclusive than treatment as usual, with one parent explaining that ‘it possibly gave me an avenue to
contribute more than treatment, um, which I wouldn’t normally, or I hadn’t previously been asked from
my perspective what I thought, um, how I thought [the young person] was doing.’ (Phoebe).

To summarise, parents were aware that young people found some aspects of the intervention challenging,
but they noticed and valued the benefits (especially the functional improvements). They found the
approach taken by their therapists to be open and inclusive, and they noticed and appreciated that the
therapeutic process was an active one. They accepted the need for measures. They reflected on some of
the relational impacts of treatment, and some felt that they would have liked to have received support
for themselves as well.

60
NIHR Journals Library www.journalslibrary.nihr.ac.uk
 DOI: 10.3310/hta25200 HEALTH TECHNOLOGY ASSESSMENT 2021 VOL. 25 NO. 20

Chapter 9 Results 5: therapists’ and service

manager/leads’ experiences

Child and adolescent mental health services therapists’ perspectives

Engaging and expert training

On the basis of their interviews, therapists generally appeared to find the training helpful. The therapists
were very positive about the detail, format, focus and utility of the training (‘I think they were all great,
I think there wasn’t one bad bit’; Caroline). The video clips illustrating particular treatment procedures
were universally seen as ‘really, really useful’ (Carol).

The style of the training workshops was perceived to be very interactive, and therapists appreciated that
the training was delivered by people who were acknowledged as experts in the field (e.g. ‘they are the
bee’s knees’; Christopher). The appreciation of the trainers’ expertise and the interactive style of training
was consistent. For example, Conner described how an advantage of ‘[B]eing trained by the person who,
kind of, originated the protocol [was] that there was a much, kind of, closer knowledge of it’ (Conner).
He went on to describe his appreciation of the blended teaching style, a point also made by Caroline:

Good mix of kind of didactic teaching and thinking together, some role plays, some getting out and
doing behavioural experiments, experiencing them yourself, it was a really good mix I think.
Caroline

Challenging but supportive supervision

Therapists reflected on the ways in which supervision had been provided and how they had drawn on it.
The use of supervision to support learning about and subsequently delivery of the treatment was largely
positive. For example, Charlotte appreciated her supervisors’ ‘insightful comments’, and Carol reflected on
how in supervision the ‘[A]dvice that you were given was massively helpful, so you really felt that you were
developing your skills’.

Some aspects of this form of supervision were framed as ‘different’ from their usual or previous
experiences of supervision. For example, focusing on therapy techniques in depth and using measures and
videos to inform the session were indicative of distinctive attention to the treatment of a case (rather than
as a forum for more general discussion of clinical issues). Generally, these differences were experienced
as positive and constructive by our therapists, who noticed a shift towards ‘[F]ocusing very specifically on
techniques and actual words so the minutiae . . .’ (Caroline). The videos, in particular, were an important
learning mechanism for our therapists:

[Using videos is] much more powerful when you actually see the act– part of that session.
Christopher

We would watch videos in supervision, which was helpful, and then the feedback that came back was
great, umm, going through the questionnaires was great. Yeah, I mean supervision was great.
Caroline

© Queen’s Printer and Controller of HMSO 2021. This work was produced by Creswell et al. under the terms of a commissioning contract issued by the Secretary of State for
Health and Social Care. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional
journals provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should
61
be addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science
Park, Southampton SO16 7NS, UK.
 RESULTS 5: THERAPISTS’ AND SERVICE MANAGER/LEADS’ EXPERIENCES

Supervision was thus framed as part of the wider learning opportunity provided by participation in the
research and as being helpful in skill development and the growth of competence and confidence.
This was one aspect of participation in the research that was attractive to the therapists:

I was kind of quite keen to umm, you know, see if there were things I could learn that might improve
my practice.
Claire

Helps your professional development, checks that you’re doing the right thing.
Camilla

Therapists reflected that they had been motivated to participate in the research because of this opportunity
to improve their skills. Learning from supervision was clearly understood as a component of this
opportunity, as it provided clinical insights and informed the next steps in treatment planning. Here,
Caroline and Claire both reflect on situations in which supervisory discussions helped them to plan their
next sessions:

I guess I would always be keen for feedback on [when] I didn’t think I did that very well, how could I
have changed that, how could I have done it, what would you have done in that situation, that kind
of stuff. And then help planning behavioural experiments and umm, ones that would work practically,
umm and then thinking about endings.
Caroline

I think supervision again was really, umm, necessary to help, ummhmm, you think about how you
were going to umm structure the session and, and, and do the session.
Claire

Thus, we can see that supervision was framed partly as a mechanism for identifying what could have been
done differently (areas for improvement) and partly as a mechanism for identifying what should be done
next (treatment planning). Supervision in the project was often delivered with peers, rather than in a 1 : 1
format. Therapists generally liked the opportunity to learn from, and with, peers:

When it worked [with] the three of us, it worked brilliantly, and I preferred group supervision because
of that richness.
Caroline

I think it’s useful to hear other people and how that is really, really key, especially if you’re struggling
to do it all as well as you’d like to, you’re running out of time and you hear that they’re also
doing that.
Camilla

Although therapists valued the ‘richness’ of learning alongside others and the benefits that came from
their hearing about perspectives, some suggested that larger groups might have been preferable. Groups
potentially offer the possibility of not always being in the spotlight, but that was not really provided in
these smaller configurations. Charlotte reflected that:

It would have been nice if there was a couple more people, because it felt really intense with just two
of us doing it.
Charlotte

62
NIHR Journals Library www.journalslibrary.nihr.ac.uk
 DOI: 10.3310/hta25200 HEALTH TECHNOLOGY ASSESSMENT 2021 VOL. 25 NO. 20

However, one participant also felt that group supervision did not allow enough time to discuss their cases:

I would guess that I was just worried that I’d have enough time to discuss mine as well, but it was
useful because you’re also hearing that somebody else is finding it difficult to fit it all in and
practicalities of it as well.
Camilla

Thus, small groups would appear to have been a ‘best-fit’ compromise, striking a balance between the
different preferences of the therapists. Some supervisees had not been able to meet their supervisor in
person, and they felt that this would have been helpful: ‘it’s not the same as face to face to be honest’
(Christopher); ‘from a supervisee perspective and feeling, umm, kind of comfortable sharing your
nightmare moments on video, it’s nice to have met somebody face to face’ (Caroline).

Less positive features of people’s experiences of supervision were generally related to one of two features.
First, there were some early anxieties when they anticipated being on the receiving end of critical
judgements about their competence. For example:

It was a bit nerve wracking and anxiety provoking when we first started . . . I thought supervision was
done in a really, non-threatening way and I think, I think effort was put into that and I think it
was appreciated.
Carol

I don’t think it made me seem like a particularly competent therapist at all.

Charlotte

As Charlotte’s quotation implies, these anxieties were not confined to the anticipation of scrutiny.
Therapists also expressed some worries about the high standards that they wished to be able to meet and
that they felt they would be measured against during supervision:

I found that quite difficult if I and, and that, that’s probably why one of the things I found it really
difficult to do this and feel like I was doing it well which kind of like disappointed me and I always felt
like ooo, I’m not doing this well.
Camilla

You’ve still got to deliver the video and stuff [yeah] and I think they put a lot of pressure on you to,
to get it right.
Christopher

Second, therapists struggled with practical issues associated with access to supervision. This involved
freeing up time, finding the time to prepare for supervision and making the technology work (‘I felt I
needed to get this on camera’; Christopher). Freeing up time, and then protecting and using that time as
planned, was a recurrent concern:

If I did particularly have a busy day if I had to split, you know if I’d had to slip something else in or
I’d had to see like another family urgently or something umm and then it was on the back of the
supervision session that was quite difficult.
Claire

[I needed to] have the questionnaires handed in, have the video uploaded, etc., all kinds of– and its
weekly and it’s quite– and when you’re quite busy it’s hard to get it all ready.
Camilla

© Queen’s Printer and Controller of HMSO 2021. This work was produced by Creswell et al. under the terms of a commissioning contract issued by the Secretary of State for
Health and Social Care. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional
journals provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should
63
be addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science
Park, Southampton SO16 7NS, UK.
 RESULTS 5: THERAPISTS’ AND SERVICE MANAGER/LEADS’ EXPERIENCES

Issues with technology often compounded the sense of urgency and pressure that are captured in the
accounts of Claire and Camilla above. Technological barriers included the ease and reliability of hardware
and software, and the compatibility with people’s location and local systems:

It’s the technology, but, kind of, Skype was unreliable and that just made it kind of stressful, cos,
when you’re kind of going into supervision and you got to spend 10 minutes trying to figure out how
to actually get a clear line.
Conner

Every now and again where it would click off or something like that – yeah it was a pain yeah.
Christopher

These barriers were not insignificant: therapists expressed considerable frustration about technical
problems and gave specific examples of the pressures on their time, such as ‘staying late trying to finish it
all off’ (Claire) and arranging supervision on a non-work day (Caroline).

A useful and well-used manual

The manual (and training materials) were also appraised very positively:

I liked the way it was structured umm, and umm, having the manual there and everything so you
knew exactly what you were doing.
Charlotte

The manual was really, obviously a brilliant resource.

Carol

Therapists used the materials in different ways. Some saw them as providing a grounding (‘I would have
to go back to the manual if I was doing it again’; Camilla), and others saw them as a more integral part
of growing their competence while delivering the intervention (‘I was constantly referring back to it’;
Caroline). Alongside this, the use of video materials within the package was seen as particularly helpful.
For example, Claire and Christopher both described how they used the videos to prepare for specific
sessions, and how they had found this so useful that they hoped to be able to continue to incorporate the
use of video materials into their future supervision.

There were a range of suggestions about additional materials that could be added to further enhance the
resources. These included providing some form of checklist to help prepare for supervision (therapists felt
that there was a lot to remember and bring) and including, in the training workshops, training in the use
of all the equipment and software needed for recording and uploading sessions and accessing supervision.
One therapist also suggested that it would be useful to have videos to show to the young people.

Developing competence
Therapists were confident that they understood that there was a ‘right way’ to deliver this intervention,
but they were less confident that the work they were doing was always consistent with that. Some
reported or described ways in which they experienced anxiety or pressure about meeting expectations.
These examples were discussed in the context of therapists’ uncertainties about the degree of flexibility
that was available to them. For example:

You kind of have to follow a very strict session pattern and it’s quite like clear instructions for
each session.
Charlotte

64
NIHR Journals Library www.journalslibrary.nihr.ac.uk
 DOI: 10.3310/hta25200 HEALTH TECHNOLOGY ASSESSMENT 2021 VOL. 25 NO. 20

I understand the need to get the research and to and to keep very purest in the research so that you
can then perhaps go on and adapt some of the models.
Caroline

In applying their training, therapists appreciated the opportunity to focus on something specific (while
framing this as a ‘luxury’ – ‘I think it was good, it felt like a luxury, actually’; Caroline) while also having
the opportunity to develop their skills in ‘doing one thing really well.’ For example:

I quite liked it actually. I quite liked that you had a focus. Umm, and that was what you were working
on and that you almost kind of didn’t get too distracted by anything else going on.
Claire

You’re just umm, doing social anxiety and you’re doing it slightly different, you’re getting umm, you’re
using the formulation in a much more prominent way, right from the start.
Camilla

They looked forward to being able to take their new skills into their next roles, but, as we will see in the
next section, they also worried about whether or not the flexibility required by ‘typical CAMHS’ work was at
odds with doing so. The sense of developing both specific and transferable skills was common; however:

I think it has changed my practice a lot umm, not just with social anxiety, but I think with other
treatments thinking about fidelity to the model umm behavioural experiments and how to develop a
model umm, the attention training, the video feedback I think umm, not only in social anxiety cases
where I feel a lot more confident and skilled in helping young people.
Claire

Identifying active ingredients

Therapists were able to identify features of the intervention that were particularly salient for their clinical
work. These were generally framed in terms of helpfulness, but some features seemed to resonate more
strongly than others. As with young people, behavioural experiments were singled out as a particularly
important part of treatment. For example, they were ‘probably the most helpful’ aspect of the intervention
(Camilla), ‘quite transformative’ (Conner) and ‘obviously a completely key part of it [that] the young people
found [. . .] really helpful and were then able to go away and then put it into practice’ (Carol).

The requirement to ‘get’ out of the therapy room was generally incorporated well into the treatment
sessions, even though therapists reported some practical barriers to this as ‘one of the biggest challenges’
of the trial (Claire), in terms of limitations of location and time to organise and complete the session within
the session length. The benefits of the behavioural experiments were represented in terms of them being
‘more active’ than the therapists’ usual ways of working:

I was probably much more active than the CBT I was using before, I was probably much more passive
[how do you mean?] Umm so right from the beginning you’re adding behavioural experiments [oh OK
yeah] where I probably wouldn’t have done that before.
Claire

Another important feature was video feedback, which was described as ‘a very powerful part of the
treatment’ (Claire), helping to ‘bring [. . .] alive the things we were talking about so that they could see it,
rather than, just sit there and describe it’ (Conner).

Attention training was framed more ambiguously – therapists felt that it was difficult to implement and
that the benefits were less tangible than for some of the other components. It was not ‘as helpful as some
of the other techniques’ (Caroline), and therapists found it ‘hard to gauge how much a young person

© Queen’s Printer and Controller of HMSO 2021. This work was produced by Creswell et al. under the terms of a commissioning contract issued by the Secretary of State for
Health and Social Care. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional
journals provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should
65
be addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science
Park, Southampton SO16 7NS, UK.
 RESULTS 5: THERAPISTS’ AND SERVICE MANAGER/LEADS’ EXPERIENCES

actually takes that on board’ (Conner). It could be difficult to implement too (Charlotte). One therapist did
report more positively about attention training:

I always wondered how they were going to respond to this. Umm, just because it’s something that
seems really different from the previous sessions and it sort of slowed, the pace has slowed down.
Umm, but actually everyone seemed to engage well with it.
Carol

Affinity with the model

Some therapists appeared to endorse the model more confidently and consistently than others:

It is a great model, I love it, I love it from a theoretical and technical point of view, erm, I love its
application and how much success you can get with it.
Conner

So, I would have been using Clark’s model anyway – social anxiety, but it was the, it was me applying
it in the way that I think it was meant to be applied.
Carol

For others, it was one of the attractions of joining the trial, to be:

. . . trained in a model for social anxiety for young people. Umm, and then obviously being one of the
therapists delivering the model.
Caroline

Some of this is reflected in variations in what therapists say about their confidence in the model, but much
of it is illustrated more indirectly, through the way in which people talked about the model. For example,
one therapist described the research as an opportunity to ‘tighten up’ (Caroline) on perceived inconsistencies
in their CBT practice; another stated that ‘I kind of didn’t always really draw out the cognitive element of the
learning enough. I think that was what supervision helped’ (Conner), and thus saw themselves as moving
towards increasing consistency with the model:

[Developing] more of a sense of competence around was, of the cognitively driven element of that
and targeting [pause] beliefs with behavioural experiments and selling that way of working to the
young person.
Conner

By contrast, another therapist described sharing the young people’s doubts about engaging in some
aspects of the activities or described times when they quickly abandoned a particular strategy because it
did not appear to be working. Others expressed concerns about their grasp of some of the components
(e.g. attention training, as we have seen). Some of these worries resurfaced in the desire for further
confirmation that they were ‘doing it right’ or in requests for considerably more flexibility around complex
cases: ‘the main thing is that it has to be a lot more flexible working with children’ (Charlotte).

Concerns about implementation

On the basis of their interviews, therapists generally appeared to find the delivery acceptable, but they
expressed concerns about whether or not it was implementable within the current CAMHS context, which was
consistently represented as ‘really complex’ (Charlotte). Their concerns included the views that CAMHS was too
‘stressful’ to support this form of intervention, and that intervention was too time-consuming. For example:

Perception would be that, even if the model had been proved to be more effective the actual
deployment of that in everyday clinical CAMHS would [pause] be less likely.
Caroline

66
NIHR Journals Library www.journalslibrary.nihr.ac.uk
 DOI: 10.3310/hta25200 HEALTH TECHNOLOGY ASSESSMENT 2021 VOL. 25 NO. 20

If you’re working in a, kind of [pause] a day-to-day CAMHS clinic as opposed to a research clinic,
it’s more difficult, it’s more stressful, in my perception um, in terms of the, the kind of conflict with
everything else.
Conner

When it’s taking more time, then maybe it looks on paper. In a context where you’re, you’re massively
overstretched already.
Carol

In the examples above, note that Conner sees the stress arising from a perceived conflict between the
intervention and ‘everything else’ in CAMHS. In the wider context of these discussions, it was evident that
this conflict arose from different priorities (CAMHS was generally framed as having to prioritise responses
to acute crises), and the requirement to adopt a responsive and flexible way of working to accommodate
these demands. It is worth noting that Caroline (above) presents these conflicts as being so entrenched
that, even if the evidence showed that the intervention was ‘more effective’ (than alternatives), she would
still not be confident that it could be offered in CAMHS.

Therapists did recognise that social anxiety could have a significant impact and that CAMHS services
needed to be able to offer effective interventions. For example, Carol said that it was ‘something that was
needed’ and ‘the model’s great, I think the treatment is great’.

They were positive about the model, but worried that the majority of current CAMHS service users would
present with more complex problems than the clients with whom they had worked during the research.
For example:

I have massive admiration for it as a model and, and kind of think, y’know it would be, kind of, life
changing for many young people that do have social anxiety, that there’s a kind of rub with um
[pause] with the presentations that we see, I suppose.
Conner

The therapists were therefore keen to reflect on how much flexibility would be required to deliver the
intervention with these service users and to work out how much flexibility would be consistent with
adherence to the model. On the whole, they concluded that the model was not sufficiently flexible for
CAMHS at present, but some of the barriers (time, technology) may be more easily addressed than others
(structure, complexity).

An unexpected issue here was that therapists were concerned that the research effectively provided an
intervention to a ‘less serious case,’ when time could have been devoted to addressing the needs of a
young person whose situation was more pressing. For Caroline, this felt like an ethical dilemma, and she
felt guilty about it:

[The] fact that I was only working with people who shouldn’t be in my team and there was a huge
waiting list of CBT cases. So that guilt was quite big [. . .] I was working with people that I wouldn’t
normally work with. I was giving them a lot of my time.
Caroline

‘Typical CAMHS’ cases were construed by all the therapists as more complex (usually meaning ‘with
comorbidities’) and more acute. In some localities, cases were drawn from other CAMHS teams, or drawn
up from tier 2 services to tier 3 (CAMHS) for the research, and so the concern was sometimes based on an
accurate perception that the young people would not meet the threshold for the therapists’ usual service

© Queen’s Printer and Controller of HMSO 2021. This work was produced by Creswell et al. under the terms of a commissioning contract issued by the Secretary of State for
Health and Social Care. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional
journals provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should
67
be addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science
Park, Southampton SO16 7NS, UK.
 RESULTS 5: THERAPISTS’ AND SERVICE MANAGER/LEADS’ EXPERIENCES

or were not typical of those who would. The comments from Camilla and Conner here are typical of the
concerns that were raised by all therapists:

There is usually something that is comorbid with it so umm, although it was, it appeared effective on
social anxiety, there is other stuff going on for some people.
Camilla

Most young people who come into CAMHS have multiple problems, some of which aren’t clear
diagnosis er, kinda problems, but are more behavioural, or emotional, or kind of systemic in nature.
Conner

Thus, the therapists were concerned that the intervention might not be suitable for CAMHS as it is
currently configured. We have already seen how this conflict was framed in terms of the tension between
acute-complex and, by implication, chronic-simple presenting problems. The conflict was also evident in
some very practical, structural features of the environment. CAMHS was configured to measure out time
(clinical appointments, room bookings, etc.) in units of 1 hour. This meant that those therapists delivering
the intervention carried reduced caseloads (‘Where’s that time coming from?’ Carol), owing to the
‘onerous [. . .] number of sessions [and] the length of the sessions, 90 minutes’ (Conner). This ‘threw’ the
‘diary structure’ (Conner).

Despite this, therapists generally appeared to find the research participation acceptable. In places, it was
difficult to disentangle this from other elements of their experience:

I think it was clear, although I think it did, I think it did merge as well.
Claire

For example, workload issues were related to research and delivery of the therapy and were often
presented as stressful to manage. It was stressful not only because of the ‘demands’ made on the therapist
in terms of adhering to the therapeutic procedures (Conner), but also because of the requirements to
complete the parallel research procedures (Claire). All therapists felt that they did not have sufficient time
to meet these sets of demands. For example, preparation time was more extensive than for ‘normal
working’, and the supervisory commitments were also more demanding. Efforts to provide the therapists
with this time were undermined by the unpredictable nature of other routine work.

As we have seen above, there were also concerns about the time needed to manage the data extracted
from measures, and the technological barriers to accessing supervision and making and uploading video
recordings. The effect of trying to meet these ‘abnormal demands’ was generally reported as stressful for
the therapists. Notably, the interviews were taken from the therapists after their training phase, rather than
after the trial as planned. It would not, therefore, be surprising if, for example, preparation were to take
longer, confidence were to be lower and anxiety about video in supervision were to be higher at this stage
in the training.

Worrying about the number of measures

Concerns about the number of measures were related to both the intervention and the research. It was
not always clear to therapists which measures were integral to the intervention and which were specific to
its evaluation. For example, Claire described the ‘sheer number’ of measures as ‘overwhelming’ at the
start. Here, Christopher describes his own initial concerns:

Oh my God this is what we’re gonna have to do every case, every client which is not true, but I don’t
know if that was made explicit.
Christopher

68
NIHR Journals Library www.journalslibrary.nihr.ac.uk
 DOI: 10.3310/hta25200 HEALTH TECHNOLOGY ASSESSMENT 2021 VOL. 25 NO. 20

Therapists also tended to frame their worries about the measures in terms of their perceptions of the
young people’s experiences of completing the measures; they worried that the young people would find
the measures boring or not helpful. Camilla, for example, felt that her clients were ‘a bit resistant to
completing them’, because they found them to be ‘quite a chore’. Similarly, Carol felt that the young
people found the measures to be ‘a bit of a pain.’ As we have seen, these concerns were not shared with
us in the interviews with the young people.

There were also therapist concerns about the workload associated with managing the data, for example
uploading the measures:

In real sort of you know day-to-day work, you haven’t got the capacity, no, that you can do the
uploading and the interpreting and it’s err you know and if you’re doing four, five, six people a day
you–you’re gonna struggle with that.
Christopher

Some measures were singled out as being particularly helpful in treatment. Most therapists preferred
the anxiety-specific measures, which ‘felt much more relevant’ (Claire). The weekly measure of social
cognitions (SCQ) was agreed as being one of the most useful, along with the less frequently used social
behaviours measure (SBQ). The LSAS-CA-SR was talked about in terms of its use in treatment:

. . . kind of [to] do a comparison from week to week of what had changed and we’d pick things up.
Caroline

Some therapists did reflect on how they would make more use of the measures in their future work
because of how useful they had been (e.g. Claire); others felt that the burden of including the measures
was not acceptable to them because of the additional administrative load. It was evident that, for most of
the therapists, the use of measures was one of the strongest distinctions from ‘normal working’.

In summary, the therapists were very positive about the training and the model. They found the process of
developing competence through delivery and supervision challenging, but also constructive, acceptable
and achievable. They framed the therapy as an active form of intervention and singled out behavioural
experiments as a key component. Their confidence in other components of the therapy was more varied.
They expressed concerns about the volume of measures involved and about the viability of delivering the
therapy as part of routine CAMHS work. They identified features that they would incorporate into their
future work, including the use of measures within treatment and the use of video in supervision.

Child and adolescent mental health services managers’/leads’

perspectives

On the basis of their interviews, managers generally appeared to support the idea of being involved in
research (and one, who was specifically interested in the model, was ‘really excited’):

I was really keen to support the therapists, and the therapists were really enthusiastic and, and I
thought from a CBT point of view it would be really good.
Monica

I was aware that the people who were allocated that condition were going off and getting this
training and supervision which was quite excellent I was pleased that those skills were coming into the
team and being developed.
Marian

© Queen’s Printer and Controller of HMSO 2021. This work was produced by Creswell et al. under the terms of a commissioning contract issued by the Secretary of State for
Health and Social Care. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional
journals provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should
69
be addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science
Park, Southampton SO16 7NS, UK.
 RESULTS 5: THERAPISTS’ AND SERVICE MANAGER/LEADS’ EXPERIENCES

Worrying about releasing staff time

Despite their support, they all expressed some concerns about being able to implement the intervention
and research in the current CAMHS context (‘the reality was very, very tricky’; Marian). These concerns
were generally framed in terms of the perceived pressure on staff, and specifically, the structural barriers to
freeing up their time:

I did think it was going to put the staff members under pressure, if anyone piped up and said yes
please, I’d like to be involved.
Margaret

There’s no way we could free them up for more than that time in the team, because they already had
existing caseloads.
Monica

It was clear that some of these concerns pre-dated the managers’ agreement to be involved in the trial.
The focus of their initial concerns was largely about the time needed by the therapists and the impact of
protecting time for a specific client group (‘There is always a waiting list for CBT’; Margaret). At the outset,
this took the form of uncertainties about what was involved and about what to expect. It was suggested
that more explicit and precise information about the time needed at the outset would have been helpful.
It was difficult for managers and therapists to anticipate what it is like to participate in research, and they
emphasised the need for realistic information. For example:

[It’s important to be] really explicit up front because I suppose, we’ve learned from IAPT [Improving
Access to Psychological Therapies] training, for example, that it’s often the amount of time people say
is needed bears no resemblance to reality.
Monica

I don’t think it was ever set down clearly how much time the trial was supposed to take so my
understanding is that was supposed to take half a day a week, but I think that wasn’t the same
understanding as the people who were running the trial or the people who were involved in the trial
who thought that they should just carve out as much time as it needs.
Marian

However, the managers’ worries about time appeared to expand as the process went on. This coloured
the way in which they talked about every aspect of the project: the training phase (‘Oh my God this is
much bigger than I ever anticipated’; Margaret), the therapists’ use of supervision, the research-specific
aspects and the delivery of the therapy itself.

Freeing up staff to deliver the therapy and receive training and supervision was seen as a challenge, even
when time was allocated for this (‘They had time allotted in their job plans for the trial and they were
excited about the trial’; Marian). It was not always clear whether it was the managers or the therapists
themselves who were responsible for negotiating (and working to) realistic job plans:

[Therapists should] make sure it’s actually job planned in before they embark on it, not after when
they realise ‘oh my God I’ve, it’s a, I’m not able to do the job and the trial at the same time you know
it’s too much’.
Margaret

I was really keen to umm, kind of, negotiate for therapists, a realistic amount of time to do it] in their
job plans.
Monica

70
NIHR Journals Library www.journalslibrary.nihr.ac.uk
 DOI: 10.3310/hta25200 HEALTH TECHNOLOGY ASSESSMENT 2021 VOL. 25 NO. 20

Some managers were more concerned than others about their ability to meet the requirements of the
research (e.g. one felt ‘massively relieved’ when they heard the trial was closing). Concerns by this point
had expanded to include the identification of appropriate cases, the impact of the work on therapists, the
secondary impact on waiting lists and managers’ perceived lack of acceptability of the intervention to
young people. For example, one manager expressed doubts about whether or not there were young
people in CAMHS who were ‘suitable and motivated and ready for a very, very kind of disorder-specific
intervention’ (Monica). Another expressed concerns about the acceptability of the measures for young
people, stating that ‘it actually had an adverse effect on the child’s anxiety because of all the measures
that were needing to be done’ (Margaret). The context and origins of these concerns were unclear.

Struggling to identify suitable cases

The identification of cases was perceived to be problematic, because (much as above, with the therapists), the
availability of suitable cases was felt to be limited in Tier 3 CAMHS. For example, Monica described how it
was ‘quite tricky’ to find cases that matched the study requirements. As a consequence, managers expressed
concerns about ‘less serious’ cases ‘leapfrogging’ the waiting list. This was framed as an ethical issue:

It feels wrong, doesn’t it, if you’ve got these people phoning up and you’re saying I’m really sorry
you’re waiting and they’re desperate to be seen umm, and then the other people are being seen
quickly who don’t have the same severity of difficulties.
Marian

The availability of cases varied across sites. One of the managers interviewed did not report finding cases
problematic, had a ‘readymade case load’ and felt that this ‘is a very relevant service for people to do this
sort of trial in’.

Managing staff workloads

Managers shared their concerns about the impact of the work on the therapists’ other commitments. Margaret
described how ‘it nearly sinks them’, and Monica worried about her staff’s capacity to cope with the workload
but ‘didn’t feel that I could be asking on their behalf or saying “actually they need to stop doing it” ‘.

Managers were aware that therapists had to meet the standards required for participation in the research.
They were represented as being under considerable pressure to perform:

They were worried about the demands on them.

Marian

It’s quite tough that I think.

Margaret

Technical problems were perceived as undermining therapists’ confidence – ‘you look rubbish in front of
children and families’ (Margaret). Consequently, there was a call for more training around the practical issues
associated with the treatment and research (such as recording and uploading videos). This call underlines the
challenges of including technological components in the implementation of an intervention. The research
team did provide support to the therapists (i.e. they provided step-by-step written guidance on how to use all
the technology – including video cameras and Skype – and face-to-face support where needed), but it was
still difficult for the therapists to manage these aspects of the trial in their day-to-day work.

In another example, the postponement of other commitments to accommodate supervision was framed as
‘cancellation’ of ‘core’ work:

People felt such obligation to be part of this treatment trial that they cancelled their prior clinical
commitments which are a key part of their job.
Marian

© Queen’s Printer and Controller of HMSO 2021. This work was produced by Creswell et al. under the terms of a commissioning contract issued by the Secretary of State for
Health and Social Care. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional
journals provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should
71
be addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science
Park, Southampton SO16 7NS, UK.
 RESULTS 5: THERAPISTS’ AND SERVICE MANAGER/LEADS’ EXPERIENCES

Managers struggled to support the delivery of supervision, owing to the perceived pressure that therapists
may have felt to meet the demands, in terms of both performance and expectations on their time,
although one commented that they weren’t ‘. . . involved in any of that’ (Margaret). For example:

I think also the fact that the supervision was coming from such experts and the training was coming
from experts made them feel umm, slightly more worried about how whether they couldn’t ‘you know’
meet the demands.
Marian

Extra stuff I suppose that hadn’t been factored in like umm, you know watching, watching videos or
reading protocols or whatever, that, that they couldn’t fit into the half a day or whatever it was that
we’d negotiated and I don’t think people were able, necessarily able or, or, or willing to kind of do it
in their own time.
Monica

Concerns were also raised about the therapists’ perceived stretching of their usual professional boundaries
to attend supervision (e.g. attending at times when they would not usually work) and whose responsibility
it was to protect this:

. . . in a way it’s my, you know, who’s responsibility is it to say that that shouldn’t have happened?
Marian

Doubts were raised about whether or not the intervention was appropriate for CAMHS clients but not
universally. Margaret, for example, reported that ‘I do not think we have a problem with that. We get lots
of children with social anxiety’, while Monica expressed doubts about whether or not there were young
people in CAMHS who were ‘suitable and motivated and ready for a very, very kind of disorder-specific
intervention’ (Monica).

However, on the whole, the delivery of the therapy itself was not a huge concern for managers (‘People
that had the treatment umm, got on well and they made a lot of progress’; Marian), except in the context
of the impact on waiting lists (as above) and backfilling therapist time.

Problems with backfill

The limited utility of excess treatment costs meant that ‘there wasn’t anybody [else] to do the work’ that
the therapist was not doing, because ‘you can’t actually do anything with little tiny bits of money’. This
was a significant factor in the managers’ preoccupying concern with therapist’s time and capacity:

It wouldn’t have felt so difficult because all those people [. . .] who’d been waiting on the waiting list
and then got triaged as not suitable for the trial could have had somebody else to see.
Marian

Managers expressed relief that the research was brought to a premature close, despite sharing in the
disappointment of their therapists, who had ‘gained a lot in terms of the skills and supervision’ (Marian):

[I] felt really disappointed for the research team umm, and also for the clinicians I suppose on, because
you know the clinicians had been really enthusiastic about it.
Monica

I thought ‘great, some of our patients who are really needy and risky can be seen now’.
Marian

72
NIHR Journals Library www.journalslibrary.nihr.ac.uk
 DOI: 10.3310/hta25200 HEALTH TECHNOLOGY ASSESSMENT 2021 VOL. 25 NO. 20

To summarise, managers were sceptical about the viability of both the research and the intervention. They
were concerned about the pressure on staff and used ‘time-as-a-limited-resource’ as the main means of
framing this problem.

When we consider the combined perspectives across the two samples, it is notable that managers appear
to overestimate the difficulty of therapists delivering the treatment. Therapists do notice the burden of the
additional work, but they also accept and value the intervention and the opportunity to develop their skills.
However, interestingly, they also mirror their managers in apparently overestimating the burden of the
outcome measurement on the young people with whom they work.

© Queen’s Printer and Controller of HMSO 2021. This work was produced by Creswell et al. under the terms of a commissioning contract issued by the Secretary of State for
Health and Social Care. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional
journals provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should
73
be addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science
Park, Southampton SO16 7NS, UK.
 DOI: 10.3310/hta25200 HEALTH TECHNOLOGY ASSESSMENT 2021 VOL. 25 NO. 20

Chapter 10 Adverse events

N o adverse events were reported that resulted from the treatment that was delivered.

Three adverse events were reported that related to the research procedures. In all three cases, NHS
therapists who were participating in the study e-mailed identifiable patient information to the research
team’s university e-mail address. This was a breach of our data protection procedures.

To avoid this situation, we had informed therapists in person and in writing, as they joined the study, that
they should not e-mail any identifiable patient information to non-NHS e-mail addresses, as these are not
secure. Therapists signed an agreement to demonstrate that they had read, understood and agreed to this
(and other relevant) procedures.

The following steps were taken when the protocol breaches occurred:

1. The research team immediately contacted the therapist to remind them that university e-mail addresses
are not secure and not to send identifiable information.
2. The e-mail containing identifiable information was deleted.
3. The University Information Management and Policy Services were informed. They advised that, as the
breach came from the NHS trust, NHS managers should be informed, and NHS procedures should
be followed.
4. Managers of the relevant team were informed.
5. All participating therapists and managers were reminded of the study procedures with regard to
data protection.
6. The SSC were informed.

© Queen’s Printer and Controller of HMSO 2021. This work was produced by Creswell et al. under the terms of a commissioning contract issued by the Secretary of State for
Health and Social Care. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional
journals provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should
75
be addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science
Park, Southampton SO16 7NS, UK.
 DOI: 10.3310/hta25200 HEALTH TECHNOLOGY ASSESSMENT 2021 VOL. 25 NO. 20

Chapter 11 Discussion
T his study was conducted in response to an NIHR Health Technology Assessment commissioned call
to address the question of ‘How feasible is a study to investigate the clinical effectiveness and the
cost-effectiveness of psychological therapy specific to adolescents with social anxiety disorder compared to
generic psychological therapy within Child and Adolescent Mental Health Services (CAMHS)’. The proposed
design was to (1) develop and manualise an appropriate intervention, and (2) evaluate the intervention
within a pilot randomised controlled trial.

We produced a manual and training materials for NHS therapists to deliver an adapted form of CT-SAD-A
(see https://oxcadatresources.com) and undertook a phase of work focused on training CAMHS therapists
in this treatment approach. Unfortunately, during this phase of work, it became clear that the proposed
feasibility trial would not be feasible because of high CAMHS staff turnover, an insufficient number of
available therapists to participate, restrictions on the amount of time therapists had available (and hence
study cases that could be seen) owing to high service demand and unfilled posts within participating
services, and a lack of suitable young people reaching the services. Because of long waiting lists, a shift in
the nature of referrals to cases associated with high risks and limited resources within teams, we were not
able to identify potential study cases beyond the usual service referral routes. As a result of the difficulties
with therapist retention and participant identification, we concluded that the proposed RCT would not be
feasible in the current CAMHS context, but we have reported on the outcomes of the training case series,
the cost of delivering CT-SAD-A within CAMHS, and the experience of participating young people,
parents, therapists and managers. We will now discuss each of these parts of the study in turn.

Treatment of social anxiety disorder in adolescents within child and

adolescent mental health services

The findings of the training case series provide preliminary evidence that good outcomes can be achieved
from the delivery of CT-SAD-A in standard NHS CAMHS. Patients were predominantly routine referrals
to CAMHS services, and all presented with a primary diagnosis of SAD and received cognitive therapy
provided by CAMHS therapists as part of their training in this novel treatment. The majority had comorbid
difficulties. The group as a whole demonstrated large and significant improvements in social anxiety
symptoms (across a range of measures) as well as associated clinical outcomes, including symptoms of
depression, generalised anxiety, social functioning and idiosyncratic appraisals of important areas in their
lives. Substantial improvements in the psychological mechanisms that are proposed to drive change in this
treatment (social beliefs and behaviours) were also reported, and treatment acceptability was high.

Outcomes in the training case series are broadly comparable with those reported in the first case series of
the treatment undertaken by our group,29 for example a drop in the LSAS of 67% at follow-up here
compared with 88% at follow-up in Leigh and Clark.29 The mean change of 61 points on the LSAS
reported here compares favourably with the figures reported for adult CT-SAD. For example, mean
changes of 43 and 46 on the LSAS were reported by Clark et al. in two RCTs (200320 and 2006,19
respectively).

From visual inspection of the most detailed and commonly used specific measure of social anxiety symptoms
within our battery (LSAS-CA-SR), it is clear that 9 out of the 12 patients reported considerable improvement
in symptoms over the course of therapy. These nine participants reported an average reduction of 85% on
the measure (vs. 13% for the other three patients), and also attained a clinically significant improvement.
Seven of the nine patients lost their primary SAD diagnosis and the other two showed reductions on their
ADIS CSR SAD rating (a reduction in CSR from 7 to 4 and from 7 to 5). All nine patients improved to below
clinical thresholds on the social anxiety and GAD subscales of the measure that is typically used to measure
anxiety/depression outcomes in CAMHS (RCADS). All but one fell to below the threshold on the depression

© Queen’s Printer and Controller of HMSO 2021. This work was produced by Creswell et al. under the terms of a commissioning contract issued by the Secretary of State for
Health and Social Care. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional
journals provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should
77
be addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science
Park, Southampton SO16 7NS, UK.
 DISCUSSION

subscale (although, notably, four met criteria for a depression diagnosis). These findings appear to compare
favourably with outcomes of children and young people with SAD from trials of more general forms of
CBT, for example Ginsburg et al.17 reported recovery rates of 40% among children and young people
with SAD who received 14 sessions of an individual CBT treatment that can be applied across a range of
anxiety disorders.

Importantly, scores on measures of the psychological mechanisms that are targeted in CT-SAD-A [i.e. social
cognitions and safety behaviours (SCQ, SBQ, and SAQ)] all reduced substantially over the course of therapy
for the nine improvers, in contrast to the non-improvers. On the SCQ, the average reduction was 96%
(range 89% to 100%) for the nine improvers vs. 32% (range 23% to 46%) for the three non-improvers;
on the SBQ, it was 77% (range 68% to 100%) vs. 7% (range –3% to 25%); and, on the SAQ, it was
43% (range 2% to 75%) vs. –4% (range –48% to 22%).

Although we would not expect any treatment to be effective for all patients, it is interesting to consider why
the three remaining patients responded considerably less to the therapy than the other nine. It does not
appear to be a function of SAD severity. Possible explanations include factors relating to how the treatment
was delivered. It may be that the particular therapists did not gain as much from the training as the others or
had less time available, so had less time to prepare, were less able to conduct full length sessions 90 minutes)
and were therefore less able to facilitate behavioural experiments in socially challenging situations outside
the therapy room. Examination of different elements of therapist competence by rating therapist competence
while blind to patient outcome will shed light on this hypothesis. For one of the non-responders, the presence
of comorbid PTSD may have been associated with the lack of treatment success. Despite SAD appearing to
meet criteria for ‘primacy’ at the initial assessment, it became clear during the treatment that PTSD warranted
treatment in its own right and may have interfered with the ability of the young person to make the most of
the treatment for SAD. Larger trials would provide the opportunity to examine potential predictors of response
to treatment more systematically.

Strengths of the study included the thorough assessment and training procedures that were implemented
within the CAMHS context. However, there are, of course, important limitations of the training case series,
including the small homogeneous sample, which included only two males out of a total of 12 participants,
and the uncontrolled design lacking a baseline assessment phase, which means we cannot rule out the
possibility of symptom change as a result of the passage of time, or non-specific therapy effects – although
the qualitative findings would suggest that this is not the case. We would suggest that it is unlikely that
improvements are due to the passage of time, as the patients had all experienced symptoms for at least
6 months to qualify for a diagnosis, and in many cases social anxiety had been present for much longer,
reflecting the long time it can take to travel through the standard CAMHS referral process. However, we
cannot rule out potential non-specific therapeutic benefits of therapist contact and support, and, therefore,
a controlled comparison is needed to test this alternative explanation. We also did not have the resources
to conduct longer-term assessments to monitor young people’s outcomes over the subsequent months
and years. Longer-term outcomes should certainly be included in any further systematic evaluation.

It should be noted that, while a strength of the study was the ‘real-world’ context in which it was
conducted, in one of the participating CAMHS services, participants who may not have ordinarily met
service entry threshold were included, which somewhat compromised the real-world validity. This context
also meant that we were reliant on therapists to identify potentially suitable study participants. Services
were provided with screening resources, but it was difficult to establish how reliably and effectively these
were implemented. Furthermore, while we provided guidance on our definition of recurrent self-harm
(to capture those young people with a level of self-harm that should be the focus of intervention in its
own right to keep the young person safe), it is unclear the extent to which therapists adhered to this
guidance and if young people may have been excluded unnecessarily. It is also important to note that
these guidelines were based on clinical experience and consultation, and the circumstances in which
targeted psychological interventions for anxiety disorders can be implemented in the context of self-harm
have not been firmly established.

78
NIHR Journals Library www.journalslibrary.nihr.ac.uk
 DOI: 10.3310/hta25200 HEALTH TECHNOLOGY ASSESSMENT 2021 VOL. 25 NO. 20

Notably, the patient outcomes described are for those treated during the CAMHS therapists’ training, and so
it is possible that the outcomes reported may be an underestimation of the potential gains patients could
experience, because we would expect therapists to achieve better outcomes as they gain more experience
delivering the therapy. Future research would benefit from establishing the extent of and optimum modalities
for delivering therapist training in CT-SAD-A to achieve optimal outcomes. Notwithstanding these issues,
the findings highlight the potential benefit of delivering CT-SAD to adolescents.

Costs of delivering cognitive therapy for social anxiety disorder in

adolescents in child and adolescent mental health services

The results of our economic analysis indicated that delivering CT-SAD-A in CAMHS, as delivered in this
training series, would cost the NHS, on average, £4047 (SD £1003) per adolescent treated. To the best of
our knowledge, this is the first detailed cost analysis of treatment for social anxiety specifically tailored to
young people. NICE guidelines24 for SAD reported a lack of studies assessing cost and cost-effectiveness of
treating children and young people with SAD. Our results, therefore, start filling this gap in the economic
evidence for young people affected by SAD.

The estimated cost of delivering treatment (excluding training and supervision of therapists) was £1861 per
young person treated within our study, which was admittedly based on a small sample size. Nevertheless,
in a tentative comparison with treatments for adults, our cost compares favourably to the estimated cost of
delivering Clark and Wells’ cognitive therapy for SAD per patient with adults (£2341).23 Notably, where the
relative cost-effectiveness of different psychological and pharmacological interventions for SAD (in adults) has
been compared, individual psychological interventions (particularly Clark and Well’s cognitive therapy) have
been found to be most cost-effective, in terms of the net monetary benefit at a willingness to pay threshold
of £20,000 (in line with NICE guidance),24 despite the relatively high intervention costs. This appears to be
due to the combination of high effectiveness and low relapse rates that ultimately bring both higher health
benefits and lower health and personal social service costs.23

However, our economic results need to be interpreted in light of some limitations. Although the response
rate of supervisor-completed economic logs was 100% throughout the study duration, therapist-completed
economics logs had rates of missing values that ranged from 50% to 67% for various treatment sessions
across all patients, and imputation methods had to be used to handle missing data. Using mean imputation
reduced sample variability, with the consequence that uncertainty around our cost estimates was
underestimated. Therapists’ feedback on the economic logs indicated that the economic forms were easy to
fill in, but therapists struggled more generally to carve out and then protect enough time to complete the
forms requested by the study protocol. Economic logs were usually left until last and were therefore often
left incomplete because of multiple pressures on the therapists’ scarce time.

It also has to be noted that the way in which the training phase was conducted was less than optimal,
because new therapists were recruited at different times throughout the training programme. Consequently,
two cohorts of therapists were trained separately, causing a duplication of supervisors’ time in the training
phase, which led to inflated costs. In ‘real-world’ practice, a day-long workshop could be delivered to a much
larger number of therapists (DC has conducted similar workshops with up to 250 therapists). As an example
of the potential cost savings, if only 30 therapists attended a training workshop at once, the amount of
supervisor time for a 7-hour workshop would amount to 28 minutes per therapist – which is about 10 times
lower than the time that was found in the study, that is, 287 minutes – and would cost the NHS £29 per
therapist (in the study the corresponding cost was £287). However, at this stage, these are only speculative
estimates, which should be validated in future studies. Furthermore, it has to be noted that, given the
difficulty encountered when implementing this new treatment in the study, the actual administrative costs to
the NHS (e.g. expanding capacity, system changes, workload rearrangement, setting up new procedures),
if the intervention were to be implemented in practice, would probably be relatively high at the beginning,

© Queen’s Printer and Controller of HMSO 2021. This work was produced by Creswell et al. under the terms of a commissioning contract issued by the Secretary of State for
Health and Social Care. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional
journals provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should
79
be addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science
Park, Southampton SO16 7NS, UK.
 DISCUSSION

which decision-makers would need to take into account when assessing the intervention. Finally, the lack of
a control condition precludes any conclusions about cost-effectiveness in comparison with current usual practice.

Participant experiences

Young people, parents and therapists on the whole liked and valued the intervention. There were lots of
concrete examples of positive change and positive experiences of the process. The active aspects of the
intervention, and specifically the use of behavioural experiments, were consistently identified as having the
most impact. Therapists and their managers found it difficult to implement the intervention within their
CAMHS teams because of a range of competing pressures.

There were a number of areas in which the accounts drawn from different perspectives did not overlap.
Where young people commented on the involvement of parents, they were positive about their individual
experience of it, but involvement of parents was not something generally discussed by many of the young
people. However, for parents, their concern about how to best support the young person was discussed
consistently. In particular, they wanted to know more about the intervention so that they could provide
support that was consistent with the work carried out during therapy. There was variation in parents’
experiences of this, apparently because the therapeutic dyads made different decisions about how much to
involve parents.

The issue of the meaning and usefulness of measures was particularly contentious. Young people and
parents were generally positive about the measures; even if they found them tedious, they were positive
about their utility. By contrast, therapists and their managers appeared to be more sceptical. Some of these
concerns were expressed in terms of time pressures and practicality, and they did not challenge utility
directly. Others were based on projected inferences about their acceptability to young people; on the
whole, the professionals’ concerns here were not borne out by the young people’s own accounts.

The challenging context of CAMHS (or specifically, ‘CAMHS as it is currently configured’) loomed large in
the accounts of therapists and managers. For young people, CAMHS was not particularly significant: they
did not talk about the intervention in ways that suggested awareness of the service context. For parents,
CAMHS was relevant in terms of some of their prior difficulties with accessing support and an appreciation
that the practicalities of treatment times in the trial were suited to family life. For therapists and managers,
there were concerns that converged on the greater ‘complexity’ of typical CAMHS cases, the urgent needs
of many of those cases, which were typically referred at times of crisis, the potential impact of research on
the fairness of waiting lists, and the problem of resolving the practical issues associated with delivering this
intervention within a system that has specific limitations around units of time and the resources allocated
to a given client. The issue of ‘complexity’ may have been less meaningful for differentiating between
‘trial’ cases and ‘typical’ cases than the issue of urgency. Many of the training phase participants also had
other comorbidities. Thus, CAMHS is often implicitly represented as a service for managing risk and crises,
rather than a service for delivering treatments.

Therapists and managers also varied in terms of whether they thought that finding suitable cases was a
problem or not, indicating variation in the sorts of young people who are able to successfully access
CAMHS in different localities.

Managers expressed concerns about the pressure and expectations placed on their staff by participating in
the research, owing to the demands on time and the level of scrutiny. Some of the therapists reflected on
feeling this pressure, and others described working hard to meet expectations. The status of the trainers
and supervisors was attractive in terms of motives to take part, but it also raised the stakes in terms of
wanting to perform well. The concept of ‘good enough’ practice did not always filter through to the
therapists. However, the therapists were generally very positive about their experience of training,
supervision and subsequently delivering the therapy. They also reflected on the skills that they had added

80
NIHR Journals Library www.journalslibrary.nihr.ac.uk
 DOI: 10.3310/hta25200 HEALTH TECHNOLOGY ASSESSMENT 2021 VOL. 25 NO. 20

to their repertoire for future use. Overall, it seemed that there was some discrepancy between managers
and their staff, in terms of the ‘burden’ of the trial work.

It was difficult for managers and therapists to anticipate what it is like to participate in research. For
feasibility studies in which procedures are often being tried out for the first time, this presents a challenge;
trials may benefit from the insights of ‘graduates’ of earlier stages of research. Concerns from therapists
and managers about the implementation in the current CAMHS context appear to confirm that early
closure of the trial was the correct decision.

It is important to reflect on the strengths and limitations of the qualitative work. The qualitative analysis
was based on 22 interviews. This is a relatively large sample for an IPA study, although it is typical of the
sample sizes found in multiple perspective designs. The construction of the sample supports the reflection
on how different experiences might relate to one another, and this has proven useful (e.g. with regard to
differing perspectives on measurement). The opportunity to triangulate the findings from the point of view
of the three samples (young people, parents, professionals) was taken up many times during the analysis
process, to make sense of ambiguities and uncertainties.

The sample includes all but one of the eligible therapists and all but one of the eligible managers. It
includes 6 parent–child dyads from a possible 14, which is a good recruitment rate for a qualitative study;
nevertheless, one potential weakness of our findings is that the experiences of the other eight dyads are
not available to us.

Interviews were of varying length. Interviews with young people are often shorter than interviews with
adults, and they were relatively short in this case too. However, in our study the interviews with parents
were also quite brief. Despite this, the level of description and reflection in both of these sets of interviews
was generally rich, and there was sufficient depth to support a detailed analysis. In particular, the steps
that we took to support participants in reflecting on the details of therapy were notably successful, and we
have been able to offer insights into which features the participants found helpful and why.

We have presented a preliminary analysis here, but this preliminary analysis was conducted by a
collaborating team, rather than by a single analyst, and it has thus benefited from multiple opportunities
for sense-checking. The analysis was overseen by a team member who does not deliver psychological
therapies and who has been involved in the study of experiences of therapies across a range of modalities
(e.g. cognitive, systemic, psychodynamic and art therapy). During the development of the analysis, the
team received considerable insights from the wider study team and management group, including the
experts by experience, at two large review meetings.

© Queen’s Printer and Controller of HMSO 2021. This work was produced by Creswell et al. under the terms of a commissioning contract issued by the Secretary of State for
Health and Social Care. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional
journals provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should
81
be addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science
Park, Southampton SO16 7NS, UK.
 DOI: 10.3310/hta25200 HEALTH TECHNOLOGY ASSESSMENT 2021 VOL. 25 NO. 20

Chapter 12 Conclusions and recommendations

F rom our experience of working closely with young people and their parents/carers, NHS CAMHS
therapists and their managers, a number of conclusions can be drawn, with implications for (1) the
treatment of SAD in adolescents, (2) treatment delivery in CAMHS more broadly and (3) conducting
research within CAMHS settings.

Implications for the treatment of social anxiety disorder in adolescents

Our preliminary findings suggest that therapists working in busy CAMHS settings can achieve good
outcomes for routinely referred SAD patients, even while undergoing their training in the therapy, and our
results point to the potential value of delivering the training more widely. Based on our small sample and
working within the current CAMHS context, the cost of delivering CT-SAD-A appeared to be no more than
the cost of delivering CT-SAD with adults and would be expected to bring greater economic advantages
because of the earlier onset of benefits, including those potentially related to increased engagement in
education. Given the promising clinical outcomes, it was a great shame that we experienced such substantial
problems implementing the training and delivery of CT-SAD-A within participating CAMHS teams as a result
of severe restrictions on their capacity. There were three particular challenges for CAMHS in implementing
the training and intervention: (1) identifying patients for whom the treatment was appropriate; (2) retaining
therapists for a long enough period within services to receive training and go on to deliver treatment; and
(3) participating therapists being able to carve out sufficient time/flexibility to participate in training and
supervision and to deliver core aspects of the treatment.

In relation to identifying patients for whom the treatment was appropriate, it is clearly essential that the
treatment is offered only to those patients for whom it is intended and does not prevent young people
from accessing a more appropriate treatment to meet different sets of needs. Through conducting this
study, it became clear that the majority of young people were presenting to CAMHS with high levels of
risk and that those young people who were experiencing psychological disorders, such as SAD, in the
absence of risk were not reaching CAMHS. Notably, risk is typically much more time limited than SAD,
which runs a chronic course,52 highlighting the need to address not just the presenting risk but also the
underlying mental health problems. The difficulty that young people and families have accessing CAMHS
has been well documented.53 Initiatives have recently been launched to begin to address this, for example
through School Mental Health teams.54 It will be essential that these teams provide the means to identify
and facilitate or provide psychological therapies for young people with primary SAD, given its chronicity,52
and particularly given the substantial impact SAD has on educational engagement and outcomes.55

Given the high service demands and relatively low capacity within CAMHS, therapists and managers
struggled to carve out and then protect the time needed to participate in training and deliver the treatment.
Difficulties were compounded by a traditional diary system, which meant that delivering sessions for
90 minutes, for example, caused problems in the context of overstretched room booking systems, which
allowed only 1 hour per week per patient. Young people and parents particularly emphasised the
importance of the ‘active’ approach to treatment, in which the young person and the therapist would go
out of the therapy room to test out the young person’s negative predictions in a relevant social context.
However, this also presented challenges for therapists, as it required more time and preparation, for
which they felt they did not have the capacity. Some of these difficulties may be overcome by making
modifications to how treatment is delivered, for example using self-study modules in either booklet56 or
online form57 so that face-to-face therapist contact can be reduced. These approaches have been found to
be associated with similar outcomes of more intensive face-to-face modalities, while reducing therapy costs,
when implemented with young people with other anxiety disorders.58 They also bring the advantage of
providing core content in a structured way, which may somewhat reduce the training needs for therapists
and ensure high levels of treatment fidelity. Online treatments can also easily embed routine measures,

© Queen’s Printer and Controller of HMSO 2021. This work was produced by Creswell et al. under the terms of a commissioning contract issued by the Secretary of State for
Health and Social Care. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional
journals provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should
83
be addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science
Park, Southampton SO16 7NS, UK.
 CONCLUSIONS AND RECOMMENDATIONS

to ensure that these are completed consistently. However, it is important to highlight that young people and
therapists emphasised the importance of working together outside the therapy room to test out negative
predictions, and it seems critical that future developments ensure that therapists have the opportunity to train
in this aspect of the intervention. We would also argue against simply shortening treatments, as considerable
gains were made between the middle and end of the treatment for many young people in the training case
series. Furthermore, booster sessions were well used within the training case series, and appear to provide a
safety net for young people and an opportunity to encourage them to continue to apply their learning from
within the therapy sessions.

Broader implications for treatment delivery within child and adolescent

mental health services

In this study, young people completed a number of measures both before and after treatment and prior
to each session. Measure completion was high; for example, for our most thorough session-by-session
measure of social anxiety (LSAS-C/A), 96% of all the sessional measures were collected. This is notable,
as implementation of sessional measures within the CYP IAPT programme has been a challenge, with only
7–20% of cases having paired data (compared with 99%59 in adult IAPT services). Previous studies have
identified a range of barriers including increased time demands on therapists and administrative staff, a
lack of therapist training on how to integrate measures into practice, a lack of data management systems
with which to record, review and audit the data, and a perceived lack of clinically useful information,60
for example if they did not include items to assess suicidal risk.61

The qualitative interviews that were conducted within this study indicated that therapists recognised the
value of sessional measures that accurately captured the problem and helped guide treatment, but they
expressed worries about the measures’ acceptability to young people. Notably, in addition to the social
anxiety and intervention process measures, we added a further depression measure to the one that is used
within the CYP IAPT data set (RCADS) to ensure that we routinely asked about suicidal risk (via the PHQ-9).
However, therapists were concerned that the young people found the measures burdensome and,
as in previous reports,62 the therapists themselves struggled to make time to manage and interpret the
questionnaire data between sessions. One manager was particularly concerned about the number of
measures that were administered on a sessional basis, seeing this as overly burdensome for young people.
However, while frequently commenting on the length and number of measures, the young people
appeared to find them acceptable. They recognised that the measures needed to be comprehensive for
and applicable to different young people. Young people particularly appreciated the value of sessional
measures to help them track progress and to also guide the content of sessions. Indeed, for some young
people, the use of measures to guide therapy stood out as a particularly important, positive component.

These findings suggest that high levels of routine data completion can be achieved within CAMHS; however,
for this to be facilitated, the measures need to be relevant and integrated with treatment for young people
and therapists (and ultimately managers) to recognise their value. We had initially intended to use existing
CAMHS data management systems to gather sessional measures electronically, but unfortunately participating
services did not integrate these systems into their services and were reluctant for additional systems to have
to be learned/used for this study. Efficient, reliable systems for sessional data capture are clearly needed,
co-designed with CAMHS and young people and their parents to ensure that they meet their needs and are
sufficiently flexible to include disorder/treatment relevant measures.

Implications for research in child and adolescent mental health services

This study set out to evaluate a novel psychological intervention within CAMHS settings, as required by the
commissioned call. As it was a novel intervention, it was important to establish that the CAMHS therapists
who were delivering the intervention were able to deliver it competently and with fidelity, and so we built

84
NIHR Journals Library www.journalslibrary.nihr.ac.uk
 DOI: 10.3310/hta25200 HEALTH TECHNOLOGY ASSESSMENT 2021 VOL. 25 NO. 20

a treatment training phase into the programme of work. Through this phase, it became clear that it would
not be possible to run a trial that required participating therapists to have received training in the novel
intervention and to have demonstrated competency in the treatment because of the high rates of staff
turnover and restrictions on therapists’ time within the current CAMHS context. This was disappointing, as
those therapists who participated in the training phase reflected on how valuable this had been, not just in
delivering treatment within the study but also for their broader clinical work. It appeared that the training
phase in this study provided an opportunity for therapists to discuss the same cases in detail and regularly
throughout the treatment, with a particular focus on the specific disorder and psychological model.
Indeed, for some therapists this felt like a ‘luxury’, which, despite its clear value, they felt would be difficult
to sustain given their usual caseload.

An alternative approach would have been to offer therapists a very brief (e.g. 1–2 days) training session
and then let them get on with treating young people in the trial. However, this treatment was new for the
therapists and, if we had followed this approach, we would not have known whether or not we were
actually testing the treatment as it should be delivered. A phase of supervised practice is a core part of
clinical training (e.g. adult IAPT, clinical psychology, CYP IAPT) and training therapists for most RCTs, so it
does not seem unreasonable. Indeed, it is considered essential in NHS England’s IAPT manual, which lays
out detailed quality criteria for the delivery of NICE-recommended psychological treatments within the
NHS. Notably, in CYP IAPT training, all training time is fully backfilled. In contrast, much of the time
involved in training in a new intervention within a research study is considered to be a treatment or excess
treatment cost, which means that participating NHS trusts have to either cover the costs themselves or
wait for any ‘excess’ costs to be covered at the end of the financial year. For overstretched services that
typically have unfilled therapist vacancies, the lack of upfront backfill provides unmanageable demands.

The allocation and management of costs incurred by NHS trusts also creates problems in delivering
interventions that go above and beyond what services are currently able to deliver. We successfully secured
excess treatment costs to support this study, and successfully applied for an increase in these when the full
extent of the time required for therapists became clear from their completion of health economic logs.
However, as noted above, overstretched services were unable to make use of the excess treatment costs
when they needed them, because they were provided at the end of each financial year. Equally, where
aspects of therapist time had been allocated as research costs (e.g. time to complete economic measures),
services could not use these funds, as there were already unfilled vacancies within services, so there was
no spare capacity to increase therapist availability. Therapists also reflected on how, even though they
might have dedicated time for the research activities, this time would often get taken, because an urgent
issue arose with a (non-research) case. It appears that, for services such as CAMHS to be able to participate
and support research, the research needs to bring immediate financial/resource benefits, for example
funding for dedicated research therapists who are able to accommodate the additional demands of taking
part in research.

It was clear that, given the many demands that participating therapists and managers already faced in
their services, it was important to try to fit within routine CAMHS ways of working wherever possible.
Unfortunately, this presented huge challenges owing to issues such as restricted information technology
within those settings. For example, many clinical settings had poor internet access, which created barriers
to conducting clinical supervision remotely via the internet with an expert supervisor, uploading video
recordings to NHS systems efficiently (with this taking up to 4 hours in some services) and being able to
collect routine measures electronically in a way that therapists would be able to access them and use them
within sessions.

As described in Chapter 3, this study benefited enormously from a dedicated PPI co-applicant, who was
able to input into all aspects of the study, as well as a PPI member of the SSC and consultation with two
groups of young people and carers (one who had received CT-SAD-A and one who had not). The PPI
co-applicant observed that a greater amount of costed time would allow for greater flexibility in availability

© Queen’s Printer and Controller of HMSO 2021. This work was produced by Creswell et al. under the terms of a commissioning contract issued by the Secretary of State for
Health and Social Care. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional
journals provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should
85
be addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science
Park, Southampton SO16 7NS, UK.
 CONCLUSIONS AND RECOMMENDATIONS

to contribute to the study, and that training that provides examples of good practice in PPI would enable
the development of more creative PPI approaches going forward.

Summary and recommendations

Young people who received CT-SAD-A in routine CAMHS achieved good outcomes and were generally
positive about the treatment approach. The cost of delivering CT-SAD-A compared favourably with the
costs of delivering CT-SAD in adults and has potential to bring even greater economic benefits. Therapists
valued the training and the treatment, but they struggled with implementing it within the current CAMHS
context. Managers reinforced the difficulties in accommodating the treatment and broader research
procedures within the current CAMHS context.

On the basis of these findings, we make the following recommendations for future research:

l Consideration should be given to how to create therapeutic settings, in schools or elsewhere, that can
predominantly focus on treating long-standing mental health problems, as opposed to dealing with
crisis phases in an adolescent’s life.
l Future research should consider how therapists can be best trained so they are competent to deliver
CT-SAD-A and other specialised evidence-based psychological therapies.
l A RCT of CT-SAD-A should be conducted within the most appropriate setting for the identification of
suitable young people and where there is scope for the treatment to be delivered as intended.
l Research funding should be made available to cover the full costs of efficacy trials for novel
psychological treatments, prior to these being rolled out in routine care settings.
l Full costs of research (including treatment costs) should be provided as part of the research grant to
enable resource-limited services to participate.
l Future studies would benefit from costing more substantial PPI time and providing access to dedicated
training to help PPI co-applicants in leading on developing creative and effective PPI approaches.

86
NIHR Journals Library www.journalslibrary.nihr.ac.uk
 DOI: 10.3310/hta25200 HEALTH TECHNOLOGY ASSESSMENT 2021 VOL. 25 NO. 20

Acknowledgements
W e would like to express wholehearted thanks to the participating families, therapists, their service
managers and clinical leads, and the R&D teams at the Oxford Health and Berkshire Healthcare NHS
Foundation Trusts.

We are also extremely grateful to the independent members of the SSC [Roz Shafran (chairperson),
Vicki Curry, Sam Cartwright-Hatton, Chris Foy and Ellie Hughes] and to our NIHR Programme Director,
Lesley Dodd, for their advice and support throughout the study. We also thank Philip Kendall for his
support with preparations to deliver the CAT project arm.

We would like to thank Laura Johnson, Frankie Glover, Rachel Evans, Helen Stockhill and Jessica Benyon
for research support during the study, Ray Percy for diagnostic supervision, and Liz White for administrative
support.

We acknowledge the support of the National Institute for Health Research Clinical Research Network
(NIHR CRN).

Contributions of authors

Cathy Creswell (Professor of Developmental Clinical Psychology) was the chief investigator, oversaw all
aspects of the study and was the lead author on report.

Eleanor Leigh (Clinical Research Fellow & Principal Clinical Psychologist) developed the CT-SAD-A manual
and training programme, delivered CT-SAD-A training and supervision, analysed and drafted clinical
outcomes and contributed to all aspects of the draft.

Michael Larkin (Reader in Psychology) oversaw all aspects of the qualitative components and contributed
to all aspects of the draft.

Gareth Stephens (High-intensity Cognitive–Behavioural Therapist) oversaw all PPI and contributed to all
aspects of the draft.

Mara Violato (Senior Researcher and Lecturer in Health Economics) oversaw and analysed all health
economic components, and contributed to all aspects of the draft.

Emma Brooks (Clinical Research Assistant) contributed to data collection and management; qualitative
analysis; and all aspects of the draft.

Samantha Pearcey (Clinical Research Assistant) contributed to protocol development; data collection and
management; qualitative analysis; and all aspects of the draft.

Lucy Taylor (Clinical Trials Manager) oversaw and contributed to data collection and management, and
contributed to qualitative analysis and all aspects of the draft.

Paul Stallard (Head of Psychological Therapies) was the local lead for Oxford Health NHS Foundation
Trust. He was the lead for the C.A.T. project arm and contributed to all aspects of the draft.

Polly Waite (Associate Professor) was the local lead for Berkshire Healthcare NHS Foundation Trust.
She contributed to all aspects of the draft.

© Queen’s Printer and Controller of HMSO 2021. This work was produced by Creswell et al. under the terms of a commissioning contract issued by the Secretary of State for
Health and Social Care. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional
journals provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should
87
be addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science
Park, Southampton SO16 7NS, UK.
 ACKNOWLEDGEMENTS

Shirley Reynolds (Professor of Evidence Based Psychological Therapies) was the lead for linking to CYP IAPT
programme/training delivery and contributed to all aspects of the draft.

Gordon Taylor (Professor in Medical Statistics) oversaw statistical aspects of the study and contributed to
all aspects of the draft.

Emma Warnock-Parkes (Postdoctoral Research Associate) delivered CT-SAD-A training and supervision,
and contributed to all aspects of the draft.

David M Clark (Professor and Chairperson of Experimental Psychology) oversaw all CT-SAD-A components
of the study and contributed to all aspects of the draft.

Publications

Leigh E, Creswell C, Stallard P, Waite P, Violato M, Pearcey S, et al. Delivering cognitive therapy for adolescent
social anxiety disorder in NHS CAMHS: a clinical and cost analysis [published online ahead of print]. Behav
Cogn Psychother 2021.

Taylor L, Creswell C, Pearcey S, Brooks E, Leigh E, Stallard P, et al. Delivering cognitive therapy for adolescent
social anxiety disorder in NHS CAMHS: a qualitative analysis of the experiences of young people, their
parents and clinicians-in-training [published online ahead of print]. Behav Cogn Psychother 2021.

Data-sharing statement

All data requests should be submitted to the corresponding author for consideration. Access to
anonymised data may be granted following review and ethics approval.

Patient data

This work uses data provided by patients and collected by the NHS as part of their care and support. Using
patient data is vital to improve health and care for everyone. There is huge potential to make better use of
information from people’s patient records, to understand more about disease, develop new treatments,
monitor safety, and plan NHS services. Patient data should be kept safe and secure, to protect everyone’s
privacy, and it’s important that there are safeguards to make sure that it is stored and used responsibly.
Everyone should be able to find out about how patient data are used. #datasaveslives You can find out
more about the background to this citation here: https://understandingpatientdata.org.uk/data-citation.

88
NIHR Journals Library www.journalslibrary.nihr.ac.uk
 DOI: 10.3310/hta25200 HEALTH TECHNOLOGY ASSESSMENT 2021 VOL. 25 NO. 20

References
1. Kessler RC, Berglund P, Demler O, Jin R, Merikangas KR, Walters EE. Lifetime prevalence and
age-of-onset distributions of DSM-IV disorders in the National Comorbidity Survey Replication.
Arch Gen Psychiatry 2005;62:593–602. https://doi.org/10.1001/archpsyc.62.6.593
2. Beesdo K, Bittner A, Pine DS, Stein MB, Höfler M, Lieb R, Wittchen HU. Incidence of social anxiety
disorder and the consistent risk for secondary depression in the first three decades of life.
Arch Gen Psychiatry 2007;64:903–12. https://doi.org/10.1001/archpsyc.64.8.903
3. Magee W, Eaton W, Wittchen H. Agoraphobia, simple phobia, and social phobia in the National
Comorbidity Survey. Year Bk Psychiatr Appl Ment Health 1997;1997:369.
4. Wittchen HU, Fehm L. Epidemiology and natural course of social fears and social phobia.
Acta Psychiatr Scand Suppl 2003;417:4–18. https://doi.org/10.1034/j.1600-0447.108.s417.1.x
5. Beesdo K, Knappe S, Pine DS. Anxiety and anxiety disorders in children and adolescents:
developmental issues and implications for DSM-V. Psychiatr Clin North Am 2009;32:483–524.
https://doi.org/10.1016/j.psc.2009.06.002
6. Essau CA, Conradt J, Petermann F. [Frequency and comorbidity of social anxiety and social phobia
in adolescents: results of a Bremen adolescent study.] Fortschr Neurol Psychiatr 1998;66:524–30.
https://doi.org/10.1055/s-2007-995294
7. Ranta K, Kaltiala-Heino R, Pelkonen M, Marttunen M. Associations between peer victimization,
self-reported depression and social phobia among adolescents: the role of comorbidity. J Adolesc
2009;32:77–93. https://doi.org/10.1016/j.adolescence.2007.11.005
8. Wittchen HU, Fuetsch M, Sonntag H, Müller N, Liebowitz M. Disability and quality of life in pure
and comorbid social phobia: findings from a controlled study. Eur Psychiatry 1999;14:118–31.
https://doi.org/10.1016/S0924-9338(99)80729-9
9. Whisman MA, Sheldon CT, Goering P. Psychiatric disorders and dissatisfaction with social
relationships: does type of relationship matter? J Abnorm Psychol 2000;109:803–8. https://doi.org/
10.1037//0021-843X.109.4.803
10. Stein MB, McQuaid JR, Laffaye C, McCahill ME. Social phobia in the primary care medical setting.
J Fam Pract 1999;48:514–19.
11. Patel A, Knapp M, Henderson J, Baldwin D. The economic consequences of social phobia. J Affect
Disord 2002;68:221–33. https://doi.org/10.1016/S0165-0327(00)00323-2
12. Katzelnick DJ, Kobak KA, DeLeire T, Henk HJ, Greist JH, Davidson JR, et al. Impact of generalized
social anxiety disorder in managed care. Am J Psychiatry 2001;158:1999–2007. https://doi.org/
10.1176/appi.ajp.158.12.1999
13. Chartier MJ, Walker JR, Stein MB. Considering comorbidity in social phobia. Soc Psychiatry Psychiatr
Epidemiol 2003;38:728–34. https://doi.org/10.1007/s00127-003-0720-6
14. Schutters SI, Dominguez MD, Knappe S, Lieb R, van Os J, Schruers KR, Wittchen HU. The
association between social phobia, social anxiety cognitions and paranoid symptoms. Acta
Psychiatr Scand 2012;125:213–27. https://doi.org/10.1111/j.1600-0447.2011.01787.x
15. Bruce SE, Yonkers KA, Otto MW, Eisen JL, Weisberg RB, Pagano M, et al. Influence of psychiatric
comorbidity on recovery and recurrence in generalized anxiety disorder, social phobia, and panic
disorder: a 12-year prospective study. Am J Psychiatry 2005;162:1179–87. https://doi.org/10.1176/
appi.ajp.162.6.1179

© Queen’s Printer and Controller of HMSO 2021. This work was produced by Creswell et al. under the terms of a commissioning contract issued by the Secretary of State for
Health and Social Care. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional
journals provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should
89
be addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science
Park, Southampton SO16 7NS, UK.
 REFERENCES

16. Katzelnick DJ, Greist JH. Social anxiety disorder: an unrecognized problem in primary care. J Clin
Psychiatry 2001;62(Suppl. 1):11–15.
17. Ginsburg GS, Kendall PC, Sakolsky D, Compton SN, Piacentini J, Albano AM, et al. Remission after
acute treatment in children and adolescents with anxiety disorders: findings from the CAMS.
J Consult Clin Psychol 2011;79:806–13. https://doi.org/10.1037/a0025933
18. Clark DM. A Cognitive Perspective on Social Phobia. In Crozier WR, Alden LE, editors. The Essential
Handbook of Social Anxiety for Clinicians. New York, NY: John Wiley & Sons Ltd; 2005. pp. 193–218.
19. Clark DM, Ehlers A, Hackmann A, McManus F, Fennell M, Grey N, et al. Cognitive therapy versus
exposure and applied relaxation in social phobia: a randomized controlled trial. J Consult Clin
Psychol 2006;74:568–78. https://doi.org/10.1037/0022-006X.74.3.568
20. Clark DM, Ehlers A, McManus F, Hackmann A, Fennell M, Campbell H, et al. Cognitive therapy
versus fluoxetine in generalized social phobia: a randomized placebo-controlled trial. J Consult Clin
Psychol 2003;71:1058–67. https://doi.org/10.1037/0022-006X.71.6.1058
21. Clark DM, Wild J, Grey N, Scott R, Liness S, Deale A, et al. Self-study enhances the effects of
Cognitive Therapy for social anxiety disorder: a randomized controlled trial. Pre-publication report
registration: ISRCTN11178360. 20122109. 2012. URL: www.isrctn.com/ISRCTN11178360?q=
11178360 (accessed 7 February 2019).
22. Mayo-Wilson E, Dias S, Mavranezouli I, Kew K, Clark DM, Ades AE, Pilling S. Psychological and
pharmacological interventions for social anxiety disorder in adults: a systematic review and network
meta-analysis. Lancet Psychiatry 2014;1:368–76. https://doi.org/10.1016/S2215-0366(14)70329-3
23. Mavranezouli I, Mayo-Wilson E, Dias S, Kew K, Clark DM, Ades AE, Pilling S. The cost effectiveness
of psychological and pharmacological interventions for social anxiety disorder: a model-based
economic analysis. PLOS ONE 2015;10:e0140704. https://doi.org/10.1371/journal.pone.0140704
24. National Institute for Health and Care Excellence. Social Anxiety Disorder: Recognition, Assessment and
Treatment. London: National Institute for Health and Care Excellence; 2013. URL: www.nice.org.uk/
guidance/cg159 (accessed 7 February 2019).
25. Leigh E, Clark DM. Understanding social anxiety disorder in adolescents and improving treatment
outcomes: applying the Cognitive Model of Clark and Wells (1995). Clin Child Fam Psychol Rev
2018;21:388–414. https://doi.org/10.1007/s10567-018-0258-5
26. Hodson KJ, McManus FV, Clark DM, Doll H. Can Clark and Wells’ (1995) cognitive model of social
phobia be applied to young people? Behav Cogn Psychother 2008;36:449–61. https://doi.org/
10.1017/S1352465808004487
27. Parr CJ, Cartwright-Hatton S. Social anxiety in adolescents: the effect of video feedback on anxiety
and the self-evaluation of performance. Clin Psychol Psychother 2009;16:46–54. https://doi.org/
10.1002/cpp.599
28. Schreiber F, Höfling V, Stangier U, Bohn C, Steil R. A cognitive model of social phobia: applicability in
a large adolescent sample. Int J Cogn Ther 2012;5:341–58. https://doi.org/10.1521/ijct.2012.5.3.341
29. Leigh E, Clark DM. Cognitive therapy for social anxiety disorder in adolescents: a development case
series. Behav Cogn Psychother 2016;44:1–17. https://doi.org/10.1017/S1352465815000715
30. Ingul JM, Aune T, Nordahl HM. A randomized controlled trial of individual cognitive therapy, group
cognitive behaviour therapy and attentional placebo for adolescent social phobia. Psychother
Psychosom 2014;83:54–61. https://doi.org/10.1159/000354672
31. Clark DM, Wells A. A cognitive model of social phobia. Soc Phobia: Diagn Assess Treat
1995;41:00022–3.

90
NIHR Journals Library www.journalslibrary.nihr.ac.uk
 DOI: 10.3310/hta25200 HEALTH TECHNOLOGY ASSESSMENT 2021 VOL. 25 NO. 20

32. Silverman W, Albano A, Barlow D. Manual for the ADIS-IV-C/P. New York, NY: Psychological
Corporation; 1996.
33. Warnock-Parkes E, Wild J, Stott R, Grey N, Ehlers A, Clark DM. Seeing is believing: using video
feedback in cognitive therapy for social anxiety disorder. Cogn Behav Pract 2017;24:245–55.
https://doi.org/10.1016/j.cbpra.2016.03.007
34. Masia-Warner C, Klein R. The Liebowitz Social Anxiety Scale for Children and Adolescents (LSAS-CA).
Available from Carrie Masia-Warner, NYU Child Study Center. 1999;215.
35. Masia-Warner C, Storch EA, Pincus DB, Klein RG, Heimberg RG, Liebowitz MR. The Liebowitz social
anxiety scale for children and adolescents: an initial psychometric investigation. J Am Acad Child
Adolesc Psychiatry 2003;42:1076–84. https://doi.org/10.1097/01.CHI.0000070249.24125.89
36. Chorpita BF, Yim L, Moffitt C, Umemoto LA, Francis SE. Assessment of symptoms of DSM-IV
anxiety and depression in children: a revised child anxiety and depression scale. Behav Res Ther
2000;38:835–55. https://doi.org/10.1016/S0005-7967(99)00130-8
37. Chorpita BF, Moffitt CE, Gray J. Psychometric properties of the Revised Child Anxiety and
Depression Scale in a clinical sample. Behav Res Ther 2005;43:309–22. https://doi.org/10.1016/
j.brat.2004.02.004
38. Silverman WK, Nelles WB. The anxiety disorders interview schedule for children. J Am Acad Child
Adolesc Psychiatry 1988;27:772–8. https://doi.org/10.1097/00004583-198811000-00019
39. Alden LE, Taylor CT. Relational treatment strategies increase social approach behaviors in patients
with Generalized Social Anxiety Disorder. J Anxiety Disord 2011;25:309–18. https://doi.org/
10.1016/j.janxdis.2010.10.003
40. Miller SD, Duncan BL. The Outcome and Session Rating Scales: Administration and Scoring
Manual. Chicago, IL: Institute for the Study of Therapeutic Change; 2000.
41. de Graaf LE, Gerhards SA, Arntz A, Riper H, Metsemakers JF, Evers SM, et al. Clinical effectiveness
of online computerised cognitive–behavioural therapy without support for depression in primary
care: randomised trial. Br J Psychiatry 2009;195:73–80. https://doi.org/10.1192/bjp.bp.108.054429
42. Allgaier AK, Pietsch K, Frühe B, Sigl-Glöckner J, Schulte-Körne G. Screening for depression in
adolescents: validity of the patient health questionnaire in pediatric care. Depress Anxiety
2012;29:906–13. https://doi.org/10.1002/da.21971
43. Staniszewska S, Brett J, Simera I, Seers K, Mockford C, Goodlad S, et al. GRIPP2 reporting
checklists: tools to improve reporting of patient and public involvement in research. Res Involv
Engagem 2017;3:13. https://doi.org/10.1186/s40900-017-0062-2
44. National Institute for Health Research (NICE). National Institute for Health Research National
Standards for Public Involvement. London: National Institute for Health Research; 2018.
URL: www.invo.org.uk/posttypepublication/national-standards-for-public-involvement/
(accessed 7 February 2019).
45. Kendall PC, Choudhury M. The CAT Project: For the Cognitive–behavioral Treatment of Anxious
Adolescents. Ardmore: Workbook Publishing; 2002.
46. Cohen J. The Effect Size Index: d. In Statistical Power Analysis for the Behavioral Sciences.
Hoboken: Taylor & Francis; 1988. pp. 284–8.
47. Curtis LA, Burns A. Unit Costs of Health and Social Care. Canterbury: Personal Social Services
Research Unit (PSSRU), University of Kent; 2017. URL: https://kar.kent.ac.uk/65559/ (accessed
7 February 2019).

© Queen’s Printer and Controller of HMSO 2021. This work was produced by Creswell et al. under the terms of a commissioning contract issued by the Secretary of State for
Health and Social Care. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional
journals provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should
91
be addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science
Park, Southampton SO16 7NS, UK.
 REFERENCES

48. Smith JA, Flowers P, Larkin M. The Theoretical Foundations of IPA. In Smith JA, Flowers P, Larkin M,
editors. Interpretive Phenomenological Analysis: Theory, Method and Research. London: SAGE;
2009. pp. 11–39.
49. McManus F, Peerbhoy D, Larkin M, Clark DM. Learning to change a way of being: an
interpretative phenomenological perspective on cognitive therapy for social phobia. J Anxiety
Disord 2010;24:581–9. https://doi.org/10.1016/j.janxdis.2010.03.018
50. Larkin M, Shaw S, Flowers P. Multi-perspectival designs and processes in interpretative
phenomenological research. Qual Res Psychol 2019;16:182–98. https://doi.org/10.1080/14780887.
2018.1540655
51. Yardley L. Dilemmas in qualitative health research. Psychol Health 2000;15:215–28. https://doi.org/
10.1080/08870440008400302
52. Cairney J, McCabe L, Veldhuizen S, Corna LM, Streiner D, Herrmann N. Epidemiology of social
phobia in later life. Am J Geriatr Psychiatry 2007;15:224–33. https://doi.org/10.1097/01.JGP.
0000235702.77245.46
53. Reardon T, Harvey K, Baranowska M, O’Brien D, Smith L, Creswell C. What do parents perceive are
the barriers and facilitators to accessing psychological treatment for mental health problems in
children and adolescents? A systematic review of qualitative and quantitative studies. Eur Child
Adolesc Psychiatry 2017;26:623–47. https://doi.org/10.1007/s00787-016-0930-6
54. Department of Health and Social Care. Transforming Children and Young People’s Mental Health
Provision: A Green Paper. London: Department of Health and Social Care; 2017. URL: www.gov.uk/
government/consultations/transforming-children-and-young-peoples-mental-health-provision-a-
green-paper (accessed 7 February 2019).
55. Stein MB, Kean YM. Disability and quality of life in social phobia: epidemiologic findings. Am J
Psychiatry 2000;157:1606–13. https://doi.org/10.1176/appi.ajp.157.10.1606
56. Clark DM, Salkovskis PM, Hackmann A, Wells A, Ludgate J, Gelder M. Brief cognitive therapy for
panic disorder: a randomized controlled trial. J Consult Clin Psychol 1999;67:583–9. https://doi.org/
10.1037/0022-006X.67.4.583
57. Hill C, Creswell C, Vigerland S, Nauta MH, March S, Donovan C, et al. Navigating the development
and dissemination of internet cognitive behavioral therapy (iCBT) for anxiety disorders in children
and young people: a consensus statement with recommendations from the #iCBTLorentz Workshop
Group. Internet Interv 2018;12:1–10. https://doi.org/10.1016/j.invent.2018.02.002
58. Jolstedt M, Wahlund T, Lenhard F, Ljótsson B, Mataix-Cols D, Nord M, et al. Efficacy and
cost-effectiveness of therapist-guided internet cognitive behavioural therapy for paediatric anxiety
disorders: a single-centre, single-blind, randomised controlled trial. Lancet Child Adolesc Health
2018;2:792–801. https://doi.org/10.1016/S2352-4642(18)30275-X
59. Clark DM. Realizing the mass public benefit of evidence-based psychological therapies: the IAPT program.
Annu Rev Clin Psychol 2018;14:159–83. https://doi.org/10.1146/annurev-clinpsy-050817-084833
60. Hall CL, Moldavsky M, Taylor J, Sayal K, Marriott M, Batty MJ, et al. Implementation of routine
outcome measurement in child and adolescent mental health services in the United Kingdom:
a critical perspective. Eur Child Adolesc Psychiatry 2014;23:239–42. https://doi.org/10.1007/
s00787-013-0454-2
61. McCarthy-Hoffbauer I, Leach C, McKenzie I. Deliberate self harm in children and adolescents.
Ment Health Learn Disabil Res Pract 2006;3:107–25. https://doi.org/10.5920/mhldrp.2006.32107
62. Johnston C, Gowers S. Routine outcome measurement: a survey of UK child and adolescent
mental health services. Child Adolesc Ment Health 2005;10:133–9. https://doi.org/10.1111/
j.1475-3588.2005.00357.x

92
NIHR Journals Library www.journalslibrary.nihr.ac.uk
 DOI: 10.3310/hta25200 HEALTH TECHNOLOGY ASSESSMENT 2021 VOL. 25 NO. 20

Appendix 1 The National Institute for Health and

Care Excellence screen for social anxiety
A s there is no brief screening measure available for social anxiety (and no data on specificity/sensitivity
of subscales from RCADS/SCAS, etc.), the NICE screening questions were used, as detailed in Table 20.

TABLE 20 The NICE screening questions for social anxiety

Sometimes people get very scared when they have to do things with other people, especially people they do not know.
They might worry about doing things with other people watching. They might get scared that they will do something silly
or that people will make fun of them. They might not want to do these things or, if they have to do them, they might get
very upset or cross.

Do you/does your child get scared about doing things with other people, like talking, eating, going to parties, or YES/NO
other things at school or with friends?

Do you/does your child find it difficult to do things when other people are watching, like playing sport, being in YES/NO
plays or concerts, asking or answering questions, reading aloud, or giving talks in class?
Do you/does your child ever feel that you/your child cannot do these things or try to get out of them? YES/NO

If a young person and/or parent respond YES to any of the items, proceed to ADIS-C/P.

Young people and parents will also be administered the RCADS as routine, so any young people that are
above clinical cut-off points on the RCADS social anxiety scale who DO NOT screen positive on the above
will also proceed to ADIS-C/P.

© Queen’s Printer and Controller of HMSO 2021. This work was produced by Creswell et al. under the terms of a commissioning contract issued by the Secretary of State for
Health and Social Care. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional
journals provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should
93
be addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science
Park, Southampton SO16 7NS, UK.
EME
HS&DR
HTA
PGfAR
PHR
Part of the NIHR Journals Library
www.journalslibrary.nihr.ac.uk

This report presents independent research funded by the National Institute for Health Research (NIHR).
The views expressed are those of the author(s) and not necessarily those of the NHS, the NIHR or the
Department of Health and Social Care

Published by the NIHR Journals Library