7th OECD World Forum on Well-being, Rome, SCHIRMER
- 1. Jacki Schirmer & Suzanne Carroll WellRes Unit, Health Research Institute, University of Canberra Using machine learning to support wellbeing monitoring: ACT Wellbeing Framework case study
- 2. The University of Canberra acknowledges the Ngunnawal people, traditional custodians of the lands where Bruce Campus is situated. We wish to acknowledge and respect their continuing culture and the contribution they make to the life of Canberra and the region. We also acknowledge all other First Nations Peoples on whose lands we gather. The University of Canberra acknowledges the Ngunnawal people, traditional custodians of the lands where Bruce Campus is situated. We wish to acknowledge and respect their continuing culture and the contribution they make to the life of Canberra and the region. We also acknowledge all other First Nations Peoples on whose lands we gather.
- 3. We work with governments, not-for-profit and private sectors to: • Fill in gaps in wellbeing data availability for underserved regions and groups • Improve quality of wellbeing and resilience data • Evaluate effectiveness of wellbeing & resilience interventions and actions • Inform design of intervention and actions to support wellbeing and resilience. WELLRES WELLBEING & RESILIENCE UNIT HEALTH RESEARCH INSTITUTE UNIVERSITY OF CANBERRA The Living well in the ACT region monitors the wellbeing of people living in the Australian Capital Territory, home to the ‘bush capital’ city of Canberra, and provides data for a number of indicators in the ACT Government’s ACT Wellbeing Framework. The annual Carer Wellbeing Survey, conducted in partnership with Carers Australia, monitors the wellbeing of Australia’s 3 million unpaid carers, and identifies actions that can help support wellbeing. Living Well in the ACT Region survey RegionalWellbeing Survey Carer Wellbeing Survey The annual Regional Wellbeing Survey examines the wellbeing and resilience of the 12.6% of Australians living in rural and remote communities, while also collecting a comparison sample from urban and inner regional areas. It provides insights into the diverse wellbeing strengths and challenges of these communities, and how they differ to those of urban communities.
- 4. Case study: ACT Wellbeing Framework The Australian Capital Territory is a small region in Australia, home to the capital city of Canberra, and a rapidly growing population (300,000 in 2000 ➔ >460,000 in 2024) ACT Wellbeing Framework (launched 2020) • 12+1 domains, 57 indicators • WellRes annual Living well in the ACT region survey used to populate 1/3 indicators • Used to inform Government priorities, policies & investment decisions “The ACT Government is using the Wellbeing Framework and the information it provides to inform Government priorities, policies and investment decisions – including through Budget and Cabinet processes.” - https://www.act.gov.au/wellbeing/wellbeing-framework/embedding- wellbeing
- 5. Common questions from policy makers seeking to use indicator data to inform decision making: • What should our target be for a given indicator? • What level of a given indicator is ‘enough’ to support good levels of personal wellbeing or community wellbeing? • What does this tell us about the types of interventions/ actions we should be focusing on?
- 6. Personal wellbeing Measure: Personal wellbeing index Description: People rate their satisfaction with 7 aspects of their life, on a scale from 0 to 10. Responses are combined to give a score from 0 to 100 (see International Wellbeing Group 2013). Currently, the ACT Wellbeing Framework reports three categories • Low wellbeing (score <60) • Typical/healthy wellbeing (score 60-79) • High wellbeing (score 80+) Are these thresholds meaningful? How should a policy maker interpret them? Thinking about your own life and personal circumstances, how satisfied are you with the following at the moment? Completely DISSATISFIED Completely SATISFIED ⓪ ① ② ③ ④ ⑤ ⑥ ⑦ ⑧ ⑨ ⑩ Your standard of living ⃝ ⃝ ⃝ ⃝ ⃝ ⃝ ⃝ ⃝ ⃝ ⃝ ⃝ Your health ⃝ ⃝ ⃝ ⃝ ⃝ ⃝ ⃝ ⃝ ⃝ ⃝ ⃝ What you are currently achieving in life ⃝ ⃝ ⃝ ⃝ ⃝ ⃝ ⃝ ⃝ ⃝ ⃝ ⃝ Your personal relationships ⃝ ⃝ ⃝ ⃝ ⃝ ⃝ ⃝ ⃝ ⃝ ⃝ ⃝ How safe you feel ⃝ ⃝ ⃝ ⃝ ⃝ ⃝ ⃝ ⃝ ⃝ ⃝ ⃝ Feeling part of your community ⃝ ⃝ ⃝ ⃝ ⃝ ⃝ ⃝ ⃝ ⃝ ⃝ ⃝ Your future security ⃝ ⃝ ⃝ ⃝ ⃝ ⃝ ⃝ ⃝ ⃝ ⃝ ⃝
- 7. Loneliness Measure: Loneliness index Description: People identify how frequently they have experienced three aspects of loneliness in the last four weeks (never to all of the time). Turned into loneliness score from 1 (never lonely) to 5 (lonely all of the time). Framework currently reports: • Hardly ever lonely: Score 1 to 2.49 • Sometimes lonely: Score 2.50 to 3.49 • Often lonely: Score 3.50 or higher. These thresholds are based on the question structure – not on evidence of what level of loneliness is associated with differing levels of personal wellbeing. Are they the right thresholds? Do these thresholds give the right guidance to policy makers? Thinking about your experiences in the last four weeks… Never Hardly ever Occasionally/ sometimes Often All of the time How often do you feel that you lack companionship? ⃝ ⃝ ⃝ ⃝ ⃝ How often do you feel left out? ⃝ ⃝ ⃝ ⃝ ⃝ How often do you feel isolated from others? ⃝ ⃝ ⃝ ⃝ ⃝
- 8. Limitations of standard regression: • Doesn’t cope with large numbers of predictors • Difficult to model and interpret multiple higher-order (3 or more) interactions between predictors ML: decision/regression trees (supervised ML approach): • Handle large complex datasets • Incorporate complex (higher-order) interactions in a way that is interpretable • Cope with non-linear effects • Avoid collinearity issues • Exploratory – no need for a priori specification (not hypothesis testing) • Can uncover unknown ‘prognostic groups’ • Can identify key thresholds of interest to inform public health/policy • Can identify relative importance of predictors Machine learning – why use it & what we did Decision trees • Used well-established ‘Classification and Regression Tree’ (CART) algorithm (in R using rpart) • Analytic dataset split into 70/30 training and testing generalisation & accuracy • Training: • developed overgrown ‘tree’ • pruned tree based on smallest cross-validation error (from 10-fold cross-validation) • Predictive accuracy in training and testing sets calculated as the variance explained (r-squared) • Input variable importance calculated as part of the estimation process Data source Today’s data comes from Wave 6 of the Living well in the ACT region survey, a sample of 2,200 people, collected in 2023. Similar analysis is being conducted on other waves of data, and our other surveys, to identify consistency of findings.
- 9. We started by including all wellbeing determinants in a model – giving us distinct groups of people within the first two steps Predictors of PWI amongst those with high/low distress Psychological distress (K6) All people: Average PWI score (out of 100) Average PWI: 71 Prevalence: 100% High distress (12+) Average PWI: 58 Prevalence: 30% Loneliness (>3.8) Average PWI 39 Loneliness (<3.8) Average PWI 65 Lower distress (<12) Average PWI: 77 Prevalence: 70% Overall mental health (fair/poor/good) Average PWI: 70 Overall mental health (v. good/exc) Average PWI: 87 Very high wellbeing (avg 87): Low psychological distress (<12/30); Very good/excellent self-rated mental health ‘Typical’ wellbeing (avg 70): Low psychological distress (<12/30); Poor, fair, or good self-rated mental health Very low wellbeing (avg 39): High psychological distress (12+); High loneliness (3.8+) Low wellbeing (avg 65): High psychological distress (12+); Less lonely (<3.8)
- 10. VERY HIGH WELLBEING group (average PWI of 87, low distress & very good/excellent self-rated mental health) Have lower wellbeing (75- 86) if… Have higher wellbeing (87+) if… Lower sense of belonging <6.6 Higher sense of belonging >6.6 Sometimes/often/always lonely >2.2 Never/rarely lonely <2.2 Household financial position poor/very poor Household financial position just getting along, comfortable or prosperous Find Canberra moderately liveable (<6 out of 7) Find Canberra very liveable (6+) Find local area somewhat liveable or worse (<5) Find local area moderately or highly liveable (5+ out of 7) Do not feel safe when home alone (<2) Feel somewhat or very safe when home alone (2+) Spend time outdoors less than 2 times a week Spend time outdoors 3+ times a week What predicts differences in wellbeing of our ‘HIGH WELLBEING’ group? If you want to build Canberrans from HIGH to REALLY REALLY HIGH wellbeing, it’s about further improving often already good liveability, belonging, social connection, household finances, and increasing time spent outdoors.
- 11. What predicts differences in wellbeing of our ‘TYPICAL WELLBEING’ group? High wellbeing group (average PWI of 70, psychological distress <12, self-rated mental health poor, fair or good Have lower wellbeing (50-69) if… Have higher wellbeing (70-84) if… Lower sense of belonging (<6.2) Higher sense of belonging >6.2 Higher loneliness >2.2 Lower loneliness (<2.2) Lower social cohesion in local area <5 Higher social cohesion 5+ Lower confidence in effectiveness of local groups and organisations <5 Higher confidence in effectiveness of local groups and organisations 5+ Have poorer general health (very poor, poor, fair, good) Have better general health (very good, excellent) Have poorer household financial position (very poor, poor, just getting along) Have comfortable household financial position (reasonably comfortable, very comfortable, prosperous) Do not feel safe to walk alone in local neighbourhood Feel safe to walk alone Do not feel confident to cope with storms, floods, fires Feel confident to cope with storms, floods, fires If you want to protect TYPICAL LEVELS of wellbeing, focus on interventions that support social connection, cohesion and belonging; supporting health; supporting those with poorer household finances; and helping build ability to cope with natural hazards like storms, droughts, bushfire.
- 12. Machine learning analysis Low & very low wellbeing groups (average PWI of 58, psychological distress 12+) Have even lower wellbeing (30-60) if… Have less low wellbeing (60-72) if… Often or always lonely (>3.8) Sometimes, rarely, never lonely (<3.8) Low sense of belonging (<4.8) Moderate or high sense of belonging (4.8+) Poorer general health (poor, fair or good) Better general health (very good or excellent) Poorer household financial position (poor, just getting along, reasonably comfortable) Better household financial position (very comfortable, prosperous) Find Canberra less liveable (<5) Find Canberra more liveable (5+) Find living costs very unaffordable (<2) Find living costs slightly unaffordable or affordable (3+) If you want to build people from VERY LOW to SOMEWHAT BETTER wellbeing, it’s about improving loneliness, belonging, general health, household financial position and living cost affordability. For most of these, you would focus on improving from very low to average levels. You can also focus on improving liveability from ‘good’ to ‘very good’.
- 13. We also examined how individual indicators predicted wellbeing (i) overall, and (ii) for different groups Loneliness example Loneliness index All adults: Average PWI score (out of 100) Average PWI: 71 Prevalence: 100% Higher loneliness (>2.6) Average PWI: 60 Prevalence: 36% Lower loneliness (<2.6) Average PWI: 78 Prevalence: 64%
- 14. What can we tell our policy makers from all of this? To make a difference to LONELINESS, design interventions that seek to reduce loneliness: • From always/often to sometimes: For those with high distress, poor mental health, known barriers that reduce social connection (unpaid carers, some people living with disability). • From sometimes to rarely/never: For those with positive mental health and lower distress, wellbeing can be increased by supporting growth in regular social contact that reduces loneliness from something experienced sometimes, to something experienced rarely or never. Never lonely (1) Rarely lonely (2) Always lonely (5) Often lonely (4) Sometimes lonely (3) Critical threshold 2.5 Critical threshold 3.8
- 15. Sense of belonging Extent to which person agrees that they feel welcome in their community, part of their community, and do not feel like an outsider People need a strong sense of BELONGING. Even a small decline in feeling welcome & included is associated with lower wellbeing. Most ACT residents have a good sense of belonging – particularly older residents who have lived in the region a longer time. However, less than a strong sense is associated with lower wellbeing. Interventions should focus on further strengthening already positive sense of belonging, while helping those with a lower sense of belonging (recent residents, those aged 30-49, women with poorer mental health, some of those living with disability) to connect to others in their community. Strongly agree (7) Agree (5) Strongly disagree (1) Disagree (3) Neither agree/ disagree (4) Critical threshold 5.2 Critical threshold 3.8 Critical threshold 5.8 Threshold 6.2
- 16. Key takeaways We have identified a subset of wellbeing indicators that are particularly strong predictors of PWI for most groups at most points in time – and some that are less strong. Less strong predictors might reflect poorly designed measures or might indicate that this aspect is less influential on PWI. Machine learning provides important insights, different to those we can gain from other methods. However, machine learning on its own does not tell you what to do. It provides new lines of evidence we can draw on to co-design wellbeing intervention and action. Consistently strong predictors of Personal Wellbeing Index Psychological distress General mental health General health Household financial position Loneliness Belonging Sometimes strong predictors Large number of indicators are important, but not at all times or to all groups (e.g. liveability, living costs, safety, community participation, housing suitability, connection to Canberra, access to places, inclusion, discrimination, trust in government) Not so strong Time use Sleep hours Heatwave resilience