Presenting preliminary findings – a high risk strategy?!
At the time of writing, it is month 31 of 36 in our Common Health Assets project, researching how community led organisations (CLOs) create health and wellbeing, for whom and in what contexts; and looking at the sustainability and scalability of CLOs.
We have achieved a huge amount. A key achievement has been the identification of 10 ‘programme theories’, 5 about how CLOs create health and wellbeing in deprived areas, and 5 about how they manage to operate effectively. We have developed these theories, with partners and our lived experience panel, drawing on lots of different sources of information including evidence from literature reviews an analysis of UK policy, participatory photography with CLO members, and stakeholder interviews and workshops. Here is a picture of one of our work-in-progress programme theories:
Example of a preliminary programme theory
The next step, which involves analysis and further data collection, is investigating those theories in our project data (surveys, interviews, card sorts), and looking to see whether we observe patterns in data that support and validate our theories, and whether participants and beneficiaries of CLOs describe their experiences in ways which are consistent with our theories.
One important way that we’ll be testing strands of our programme theories is through our longitudinal survey with CLO participants from across the UK. Using this approach we have captured participant outcomes over the last 12 months, using measures of capability, health-related quality of life, mental wellbeing, and social connectedness alongside information about their characteristics, like gender, age, and how they have been interacting with their CLOs over the course of a year - what activities, groups, volunteering or work they have been doing and what services and facilities they have been using. More than 250 out of the 350 initial participants have stayed with us to complete their follow up questionnaire at 12 months, so we have lots of rich data.
We are now in the thick of data analysis, making sense of and exploring patterns in the data. Do things work as expected, for the people involved? Or perhaps only some people, in some circumstances, respond to activities and engagement with CLOs in a positive way. Are there unexpected outcomes, or even negative outcomes? Is it really about what CLOs do? Or how they do what they do – which might be harder to ‘get at’ in our data. To help us understand this, we’ll been working with stakeholders and community experts as well as our lived experience panel throughout.
Given our theories are preliminary and still being tested, when should we start talking about it out with the project team? At conferences, with practitioners and policy makers and in other public spaces? How open should we be about provisional theories that might change? How can we best share them?
As we approach the final months of analysis it makes sense to ‘get out and about’ and talk to as many people as possible about what we’re doing and to build towards a launch of our final findings. Getting out there is a way to ensure that research is relevant to CLOs and has best chance of having impact on the ground, and talking to stakeholders as far and widely as we can will improve our interpretations and might suggest avenues for data analysis we haven’t yet considered.
However, it is a natural inclination for researchers to want their work to be polished up to a shine and presented in perfect prose before sharing. That way, the results are less likely to be misquoted or misunderstood. What if provisional and ‘untested’ theory is circulated onwards as if it were findings? What if things change along the way and some of our theories do not hold up, or change shape? In academic conferences (where audiences are used to warnings about ‘work in progress’, ‘not for quotation’, ‘preliminary findings’) we might be on familiar ground, but what about when we talk to policy makers and practitioners?
I recently spoke at two fantastic gatherings of community organisations, community development practitioners, policy actors, and academics, in Scotland: The Big Gathering organised by Scottish Communities for Health and Wellbeing (SCHW) in November 2023; and the Community Health Exchange (CHEX) national conference in March 2024. At both events I only had very early findings (programme theories) to present and I found myself reminding the audience several times that the theories were provisional and untested. My impression was that the audiences appreciated that the work was in progress, although I was sorry not to have more big punchlines and analysis of data to support or refute our programme theories. I hope that it has profiled the project and the work, maybe started some conversations, and perhaps will help people to recognise Common Health Assets when the findings are published and recall the build-up and explanation of what we were trying to achieve.
I’m looking forward to being asked back in months to come with some clearer messages grounded in mixed-method, qualitative and quantitative analyses!
