Presenter:
Liz Ferguson, Museums Northumberland (eferguson@museumsnorthumberland.org.uk),
Annie Padwick, Evaluation lead NUSTEM (Annie.padwick@northumbria.ac.uk),
Joe Shimwell, Primary Outreach lead, NUSTEM (Joe.shimwell@northumbria.ac.uk)
Theme: Evaluation and impact
Reporter: Laura Thomas
One sentence summary: In this session, project partners shared their experiences of using two evaluation methods in their practice: reflective practice and most significant change.
Description
Annie from NUSTEM opened the session with a description of their experiences in incorporating reflective practice in their work. This was a formalisation of something that was already being done. They were motivated to do this as they wanted to ensure learning from one project would be incorporated into others.
In addition to the introduction of reflective practice, they were also interested in identifying evaluation approaches linked to their theory of change. The method identified was ‘most significant change’ (MSC) and this is also being incorporated into their general practice at NUSTEM. Annie noted that, at a glance, MSC allows you to take perspectives from project participants and take them ‘up to the top’ ensuring that outcomes important to participants aren’t missed.
Following this overview from Annie, Joe from NUSTEM spoke in more detail about how they do this in practice. After an activity there is time set aside for Joe to complete a reflective diary. This helps them assess what happened in the workshop. Joe felt this speeds up the iteration of workshops and has helped to identify opportunities for simple changes but also it can take pressure off participants as the subjects of the evaluation. MSC has been integrated into their STEM Communities project where it is being used in both a summative and formative way. This project is focused on the local history in the North East, supporting families to explore STEM topics in collaboration with researchers.
Liz from Museums Northumberland and Woodhorn Mining Museum then took over to discuss how they have used the reflective diaries as they work to set up the STEM Communities project. Early activity included meeting with senior management in local schools to recruit them to the project and discuss how it would work. In using the reflective diary, Liz was able to see what schools had in common but also where there were differences in the ways they worked, even in the same academy trust. For example, some parents had trauma relating to their own school experiences, so headteachers were able to advise and edit language of project communications going out to families. The reflective diary helped to document this but also influenced questions headteachers were asked in subsequent meetings.
Joe then described how they used the MSC approach with Liz. An example MSC question prompt was “In your opinion what has been the most significant change that you have seen in your thinking or practice since working on STEM Communities.” Joe felt that with this method there isn’t one voice which is the loudest, the thoughts and experiences are levelled out and this helps to break down power imbalances. Joe noted that when you sit down with the stories from a project it is incredible to see the outcomes identified and what people have learned. Annie shared that MSC was about to be undertaken with the STEM Communities participants. The project team want to know what is important to the participants and why they keep coming.
At the conclusion of the session, Joe shared the impact of the reflective diaries in team meetings: as they have spent time thinking ahead of the meeting, they are more productive and enjoyable. Participants in this session were then given copies of the reflective diary and encouraged to review it and consider writing up something based on their experiences today.
Key messages
- NUSTEM shared examples of two practices they have incorporated into their work: reflective diaries and most significant change.
- The benefit of the reflective diaries was that they ensured time was available for reflection following session delivery and this encouraged iteration of the activities and provided a source of information for evaluation, taking the pressure off the participants as the subjects of evaluation. Lessons learned were also able to be shared between staff members and projects.
- The key benefit of the most significant change was to ensure outcomes important to the participants were heard and recognised.
Resources
N/A
Symposium Partners
