Evaluation

Evaluation

Pandemic Prevention Debate Kit – Evaluation Report

The Pandemic Prevention Debate Kit was funded by The Royal Institution, UK Research and Innovation, and the Reckitt Global Hygiene Institute.

The Kit asks the question: Were all treated fairly?

The Kit was disributed from January 2022, and evaluated in July 2023. Continue reading

Posted on July 31, 2023 by modjosh in Evaluation | Comments Off on Pandemic Prevention Debate Kit – Evaluation Report

Royal Institution Online Youth Summit 2022 – Report

A screenshot of the reportSince 2020 the RI have commissioned an online youth summit on a subject related to the CHRISTMAS LECTURES.

The Summit takes place at the RI and consists of a series of talks, and provocations in the Faraday Theatre interspersed with breakout sessions in various rooms around the building.

Continue reading

Posted on March 2, 2023 by modemily in Evaluation | Tagged | Comments Off on Royal Institution Online Youth Summit 2022 – Report

Darwin Tree of Life activity Report March 2021 to July 2022


The Darwin Tree of Life activity ran from March 2021 to July 2022 and was funded by the Wellcome Trust.

The activity featured 8 teams of scientists working on the Darwin Tree of Life project with the goal of sequencing the genomes of all eukaryotic species in Britain and Ireland.

Continue reading

Posted on January 30, 2023 by modbella in Evaluation | Tagged | Comments Off on Darwin Tree of Life activity Report March 2021 to July 2022

Climate Warming (Ban Flying) Debate Kit – Evaluation Report

The Climate Warming (Ban Flying) Debate Kit was funded by the Royal Institution, IBM, Lloyd’s Register Foundation, and UK Research and Innovation. It was produced alongside the CHRISTMAS LECTURES Zone, in partnership with the 2020 RI CHRISTMAS LECTURES series: Planet Earth: A User’s Guide.

The Climate Warming Debate Kit asks the question: Should flying be banned for ten years?

The Kit was distributed from March 2021, and evaluated during December 2021 and January 2022.

Summary:

  • We estimate that at least 221 Debate Kits have been used with students.
  • Feedback from teachers who have used the Kit is highly positive. Teachers value the ways in which the Kit links to the curriculum, provides varied perspectives around the issue, and encourages students’ lateral thinking.
  • Every teacher responding to the feedback survey, reported that they intend to use the Debate Kit again.
  • The COVID-19 pandemic has had significant and ongoing impacts on schools and teaching. Teacher feedback across our projects indicates a need to focus on the core curriculum due to lost time from school closures, students isolating, and navigating changing restrictions. This is reflected in feedback we have received which indicates that many teachers have not yet had time to make use of the Kit.
  • 75% of teachers who told us they haven’t yet used the Kit report that they intend to use it in the future.

Read the full report [PDF] ❯ Continue reading

Posted on February 24, 2022 by modjosh in Evaluation | Comments Off on Climate Warming (Ban Flying) Debate Kit – Evaluation Report

Food Hygiene Debate Kit – Evaluation Report

The Food Hygiene Debate Kit was funded by e-Bug and Safe Consume.

The Kit asks the question: Is it safer to eat at home or out at a restaurant?

The Kit was distributed from June 2020, and evaluated during December 2021 and January 2022.

Summary:

  • We estimate that at least 308 Debate Kits have been used with students.
  • Feedback from teachers who have used the Kit is consistently positive. Teachers value the ability of the Kit to encourage critical thinking and appreciation of others’ opinions.
  • 94% of teachers responding to the feedback survey, reported that they intend to use the Debate Kit again.
  • The COVID-19 pandemic has had significant and ongoing impacts on schools and teaching. Teacher feedback across our projects indicates a need to focus on the core curriculum due to lost time from school closures, students isolating, and navigating changing restrictions. This is reflected in feedback we have received which indicates that many teachers have not yet had time to make use of the Kit.
  • 57% of teachers who told us they haven’t yet used the Kit report that they intend to use it in the future.

Read the full report [PDF] ❯ Continue reading

Posted on February 24, 2022 by modjosh in Evaluation | Comments Off on Food Hygiene Debate Kit – Evaluation Report

⭐ Schools @ Darwin Tree of Life Progress Report March to July 2021

Schools @ Darwin Tree of Life runs from March 2021 to July 2022. This work was supported by the Wellcome Trust, grant reference 206194/Z/17/Z and 218328/Z/19/Z. This report will detail progress of the Zone from 15 March 2021 to 14 July 2021. The activity features 35 people working on the Darwin Tree of Life Project. Download the Zone Report (pdf) ❯ Scientists 35 scientists and researchers created profiles in the Zone. See who is taking part ❯ The winner of the Summer Term was Code Crackers. 🏆 The Schools @ Darwin Tree of Life trophy is on its way to the team! 🏆 Students 291 students from 14 schools logged into the Zone. 87% of active students were from target schools: 29% from widening participation schools and 62% from underserved schools. Live chats 32 live chats took place. 13 were booked by teachers and 19 were additional chats, open to … Continue reading

Posted on September 22, 2021 by in Evaluation, Update | Comments Off on ⭐ Schools @ Darwin Tree of Life Progress Report March to July 2021

Self-Driving Cars Debate Kit – Evaluation Report

The Self-Driving Cars Debate Kit was funded by the Institute of Physics, the Royal Institution, and Lloyd’s Register Foundation. It was produced alongside the CHRISTMAS LECTURES Zone, in partnership with the 2019 RI CHRISTMAS LECTURES series: Secrets and lies?

The Self-Driving Cars Debate Kit asks the question: Should our town centre be for self-driving cars only?

Through our evaluation we have learnt:

  • We estimate that at least 515 Debate Kits have been used with students.
  • Teachers valued the Debate Kit. Teachers report that the Kit has encouraged debate and respect for each other’s opinions. Teachers value the way in which the Kit “puts STEM learning in a real-life context”.
  • Kits have been used across a range of age groups from KS2 to KS5, and with adult learners.
  • Reception to competitions were mixed. 21 entries were received for the KS2/3 competition to design a road sign, however no entries were received for the KS4/5 competition to report on the debate.

Read the full report [PDF] ❯ Continue reading

Posted on March 12, 2021 by modjosh in Evaluation | Comments Off on Self-Driving Cars Debate Kit – Evaluation Report

Unisex Toilets Debate Kit – Evaluation Report

Unisex Toilets Debate Kit Evaluation Report Cover The Unisex Toilets Debate Kit asks the question: Should schools make all their toilets unisex?

The Debate Kit was funded by the Royal Institution and Lloyd’s Register Foundation. It was produced alongside the CHRISTMAS LECTURES Zone, in partnership with the 2018 RI CHRISTMAS LECTURES series: Who am I?. Continue reading

Posted on January 5, 2021 by modjosh in Evaluation | Comments Off on Unisex Toilets Debate Kit – Evaluation Report

Royal Institution Online Youth Summit 2020 – Report

Screenshot of home page of RI Youth SummitEach year the RI run a youth summit on a subject related to the CHRISTMAS LECTURES. In 2020 it was decided to run the Summit prior to the filming of the lectures so that the views of the delegates could be communicated to the lecturers. However the Covid-19 pandemic intervened and the RI building remained closed to the public.

The decision was taken to bring the summit online and I’m a Scientist was chosen as the platform because of a strong record of safeguarding and working with the RI on the Lectures. Continue reading

Posted on October 30, 2020 by ModShane in Evaluation | Tagged | Comments Off on Royal Institution Online Youth Summit 2020 – Report

COVID-19: Students’ questions

Word cloud of themes students ASK about COVID-19With schools across the UK due to close this week, the impact on the daily lives of students and young people of the COVID-19 outbreak is clear. We wanted to do a little analysis to look at what young people are concerned about, what are they asking?

Finding ourselves in the midst of the March I’m a Scientist (IAS) and I’m an Engineer (IAE) events, means that for the past few weeks, young people across the UK and Ireland have had the opportunity to ask any questions they like to groups of scientists and engineers.

Continue reading

Posted on March 19, 2020 by modjosh in Evaluation | Tagged | Comments Off on COVID-19: Students’ questions

Are Single Sex schools more likely to take part in I’m a Scientist?

Slightly. A recent request for help resulted in a surprisingly large number of girls schools responding. We wondered if participating schools were biased along single sex education lines. A quick analysis gave us the following percentages of UK schools taking part: 20.4% of secondary schools 19.9% of mixed schools 27.7% of Girls Schools 19.0% of Boys Schools So we have a slight bias towards girls schools, but as they constitute only 7% of secondary schools, we can live with that bias. And here’s the Venn diagram that illustrates the figures, and illustrates how dreadful venn diagram generators are.

Posted on January 15, 2020 by modemily in Evaluation | Comments Off on Are Single Sex schools more likely to take part in I’m a Scientist?

Reflecting on I’m a Scientist participation: Academy Zone pilot

“

 

I found a lot of students were asking questions that weren’t directly related to my work, so initially I kept quiet, but then talking about it with Hannah and the other scientists, and reflecting on the Academy Zone questions made me consider my work in terms of ‘social influence’

Ian, PhD researcher

As part of the ChallengeCPD@Bath project, last year we showed that engaging with school students in I’m a Scientist (IAS) functions as effective, accessible, experiential learning, especially for communication skills.

We wanted to further improve IAS as a training opportunity for researchers. The Academy Zone was created to help participating scientists better understand principles of good public engagement through reflecting on their IAS experience while it was happening. The ultimate goal: help build capacity for effective, high quality, public engagement in the research community. Continue reading

Posted on July 25, 2019 by modemily in Evaluation, Scientist Benefits | Tagged | Comments Off on Reflecting on I’m a Scientist participation: Academy Zone pilot

I’m a Scientist – Live! at the Ridgeway School for STFC Bringing Science to Swindon programme

On Tuesday 19th March two shows of I’m a Scientist – Live! came to the stage of the Ridgeway School as part of the Science and Technology Facilities Council’s Bringing Science to Swindon programme. The event helped STFC reach a new audience and engaged school students with the people behind the science funded by UK Research and Innovation (UKRI) councils, all based in their hometown.

In total, 200+ secondary school students from 5 different Swindon schools (Ridgeway, Kingsdown, Abbey Park, Commonweal, and Warneford) attended the shows.

They put their questions to 12 members of the UKRI community, between them representing STFC, EPSRC, BBSRC, NERC and MRC: scientists, engineers and staff from the research council head offices at Polaris House. Continue reading

Posted on March 28, 2019 by in Evaluation, I'm a Scientist - Live | Comments Off on I’m a Scientist – Live! at the Ridgeway School for STFC Bringing Science to Swindon programme

Who do school students want to see on the new ÂŁ50 Note?

£50 Note Zone LogoAccording to the I’m a Scientist poll in the £50 Note Zone, the scientist who should appear on the new £50 Note is:

Stephen Hawking

Championed by Lydia James and Ryan Cutter, Stephen Hawking won the most votes in our poll; students and members of the public believe he should be the scientist featured on the new ÂŁ50 Note. Continue reading

Posted on February 7, 2019 by modjosh in Evaluation | Comments Off on Who do school students want to see on the new ÂŁ50 Note?

Privacy Debate Kit – Evaluation Report

Privacy Debate Kit Evaluation Report CoverThe Privacy Debate Kit asks the question: Should mobile phones always be listening?

The Privacy Debate Kit was funded by the Royal Institution and Lloyd’s Register Foundation. It was produced alongside the CHRISTMAS LECTURES Zone, in partnership with the 2017 RI CHRISTMAS LECTURES series: The language of life.

2,000 Kits were produced, and distributed from January 2018. As of September 2018, all but a handful of Kits have been distributed. Continue reading

Posted on October 26, 2018 by modjosh in Evaluation | Comments Off on Privacy Debate Kit – Evaluation Report

Experiential public engagement training through I’m a Scientist

We looked at whether taking part helped scientists from the March 2018 event improve their communication skills and if so, explore why IAS worked as experiential training. The results showed scientists became more confident communicating with public audiences and helped them develop effective ways of talking about their work. The opportunity for frequent practise, honest student feedback, and the text based nature of interaction were among the elements highlighted by scientists as particularly helpful. Continue reading

Posted on October 12, 2018 by in Evaluation, Scientist Benefits | Comments Off on Experiential public engagement training through I’m a Scientist

Scientist Interview Narratives

Here are narratives for each of the five interviews carried out as part of the I’m a Scientist element of the ChallengeCPD@Bath project, funded by UKRI. These interviews investigated how I’m a Scientist works as experiential training for scientists communicating their research Continue reading

Posted on October 11, 2018 by in Evaluation, Scientist Benefits | Comments Off on Scientist Interview Narratives

MRC Festival Zone 2018 — Zone Report

Read the full report→ The MRC Festival Zone 2018 was a four-week-long event funded by the Medical Research Council (MRC) as part of the MRC Festival of Medical Research. There were 25 MRC scientists and support staff taking part. 32 schools from all over the UK participated in the zone, with 1,404 students logging in over the four weeks. Over half the schools were widening participation and/or under-served. Teachers from 45 schools accepted places in the zone. However, several were unable to participate, mostly due to scheduling difficulties. In the Ask section students asked 740 approved questions. Of these, over 50% were relating to science and medical research topics, a figure higher than the average for an I’m a Scientist two-week, general science zone. 76 online, text-based live chats took place during the MRC Festival Zone, three of which were evening chats, where students could access the chat from home to … Continue reading

Posted on August 3, 2018 by in Evaluation | Tagged | Comments Off on MRC Festival Zone 2018 — Zone Report

Researcher Development Framework and I’m a Scientist

It’s well established that scientists doing I’m a Scientist gain just as many positive outcomes as the students they talk to. Using the Vitae Researcher Development Framework to frame these benefits allows us to articulate them in a way that resonates with universities.

Recently, we surveyed event alumni from centres of doctoral training (CDTs) to quantify the effect of taking part on relevant RDF descriptors. This is part of our ongoing strategy to build relationships with these institutions. In total, 37 alumni who had taken part in I’m a Scientist or I’m an Engineer at least 3 months prior responded. Continue reading

Posted on May 15, 2018 by in Evaluation, Scientist Benefits | Comments Off on Researcher Development Framework and I’m a Scientist

Increasing students’ confidence with I’m a Scientist

“Many of our students lack confidence in their academic ability so they were buzzing when they realised they can hold their own in a conversation with intelligent, educated people and this helped them realise they are all scientists too!” – Julia Anderson, FE College Biology Lecturer


This general further education college is split across 3 sites in a large, post-industrial, non-university town. Students at the college took part in the I’m a Scientist Immune System and Genes Zones in March 2018. Julia tells us how meaningful engagement with the scientists increased her students’ confidence. Continue reading

Posted on May 14, 2018 by admin in Evaluation, School, Teachers, Widening Participation | Tagged , | Comments Off on Increasing students’ confidence with I’m a Scientist

Using I’m a Scientist to increase participation in higher education

“Traditionally, not many of our students go on to university. I’m a Scientist helps by allowing students to relate to scientists and helping them see the value of studying at a higher level.”

– Mark McNally, Science Teacher


A mixed 2-19 academy, where over two thirds of the school population are students from disadvantaged backgrounds, took part in I’m a Scientist in March 2018. Mark tells us how the activity helped interest his students in science careers and consider higher education.

Continue reading

Posted on May 2, 2018 by admin in Evaluation, science_capital, Teachers, Widening Participation | Tagged , , | Comments Off on Using I’m a Scientist to increase participation in higher education

Providing STEM opportunities for distant schools

Baltasound_Junior_High_School_in_the_snow_-_geograph.org.uk_-_1725784“As we are a remote rural community we do not have a huge variety of careers on our doorstep but these events help to bring them closer to pupils. More students should be getting these funded opportunities across the UK.”

– Emily Tulloch, Science teacher on the island of Unst.


The most northerly school in the UK is located in one of our most distant areas in the Shetland Isles. Emily tells us how I’m a Scientist allowed her remote students to explore a range of STEM careers and increased motivation to learn science. Continue reading

Posted on April 6, 2018 by admin in Evaluation, science_capital, Teachers, Widening Participation | Tagged , | Comments Off on Providing STEM opportunities for distant schools

Broadening horizons for students in a deprived area

“It was a really easy project to engage with for both children and teachers; it only took me about 30 minutes to prepare for all 3 lessons. I was pleasantly surprised with just how excited students were to get responses from serious adults about their work and the adults’ interests.” – Vicky Heslop, Year 6 teacher


A junior school that meets our widening participation criteria took part in the Climate Zone of I’m a Scientist for the first time in March 2018 with their three Year 6 classes. The activity broadened student aspirations, improved enquiry skills and challenged their perceptions of scientists.

Continue reading

Posted on March 27, 2018 by admin in Evaluation, Teachers, Widening Participation | Tagged , | Comments Off on Broadening horizons for students in a deprived area

School engagement in STEM enrichment: Effect of school location

In recent years funders of public engagement and outreach activities have made a priority of reaching underserved audiences.

Wherever we looked we found anecdotal evidence that while, as a sector we were becoming increasingly effective at reaching schools in deprived parts of our metropolitan areas, rural communities continued to miss out.

But anecdotal data only gets you so far. We wanted to find out just how much the more remote schools were missing out. We also wanted to know what constitutes a remote school in this context. Continue reading

Posted on November 30, 2017 by modjosh in Evaluation, Widening Participation | Tagged | Comments Off on School engagement in STEM enrichment: Effect of school location

Thinking about Science Capital

Science Capital is a concept that measures the combination of experiences, personal connections, knowledge and attitudes that contribute to how much a young person identifies as a “science person”.

The ASPIRES researchers identified eight dimensions of Science Capital, including scientific literacy, science-related attitudes, values and dispositions, and knowledge about the transferability of science. I’m a Scientist aims to contribute to young people’s Science Capital through interactions with real STEM professionals. In particular, focusing on three dimensions of Science Capital: knowing people in science-related jobs, scientific literacy, and knowledge about the transferability of science.

Continue reading

Posted on October 6, 2017 by in Evaluation, science_capital | Tagged | Comments Off on Thinking about Science Capital

How do students from different schools engage with IAS?

Thousands of school students meet scientists through I’m a Scientist every year, and they ask thousands of questions. In June 2017, over 3,000 students took part, asking scientists more than 2,500 questions in the ASK section alone. This is also the event that we implemented our question coding system across all the zones to see what students are asking about. This all got us thinking: Do students from different types of schools ask more or less of certain question types? We’ve identified two groups we want to look at: Under-served: Schools more than 30 minutes travel time from a major research HEI Widening Participation: Schools with an above average number of students eligible for free school meals Taking the questions from the I’m a Scientist zones in June 2017 it appears that: Overall, the split of questions is similar across all groups of students Under-served students ask more “science topics” … Continue reading

Posted on September 14, 2017 by in Evaluation, Widening Participation | Tagged | Comments Off on How do students from different schools engage with IAS?

“I’m a Scientist is great, but wouldn’t it be better if students could see and hear the scientists too?”

This is a question we get asked from time to time. Here we explain why we’re confident that text interaction remains the best format for effective, inclusive, online engagement because it makes students and scientists more comfortable, levels the playing field between adults and children, makes the events accessible to a wider audience. Students are more familiar with text-based chats. There is growing evidenceš that young people communicate most via text and less and less through phone or video and we’re hearing that anecdotally too. Feedback from teachers has pointed out that students are not only more familiar with a text format but also more comfortable with it. All parties feel more confident about not being visible. As an ex-teacher myself, the thought of making a class visible online to an unknown person via a webcam makes me uncomfortable. I’d also be concerned about scientists inadvertently displaying confidential or inappropriate material … Continue reading

Posted on August 4, 2017 by modemily in Evaluation, IAS Event, Science Engagement | Tagged | Comments Off on “I’m a Scientist is great, but wouldn’t it be better if students could see and hear the scientists too?”

What do students ASK about? [REDIRECT]

Our events generate huge amounts of interesting data, which we know contains all sorts of valuable insights. When there’s so much of it, it can take a while to work out how to make best use of it. One example of this is thematic analysis of the questions students post in ASK. Over the years of running the event we’ve developed a pretty good sense of the types of things students ask about and will always showcase great examples of questions. Converting that into a systematised, reportable analysis is more difficult, but can be done. So, we have developed a system for coding questions based on a set of themes that students often ask about, and after some trials in March, we’ve applied it to all zones in the June event (read the latest reports here). At their broadest level, most student questions fall into one of three areas: questions about the … Continue reading

Posted on July 17, 2017 by in Evaluation | Tagged | Comments Off on What do students ASK about? [REDIRECT]

Gender differences in online engagement

“I thought scientists just looked like they do in the film Flubber and experimented on aliens or weird stuff but when I found out you liked Taylor Swift I realised you are more down to earth and not like mad scientist :)” – Student, November 2015 I’m a Scientist is about connecting pupils with real scientists. Something we want to find out is what effect this interaction has on different groups of students. Recent research by the Institute of Physics has done just that. The IOP has carried out an independent research project to gain an insight into the behaviour and attitudes of boys and girls who do I’m a Scientist. As part of their Improving Gender Balance project, the IOP funded two zones: Terbium Zone and Osmium Zone. The researchers anonymously surveyed the students’ attitudes to science and scientists before and after the event. We also provided them with all the student’s interaction … Continue reading

Posted on October 21, 2016 by in Evaluation | Tagged | Comments Off on Gender differences in online engagement

Careers Zone

The Careers Zone is a new zone where students attending a careers fair at their school can talk online with a range of scientists and engineers, based all over the UK. A lot of the time, school careers fairs are only able to accommodate businesses and organisations in the local area. We want to give students an opportunity to ask their questions to experts working in a multitude of areas, letting them see how diverse STEM careers can be. In turn, STEM professionals can pass on honest advice and information, showing students what it’s like to be a real engineer or scientist. We selected a range of experts to take part, including a deep-sea researcher, a scientist for the NHS and a Jaguar Land Rover engineer. There were also engineers who had gotten into their professions through an apprenticeship. It was important for us to let students see the different … Continue reading

Posted on October 6, 2016 by in Evaluation, Project News | Tagged | Comments Off on Careers Zone

Osmium Zone – I’m a Scientist, not just for scientists?

The Institute of Physics collaborated with I’m a Scientist to create the Osmium Zone that included four people who had studied physics and now worked outside of academia and research, plus one physics researcher, to demonstrate the diversity of STEM careers to school students.

The goal was to make students aware that studying science gives transferable skills valuable in fields outside of traditional research. The Osmium Zone had high engagement and positive feedback from students, and a survey showed a slight increase in students wanting a job that uses science skills and knowledge.

Continue reading

Posted on September 1, 2016 by in Evaluation, Event News, science_capital | Tagged | Comments Off on Osmium Zone – I’m a Scientist, not just for scientists?

Demand vs. Capacity — An update for June 2016

Graph of I'm a Scientist UK class requests by event in academic year — March 2013 to June 2016 Any avid readers of our project blog — there must be at least one of you — will have noticed we’ve written a lot recently about our increasing demand for classes, and our over-subscription rates.

In January, we published some numbers. Since then we have run the March 2016 event, and finalised class places in the June 2016 event. Here are some updated numbers:

Continue reading

Posted on May 23, 2016 by modjosh in Capacity, Evaluation, IAS Event, Project News | Tagged | Comments Off on Demand vs. Capacity — An update for June 2016

On student activity and simplified registrations

TL;DR: We made the site easier for students to access and the proportion of our audience engaging increased. Like any company, we have a set of Key Performance Indicators which we use to keep an eye on how well we’re doing. One of those KPIs (professionals use abbreviations) is the percentage of active students during an event, what we will call, %AS. %AS shows the number of students who log in to the site, and go on to — at the very least — ask a question, write a line of text in a live chat, cast a vote, or leave a comment. Basically, it shows the proportion of our audience who are actively engaging with the activity. In July 2014 Rosie posted a message on our project management app of choice, pointing out that the %AS for the previous events had been falling to the level it was at during the … Continue reading

Posted on May 23, 2016 by modjosh in Evaluation, Event News, IAS Event, Project News | Tagged | Comments Off on On student activity and simplified registrations

Vaccinations Debate Kit – Evaluation Report

Download the full Vaccinations Debate Kit - Evaluation Report The Vaccinations Debate Kit discusses whether children should be required to have all their vaccinations before they can go to school.

This debate kit was funded by e-Bug (a project operated by Public Health England) and Science Foundation Ireland. Gallomanor collaborated with E-Bug to research the kit, and asked for advice from teachers to ensure its educational suitability.

3,000 kits were printed in March 2015: 1,000 were distributed by e-Bug and 2,000 by Gallomanor. We have just finished our evaluation on these kits. And this is what we’ve learnt:

Continue reading

Posted on February 22, 2016 by modangela in Evaluation | Comments Off on Vaccinations Debate Kit – Evaluation Report

Demand for classes

When we started I’m a Scientist we used our March events to allow teachers to test the event and for them to come back in June with more classes. That time after school exams when teachers looked for something to inspire kids was the ideal time. Times have changed since then. March remains popular as teachers look for activities to coincide with British Science Week, but June is no longer the most popular time of year. We think this is due to increasing numbers of schools starting the next year’s timetable and curriculum after exams and that fallow period is no longer fallow. November is where we saw some spectacular growth in 2015. We’re not sure what has driven this – we’ll investigate and we’ll rebalance when in the year we run zones. The other clear narrative from this graph is that excluding the lower than expected level of requests in … Continue reading

Posted on January 12, 2016 by ModShane in Capacity, Evaluation | Comments Off on Demand for classes

November 2015 – Bigger than ever

We’ve just published the latest batch of zone reports for the recent November 2015 round of I’m a Scientist. You can find them here, or at the bottom of this post, but before you dive into the pretty pie charts and wicked wordles, we felt it would be useful to provide some context to keep in mind as you read: We had an inkling that November’s eight zones were going to be big. As we reported beforehand, due to extremely high demand from teachers, it was the first time ever that we had to turn schools away from I’m a Scientist. We also had to limit the number of classes of those taking part to one or two per zone to fit as many schools in as possible. Without doing this we would have had to run double the number of zones to cover the demand. So, after the hype, exactly how big were the zones in November? Here … Continue reading

Posted on December 17, 2015 by in Capacity, Evaluation, Event News | Tagged | Comments Off on November 2015 – Bigger than ever

Antibiotic Resistance Debate Kit – Evaluation Report

picThe Antibiotic Resistance Debate Kit discusses whether the NHS should tell GPs to give back-up prescriptions instead of immediate antibiotics wherever possible.

The kit was funded by e-Bug (a project operated by Public Health England) and Science Foundation Ireland. Its main distribution took part between October 2014 and January 2015, and we have evaluated it between September and November 2015. After the evaluation, we’ve learnt: Continue reading

Posted on December 1, 2015 by modangela in Evaluation | Comments Off on Antibiotic Resistance Debate Kit – Evaluation Report

3 words to describe a scientist

In March 2014 we started running zones for primary students only, increasing the number of primary school students who take part in I’m a Scientist. And we wanted to know more about how the event affects them.

This time, we drifted away from our usual online environment and sent printed surveys to teachers taking part in the Colour Zone and the Thulium Zone, the two primary-only zones we ran in June 2015.

We asked students which 3 words they would use to describe a scientist, before and after taking part. We got responses from over 300 students from 9 different schools. The students used more that 230 different words to describe scientists. They used 188 different words before taking part and 127 after, maybe indicating that they are clearer about what a scientist is after taking part. Awesome, cooperative, inquisitive, life-saving… are just some examples of the words they used Continue reading

Posted on November 2, 2015 by modangela in Evaluation | Comments Off on 3 words to describe a scientist

Which browsers and devices are schools using?

Every so often we take all our data, and look at how schools use the site; looking at how visitors access the site and how much that has changed in the past couple of years.  This post is for data up to 2015. See also data for 2014, 2018 and 2019. We’ve made a lot of changes in recent events, especially when it comes to registering accounts and using the site on mobile devices. We now pre-register all teachers and students, giving them usernames and passwords, so they can start asking questions immediately. We can assume that the student visitors give a fair reflection of the general school IT facilities and system capabilities. The graphs show the student data represented by dashed lines. We saw last year, that visits from mobile and tablet devices were increasing. and no surprise they continue to grow, while desktop usage drops slightly. For student tablet users … Continue reading

Posted on October 30, 2015 by modemily in Evaluation, School | Comments Off on Which browsers and devices are schools using?

Research vs Evaluation

Science Learning+ is a significant funding scheme provided jointly between the Wellcome Trust and the National Science Foundation. Learning can happen anywhere and at any time. Science Learning+ is an international initiative that aims to understand the power of informal learning experiences inside and outside of school. The second aim of the scheme is to “bridge the practice and research gap” At a seminar in July aimed at providing an update on the Phase I project an interesting conversation developed about that gap between Science Communication practitioners and researchers. I heard one speaker talk about practitioners wanting to know if a hypothetical red headline would give a 3% uplift in visitors. I responded on twitter: Disagree that practitioners want efficacy. I use eval. for that. I want research to tell me if the activity provides good outcomes #slplus — Shane McCracken (@ShaneMcC) July 27, 2015 Not all practioners agreed with … Continue reading

Posted on August 27, 2015 by ModShane in Evaluation, Science Engagement | Comments Off on Research vs Evaluation

Harwell Open Day Zone Report

On Saturday July 11th the Harwell Research Campus opened its doors to the public. 10,000 people were expected to come and see some of the most complicated and cutting edge science equipment in the world. Hundreds of volunteers working with that equipment were on hand to speak with the public and show them what happens. We created a Harwell Zone to allow visitors to text in questions. It was promoted using posters as above. The logic was that with 10,000 visitors some would not get to ask their questions, or may think of it later, or possibly be too shy to ask in person. It was a family day. Working with the over-worked (understatement klaxon just sounded) outreach team at Diamond and Harwell we set the zone up and promoted it to exhibitors. It wasn’t a great success. A mere 14 questions were sent. However in the interest of continual … Continue reading

Posted on August 24, 2015 by ModShane in Evaluation | Tagged | Comments Off on Harwell Open Day Zone Report

Electricity Distribution Debate Kit – Evaluation Report

It’s been a year since we launched our Electricity Distribution Debate Kit. We emailed all the 810 teachers who requested it, and followed them up with reminder emails and letters to collect feedback. We then analysed it and this is what we’ve found out:

  • Hard copies of the kit were sent to 810 teachers who requested them,  687 were sent to STEM Contract Holders, and the rest were distributed among different teacher networks. The kit was also downloaded by 242 different users.
  • We estimate that the kit was used by 29% of the teachers who requested it.
  • Every teacher who filled in the feedback survey found the kit to be an “excellent” or “pretty good” educational resource.
  • 97% of teachers feel their students know more about Electricity Distribution after using the kit.
  • We measured how the students reflected on the issues surrounding electricity distribution and actually changed their minds throughout the debate.

You can read the full evaluation report here (PDF). Continue reading

Posted on March 18, 2015 by modangela in Evaluation | Comments Off on Electricity Distribution Debate Kit – Evaluation Report

Understanding Animal Research – Google Hangout

Before our most recent June 2014 I’m a Scientist event we partnered with Understanding Animal Research to run a Google Hangout for scientists who work with animals. Students often ask the scientists for their views on animal testing, and we want to help give scientists the confidence to be open and talk about working with animals in their research. John Meredith, Education Manager at UAR, ran this session with six scientists who were taking part in I’m a Scientist the following week: The A word: how to talk about animals in medical research  This informal webinar presented by Understanding Animal Research will look at how and why we should talk about animals in medical research. It will cover current public attitudes to animal research, the facts and figures, the value of openness and how to answer the tricky questions or deal with confrontation. There will be plenty of time for questions … Continue reading

Posted on August 6, 2014 by modjosh in Evaluation, IAS Event | Tagged | Comments Off on Understanding Animal Research – Google Hangout

Wellcome Trust engaging science day

How to share and disseminate the learning from your project and evaluation. On 14th May the Wellcome Trust held one of their twice-yearly Engaging Science days. They’re an opportunity for grantholders to meet with each other and Wellcome staff, to share learning and ideas. And to sample Wellcome’s renowned lunch offerings (side salad complete with edible flowers and quail eggs, anyone?). Gallomanor were invited to speak on the panel of the session convened by Ben Johnson of Graphic Science, with the blurb “Once you have completed your project how do you widen its impact by ensuring others learn from what did (and didn’t) go right? An effective evaluation can also help with securing further funding and a wider roll-out of your project.” Other panelists were Manisha Lalloo from the Royal Academy of Engineering giving a funder’s perspective, and Becky Parker & Dave Colhurst from Simon Langton Grammar School. Evaluating the … Continue reading

Posted on May 19, 2014 by admin in Evaluation | Tagged | Comments Off on Wellcome Trust engaging science day

I’m a Scientist zones for primary students

We’ve had plenty of primary schools take part in I’m a Scientist zones in the past, nestled in among secondary schools. Feedback from primary teachers and students shows they get a lot out of taking part. Primary schools are looking for science enrichment activities too, and talking with real scientists is exciting at primary school as well as secondary. Scientists often wouldn’t know the students were Year 5 & 6, not Year 7 & 8, if we didn’t tell them. This March we decided to run 2 primary school only zones, for Year 5 & 6 students. In part to avoid possible situations of primary students reading questions on non-primary-friendly topics asked by secondary students in their zone (think sex, drugs, rock’n’roll). Partly to open up I’m a Scientist to a wider group of UK schools and students. And importantly because one of the conclusions from the ASPIRES project about young people’s … Continue reading

Posted on April 10, 2014 by modjosh in Evaluation | Tagged | Comments Off on I’m a Scientist zones for primary students

How does I’m a Scientist affect students’ attitudes to science?

We’re always evaluating the online STEM enrichment activity, I’m a Scientist, to explore its impact and how it can be improved. Check out latest evaluation posts. We know anecdotally that participating in I’m a Scientist has a positive effect on students’ attitudes to science. Teachers tell us that their students were buzzing, and that they understood more about science. Students thank scientists in live Chats for an interesting lesson. They tell us that it is “better than Facebook”. But, being data geeks, that isn’t good enough. We wanted to know just how much we were affecting the students. Were we affecting them all by the same amount? Girls, boys, year groups? Did being more active in the event mean a greater change in attitude? Questions we ask students Thanks in part to the pilot work by Robin Longdin (then a Science Communication masters student at UWE) we ask every student registering for I’m … Continue reading

Posted on April 9, 2014 by ModShane in Evaluation | Tagged | Comments Off on How does I’m a Scientist affect students’ attitudes to science?

March, June or November?

When teachers ask me what aged students they should bring on I’m a Scientist, my answer is often something along the lines of “well, it’s developed for Year 9 students but all ages get something different out of taking part, so take your pick”. The good news is that our data supports this. When students register we ask them what year group they’re in. There are Year 5s. There are also Year 13s. Around two thirds of the 30,000+ students registered so far are KS3 (Years 7, 8 & 9). The bad news? It’s not really bad news as such, but it looks like I should also be advising teachers on the time of year they should take part with different year groups. March and June are great for pre-GCSE students. Teachers tell us they’re looking for something fun to do with students at the end of the year. They’re less good … Continue reading

Posted on March 14, 2014 by modjosh in Evaluation | Tagged | Comments Off on March, June or November?

Girls vs Boys

I was asked recently what the gender split was for students taking part in I’m a Scientist. I replied that we’d never looked on the assumption that since teachers took their students online it would reflect the school population. I’m never one to turn down the chance to crunch some numbers so I took a look. And was surprised. At registration we ask if the person registering is a Boy, Girl or A group of students (where more than one pupil is sharing a log-in). We have data going back to June 2012 and nearly 15,000 respondents to the question. Surprisingly it wasn’t a straight 50:50 split. So I then decided to look at it by year group. What we saw was that year 9 & 10 were significantly skewed towards girls. And that bias continued through to 6th form. Is that common for science outreach events?

Posted on March 11, 2014 by ModShane in Evaluation | Comments Off on Girls vs Boys