Evaluation

I’m a Scientist so far in numbers

How many zones have we run? How many students engaging with how many scientists? Asking how many questions? What year group are they? Where in the UK are they? These are some of the questions we get asked a lot, and until now haven’t had all the answers in one place to roll off the tip of our tongues. We’re pulling together some slides summarising I’m a Scientist to date and once they’re done we’ll post them up here. In the mean time, I’ll be posting snippets that show just how far we’ve come since our first big Wellcome Trust grant in 2010. To start us off (and to set the scene for the graphs and maps to come in future posts) here are some of the numbers so far, up to February 2014: Zones: 118 51 general science zones, 67 themed zones Spread across 10 events, in March, June … Continue reading →

Posted on February 14, 2014 by in Evaluation | Tagged News | Comments Off

Visits to IAS by technology type

In 2014, as part of a review of what we need to do over the next 5 years with I’m a Scientist, we looked at how visitors accessed the site and how that has changed year by year over the last 4 years. We also looked at how visitors registered as students differed from All visitors. The Student visitors probably give a fair reflection of school IT capabilities. In the graphs the student data is represented by dashed lines. We only have student data from 2011 when we started tracking registered users separately. View more data up to 2019, 2018, 2015. It was no surprise to see that visits from mobile and tablet devices were increasing but having 23.6% of visits from mobile devices in 2013 was a shock. Delving deeper it is clear that it is mostly questions that mobile users visit. Tablets tell a different story. For student tablet users … Continue reading →

Posted on January 22, 2014 by ModShane in Evaluation | Comments Off

I’m a Scientist acts as a public engagement booster for scientists

Anecdotally, we’ve heard of how I’m a Scientist can be a good starting point for science communication activities. This was the case of Suzi Gage, Tom Crick or Suze Kundu, who took part in I’m a Scientist in June 2011 and are now putting a great emphasis on the communication side of their scientific careers or even fully devoting to it. However, we were still curious to know to what extent we could extrapolate this to the wider community of scientists that have participated in the event during the last years. How could we know if I’m a Scientist had encouraged them to do more science outreach? Well, we decided to ask them. We sent a survey to all the scientists that had participated in I’m a Scientist until 2012, leaving a gap of at least one year since they took part in the event. Approximately a quarter of the … Continue reading →

Posted on January 8, 2014 by modangela in Evaluation, Event News, Science Engagement, Scientist Benefits | Tagged News, Scientists | Comments Off

How does I’m a Scientist change students’ perceptions of science?

We’ve just run I’m a Scientist in Ireland and are curious to know how our event actually affects students’ attitudes towards science. In order to do this, we included a short and compulsory pre-event survey in the form students used to register. We then asked students to fill in the exact same survey on their profile page after the event. When we matched the data from the two surveys, 92 students (7% of 1,247 students that participated in I’m a Scientist) had filled in both surveys. Importantly, data from the total number of students that filled in the pre-event survey very closely correlates with the pre-event data of this 92 student sample. We were very happy to find out that students’ interest in science and science related careers is clearly increased after taking part in I’m a Scientist. This is what we have learnt: The amount of students that say they love science doubled after taking part in … Continue reading →

Posted on December 19, 2013 by modangela in Evaluation, Event News, IAS Event, Science Education, Science Engagement, Teachers | Tagged News, Teachers | Comments Off

IAS Live: Antibiotic Awareness Day

Yesterday afternoon, on European Antibiotic Awareness Day, we joined the Royal Society of Chemistry (RSC) at Burlington House in London, Piccadilly for a foray into antimicrobial research. We took five scientists — three of whom were I’m a Scientist alumni, having taken part in previous online events — whose research looks at the uses of antibiotics and antimicrobials, put them in a room in-front of around 100 sixth-formers, and had them answer questions on everything from Typhoid Mary, to zombie apocalypses. Our scientists were Rob Shorten, Jess Bean, Mark Roberts, Clare Taylor, and Emma Newton. Helen Arney, our MC for the afternoon kicked off by introducing the scientists before launching into a round of Science Fact or Fiction. The facts covered everything from bacteria flavouring cheese and yogurt, to our being only 10% human, with most of the cells in the body (by number) actually being bacteria. Clare introduced the … Continue reading →

Posted on November 20, 2013 by modjosh in Evaluation, I'm a Scientist - Live | Comments Off

IAS Live: Drugs, Bugs and Infections

Last night five scientists researching different aspects of antimicrobials took to the floor at the FIS Conference at the ICC in Birmingham to take part in I’m a Scientist Live: Drugs, Bugs and Infections. Our five were: Mark Webber Richard Bax Amy Wedley Anne-Marie Krachler Peter Hawkey Our MC for the evening, Simon Watt, kicked off by spending 10 minutes interviewing Laura Piddock, Chair of Public Engagement at BSAC about the current situation in research. The scientists introduced themselves and gave their Science Facts and Fictions before the 100-strong audience launched with their questions. 30 mins later with time running out, Simon brought the event to a climax by asking the scientists to spend 30seconds telling the audience why they should get the votes and the winner’s trophy. The vote that followed confirmed that Anne Marie’s promotion of the IMI’s Summer School for Year 12 students was enough to edge … Continue reading →

Posted on November 20, 2013 by ModShane in Evaluation, I'm a Scientist - Live | Comments Off

What criteria do students judge scientists on?

We’re always evaluating the online STEM enrichment activity, I’m a Scientist, to explore its impact and how it can be improved. Check out latest evaluation posts. When students take part in I’m a Scientist they get to vote for their favourite scientist to win £500 to spend on communicating more science. It gives them ownership of the project and they decide who gets, in effect, a small grant for public engagement. But what are students’ votes based on? This is how 855 students (or groups of students) have ranked certain criteria from most to least important when considering how to vote for scientists, between January 2012 and November 2013. The results come from a Drag & Drop ranking activity in the first lesson plan “You’re the Judges” that teachers run to introduce their students to I’m a Scientist. The top ranked criteria are all linked to health or the scientific method. This is reflected in the number of … Continue reading →

Posted on November 8, 2013 by admin in Evaluation, IAS Event, Teachers | Tagged News | Comments Off

Using teacher feedback to plan future themed zones

A question I sometimes get asked when I tell people about I’m a Scientist, is how do we choose which zones to run? In short: we ask teachers what they’d like. In July we emailed all the teachers signed up for I’m a Scientist to ask them what zones they’d like to see in our next three events: November 2013, March 2014 and June 2014. Just over 40 teachers filled in each survey by ticking as many of the 40 zone choices as they’d like (some are zones we’ve run before, others are new). Which zones came out top? The graph shows the top 20 zones, by the number of teacher requests across the 3 surveys for November, March and June. Forensic Science and Genes zones came out top, closely followed by Stem Cells, Health, Organs and Sport Science. Most of the top requested zones are biomedical; this might be because there … Continue reading →

Posted on October 30, 2013 by in Evaluation, IAS Event | Tagged News | Comments Off

We’ve published all our zone reports from June

We’ve published all our individual zone reports from the June 2013 event. If you delve further back into the archives, you’ll find them for the zones we ran in March 2013 & November 2012 too. All the reports are tagged (funnily enough) under “Evaluation Reports“. Each report sums up the activity in that zone. They provide some information for the scientists and teachers in that zone about how much the students and scientists interacted. How many questions were asked and answered? What were the popular topics asked by students? What examples of great engagement stood out? What did students, scientists & teachers say about taking part? How does the winner plan to spend their £500 on communicating more science? We email out each report to all the scientists and teachers in that zone. Scientists tell us the reports really help justify their participation to supervisors and colleagues. The detail tells them … Continue reading →

Posted on July 17, 2013 by in Evaluation | Tagged News | Comments Off

Who are you Twitter?

We’re always evaluating the online STEM enrichment activity, I’m a Scientist, to explore its impact and how it can be improved. Check out latest evaluation posts. As we prepped for our Science Communication Conference 2013 session on how we evaluate I’m a Scientist, the occasional sidetrack leads to interesting stuff. A while back we used Tony Hirst‘s scripts and know how to download our twitter followers network. We’ve used Gephi to map out the clusters within our network which helps us work out who is following us and how diverse our network is. We were delighted to see an Australia cluster thanks to our friends in Adelaide. More about that later. Another useful benefit of downloading your followers is the ability to search their bios to find specific followers who are interested in certain subjects. This evening we identified 83 followers with “food” in their bio. We’ll be contacting some of them (excluding those … Continue reading →

Posted on April 18, 2013 by admin in Evaluation, Science Engagement | Comments Off

IAS Outreach by STEMNet Area

We’ve selected the scientists and schools (all will be revealed next week) and we’ve looked at where they are across the country. We want to work closely with STEMNet contract holders to promote our events to teachers, scientists and engineers, so we’ve calculated how many schools, classes and scientists are in each STEMNet contract area. From that we’ve calculated the estimated number of interactions (or Live Chats as we call them) we expect per area. On average each class has one live chat and each scientist attends seven. The best news was that there are only 3 areas where there is no activity. We are doing science engagement and enrichment across the country. But where do we have most activity? Most classes signed up: Bristol, Bath and Somerset – 25 Lancashire – 19 Surrey, Central and East Berkshire & West Yorkshire – 18 Special Kudos to Liz Lister in Bristol, … Continue reading →

Posted on February 7, 2013 by in Evaluation, Event News | Comments Off

What themed zones do teachers want?

For I’m a Scientist to work we need to run zones that teachers and students want. So when we’re deciding on zones to run in the next event we ask teachers to tell us what they want. Teachers voted on a longlist of zones we drew up – they could select as many as they’d be interested in taking part in. We also asked them for suggestions for other zones. Over 100 teachers told us what zones they’d like to see. The results are below.

Posted on December 12, 2012 by in Evaluation, IAS Event, Science Education | Comments Off

IDACI Index Profile

It it important to know who we are reaching with our events. Funders are keen to know and we want to know that we aren’t just reaching the usual suspects. We’ve recently done some geographic analysis, which we’ll share soon. But general location is only part of the story. Nice schools in nice areas tend to do plenty of science engagement and send lots of students to university. Therefore it is important to see if we are reaching those schools who don’t normally participate; those hard to reach schools. But how do you measure that? Does the deprivation of the area in which the school is located provide a good proxy? Some studies would suggest so: “The Characteristics of Low Attaining Pupils” DfES Statistics Bulleting 02/2005 June 2005 We plotted the IDACI Score of the schools who participated in IAS and compared the profile (based on deciles) against that for … Continue reading →

Posted on November 23, 2012 by in Evaluation | Comments Off

SpotOn London 2012 – Background

As part of ScienceOnline (or SpotOnLondon2012) Karen Bultitude and I have arranged a session called: Can we work together to better evaluate online engagement? It’s taking place in the Steel Room on Monday 12th November at 2.30pm. We wanted to provide some context to the session. Here goes: Since you’re interested in ScienceOnline it’s fairly likely that you think online science engagement is important and going to get more important. Me too. I run a few online science engagement projects, https://imascientist.org.uk is the most prominent. It’s pretty important to me to know how well the project is performing. We need to know in order to improve it, we need to be able to tell our funders and we feel obliged to be able to demonstrate to all stakeholders that the project is effective. So we evaluate our projects using a wide range of methods: surveys, interviews, personal meaning mapping and … Continue reading →

Posted on November 9, 2012 by ModShane in Evaluation | Comments Off

I’m a Scientist Live – evaluation

We like to experiment with the I’m a Scientist brand. In the last 12 months we have adapted the site to accommodate unlimited number of scientists in a zone, we’ve run In the Zone across a 12 week period and opened it up to the general public with the GM Food zone. Yesterday show another variation when we took the show live in front of an audience. Thanks to the boundless energy and eternal optimism of Rhys Phillips a former IAS participant we agreed to let him use the format for the Cardiff Science Festival. Sunday 15th July 2012 at 5pm on the Glanfa stage in the foyer of the Wales Millenium Centre was scheduled. A quick email out to scientists who had applied to take part in IAS resulted in 6 scientists volunteering to take part. Dr Mark Lewney was roped into MC’ing the session. We were given a … Continue reading →

Posted on July 16, 2012 by ModShane in Evaluation | Tagged News | Comments Off

We've cut our teacher survey from 41 to 18 questions

This morning I cut down the survey we ask teachers to fill out after taking part in I’m a Scientist. The old survey was a hefty 41 questions long, and the new one has just 18 questions. By looking at teachers’ previous responses to the 41 questions we were able to identify the questions which worked, and the questions which didn’t. Using this information we’ve made the survey simpler. It’s also more quantitative. Funding from the Wellcome Trust for the next 3 years allows us to improve the way we evaluate I’m a Scientist, and move to evaluating outcomes through more quantitative measurements. My next task is to do the same for the scientist and student surveys, and cut them down from 28 and 25 questions.

Posted on November 23, 2011 by in Evaluation, IAS Event | Comments Off

Evaluating the Impacts of engagement

How can we evaluate the impact on students taking part in I’m a Scientist? Can we measure if they’re more likely to take a STEM subject at A Level? If they’re more likely to study science at University? How should we use the large amounts of data generated by online projects? How can we share our evaluation in a more useful way? These are just some of the questions we’re trying to answer about evaluating I’m a Scientist and other Gallomanor run projects. Judging from the first in a series of seminars looking at Evaluating Impacts of Public Engagement and Non-Formal Learning, last Friday 4th November, others are thinking along the same lines. The Core Issues & Debates seminar kicked off the series at the Dana Centre in London, and bought together a range of researchers, evaluators and learning and communication practitioners. Future seminars focus on areas such as how … Continue reading →

Posted on November 9, 2011 by in Evaluation, IAS Event, Science Engagement | Comments Off

Evaluation in 2011

Evaluation in 2011 In June 2011 we came to the end of a grant from the Wellcome Trust to run I’m a Scientist in 2010 and 2011. We’ve evaluated throughout the project and the final evaluation report is available to download below, as well as a shorter executive summary and summary sheet. 2010-2011 Evaluation Report– at 75 pages long it contains a lot information including the methodology, aims, key figures and data, findings and recommendations. Executive Summary – this outlines the main findings from the 2010-2011 Evaluation Report and looks at whether the objectives set at the start of the project have been met. Summary sheet of the event in 2010-2011 – this highlights some of the key numbers and quotes from evaluations, and reasons why the event works so well. We also produce summary reports of each zone and send them to the scientists and teachers in the zone. These … Continue reading →

Posted on February 1, 2011 by modemily in Evaluation | Comments Off

Read about our session at Science Online conference

We’d never been to the Science Online conference. And we’d never given a presentation about I’m a Scientist, Get me out of Here! But when the organisers asked if we’d like to do a session on the event we jumped at the chance. We then closed the office for the whole of August, so we had to write and organise our highly interactive, possibly risky session in 2 days when we got back last Wednesday. At the same time as do all our catching up from the holidays. Last week was an interesting week… So here is a summary of our session (last Fri, 3rd Sept) and what we, and others, said. Our presentation At Gallomanor we like to walk the walk, as well as talk the talk. So we wanted to make the session as interactive and audience-led as possible. After a short (2 min) intro, we showed the … Continue reading →

Posted on September 7, 2010 by in Evaluation, IAS Event | Comments Off

Students! Tell us what you thought about I’m a Scientist, and win a prize.

This is a last reminder: We really want to hear from you! What do you think, did you like I’m a Scientist? What did you like/dislike about it? Was it better or worse than normal lessons? You should fill in our survey and tell us what you think. It helps us make it better for next time. Also you’ll go into a prize draw for £20 WH Smiths vouchers. Survey is here http://www.surveygizmo.com/s/307117/i-m-a-scientist-june-2010-tell-us-what-you-think- Thanks for your help!

Posted on July 19, 2010 by in Evaluation, Event News | Comments Off

How DOES magnetism work?

Last year I saw A C Grayling talk on happiness and it’s importance. He said that he bans his students from using the word ‘happy’, that it’s a lazy portmanteau word. He thinks that if you are forced to choose a different word – hopeful, exhilarated, content – you’ll think about what you really mean far more clearly. I think the same can sometimes be true of jargon. Scientists taking part in I’m a Scientist have told us before that explaining yourself without using jargon is hard work, but unexpectedly rewarding. It means you have to think through what you mean and it exposes your mental shortcuts. We saw a great example of this in Imaging Zone. A fairly innocuous-seeming question (Why do magnets attract and repel?) pretty much lead to the scientists realising that they don’t actually know how magnets worked. Not really. Not when they tried to actually explain it to … Continue reading →

Posted on July 13, 2010 by in Evaluation, IAS Event | Comments Off

A scientist gives us the lowdown on what it’s like behind the scenes…

As you all know, there are evictions every day for the rest of the week! You get a vote every day, so remember to keep voting for your favourite scientist. But how do the scientists feel, being bombarded with questions and then facing eviction? One of the scientists who took part in I’m a Scientist in March, the lovely Louise Buckley, has agreed to tell us about what the scientists taking part are REALLY thinking… Hey there! I am Louise one of the scientists from the last “I’m a Scientist….” event. This competition is looking even more exciting than when I took part – I cannot believe how many questions are getting asked. It’s a total mash-up of questions – these scientists are going to need to take a holiday afterwards to recover! I bet the scientists are starting to feel nervous now about who’s going to get evicted. We … Continue reading →

Posted on June 23, 2010 by in Evaluation, Event News | Comments Off

Busy, busy, busy!

Well the first week is almost over already and it’s been incredibly hectic. This is already the busiest event we’ve ever run – we’ve had nearly 4,000 visitors, just in the last few days! Thank you to all the students for the interesting, funny and thought-provoking questions. And thanks to all the scientists for all the hard work they’ve put in answering them. Students – feel free to comment wherever you want to on the site. Visitors – although you can’t comment, you are very welcome to have a look round and read everything. If you want to you can follow us on twitter – our twitter feed is http://twitter.com/imascientist, or the hashtag for the event is #IAS2010 – scientists and teachers, if you are on twitter, please use the hashtag if you are tweeting about the event, then people can find all the comments together. We are having a … Continue reading →

Posted on March 18, 2010 by in Evaluation, Project News | Comments Off

These scientists were just like real people!

Hi everyone, here’s a summary of how the March 2009 event went (PDF download). I was supposed to write a couple of pages so of course it’s 12 pages long (I’m the same with cooking – if I have a couple of friends round I make enough food for ten and we’re eating leftover curry ’til thursday…). Fortunately Shane (my boss) is busy being terribly important in London running his G20 bloggers tent, and hanging out with Bob Geldof (pictured), so he can’t tell me off about it. What we found was pretty similar to the pilot (but it’s important to check everything is still working:-)). One of my favourite quotes was the one I used as the title. Students were over and over amazed that ‘scientists are just like real people’, and that they were taking time to talk to young people. They also couldn’t quite get over that … Continue reading →

Posted on April 3, 2009 by in Evaluation, IAS Event | Tagged Evaluation, March 2009, thanks | Comments Off

Supplementary questions

Lots of people have written back to me with comments on the draft plans for the teacher packs. Thanks everyone!

Posted on March 18, 2008 by in Evaluation, Science Engagement | Tagged chemistry, data handling, Democs, development, draft plan, feedback, information sheets, lesson plans, teacher packs | Comments Off

Do you think scientific speed-dating would work?

Well we’ve been beavering away here in I’m a Scientist Towers. We’ve taken on a freelancer to develop the teacher packs. Becky Davies usually works at Techniquest

Posted on March 12, 2008 by in Evaluation, How Science Works, School, Science Education | Tagged Becky Davies, development, draft plan, feedback, information sheets, lesson plans, teacher packs, Techniquest | Comments Off

View all answered questions

No comments to display

View all Frequently Asked Questions