We’d never been to the Science Online conference. And we’d never given a presentation about I’m a Scientist, Get me out of Here! But when the organisers asked if we’d like to do a session on the event we jumped at the chance.

We then closed the office for the whole of August, so we had to write and organise our highly interactive, possibly risky session in 2 days when we got back last Wednesday. At the same time as do all our catching up from the holidays.

Last week was an interesting week…

So here is a summary of our session (last Fri, 3rd Sept) and what we, and others, said.

Our presentation

Sophia presenting at Science Online coference London

At Gallomanor we like to walk the walk, as well as talk the talk. So we wanted to make the session as interactive and audience-led as possible. After a short (2 min) intro, we showed the audience the 12 topics we’d prepared and asked which 6 they wanted to here about.

Here’s the prezi presentation, with a slide on each topic. I’ll summarize below what we were going to say on each – including the ones we didn’t cover on the day.

IAS Solo on Prezi

1. Film – we showed our ‘Intro for the classroom’ film.

2. Our Philosophy

The audience didn’t vote to hear about this, which was a shame as I think it’s really important in explaining why we do what we do, and how. Fools, fools!;-) Maybe we should have called it something more exciting sounding…

Anyway, key points would have been:-

• We reverse normal power structures (kids in our event are given some decision-making power and the chance to ask what THEY want to ask). This engages pupils who are normally turned off. See more on my thinking here in this piece on the Secrets of Engaging Teens.
• Making it fun and game-like is not a gimmick and doesn’t make something not educational. It makes people pay attention and be interested! (e.g. this research)
• It’s not just about getting kids to study science. Not all teenagers will grow up to be scientists, but they will all grow up to be people, and need to have a relationship with science. More on this point in this article for Wellcome Trust blog.
• Scientists have a lot to gain from engagement too – they aren’t just doing everyone a favour. They can be energised, challenged and made to think by the huge variety, and inventiveness, of students’ questions.

“a 4.00am Eureka! moment solved a problem that’s been bugging me for the best part of a year. It came from a seed of an idea planted in my head by a simple question from a 13/14 year old, absolutely bloody marvellous! Next day I was skipping into work like a refugee Munchkin from the Wizard of Oz, bleary eyed but elated.”

3. History of event

Shane thought of the idea for our sister event I’m a Councillor, Get me out of Here! while drinking Guinness. It’s been running since 2002, helping councils and local councillors to engage with young people in their area.

We came up with the idea of doing a science version in 2007 and ran the first, pilot event in 2008.

4. Funding

The pilot was funded with a People Award from the Wellcome Trust. We then successfully applied for a follow up grant from Wellcome’s Society Award scheme.

We love Wellcome and they are fab. We approached lots of people – people like BIS and Dept of Education who you’d think would be the right people to approach, and we got nowhere. All roads led back to Wellcome.

If you’d like to apply for science engagement money from Wellcome, our advice would be:

• Make sure you understand what they value and what they are looking for, and that your project fits in with that.
• Contact them early and talk to them.
• Make sure you have really thought it through and explained your plans (not just vaguely). They will fund imaginative, even risky stuff, but they want to know you aren’t going to piss the money up the wall.
• Realise the importance of formative (and summative) evaluation.

5. Site detail

Shane was just going to run through what’s on the site and how it works. You can prob just have a look for yourself.

6. Site build

It was in WordPress, cos we like and support open source, etc. And also cos WordPress is great. Apparently the site is doing things with WordPress that no one has done before. This is of course very exciting.

Developed by total legend Mike Little. Here’s Mike’s presentation about the project at Wordcamp 2010.

7. Scientist Recruitment

We pursued as many routes as possible to recruit scientists, including contacting learned societies, universities, research institutes and companies. Personally, I also bore people to death at parties and force my card on them if they are scientists or know any.

Because we’d done formative evaluation (i.e. talking to possible end users) as part of the development, we were able to focus communication and explain what the benefits would be to the scientists.

Once we’d run the pilot, word of mouth was one of our most powerful tools, as the scientists who took part really loved it. We also had evaluation evidence to back up our claims of the benefits.

To select the scientists (as we are now oversubscribed) we involved our end users. For the last event we took each scientist’s one-sentence description of their work and put it up blind (i.e. just their words, no information on age, gender, ethnicity, organisational affiliation) on a website where students and teachers rated each description. We, and a representative of the Wellcome Trust, also rated each scientist and we combined the scores.

We still want to find new ways of recruiting scientists, and particularly reaching the scientists we aren’t reaching at the moment. If you can help, let us know! And if you want to get involved, sign up here.

8. Teacher recruitment

Timeline for debate kit sign ups, kit 1

Again, used many routes. As you can see from this graph of sign ups for debate kits, some of the most powerful methods are direct mail, the Planet Science e-newsletter, and posting on the TES message board.

Also, for the big event in June, we got a significant number of sign ups from the STEMNET newsletter and a few from many other sources. We think it’s worth casting the net widely.

Again, word of mouth incredibly powerful. Especially because we recruited a ‘teacher panel’ of teachers to help us develop the project, before the pilot, and they became very invested in the project and really helped us a lot. This emphasis on consultation with teachers also meant we really were providing something they wanted, in a way that was useful to them.

Cross-marketing from our debate kits project was also incredibly useful. It enabled teachers to get a sense of where we are coming from, and the quality of what we provide, before making the commitment of signing up to a two week event.

We are oversubscribed with teachers too, and here part of our selection procedure is to deliberately pick as varied schools as possible (geographically, type of school). This has included Special Schools, Pupil Referral Units and Bristol Hospital Education Service. And also schools in Shetland, Northern Ireland and Oman.

We also try to get a representative sample results-wise. By which I mean, high-performing grammar schools are overly represented in the schools who apply, we try to even things out by picking more of the less academically achieving schools. We think it makes a bigger difference to those kids.

As with scientist recruitment, this is still a work in progress. If you can help us get the word out to teachers (taking part is FREE to schools), please do! Or if you are a teacher who would like to get involved, sign up here!

9. Evaluation strategy

Formative evaluation has been absolutely key. We started talking to scientists and science teachers before we did anything else. Asking them about what they wanted, what would work for them, what motivates them.

For teachers, we recruited a teacher panel of people interested in the project, who could give us instant feedback via email on everything from lesson plan ideas to terminology on the site. This also meant that they were invested in the project, when the pilot came around, and understood it and what they could do with it.

We also included young people, for example testing possible designs on them. And we made several school visits to observe science lessons.

We had a limited budget for the pilot, so much evaluation had to be done in house. But we set aside money to appoint an external evaluation consultant, Yvonne Harris, to spend a few days on the project. She advised us throughout, and also conducted some independent interviews with participants at the end, and audited our report. This was absolutely invaluable as she could check things like questionnaires and methodology as we went along, and suggest solutions we would never have thought of, and bring an independent perspective to the whole thing.

It also helped that we had grown the project organically, as we had developed over the years questionnaires that worked, and found (and corrected) many useability issues as we went along.

You can read our evaluation report on the pilot here.

We now have a much bigger budget for the roll out of the project, and we have devoted far more of that to external evaluation. Kate Pontin is now our external evaluator, and she has been invaluable in helping us think with clarity about what we need to find out and how we can do it. She has also been able to do far more schools observation than we can, as we are busy running the event while it’s on! This has been extremely useful.

Kate’s interim report will be unveiled at a special event at the Wellcome Trust on 20th October. This will be part of a special ‘Beyond Blogging’ event, curated by us. Tickets will be very limited, but do get in touch if you’re interested in attending.

10 Evaluation findings

Shane outlined some topline figures so far from the 2010 events.

TopLine Web stats for IAS June 2010

• 4,667 students
• 100 scientists
• 171 teachers
• 6,580 questions
• 3,085 comments
• 4,744 votes

More in-depth evaluation results will be published on 20th October, as above.

In the meantime, you can read our full evaluation report on the pilot here.

A summary of the pilot evaluation report here.

Or our short evaluation report on the 2009 event here.

11. Summary of the strengths and weaknesses of this format

Strengths

• Power reversal truly engages and empowers.
• Online gives access to scientists (for schools) and students (for scientists), without having to go anywhere.
• Doing it online also creates intimacy (makes it easier to ask real questions and break down barriers), compared to a scientist in person giving a talk.
• It also ‘levels the playing field’ – quieter and less confident students participate more.
• You’re reaching all the students in the class – not just the very keen, as you find in science clubs, etc.
• A conversation develops over time – over two weeks, students can read about the scientists, go away and find out more, ask questions, think about the answers, ask more questions…
• For teachers, the preparation work is done for them and they can concentrate on facilitating learning.
• For scientists, it saves time – no travelling to a school, they log in from their desk. Every moment spent participating is spent in engagement.
• Scientists also find the range and energy of the questions rejuvenating, thought-provoking and inspiring.
• And scientists get into the competitive aspect and have fun.

Weaknesses

• Some scientists (and some teachers) don’t like the informality of the project, although we think it has real value in making connections and breaking down barriers.
• We’ve disguised the learning and made it fun, so some teachers/scientists/students don’t see that it’s there and think it ‘won’t help them pass exams’. (God help us if that is the only thing some people think education is about).
• Schools IT (sigh!). Often school firewalls are over-enthusiastic and we do have some problems with schools blocking the site.
• It can be hard work for the scientists. Some had ~700 questions to answer!
• It’s expensive to run (although not compared to many other projects).

12. The Question Game

Our question comedy improv game! OK, so not very comedy, and not actually improv, but kind of a game, inspired by Whose Line is it Anyway? Audience members shout out a word, any word, and we search the site to see if there are questions (or answers) containing that word. Intended to give an insight into the enormous range of questions.

Here’s some results

Search dinosaur

Search space

Search evidence

If you want to play the question game for yourself (be warned, browsing the site can be addictive!), just go to the main page and type in the search box near the top.

Live chat

After the presentation, we wanted to give the audience a feel for what live chats are like, and why they are so popular with scientists, students and teachers. So we had a live online chat, with the audience taking the place of students. We gave out log in details and everyone in the audience who had a laptop (quite a few, it being Science Online) could log in and take part. We also showed the chat on the projector.

Some of the key benefits of live chats:-

• Access to people who couldn’t be there otherwise: We had a scientist in Michigan, one in Sydney, a teacher in Shetland, and a student who was in school. All of them had taken part in the event and answered questions from the audience about what it was like.
• Immediate and friendly
• Fun

• Newer comments appear at the top, so you need to read upwards.
• Discussion isn’t threaded – we’ve found that breaks up the chat too much and stops it being a communal experience.
• Chats can be difficult to follow at first, but you get used to it quickly. Students are often quite familiar with chatrooms and don’t find it’s a problem.
• There are two chatrooms side by side, one for students, one for the scientists. (In this chat that means one for audience and one for our participants). This makes it more difficult to understand an archived chat, however, we’ve found from experience that if the scientists and students are all in the same box, the scientists’ replies get lost in an avalanche of comments from students, so this works better.

Q+A

Despite our best intentions, Q+A actually ended up spilling into all of the session and we started taking questions during some of the earlier presentations, and during the live chat. Personally, I think this was in some ways the most engaging part of the session, as we were talking directly to people and they were asking questions, rather than us lecturing them presentation-style:-).

*We made a tactical error really: so that the twitterfall in our session was a useful backchannel, (rather than a tempting view of other parallel sessions that people were missing!), we suggested a special hashtag #iassolo. I now realise it would have been better to keep the #solo10 hashtag, but just add an extra #ias, then all the hashtags from our session would have been archived along with the others from the conference. Oh well, you live and learn.

Archive of tweets relating just to our session, hashtag #iassolo.