Are we asking the right question?

Posted by Bronia Arnott

Recently I wrote about using twitter to recruit participants for a research study and how others said it couldn't be done. But were they asking the right question? Should they be asking if it SHOULD be done?

When I tweet to my followers on twitter I am sending information out to people who are like minded individuals. They probably follow me because they share some of my interests and probably because they like me have been caught up in the fallout from the weapon of mass distraction that is the Internet. So are these the kind of people I should be trying to recruit for a research study? A large proportion of them have a PhD and many work in academia (and I know how strange academics can be!). Even if they don't fall into those categories they tend to be those interested in mental or physical health, quite well educated, or my friends - which might not be very inclusive!

Reduce, reuse, retweet

But before I shoot down my research paper with the biased sample bullet let's take the finger off the trigger for a moment. Of course those things are true of my followers, but what about the followers of those who re-tweeted (RT'd) my requests for participants? Their followers may be more diverse than my own - especially those such as local radio stations who kindly RT'd. So, in theory, the further the RT was RT'd the more likely I was reaching people who were not that similar to me.

Taking another step back from the big red button, what kind of participants did I want to recruit? Well I was actually looking for adults of working age who regularly commute and own a smartphone – so if twitter users tend to be adults of working age who regularly commute and own a smartphone then perhaps that isn’t a problem?

However, while most twitterers are lovely (and recruiting this way allowed me to meet some of them who I may not otherwise have had the opportunity to meet) they are clearly different in many ways to those who don’t use twitter. Therefore while I do think people can and should recruit via twitter I don't think it should be the only method of recruitment. Unless you want to measure the effect of the weapon of mass destruction that is the Internet, then go ahead.

But enough about what I think, what are your thoughts?

They said it couldn't be done…

Posted by Bronia Arnott

Recently I needed to recruit some participants for a user engagement study that we were organising as part of our research. I had had in my mind for a while that I would probably try to recruit some participants via twitter. Anyone who knows me knows that I am a twitter advocate. I had been thinking about this idea for a while. However, when I mentioned my plan to others they said that they didn't think it could be done...

The Incredulity of Saint Thomas, by Caravaggio

Due to some unexpected delays on the project I realised that my brief recruitment timescale was going to coincide with moving house and my daughter starting preschool. I negotiated to have time off as long as long as I could recruit participants while I was on leave. The appeal of a twitter recruitment drive became even more appealing. I thought maybe I should probably have a strategy (or maybe I was just procrastinating) so I checked out what times of the day were most likely to get Re-tweets and put my academic writing skills to the test trying to summarise the user engagement studies into the 140 characters allowed by twitter. I also thought about targeting specific people who I followed or who followed me (or our research project twitter account) and asking them to Re-tweet to their followers. "Shy bairns get nowt" and all that.

Soon the RTs were coming in and so were the hits on our project website. Slowly but surely the sessions filled up. So were the doubters right? It wasn't as successful as I would have liked but I think there were a few reasons for that: we had very specific inclusion criteria, limited time, no funds, and for some of the time I had no Internet access (the provider who messed up not once but twice will remain nameless).

I think in the end researchers probably can recruit using twitter. Whether they should or not is a different question...

Routine Secondary Data

Posted by Lynne Forrest

We all know what a nightmare it is trying to recruit participants for research studies. So if you only have to get hold of some routine data that’s just sitting there, well, that’s going to be much simpler, isn’t it? You’d think....

The plan for my PhD was to look at inequalities in cancer care by linking cancer registry and Hospital Episode Statistics data for lung cancer, and also linking to audit data. This is routine data that has already been collected and so I naively assumed it was just a case of getting ethical approval to access the data, finding someone to cobble the data together and off we go. I wrote an optimistic project timetable where I would get my hands on the data about seven months into the PhD. Eighteen months in I’ve finally got hold of some unlinked data and I’m still waiting for the rest.


I don't work for News International, so what's the problem? Photo: Christian Sinibaldi

So, what went wrong?

I think my first mistake was assuming that just because the data was there it would be easy to get hold of. There are a lot of hoops you have to jump through first.

I thought that what I wanted to do was simple but it turns out that it’s not. This is apparently the most complicated linkage that the cancer registry has undertaken and the bottom line was, nobody wanted to do it. I spend a lot of time begging people to speak to me and basically being fobbed off, in the nicest possible way. Luckily I eventually found a newly-joined analyst who was willing to give it a go. I’m not sure that she’s thanking me now...

Issues then arose of whether the data I wanted might be identifiable. Variables such as date of birth and death are classed as identifiable and individual records are ‘potentially-identifiable’, even if they don’t include identifiable information (which is an excellent catch 22 – they are identifiable even though they are not identifiable...)!

Finally it seemed like it was all coming together. I’d agreed with the registry that they would supply me with anonymised data containing ages rather than dates, I’d made it through ethics, and I’d got some data. But, on checking, not exactly the data I wanted. So, currently I am discussing with the registry how it will be possible for me to calculate survival time if they won’t allow me to have data on the number of days from diagnosis to death. Survival from lung cancer is short and rounding to the nearest year isn’t going to identify survival differences with any degree of accuracy.

The sticking point is that although they are not supplying me with date of death I could theoretically work it out from this information and that makes the data (aaagh!) ‘identifiable’. However, as I don’t have an NHS number, date of birth, or place of death, I don’t know how I would identify anyone from the 140,000 records I have. Plus I’m a researcher, not a News International journalist, and I’m not interested in anyone individually, so I’m not going to attempt to do this.

Can’t I just sign something to that effect and have the data I need please?

A room of one's own

Posted by Stephanie Clutterbuck

I have never read "A room of one's own". But I thought it made a snappy title for a blog post so I did what any self-respecting early career researcher does when she wants to make sure she is not talking complete nonsense- I Wikipedia-ed it. Turns out it fits quite nicely. You see Virginia Woolf believed that "a woman must have money and a room of her own if she is to write fiction". And although I have no interest in writing fiction I can see her point, a woman must have participants and a room of her own if she is to do science.


Virginia Woolf, by Frederico Novaro

There I was, March 2012, a bright-eyed PhD student confident in my experimental design, armed only with some predictions and a dream. Against my better judgment I believed that data collection in schools would be easy and as long as I asked nicely for ‘a quiet room’ my wish would be granted leaving my data collection to roll merrily along, untainted by extraneous factors.

Fast-forward two months, seven schools and 200 participants later and I have learned an invaluable lesson: schools have very different definitions of ‘a quiet room’.

At my first school I was shown to an open corridor where I was told I would run my experiment. Admittedly, I had to suppress my inner toddler from stamping her feet and screaming ‘This is NOT a quiet room!!’, but I made do. And in fact it would have worked well if the corridor wasn’t simultaneously being used as a makeshift studio for yearbook photos. And then as a meeting place for all the Year 5 boys to earnestly discuss the moral implications of stealing the Year 6’s football during break. To be fair it was a lively debate and I could see their point, why should the older boys get the ball - isn’t that ageism?

Still, data collection continues to roll along in various schools and I have become adept at managing my inner meltdowns regarding the unreality of tightly controlled experimental conditions outside of the lab. I was able to force a smile when a teacher barged into my quiet room (which happened to also be a kitchen) and rattled the contents of every drawer and cupboard in search of a knife to cut cake. And I have learned to tune out repeated renditions of ‘Kung Fu Fighting’ and ‘Heart and Soul’ sung by jubilant five year olds in nearby music rooms.

Fortunately, for my sanity, I feel data collection enlightenment is within reach. The other day I unflinchingly accepted that there was nothing I could do about the man in muddy overalls wielding a shovel and walking through my quiet room. Twice.

Don't give up

Posted by Amy O'Donnell

Before I embarked on my PhD in public health, I spent eight long years at the coalface of policy research. My chosen specialist subject: promoting the participation of under-represented groups in the labour market.

A huge part of that research agenda was engaging with so-called ‘hard-to-reach’ groups to find out their views and experiences of economic and social exclusion. So those eight years saw me reaching out to countless ethnic minority support groups and more fieldwork with long-term benefit claimants than I care to remember. Truly, I had the whole research-engagement agenda licked.

Don't give up

Or so I thought. Right up until the point I started recruiting participants for my doctorate.

For the record, my PhD examines whether we can use routinely collected data to assess the delivery of alcohol interventions in primary care. The fieldwork involves data extraction in 20 GP practices plus interviews with GPs themselves.

I’ve done scores of lone parent interviews. I’ve tracked down Muslim community representatives in the far reaches of Scotland. I’ve recruited peer researchers to access the insular gypsy and traveller community. Bring on the full-time, paid professionals in their permanent, accessible venues!

Turns out that when you don’t have any research funds, when you’re not part of the exclusive Primary Care Research Network, and when you can’t actually get anyone to answer your emails, it’s actually pretty damn difficult to get GPs to take part in your research.

Of course I didn’t quite appreciate this at the start when I posted out my pretty, ethics committee-approved recruitment letters to a select sample of practices and waited for the offers to flood in. They’ll soon come-a-calling, I though, and I’ll go skipping merrily off to collect my data. But they didn’t, so I couldn’t.

Slowly, two key issues emerged. First, I was hardly alone in this endeavour. There are simply zillions of us health researchers trying to grab a few minutes of GPs’ precious time. Understandably, the research that makes financial sense, or reflects a particular interest of the practice tends to take priority. Second, recruiting GPs in the middle of the most significant shake-up of the NHS since its foundation was never going to be easy.

But after many long months of unanswered letters and emails, and playing phone-tag with elusive practice managers, I am delighted to announce that I now have 15 practices recruited and have actually started my GP interviews.

So what happened?

All I can suggest is that I just tried harder. So instead of tens of emails to practice managers, I sent hundreds. I forgot my pride and pestered every friend, family member and colleague I could think of who might be able to help. I fine tuned my data requirements so that I could truthfully assure practice staff that I really would take up as little as possible of their busy day. I offered to visit for a ‘quick chat’ at anytime, anyplace, and with anyone that might listen. I was friendly, positive and professional.

Most of all, I just didn’t give up.

No magic tricks required - recruiting adolescents for research

Posted by Stephanie O'Neil

At a recent event, a speaker began to muse on the difficulty of recruiting adolescent boys into research.

Having spent most of the last decade avidly researching the mystical things we call ‘youth’ and ‘sociology’ whilst trying desperately to combine the two,* I have more experience than most when it comes to working with ‘young people’. So recruiting adolescent boys – right up my street.

Three years ago, I set out on my PhD journey to examine young people’s relationship with alcohol and how they framed their choices about drinking. I knew instinctively that, to some degree, I needed to conduct qualitative research (I also did a systematic review and a Q methodology study, but those are sagas for different posts). This ruled out working in schools, who couldn’t see what was wrong with handing out a survey to the masses. Instead, I wanted to speak to young people alone for up to an hour. And I wanted to do it without parental consent (a topic for yet another post). Absolutely no go in schools, I’m afraid.

So I set out far and wide for youth centres, youth offending teams, and youth and inter-generational projects. I hung out with the youth parliament, on ‘mobile youth buses’ and toured Eldon Green on Friday evenings with youth workers. I took part in air hockey competitions to ‘win’ an interview, made home-made jewellery and watched more Emmerdale and Eastenders than is healthy for your psyche. I became known to the young people of Newcastle as ‘that Mackem alcohol bird researcher’.

Young people in Newcastle

Sometimes, I had to spend several weeks attending a youth group before anyone would agree to be interviewed. Other times, young people would agree straightaway. Regardless, I ended up with fascinating, rich interviews, from an equal split of boys and girls. And all of this was achieved with no incentives. Each young person gave up their time freely and willingly.

I don’t believe that the ability to engage young people (and I do think this is a better term than 'recruit youth') in research rests on being young, female, or a native to the North East – although I am all of these things. What I do think it rests on is how you treat the people you interview.

Young people are not a group that should be considered ‘different’. In fact, thinking of ‘adolescent boys’ as a distinct group that might be hard to recruit is probably the reason why they become difficult to recruit. Like everyone, young people just wanted to be listened to and to know what they were contributing to by taking part in research .

That’s it folks. No other magic tricks required.


*Yes, I’m a sociologist working in a public health department. Please don’t judge me.

Short listing and revealed preferences

Job hunting

Posted by Jean Adams 

Wow!  Fifty six people applied for the research post that Ihad advertised: "4 year research vacancy to support a series of studies on the acceptability and effectiveness of using financial incentives to encourage uptake of health promoting behaviours in the UK."

Four years is anice length of time for a research job.  But 56 applicants?  How's agirl supposed to do justice to the more than 600 pages of application stuffthat these generate?  Obviously, the solution is Newcastle University'sshort listing matrix grid.*  Tick here if your candidate has the qualitieslisted in the job description.

So of 56applicants, guess how many definitely looked like they had the rightqualifications and experience in an appropriate research area?  No, really- guess.  Wrong! 

Guess again. Wrong again!  

It was four. Including one person who didn't really but had such an outstandinglyimpressive academic record, and has run a successful business since the age of13, that I felt I couldn't not interview them.  Plus another six lookedpotentially like they might be good.

I don't have vastexperience of employing researchers, so I don't know if this is unusual. But I thought the general pattern was quite interesting.  The notshort listable fell into three, not mutually exclusive, groups:

1. People with aMasters in Public Health but no real-life research experience beyond their MScdissertation.  None of them had published a paper from their MSc - whichwould have counted as real-life research experience.  

2. Doctors andother health professionals from developing countries.  Some with vastclinical experience.  Some with Masters in Public Health (see 1 above). But still no research experience.

3. People withextensive laboratory research experience.  Lots of PhDs.  Even some muchmore senior people.  Some even said in their 'statement of intent' thatthey were looking for a position in Cell Biology.

Now as I writethis I'm wondering if I was too harsh.  How are people supposed to getresearch experience after their MSc if people like me won’t hire them?  Ibet you some of those foreign doctors are highly competent and could put theirhand to most things.  Why shouldn't research experiencebe transferable from lab to office?

But what really, really astounded me is that not one singleapplicant stated what their interest was in my particular researchproject.  They were all passionate about publichealth, keen to expand their skills and experience across a range of researchmethodologies, and excited to work in such a prestigious department asours.  But none of them had anything tosay about using financial incentives to encourage uptake of health promotingbehaviours.

Perhaps this is revealed preference at its sharpest.  Maybe nobody out there is interested in myresearch.

*Not quitetrade-marked yet.