Posts Tagged ‘data collection’

Interviews come to an end…

Two months has now nearly passed since I began data collection through interviews with project managers, really – how time flies.  I’ll have to admit it has been exhausting.  Arranging interviews, preparing for a interview, transport to and from interview location, performing the 90-180 minute interview, sending a thank you note, filing the notes taken and ethics signature, transcribing the recording, gathering and building sampling information e.g. duration, demographics – and then the cycle begins again.

Panic attack!  At one stage a few weeks back, I got a bit worried that my questions/themes that I had designed might not be getting even close to answering my research questions and that I should have been more disciplined and performed a pilot (thinking of @jeffreyKeefer’s recent blog posts).

Yet when I retraced my thinking I calmed down – let me put it this way, if an structured interview questionnaire is set at 0 – semi-structured at 50 and unstructured at 100, my interview design is at about 75-80!  This re-affirming my strategy of using loosely structured questions/themes to maximize my chance of creative and active interviewing removed the panic. At the end of the day how would I have piloted a conversation?

Can you imagine after completing the majority of interviews only to “think” you had no data?????    What I did do though was at the half-way point was refine my questions, introduce some new ones, and drop others, but all linked to the themes in question.

Sampling: has been interesting.  My original population for the case study included 23 project managers, of which 16 responded, however, as I went through the interviews 5 further opportunities arose from within the interviews (as either snowballing and/or theoretical sampling) to interview “extras” who might have unique perspectives, or had either left the  case study group or the company.  These opportunities did turn out some really interesting insights and often counter-claims to the other project managers – so I’m looking forward to when the analysis begins.  I think there is another important point here.  When my chance for observation was declined – I wondered about case study methodological triangulation!!!!  I still have documentary analysis (bi-angulation!!) so it seemed important to get some different perspectives from management and not just the project managers themselves….could I call this theoretical triangulation Yin, 2003.  Not sure.

Transcribing & early coding:  In the early transcripts I refrained from any coding whatsoever, but as I progressed I did start to code some times as I went along, nevertheless the data largely remains as from the interviews.

One thing I have been doing that might be quite controversial is HOW I have been doing the transcripts…so its time to come clean.

  • English was in most cases a second language for the participants.  While they use English the majority of the time within the case study site for their work, some grammatical corrections were made.
  • Given the analysis will be themed coding and discourse analysis, most of the stutters, pauses, ums and arhhs have been taken out as I have deemed these not to substantially related to my research questions.
  • Given the loosely structured nature of the interviews, there were some occasions that the interviewee and I drifted off topic.  On these occasions these drifts off topic have been left out of the transcripts and replaced with a symbol at least to show where these are.
  • My own question phrasings or ramblings were mostly shortened but care taken so that the meaning not changed, all mmmms, yes, how so etc… were largely removed.
This data collection has felt like a marathon at times, and in total I estimate that there will be 160,000 words from the 21 transcribed interviews.  The good news is I can see data analysis of these transcripts and documentary analysis beginning soon.  Which will represent for me an important milestone that calls for me to move-away from my subjects as interviewees and colleagues and put that academic hat back on – and start theorizing!
So what questions do I have for my fellow researchers and bloggers out there:
  1. Is losing the observation data a big issue for this case study’s validity and quality claims?
  2. Do you think my transcription technique is acceptable and justified?
  3. Any tips during this transition?  Coding looks like it will take a while, but discourse analysis and documentary analysis also looks like a big task.
  4. And finally, should I re-engage with the literature soon?  I was thinking of doing this later once some themes start to emerge – from a practical and coherence perspective.
Last but not least – a big thank you to the courageous project managers who took part in the interviews – without you there is no data!  I hope to be able to do it justice.
Look forward to hearing from you!  Happy researching!!!!!

Interview reflections

My musing today is about my experience so far using interviews as a data collection method.  The whole process of conversing and listening to fellow project managers has been a learning and enjoyable experience, and every single interview has left a signature or unique impression on me.

I’ve purposefully been an active interviewer to maximise my connections a) as a project manager/researcher interviewing project managers, and b) the shared company events and contexts, but what are the lessons to date?

  1. Interview schedules are a quite a time costly exercise – contacting interviewees, arranging time/place, handling re-arrangements if they no-show or change, and the follow ups before and after the interview all takes time.  I found a simple excel spreadsheet and outlook calendar helped.
  2. I offered the choice of interview location to each interviewee, hopefully they chose a place where they would feel comfortable to talk.  Abu Dhabi’s coffee shops have been popular, however, the recordings from these locations have been hard to transcribe at times – due to not being able to control the noise levels from babies screaming, music rocking, or grinding or cleaning the coffee machines!  I prefer the interviewee comfort over trying to enforce quiet locations.
  3. Interviews require quite a lot of energy and intensity.  Considering my interviews have ranged from 90-150mins in duration, I can handle no more than 2 per day – on top of my day job, plus the commute.  Being attentive, active listening and controlling my terrible tendency to interrupt is energy intensive.  One per day might be optimal for part-time researchers.
  4. Frustratingly in four out of 14 interviews, one of the recording devices (iPhone) has not work for some reason.  Having a back up recording device (Livescribe Pen) has saved my bacon.  Anyone who know how’s to convert a livescribe pencast file into mp3 so I can transcribe the backup files in NVivo – please email me!
  5. Technique: well, I’m slowly getting better……I think.  When I went through my first few interviews the number of times the interviewee and I were speaking at the same time was appalling. Embarrassing in fact!  Nonetheless, the lesson perhaps is that you can benefit from trying to transcribe the interviews as soon after the interview so that these kind of reflections on your technique can positively influence future interviews.  I have however, refrained from actually formally beginning any systematic data analysis but discourses and patterns naturally pop out.
  6. Question phrasing:  I’m noticing that some of the early phrasing of questions got blank stares and required too much explanation on my part.  Since then, I’ve tried to find simpler and more appropriate ways to phrase the question – done gradually AND at roughly the midway point in my interview schedule.
  7. Trust/Safety can be a big factor to getting good data or not.  I’ve have one particularly bad interview where I was accused of steering the interviewee to talk about something that they were not happy to talk about.  What was interesting was how their body language went defensive and closed – actually it was only at the end that this person opened up again with about 5mins to spare!  This has thankfully not been a pattern i.e. 1/14 cases.  Most actually found the interview to be really useful for them and some actually told me at the end what they got out of it.  I will admit however, that having insider knowledge does mean I’m steering a bit….but that’s got more to do with a pattern emerging from previous interviews not any pre-concieved judgment on my part, but equally perhaps hard to separate.
There is one last point ringing in my head.  Once I was described as demonstrating journalist tendencies rather than an academic ones – mainly I think from my passion and rhetoric, my comfort in taking on a political agenda, and previous seemily lack of interest in academic theory development.
Now given my interview style (choice) I find myself wondering how different research interviews and journalist interviews really are?  What has changed in me, is my understanding in the need for adding to the academic conversation, my agreement that we need rigour  research (in whatever style appropriate for the ontological / epistemological genre).
Nevertheless – in the interview – I simply feel that making the most of the connections, the relationships, the history is beneficial rather than taking a passive academic interviewer stance.  For me, its about getting to the ‘real’ interesting data underneath the veneers people and organisatons maintain.  I’m simply happy to do whatever it takes to get great data – then create the social/intellectual distance during the formal analysis phase!
What do you think?  Love to hear from you!

Plea for help!

My blog today – from my new blog site (after I spent much of Saturday transferring over the content from iWeb is a plea for help on my MRes dissertation project from the research community #phdchat.  Yesterday, I posed a question on #phdchat that got a lot of responses: what do you call participant-observation that is based from memory – and is it a valid data collection method?  Let me explain.

Recently, I lost my primary case site where I was going to do a case study with 6 day participant-observation, interviews * 2 with 3 program managers, and documentary analysis of project management methodologies, PMO charters etc.  My plan B (now Plan A) was to move the case site to a location where I have worked before.  My change to the data collection is to interview circa 15 project managers with unstructured in-depth interviews on identity, and continue with documentary analysis (no change here).

What has changed significantly is that the 5-6 days of observing project managers is impractical as I can’t observe 15 managers except in the general sense, and I can’t get close to the action due to their primary work being all over place, and I would only get to see them at temporary desks sending emails…at best.  So I got to thinking about the years I had worked there, and the events, presentations, and meetings etc.  I did get to observe most of these project managers in action in some form or another.   The key is point us that I never made any notes.

So what I’m thinking is to go back through emails, presentations, critical events (there were some) with most of the project managers I’m interviewing and construct accounts from my perspective.  The idea is still the same in that I want to see how ‘what people say’, verse ‘what I SAW them do’, and the documentary evidence converges or contradicts in order to same some thing useful and, valid – kind of like a retrospective diary.

Is this a valid data collection method?  If I was interviewing someone it could be called narrative history or oral history perhaps – but I’m writing it.  Alternatively, I could call it auto-ethnography – however, here I am not the subject.  Is there something called auto-narrative history hehehe?

I believe it is participant-observation – just an odd form of it.  I could use say Spradley’s framework (1980, p78) to make it somewhat systematic!


What would you call it?  Or is my thinking here just nonsense???  Help me find a way to use what I think could be useful data for compare/contrasting purposes.  And, most importantly, be my heroes and share some references of research projects or articles that have implemented the same strategy legitimately.  I will need some backing I would think to get this past the external examiner!

PS. I hope this blog works ok!  It is the first time I’m using WordPress.