I'm starting to worry that AI is making some PhD researchers LESS confident in their own thinking.
Last week, I had a chat with a PhD researcher that really concerned me and I want to put my take on this “out there”. The PhD researcher in question is okay with me sharing this anonymously BTW so I’m not betraying any confidences and they read and approved this before I posted it.
Here’s what was going on.
They’re in the middle of their data analysis and they’re trying to figure out what it all means. So far, so normal, right? They keep getting stuck and doubting themselves. Also, completely normal. They’ve also started chatting to their AI about their data analysis. Also normal…? Yes, I’d say so, from what other PhD researchers are telling me.
But - this is the stage at which I get a bit, “Mmm, feeling a bit uncomfortable with this”.
Once we established that they weren’t uploading any information to it that they absolutely shouldn’t be - e.g. participant identifying information, confidential research material (gulp!) – they explained that they were chatting to the AI about what their findings might “mean”. Probably worth adding here that they started their PhD several years ago, prior to the AI explosion and all the work they did in the early days was, as they said, “All from my own head.” They hadn’t outsourced any of their thinking to AI or uploaded any of their writing into ChatGPT to get its ‘Thoughts’ on it.
NOW though, they were talking to it about their interpretation of the data and kept reaching for it asking, “Might this mean…?”, “What do you think about…?”.
In the middle of me saying, “Well I guess that’s okay as long as you don’t….”, something stopped me.
I looked at this smart, accomplished human in front of me who was balancing a full-time job and a whole adult life with their PhD and wanted to say, “Step away from the f-ing AI and listen to this mini lecture I’m about to give you!”. Instead, I clumsily said the following, which I’ve since tidied up for this post.
As interpretive researchers, we’re meant to be REFLEXIVE. When we start plopping our analytical thinking into the AI – even if we are using it with the intention of genuinely examining our assumptions and we’re not trying to do anything naughty – are we even being reflexive anymore? How do we know when we're critically examining our interpretations and when we're beginning to rely on AI to validate them?
I haven’t made up my mind on that one yet. Don’t know.
I also think that some of the conversations we’re having about AI are a bit of a sideshow. A very smart person I worked with several years ago taught me that whenever there’s a ‘hot topic’ being debated, ask, “What is this distracting us from?”.
So, I asked myself that question and I think all the to-AI or not-to-AI is diverting our attention away from this:
Are we becoming less willing to trust our own analytical judgement?
I’ve done it too. Outsourcing my own judgement to AI – I did it with gay abandon in my own small business when it first came out: What videos should I make? What emails should I write? What should I cover on my blog this week? Because I DIDN’T TRUST MY OWN JUDGEMENT as I was new to the online education business world and felt like a complete imposter so THINK FOR ME, ChatGPT! GUILTY as charged.
My “enough” of all this came earlier this year when I saw that I’d started to write like AI. Bloody hell. Then I found myself (look - “found myself” FFS!) spotting the “it’s not this, it’s that” and sets of three in my writing. I was like jeez, what the hell?
Trusting our own analytical ability is important as qualitative researchers, as is our okay-ness with uncertainty. Qualitative Research 101 is basically learning that there isn't necessarily ONE correct interpretation of your data, but you'd better be able to explain how you arrived at yours.
However, chuck in a technology that's very good at producing confident-sounding answers, even when those answers aren't necessarily right, and you've got yourself a bit of a poop show.
These are PhD researchers who genuinely CARE about doing justice to their data and the people who shared their experiences with them. They care so bloody much they're spending YEARS of their lives researching those experiences.
When they start doubting their interpretations, of course they're going to look for reassurance.
Seeking that reassurance, they hop onto ChatGPT, Claude or whatever AI the cool kids are using at the moment. (Claude does sound like a cool dude though, right? I bet he smokes Gauloises and is fancied by everyone’s mum like Patrice from the Inbetweeners.)
Right, where was I? Yes. PhD researcher, good intentions, difficult analytical decision, AI to the rescue.
And so begins the reassurance-seeking loop. (Yes, I start some sentences with “And” now – FML! 😂).
Every time they feel uncertain, they ask the AI.
It responds confidently with an interpretation that sounds perfectly reasonable.
They feel reassured.
Until they encounter another difficult analytical decision.
Back they go.
I’m not saying never use AI at all, but maybe I’m suggesting that we shouldn't normalise automatically seeking external reassurance every time we encounter analytical uncertainty. Sitting with a bit of uncertainty and being okay with something in our data not being resolved yet is part of the process.
I’m going to say the thing that people who know me will roll their eyes at because I say it so much: Your biggest a-ha moments with your data analysis tend to happen when you’re NOT sitting in front of a computer, or a wall covered with Post-It notes. IYKYK.
So, anyhoo…. Just wanted to put this ‘out there’. At what point does using AI to help us think become relying on AI because we don't trust ourselves to think? Where do YOU draw that line in qualitative research? I'm still working out where I stand.