Steve Portigal ‘Symbolic Lab Coats, ASMR and Coke-Fueled Lap Dances: More User Research War Stories’

Hello and welcome to another episode of This is HCD. I’m your host Chi Ryan, and in this episode I’m speaking to researcher Steve Portigal. Steve is the principal of Portigal Consulting, and the author of two books: Interviewing Users: How To Uncover Compelling Insights and Doorbells, Danger, and Dead Batteries: User Research War Stories. Based outside San Francisco, Steve helps companies to think and act strategically as a result of human insights. Steve has his own podcast, Dollars to Donuts, where he interviews people who lead user research in their organizations.Throughout his career, Steve has interviewed hundreds of people including families eating breakfast, hotel maintenance staff, architects, rock musicians, home automation enthusiasts, credit default swap traders, and radiologists. His work has informed the development of mobile devices, medical information systems, music gear, wine packaging, financial services, corporate internet and video conferencing systems. Wowsers! Steve has really done a lot of things!

But before we get into this episode with Steve – we are actively looking for podcast sponsors with 100% of the money raised going directly to Cara Care, an incredible NGO who support children who have suffered abuse. Get in touch to find out more via our website thisishcd.com. You can also donate directly by clicking on the dollar sign inside the media player on the website.

Chi: Welcome to the show Steve! So great to have you here. Can you tell us a little bit about yourself and about your experience with human centered design.

Steve: Yeah I work for myself. I run a tiny consultancy. I guess if it’s just myself that would make it pretty tiny. Just outside of San Francisco and the work that I do is helping companies learn about their customers and figure out what to do with that information and also just get better at that as an overall practice for their organization. I’ve been running this business since 2001 so I’ve been doing this work for a while. Before that I worked at other agencies. And although I could go back to that, that’s kind of where I’m at right now.

Chi: So you started out when you were, what, 10 (years old)? (laughs) Go on. I know I was so young when I started out – I was I was literally a baby.

Steve: (Laughs) This is a good rapport building technique by the way – researchers open up with flattery.

Chi: (Laughs) It gets you everywhere! Maybe you could tell us a little bit about how you started out how you got interested in this world that we inhabit.

Steve: Yeah I sort of found my way serendipitously kind of backwards. You know I was a computer science undergraduate because I thought that was about making things and it may be nowadays but back in my day it was very heavily on the science part and it was extremely theoretical and I really struggled with that as kind of an academic direction. I remember the last year I went there was a computer graphics course that’s kind of an emerging area and. And as part of that course, they gave us a little introduction to HCI – to human computer interaction. And what blew my mind then was this this idea that – you know we take for granted now – but this idea that the person who makes a thing has not just an opportunity but also an obligation to make the thing in a way that the person who uses that thing can succeed. And I remember the Professor talking to us “Oh if you see someone that doesn’t know whether a push or pull a revolving door or get stuck in a revolving door or you know can’t see where the steps are you know going up and down a staircase and you know stone walls are trips that’s the fault of the person that made it”. And that was such a huge reframe for me. I was kind of in that era of personal computers where if you knew how to use it you would sort of lord that over people that didn’t. That was just, I don’t know, a cultural norm, but I was part of. So that really opened things up for me and the fact that there was a whole discipline about human computer interaction that was meant to learn about people and then figure out kind of how to best make things for them. I didn’t know the word ‘design’. This was before the web you know – software as an industry was very technical. I didn’t know the market research – like I really had a very limited view of what the business world and the professional world could be, but that sort of sent me off into a graduate education in human computer interaction and and then trying to figure out how to apply that in the professional world. It was certainly challenging, but I was very fortunate to end up at industrial design consultancy that was at the same time were trying to design software as part of how they were serving clients. They were also experimenting with ethnography – this buzzword that we also called it – the fuzzy front end or the front end of innovation. There were a lot of funny terms being thrown around in the 90s but it was this idea that you could help companies figure out what to make and how to make it – and you would do this, this upfront research, and you would also do evaluative research, you also would see if the thing that you were making would work for people and this was, you know, a hard sell at first. It wasn’t what companies were looking for – it wasn’t what their process was – it wasn’t what they thought this agency could provide. But you know, over time, this starts to become a standalone part of the business. And you know, I was able to learn, in an interesting way, to learn because it’s not like any of us knew what we were doing, so I was kind of all along for the ride as long as this was being developed as a practice and we were, you know, it was a Silicon Valley industrial design company so what they were typically doing was putting like plastic boxes around circuitry. That’s kind of what a lot of that business was, or you know, figure out how to create medical devices that would have some injection molded something or something, and then we started doing projects like helping a breakfast foods company think about the sort of occasion and sort of symbolism and meaning of breakfasts in a different way that led to new products. And so the business started to sort of – we started to do interesting kinds of work and have those opportunities and I started to see, you know, sort of how to find my own place in going into people’s lives, their homes and their work and interviewing them and observing them and finding what those patterns were and being able to make recommendations to companies about how they could act on what we had learned.

Chi: You make a really good case for why research is important and I wonder at what point do you think that companies start to realize the value of working in this way – of looking at their their customers and the people that are in their ecosystem in a more human way.

Steve: Yeah, that’s a thing that I’ve been interested in, or when does this happen, or what are the conditions that make it happen. And the answer to that question has changed over time. You could go back to a point where, like it was never or rarely or until something catastrophic happened or you know some failure. I think now it’s getting baked in to, you know, earlier on so that you know a tipping point might not even be the right metaphor – I think it’s so much tied to the ascendancy of design. You know, I like to separate design and research, but I think there’s a point at which they get combined into one thing as design starts to be part of how new companies are forming their ideas, being started by designers or their being started with designers or design becomes like a key hire for like big financial services companies that want to reinvent themselves or you know, try to catch up with where the culture way, their user base has moved. I think you see some element of involving users in the process coming at the same time. So I think the short answer is sort of everywhere. The tipping point has happened and it’s just it’s table stakes now. I think the more cynical part of that or the caveat to that is it’s still the way that companies are bringing users into their design processes is still maybe more naive. It’s still more closed ended. I think – I think the word testing is a word that I sort of cringe when I hear it because testing is the place holder for anything that anybody would ever do. Well yes, we really really believe in our users. We test everything with them. And I don’t mean that testing is bad but testing is not everything. Testing is not how our culture aligns around what the lives and passions and concerns and fears of their customers really are. It’s how they kind of ship things to minimize failure which is good, like we should be happy about that. So I’m still looking for that tipping point where there’s kind of a richer belief in customers and insights about them and process to make that happen that are really that well that’s a strategic tool where that’s you know a big part of how the company the organization kind of moves along. I think you see it happen sometimes when there’s a change at the highest levels like they bring in somebody who worked at another company where there was a lot of user centred design processes and they start making changes, you know, that often it’s kind of capped at of service level until something happens at a larger level higher up in the organisation where someone wants to to change the way that the company is going about it and that person is maybe not a designer, maybe not a researcher, but you know, is maybe a product person or just an overall kind of an operations person or some executive that’s really thinking about how does this company work and what are we not doing right then and trying to create some change.

Chi: I want to ask you about making that happen in organisations but before I ask you that question, you mentioned separating design and research. Could you expand on that a little bit.

Steve: Yeah, I mean I am a research person and what I do is, what I write about is, what I advocate for and I think, you know, researchers need to as individuals and as a process. I want us to fly our flag and not get sort of folded into the larger conversation about design. There’s a lot of good conversations about, you know, how do you manage designers, how do you create career paths for them. How do you hire them. How do you create an environment where designers can succeed in a tactical culture. For example there’s a lot of stuff about that. Those things are the same for researchers – research people that are drawn to research bring with them different personality types and different skills and different ways of thinking about problems and different language and different backgrounds and so forth just from a purely, you know, managing eighteen points of view. I think that’s an example where research needs to be considered separately. Yeah, and I feel like research is an activity but it’s also a skill set or a role or an individual, and I think we struggle with it. You know, there’s the activity of research which anyone can do – and designers do research, and product managers do research, in you know, any sort of role you can do that but, then also you have researchers getting hired and so what do researchers do if everyone can do research? So part of me is just wanting to sort of advocate for the specialness and the separateness. So I think, you know, research is a special skill set, and there’s, you know, when I think about that divide that I want to hold on to – is it a role or is it a skill. I believe that everyone should be doing it, but I think there is, you know, certain types of problems, certain types of situations where there’s a complexity, or you need that expertise, and I want to just acknowledge and advocate for. That’s an expertise that maybe someone who can do a decent job at a running usability tests, you know, showing their designs to somebody and getting feedback, not even called usability test, that designer may have all the skills that they require or to kind of do a great job with what they’re doing but there are other types of problems, other types of questions, other types of initiatives, where you need someone who has different skill sets and lets us call that person or researcher. So yeah, I think given the types of problems and the roles and how people are rewarded, incentivize, managed, encouraged facilitated, those are just some of the reasons why I want to consider those as separate types of things.

Chi: Well it’s interesting because on the weekend I was in Toronto at a conference called the Design Leadership Summit and Jared Spool was on stage and he spoke about everyone being a designer and this is – I think there’s a parallel between these two things because, like you, I come from a background of industrial design and architecture actually and so I really do believe in the fundamental disciplines of design whether it be visual design industrial design and so on. And I think that everybody can participate in the design process but I think that everyone being a designer is a little bit of a misnomer. So I really like what you’re saying about research being a discipline unto itself and I think, you know, you said that you have a science background. It’s no wonder, because I think research in science is so different in a lot of ways – same but different. You know, I think scientists who are out there doing research on various different things would probably think that, you know, doing a little bit of research on the side of a design project, you know, might not exactly be the same thing – so it feels very similar to the struggles that designers are going through in terms of everyone being a designer.

Steve: That’s a great analogy a great comparison.

Chi: So getting back to the other question – how can we help our clients, or the people that we’re working with, understand the value of research and have an easy answer for that?

Steve: I mean if there was easy answer I guess the question wouldn’t need to be asked! (laughs) I mean, I think some there’s some paths to success that I’ve seen, you know, people go through you know some of it is managing up and then so you know how do you successfully manage up. How do you get people’s attention a little bit. How do you show them value. How do you give them kind of experience and exposure. You know I mean I think this is maybe the researcher answer to that question or it starts with trying to understand what are the barriers, what are the blockers. You know, it’s a framing of the problem to say well they don’t see the value in research. Maybe they do see the value in the research but they’re ranking it lower than something else. Time to market is more important it’s better to get something out than get it right. Like I’ve worked with organisations that are very much about experiments which I’ve had a hard time with because we learned things that had very clear implications to what they were putting out and they said yeah that may be true but we’re going to put something out and then you know after a period of time I think they were on a kind of an annual basis after a year we can you know we can see how we perform with these different options. So even though we had you know an insight that would drive a decision their kind of product culture I guess worked very differently and was really late in the game when we kind of uncovered that. We think with thought we were more on the same page. So there is there is sort of trying to figure out what is going on why isn’t it being done. You know maybe research is just seen as validation. Like I was talking about before and not seen as a kind of a strategic tool you know who were the people that are holding those beliefs and how were they influencing you know the steps that people are taking. You know what do project timelines look like and you know what’s being allowed or what’s not being allowed. I’ve seen people be successful to a certain extent you know ask for forgiveness not permission or I mean just just doing it. Just going out and doing something scrappy and small to answer their own questions and kind of making those things available in small ways. And I think there’s some sort of best practices around that that I’ve seen. I worked with a team that would do a hack day style research that was really just about getting people out on the field and they weren’t doing a lot of analysis or really anything they were just getting people out to talk to users and so they could kind of fly under the radar with that a little bit. It wasn’t  a multi week thing and it wasn’t impacting projects it was just you know giving people that holy crap moment where they go talk to someone and realize that how they talk about their life and their goals and how they’re using your product is different in fundamental ways. And so that was you know that was sort of a slow burn kind of culture change operation that every month they would do this for a day and it grew and grew and grew and they brought more people in and they so they set it up in kind of a positive way. I thought that was really neat, where if you attended as kind of an observer or a note taker or first several times then they gave you the ‘symbolic lab coat’ and like the people that ran this went and got lab coats stitched with the company logo which clearly was a being like the brand police would not have been happy about it but they did it anyway they just had enough money to buy these things and so they had a little ceremony in their prep sessions where they would you know say now so-and-so has been with us four times they’re now able to lead their own we want to present them with this jacket. And so they created a really positive kind of “want-in” thing and they were working you know at sort of mid level and kind of leadership level and they were not making so much noise anymore or would stop them and they would run these sessions these debrief workshops that we would do after this day and we go out and talking to people. They ran them once in the kitchen because they couldn’t get a conference room. But people kept walking by and like what he’s doing. What are you guys doing. Oh we spent the day talking to customers. Coo! And then they’d kind of go about their business we started to become. You can see sort of seeding some viral aspects of this change as opposed to kind of go all the way to the top and saying we have to go about it this way. They were, you know, finding any opening and kind of just kind of filling in like in, kind of like they were trying to challenge the status quo they were trying to kind of bring people along slowly and gently you know and eventually that company ended up hiring like a director of research, and then that and I think just staffed up enormously from that. So it became an important thing for them but in the early days it seemed almost sad right. You guys are barely doing anything you’re not using the there’s so much potential here but they cannot win a battle. I think in sort of a slow and positive way I think, you know, in that circumstance that was kind of an effective technique I think we could come up with organizations we know where “that is never going to work because this and this and this!” but here I think that was you know a little bit under the radar but very much inviting people in making it fun you know looking for very small wins, letting people sort of feel and touch the world outside the building. You know I think it’s an interesting kind of slow technique and I think, you know, a principle that can kind of underlie a lot of this is is not asking for process. You know there’s a difference between saying we need to have two weeks in the schedule to set up this thing versus we want to provide information to ensure the decisions that are being made are the right ones, and I think it’s naive – it’s seductively naive – to talk about how you want to work and less so to talk in the language of the person you’re trying to persuade, and to sort of say here’s why this is better for you and less about here’s what I want to do.So it’s not a feel good sort of kind of story – but it’s an outcome story,

Chi: That’s exactly what I was thinking – I was thinking this is – it becomes less about executions and more about the outcome. And that’s a really important distinction to make, especially when you know, for designers particularly, and from I’m thinking about this from my own perspective, because often times where asked for a thing at the end and and often what is really needed is an outcome as a result of a thing or many things – something happens as an outcome and that’s an interesting juxtaposition.

Steve: And that’s even more of a challenge for those of us that work as vendors where deliverables in the procurement process is about you know articulating what the deliverables are so you can’t sell – you might be able to sell on outcomes but you can’t get through the vendor process on outcomes you have to say here is the list of deliverables when you know people are kind of gatekeeping new research projects based on deliverables not not outcomes. So it’s you sort of have to talk about both you know and I’m sure some of that is true for folks that work in-house as well.

Chi: So a few years back you wrote a book called Doorbells, Dangers and Dead Batteries about the highs and lows of research, and you’ve already told us a few war stories. Is there something that stands out in your mind that is a research trip that you did or a particular research gig that you did that’s really wild or something really interesting that you could share.

Steve: I don’t know if I have a story that’s wild that’s not in the book. You know, I mean it’s funny because we we try hard to avoid the wild ride. I think that’s. And yet it happens anyway I think that’s kind of the that’s the tension of doing the research. I don’t know if I had an interesting experience with a couple of years ago. I think I wrote about this briefly in the book but I can say a little bit more about it where we had recruited people that were running small businesses. And there was very very specific questions that they were asked about is this business your primary thing does you know all or most of your income come from is where he spends time. We’re very clear that we didn’t want somebody with you know, a part time at Etsy store, who works in some other kind of environment that we didn’t want to get to come as a small business or we wanted people that were legitimately living and running and doing small business. And so I end up in the home of a Silicon Valley executive – the guy was wearing his badge when he met me at the door from a Fortune 500 company and I sort of thought, oh he’s you know, that’s one of his customers I mean he’s an enterprise small business guy and he’s got a badge because he’s on site and he’s you know whatever is installing and configuring and that was kind of the assumption they had about why he had this badge. And the more I talked to him the more it became clear that his life was 85 percent about his full time job that he had as at an executive level and he just you know it starts to dawn on you like oh this is something is wrong here and it’s further compounded and I want to be careful what I say because you go to these environments you don’t know what’s going on but there was just there was like a tense family dynamic going on that I couldn’t understand. You know this was the home the whole family was there and so I am not able to say because I don’t know what the situation was. But you start getting some strange vibes and some comments are made and people are smiling and laughing but things seem uncomfortable. So there’s kind of that cloud hanging over it and then I start to feel kind of caught up in what is happening here. Did I make a mistake. Did the people that found him for me make a mistake. Did this person just flat out lie to somebody in order to get in this conversation which is crazy you know what would motivate someone to do that. There is not this is again an executive. There is no like 200 dollar incentive that’s going to change his life. You know that that would motivate him to spend two hours with me predicated on a lie. And so you sit there you know I think part of what’s wild is is that delta between is something wrong or something normal and I have to carry on something as normal because I can’t really ascertain it as an honest mistake or not. And it was just very very uncomfortable. You know I’ve done enough of these where I have certainly encountered people who have misrepresented themselves in order to be involved in the study and sometimes it’s just purely for money. You do everything you can to kind of prevent that from happening but it happens and it’s just it’s so unsettling. You know, when you talk about a wild thing that happened like there’s a story in the book about a guy (researcher) getting taken on a, you know, coke fuelled run to a Miami strip club (laughs) you know, in a sports car with you know, forced lap dances – like I don’t you know, that’s in the book and like that story is astonishing. You know I haven’t really had that experience this is much more of an internal experience where everything that’s happening on the surface is normal I just ran the interview like I would run the interview. But inside I am sort of churning about a thing that I can’t assess in the moment and I can’t resolve. And so that’s sort of like in a lot of internal chaos. But it’s not quite as dramatic as you know the sports car in the strip club story but then you know I like I’ll follow up with my recruiter afterwards and said Well you know what’s going on because here’s what I saw and they were so adamant that they asked him these questions and he gave them these answers and it’s it’s not resolvable or it’s not like oh it must have been for this reason. There is no explanation. This was a successful individual. Again an executive I’m sure he and I have multiple Linkedin connections in common like this. It’s someone that I’m in the same world with you know here in Silicon Valley and there is no explanation that I am able to get. There might be one but I don’t have access to get it. And so you know you left sort of with an odd feeling during and the inability to resolve it afterwards about you know how to feel what’s right. Was there a failure who to blame like all those things kind of go on the that’s again. No no cocaine (laughs) but in emotional turmoil I feel pressure to tell you like a cocaine story and I don’t have one I’m sorry.

Chi: Well I’ll tell you something weird that kind of counters that. So have you heard of AMSR?

Steve: Yes –

Chi: Autonomous Sensory Meridian Response, and if anybody doesn’t know what that is, check –

Steve: (Whispering) Maybe we should demonstrate it for everybody –

Chi: (Whispering) By whispering into the microphone – check out the documentary Brain Gasm. It’s basically the idea that you know when people whisper really softly it makes your brain get tingly. When I was a kid you know we had a thing called a telephone in our house and people would ring up to do surveys and I really liked getting called up for surveys. (Laughs) I kind of  – now I think that it was giving me a kind of ASMR feeling from doing it – I would get a little bit of a tingly brain sensation from people asking me questions. So I do know people like talking about themselves often. I’m not saying that’s the reason that happened in your story – but that’s my own story about that.

Steve: Yeah I mean I think people like talking about themselves as is the truth that this whole enterprise rests upon. But I hadn’t thought about it like ASMR – kind of response. That’s interesting.

Chi: You know maybe maybe some people just like being interviewed. I mean I’ve – I’ve definitely had a few experiences myself where similar things happen. I did go in interview one person once who was it openly stated to us from the moment that we got there that they were basically a national research participant because they got paid to do it and that was a way that they could make extra cash. And I guess that’s something that you have to be careful of when you are getting recruiters to find you people because you know, you might not necessarily get the response that you need from someone who’s recruited in that way.

Steve: Yeah, yeah. I mean I think I’ve definitely had experiences with ‘professional respondents’ as they call them who are clearly lying and it’s it’s it’s just confounding. So I think the screening question sometimes uses to ask people how often they participated or when they last participated – but if they are lying as part of the study they’re not going to answer that question honestly so.

Chi: (laughs) So it’s or does it’s all just skewed from the start.

Steve: It does assume you know, honesty and I think the good thing is I think we’re rarely fooled by those people becomes clear. The more you do this anyway you start to understand sort of who’s kind of presenting to you – you know when you ask to see a device like it’s oh that’s in my other house or you know my stepdaughter took it away you start to realize oh yeah, are those of cues of someone who’s who’s making it up. I mean you know and worse sort of fixating on that aspect of it that is it’s the rarity. I think part of the fun of telling the story is not to say all these terrible things happened but just to say interesting weird surprising funny sad upsetting stories happen as part of it it’s just part of the whole – the whole mix. And you know, if you spend your career with a very few of these kinds of things, good. But it’s a different way of talking about the reality of people and that sort of reminds us that just like deliverables vs. outcomes you know I think participants vs. real messy humans is the same kind of split. And I think you know that’s why I wrote the book why I wanted to tell other people’s stories the book is many many other people’s stories and a few of my own anecdotes of just trying to paint a more humanized picture of what research is like.

Chi: I think it’s interesting because there’s the stories that people tell you when you’re doing research and then there’s the story of you doing the research. And in my mind I have this one particular research trip that I did in the Philippines. And the thing that stands out to me I mean there was fantastic things that came out of the research that we were doing in there was a big team of us and it was amazing and we were working with people in call centers and it was really really interesting. But the thing that kind of sticks out in my mind is actually some of the things that happened to our team during the research. So you know, we had things – like missing our planes, getting caught in a typhoon. We didn’t have a car. We should have hired a car or we should have had a driver because we had no idea what the traffic was like in Manila. One of the team members went to catch their flight back to Australia and the taxi driver kind of kidnapped them because they didn’t have cash – all the things that you don’t think about when you’re going to do research – they happened to us on that on that particular research trip. And so you know, we got great insights and the client was just amazed but at the same time we sort of went through this wild adventure of our own on the back side of what was actually happening on the trip.

Steve: And I think that’s you know, there’s a sort of terrible and wonderful in there may be funny in hindsight or they just they feel differently in hindsight. And you know there’s tactical lessons to be learned from that. Like you said “next time we go to you know this kind of environment we’re going to hire a driver,” so, I think at one level the stories are just here’s things that we should remember to do better at it or for yourself or for somebody that hears your story right somebody hear your story right now and thinking Oh yeah I should get a driver. And I think at that practical level that’s a great outcome. And then there’s this sort of a softer sort of humanizing I have like we said it humanizes researchers right researchers make mistakes they’re people them they are. And so you can’t hear your story and it helps to have empathy for you and hopefully modeling having empathy for themselves the next time they don’t bring enough batteries or forget to turn on the recorder or meet someone who’s drunk or end up you know having to help someone you know use their walker to get to the taxi or whatever kind of thing end up having to do that sort of challenges what they thought their work was about. You know hopefully telling these kinds of stories has sort of the two levels of outcomes for us. It sort of softens and humanizes us. Our relationships with other people and gives us empathy for the mistakes or that to call them mistakes or just sort of the realities of those kinds of situations. And prepares us for the fact that we can’t prepare for everything. That’s also the other lesson I take away is that yeah you can remember not to hire a driver but there’s still another thing that you didn’t know that’s going to happen and so how again to deal with that. And you know I would love to always have the grace that I’m sure you and your team had you know in their sort of ideal sense as these things were going on you know to cope with that. That’s just life and research kind of exposes you to different aspects of it that maybe you wouldn’t otherwise get to sample.

Chi: So some years ago you did a talk in Hong Kong called “You’ve done all the research, now what?” What do you do after you’ve done the research? How what do you do? How do you gather all that information and make it into something bigger?

Steve: Yeah I think about it as sort of two levels: I guess I’m going to make everything into two levels that will be my framework for every question you ask me (laughs). There’s two aspects to the research: there’s the – it actually builds on I think what you’re saying about your team – there is the experience you have doing the research and then there’s the data that you gathered so the experience is it’s in the conversations you have in the car afterwards it’s in the text you send back to the office to tell somebody else what happened it’s it’s all sort of reflective it’s the shower thoughts the dogwalker reflections and so what you’re thinking about as you are kind of immersed in all this stuff and it’s very rich and powerful but it’s super biased as well. We remember things disproportionately and we forget other things and we magnify and we distorts. And so what I see teams doing is kind of not spending the time with their data. They go do some research and they are just caught like debrief you kind of sit down and you talk about what happened then you whiteboard it and use your stickies and whatever kind of sexy tool that you’re using right now and you kind of talk about what happened – then then that’s it. Like that’s the output and I think that’s you know it’s an iceberg model – the deeper you go the more mass and volume there is. And what I am a big fan of even though this takes more time – I mean it’s substantially more time – is to go back and listen to or read transcripts or watch or revisit all of the data the the recordings not your memories of the recordings and so then you’re doing a more intentional kind of analytical process. I think you – I think about analysis and synthesis I had used these words forever and until finally I looked them up. Analysis is taking a big thing and breaking it down into smaller things to kind of make sense of it. So you have a two hour interview and you pull out you know 12 kind of quotes or sort of moments in the interview that were really important that’s analysis synthesis is taking small pieces and building them up into something new. So you have to do some analysis or you have to break all these larger elements down. And so when you are by though when you are like texting somebody or debriefing or reflecting in the shower all those things are or analysis you’re polling small parts out of this sort of huge monolithic element of the, the interview of the data so that you start practicing that right away but then going back to your actual data and seeing what actually was said because it’s always different than what you think that you heard what you think you heard is really interesting and important it’s a good starting point for this process but I really want people to go back to what the data actually said and allow themselves to be surprise. “Well that’s a little different!” “Oh there’s something else here that we kind of ignored!” and do that by breaking it down into quotes and anecdotes and then building a back up and to patterns and themes and frameworks and models and you know those lovely two-by-twos that we love to create that start to show what relationships are and where gaps are. You’ve got to do the breaking down the building up and you’ve got to do it with the data not what you’re reflections you’re kind of debriefs are and that that’s an investment of time and I think you have to decide does the research question or investigating merit at that level of scrutiny maybe it’s something pretty closed ended if you’re trying to figure out you know what are the breakdowns in this process for people you might be able to sort of observe that you’re trying to understand you know how are people tracking their careers and what tools are they using to to support them along that way. And what are the opportunities there then yeah you need to go back to your data really see what was being said. What does it really mean and what are the patterns and what are we kind of what can we build from that.

Chi: There are often times I think that there’s so much pressure to certainly with analysis and synthesis to do it really quickly. Someone recently said to me you know can’t we sort of make something shorter in terms of the research and it’s it’s like well it’s really it really depends on what you’re willing to sacrifice. Right.

Steve: Yeah.

Chi: Because you know if you spend less time put pressure on people to do research in a shorter period of time or analysis in a shorter period of time or synthesis in a shorter period of time, what you’re really saying is, well, you know we we’re not putting as much importance on these things as we want you to think we are. You know it’s it seems a shame to squeeze those things. If you could get a better result by just spending a little bit more time.

Steve: I think that might be a nice answer to your earlier questions for me which was about how do you advocate for you know doing research. Well – I’m paraphrasing you – but being able to push back with the trade offs, as I think you just described, I think is really important. So you know again it’s different to say well we need, you know, eight days you know with all these people to kind of get everything out of the research that’s different than saying you know if we work through this in two days here’s what we will produce for the team if we work through it in eight days here’s what we will produce with the team so let’s just all agree that we’re comfortable with those tradeoffs.

Chi: So that’s really sage advice. I think the way that you just described that – I think that’s something that most people who – anybody who’s putting a proposal together for research could use immediately that rather than looking at at how you propose research as “I will have eight days to do that try to explain. If we do it for two days this is what’ll happen if we do it for three days this is what will happen. That’s very, very valuable.

Steve: It’s back to the outcomes versus deliverables or the outcome is “we’re going to learn this” and that’s not an us vs. them thing it’s like let’s agree together on what we want to invest in and what we need to have.

Chi: One of the most common questions that I’ve certainly been asked and I hear people ask is how much research do you do. And I always say it’s a bit like how long is a piece of string. You know, but thinking about it in that way rather than worrying about how many people do we need to go and talk to you know it’s more about well the more that you can do it the more information that you’re going to gather.

Steve: Yeah, Gregg Bernstein who works at Vox has been doing the title may of change but I think he’d be doing talks online somewhere in video or slideshare forum called “Be more certain” and you know he’s sort of describes the whole gamut of you know what available timelines are where he’s asked to provide value or give guidance or. And so this idea of you know certainty kind of on a continuum and you know not to and probably tell the story right but he had some some situation where something came up on the slack channel about like had a name some feature and he like put a slack poll out and got a handful of responses. So it was just a very quick exercise – but it gave him enough insights about a thing that was outside his own experience that he could turn around and make you know make a recommendation. That’s not a research project that’s going in anyone’s portfolio. It sort of fails on all the measures of like good research or whatever. But it was exactly the right way that he could support people in making a decision. And there again he is oriented towards the outcomes that you know that his colleagues have and that he’s being asked to support. And you, know willing to clearly he never pushes back but that he’s able to identify this sort of small moments as well as large movements. This is a nice way of thinking about it I think.

Chi: This has been an awesome conversation and I’m sure that we could keep going on and on and on. Before we go I’d like to ask you two more things. So – what do you think of all of the things that you’ve learnt? What do you think’s the most important thing that you’ve learned about research?

Steve: I mean the most important thing I learned about research is about myself. Which I don’t mean to be any sort of an egomaniacal framing but you know research is a person-to-person activity and so every time I go talk to somebody I come in with my own experiences and my own biases about being in the world as I have been in the world and my own expectations about what I’m going to see based on you know how they’ve been sampled and what the research project is about you know and so to do that well means being able to hear my own judgment. I don’t mean I can’t be anything but the product of my own experiences but I can be more open to having them challenged or dismantled or reflected back in a different way. That’s actually one of my favorite things about research is that feeling you get when you kind of get this ‘ohhhhh’ like that an assumption you have made and I don’t mean a profound assumption about the value of just parties bringing. I mean it could be about anything. You know this person that you’re meeting and what their what their life is like and how they are talking about things you know feeling that sort of thing crumble is just like it’s just so rewarding that I feel like I am learning about the world and learning about myself because I have dismantled a presumption that I didn’t know that I had. So that keeps happening. It’s not finalized in that regard – I keep discovering my own biases prejudices assumptions and there are many reasons why I want to why I love doing this work but just that one is kind of a side benefit as just so it feels like I’m always growing as a person.

Chi: Well – thank you for being on the show and for the little brain tingle that you’ve just given us.

Steve: All right thank you. Great conversation.

Chi: So there you have it. We hope you enjoy this episode with Steve and would love to get your thoughts and feedback on the topic to join the conversation. Go to thisishcd.com and register to join our slack channel where you can get in touch. We use our channel to shape future episodes and share design related content everyday. It’s also a great way to meet designers from around the globe. Thanks for listening. See you next time.

Resources

• Dollars to Donuts Podcast
• Doorbells, Dangers and Dead Batteries
• Interviewing Users
• Portigal Consulting
• Rosenfeld Media
• Steve Portigal on LinkedIn
• Gregg Bernstein – Be More Certain