Episode highlights
- 00:02:57 – Workshop and talk at uxcon vienna
- 00:04:32 – Measuring UX and design impact
- 00:17:30 – Risk mitigation through UX research
- 00:23:08 – Accessibility in design
- 00:29:32 – Conclusion
About our guest Vitaly Friedman
Vitaly Friedman is a UX leader who has been working with mid-size and large organizations for over 19 years, improving UX of complex systems, accessibility, performance, and design. He has collaborated with the European Parliament, OTTO, Zalando, Axel Springer SE, and REWE Digital. As the founder and editor-in-chief of Smashing Magazine, he organizes inclusive “Smashing Conferences” and runs live “UX training”. Vitaly thrives in solving complex UX challenges and working with stakeholders. His curiosity extends from interface design to front-end, performance optimization, and accessibility, all with the goal of creating digital experiences that people can easily understand and enjoy.
When I speak about design, when I speak about research, I speak about risk management. Because to me, design and research are fantastic tools to mitigate risk, not in terms of compliance and legal obligations and stuff like that, but spending and wasting an incredible amount of resources building the wrong thing that is never going to fly.
About uxcon vienna
uxcon vienna is a conference dedicated to UX Research and UX Design. It brings together professionals, experts, and enthusiasts in the field of UX to share knowledge, insights, and best practices. The conference is attended by both specialists from both Europe and the US, thus providing a great oppotunity for networking or professional exchange. Attendees can expect to learn about the latest trends and developments in UX research and design, gain practical skills, and connect with like-minded individuals. uxcon vienna aims to inspire and empower UX professionals to create impactful user experiences.
Podcast transcript
[00:00:00] Tina Ličková:
This is the 40th episode of UXR Geeks and the last one of our special episodes for uxcon vienna 2024. The conference takes place in September at Expedithalle in Vienna, Austria, featuring speakers like Vitaly Friedman, Nikki Anderson, Anfisa Bogomolova or Steve Portigal and Indi Young.
I am speaking to Vitaly, a UX designer, speaker, and co-founder of Smashing Magazine, and a bit of a superstar. Vitaly will be hosting a full day workshop on “Designing for complex UIs” on September the 18th, which will be the warm up day of uxcon vienna. And he will be also leading a talk on “How to measure UX and design impact” on the 19th September.
In our discussion, we touched a little bit on the aspect of UX quality, choosing the right design metrics, KPIs, and stuff like that. So tune in for a glimpse of what to expect from Vitaly ‘s workshop and his talk.
Vitaly, hi. You are pretty known from Smashing Magazine that when we were talking, you were also mentioning your client’s work. So now I’m really interested, like, what do you do every day if you are not talking at conferences, writing articles?
[00:01:29] Vitaly Friedman: Oh, thank you so much for having me, Tina. First of all, I think I wish I’d known what I’m doing with my life.
Tina, I have no idea. I’ve never known it and I don’t think it’s, it’s going to change anytime soon. I think I like one thing that has been bothering me for a while. Like maybe it was like, well, I think like eight, nine, 10 years ago, I felt like I’m writing. I’m just writing. I’m just a writer. And that’s not fair because I wanted to keep my hands dirty all the way in, in the actual work.
And so we actually do a lot of consultancy work these days. I spent quite a bit of time between client work, client projects where we try to solve different problems, actually very different problems. Sometimes they’re more on the digital nature, sometimes their physical nature. Right now, one of the bigger projects is the big revamp of the European Parliament websites, which of course is a massive information architecture issue.
There is a lot of research that has to be done, but also a lot of Other things have had to be established, like, for example, design KPIs, UX metrics, things like that. But it’s very much evidence driven design. I think most of my time these days is actually, I guess I wouldn’t say designing pixels on the screen, but designing everything but pixels on the screen to make sure that when we design pixels, they’re the right ones.
[00:02:41] Tina Ličková: Um, nice, nice framing. You are going to be at the uxcon vienna in September. You are going to have a workshop and you are going to have a talk. So what is the workshop about? And what is the talk about?
[00:02:57] Vitaly Friedman: Yes, that’s also a good point. If I look back, I was always preaching for those wonderful, beautiful things.
Let’s design the perfect carousel, perfect accordion, perfect piano. table, perfect this and that. And then I got into this world of enterprise where whitespace doesn’t exist, at least not to the same degree it exists in other, uh, environments. And so I actually ended up in a quite complex, complicated and complex, uh, systems where everything was different.
And so many things that we learned from that brought those journeys and basically try to pull together in some sort of a workshop and this is what the workshop is going to be about designing for complex UIs. So if dear friends listening to this at any point in time, if you ever got confused by nested filters, really complex flows, integrations, third parties, permissions, roles, and everything of that combined pulled together in one single place on and on top of that data grid and data tables, you’ll be mesmerized by some of the confusing bits and hopefully useful design patterns that will emerge in September.
The talk, however, will be very different. The talk will be very much about design metrics, UX metrics, design KPIs, and how to establish them. And from the perspective of literally how do we measure UX to the perspective of how do we actually establish governance for those KPIs as well. So very holistic with practical tips of how to actually establish metrics or KPIs in your own company organization.
[00:04:25] Tina Ličková: If you were to summarize in a few sentences how to measure UX.
[00:04:32] Vitaly Friedman: I think the most important part for me is to really understand the landscape. In the past, I always had this idea that if I just jump in any project, just give me a bit of time, and I’ll learn any domain without any issues at all, and then I’ll be able to do anything.
But then when you end up working in those kinds of corporations, corporate and enterprise environments, you realize that when you walk in, in such a big organization, let’s say European Parliament or so, like many others, you really have to be very careful how you actually talk about design. It cannot be just a matter of opinion.
It cannot be a matter of taste. It cannot be a matter of decision that I’m making because who am I against this? monolithic, huge mammoth ahead of me, right? So I have to be arguing very carefully and strategically about what design represents. And to me, design represents impact. That means when I walk in, I try to make an impact with the design work that we’re producing.
And so that means understanding environments and specifically understanding what people actually do in that environment. What are the tasks that they actually do on a regular basis? The first thing we often do is just. try to figure out the least of very representative tasks that people who are coming to this product to that service, uh, regularly or irregularly, irregularly.
Oh, it’s a difficult word to pronounce. Um, what do they actually do or what should they be successful at? And if they are successful, just how much time do they actually need to be successful with it? And then the point is let’s optimize the timing and let’s optimize the success rate, given that the industry standard is 80 percent success rate.
Most digital products I’ve been experiencing. are somewhere below 50%, maybe between 40 and 50. And that’s a massive opportunity that is often missed. And of course, depending on the nature of the project, sometimes it would be specific, let’s say, if we’re working on search, we’re going to measure search quality.
If we’re dealing with onboarding, we’re going to measure onboarding quality and kind of define scores. And very often this would be. Relatively quickly, it was really complicated because we’re establishing sort of design KPI trees very similarly to what marketing teams often do. Opportunity trees, KPI trees, things like that to really indicate and explain design intent and why we’re doing that design thing in the first place.
Because I don’t know about you, Tina, but very often designers come to me and they feel like they struggle to explain the impact of the design work. And so that’s what we try to basically do.
[00:07:02] Tina Ličková: Yeah, and that happens also with researchers. As you were mentioning, it becomes complex and sometimes some metrics, which should be just measuring something, people start to work towards them and to reach better metrics and better KPIs.
And sometimes what happens is that some team has a KPIs, which is in complete opposite to a team of other KPI. And I’m wondering what is your take on how to actually set the metrics landscape holistically?
[00:07:34] Vitaly Friedman: Oh, you are hitting the crux of the issue and the challenge there because sometimes you see very different views of how this KPI tree should be built.
Sometimes there are trees, but then that means that one branch might impact the other and you cannot really represent it easily. And so very often what teams do is they create this between basically overlaps or connection lines between those different KPIs. And to me, this is very sophisticated, but I also find it quite challenging to feel comfortable with those kinds of visualizations.
So what we try to do is to establish two sets of KPIs. On the one hand, if we have, let’s say, a product that has to do something, some things need to work really well. So we’re going to track how well people perceive, people actually do those tasks that they need to do. But on the other hand, if we’re, let’s say, working on a specific feature, we’re going to have local KPIs that define the impact of that particular thing that we’re working on.
So if we want to improve our navigation on level two in our, in this specific product, then we’re basically going to track how good that navigation works. So this isolates basically the scope of what you’re working on, partially. But then, of course, you might have, Some impact on some other parts of the journey or some other parts of the product, because what we’re working on here, people might be jumping from your navigation path to an entirely different set of products and all the way it creates this impact.
And so. It’s very difficult. I mean, personally, I would say that despite the fact that we can actually create these connections, it’s not about being able to point out and say, okay, we did this and this had this impact at scale and we are responsible, the sole people who are responsible for that, we see it as a collaborative effort, which means if there is a team here and there is a team over there and they’re driven to the same goal, as 80 percent success rate, that’s fine.
However, if you do have teams that have contradicting goals, then it becomes indeed a challenge. And this must be resolved together with the PM. PMs from dedicated, from those teams. That is often indeed challenging because it’s very hard to see the flow of your impact as you’re deciding. That’s really something that’s been on my mind a lot.
And I don’t think I have an answer to that question, at least not yet. So you are really good at this. It’s getting right to the crux of it.
[00:09:51] Tina Ličková: I’m trying to build a base here and I’m always thinking about our audience researchers and measuring impact in UX is a big topic, especially with all the layoffs happening right now.
So I’m wondering how researchers can contribute to measuring impact and measure it holistically as showing that their work has worth.
[00:10:18] Vitaly Friedman: I think the most critical part is, for me, probably the most impactful thing that I have learned to do is to really frame my work as an enabler of business goals. So everything we do, we’re basically enabling business goals.
On the other hand, of course, we also want to improve. the overall experience, disability, accessibility. But to me, those things are the essence of helping businesses reach their business goals. So when I walk into a big meeting where we have, let’s say, business discussions or whatever is happening, I never try to argue about the importance of white space.
I never tried to argue about the significance of, I don’t know, having 20 or 30 people to or reasonable testing. In fact, I make it almost like it’s just a way of working. This is just what we do. I never asked for permission to do those things. Instead, what to me, the most important part is to see we do our work and then we contribute to business to help business actually reach its goals.
But for that, we cannot actually make assumptions based on other things there. So we need to understand our customers better. And we also need to make sure that people don’t get locked in any way or frustrated because people don’t buy when they’re frustrated. People don’t complete tasks when they’re annoyed.
People abandon when they are really when the experience is disruptive. So we need to move all the things away. But for that, we need to understand what we need to put away. And so every single discussion that I’m having is really with the business people, right? It’s always framed from the perspective of we do this and we make this happen and we work on this, ultimately with the goal to clean up everything so we can actually reach those business goals.
And sometimes it’s very difficult to advocate for it, highlight it and explain, but there are always two kinds of frames of references that I’m using. One of them is by doing good research, by doing good design, by taking care of visibility, accessibility, and we reduce costs. That’s something I’ll bring up because this is a very easy to tell that story is very easy to tell because of course, fewer people are going to complain to customer service, fewer people are going to cancel the subscription if they’re satisfied with the experience that they’re getting, they can complete the task and so on and so forth, right?
So that’s one story. On the other hand, you can also try to tie it in with something that business cares about any kind of business KPIs that they have. From conversion, to revenue, to growing new business, to growing existing business, anything of that kind. And I think that this is something that’s really underrated among designers and researchers.
When I speak about design, when I speak about research, I speak about risk management. Because to me, design and research are fantastic tools to mitigate risk, not in terms of compliance and legal obligations and stuff like that, but spending and wasting, I should say, an incredible amount of resources building the wrong thing that is never going to fly.
And I think when I bring in this notion of risk or risk mitigation into the meeting, all of a sudden the design work that we’re doing is seen a bit differently. I mean, you still need to tell a story. In the end, Even those KPI trees that we’re going to build and I’m going to show also in the workshop, right?
This is just a storytelling tool. So we are working on this with these features, these improvements with our teams in those sprints with our researchers to find out more about where users fail and why in order to do that and then to do that and to do that. And as you keep building up, These features and these reasons why we’re actually doing it, drive and try to connect the design initiatives we do at the bottom, or research initiatives we do at the bottom, with those business goals on the top.
This is why I call it the tree. So traverse this tree from bottom, where design and research activities are, to the top. Just to give you an example, it might sound a bit abstract. So if you’re, let’s say, selling washing machines. And you may have 100 types of different washing machines. Sure, people come to your website and they look at the list of washing machines and they might find it challenging to find the right one.
So you might want to work on something like guides. You might want to look at something like configurators. You might want to have something like helpers, assistants, guides. filters, anything, any of those things could work, right? But if, for example, that business complains that people have challenges and they maybe don’t buy as much as it would like to do, would like them to buy, maybe they want to increase their revenue a little bit, right?
Then comes the question, what can designers and researchers do to make that happen, right? So we can start thinking about, okay, why do people not buy? The checkout might be impeccable. So the problem is not the checkout. It’s maybe a decision making problem that people have. That would definitely not come from business because business doesn’t know.
This will come from research. This will maybe also come from designers. And then we say, all right, so how can we help people get to the decision faster? But it’s not a random decision. We need a best fit decision for them. Ultimately, it’s a washing machine that they need to be able to fit in the room, not just a random one.
It’s not like people buy five washing machines and then say, Oh, I’m going to send four back if they don’t fit. That’s not a use case where I need to find the right washing machine. faster. Hold on. But then to find the right washing machine, they need to have at least a couple of candidates from this list of hundred washing machines, right?
So that’s a finding, filtering, sorting kind of experience. And so you visualize this entire intent of what you’re doing from the top saying, okay, we need to get people, to a selection, a relevant selection of washing machines faster. And then from that selection, we need to drive them to the washing machine faster.
And these are the things that we can do to make it happen. And then if we actually get people to the right washing machine faster, this is a hunch that we have that people will also buy in the end. Or at least they will be able to make a decision to buy or not to buy without being stuck because people don’t buy when they’re paralyzed in the choice.
So that’s this whole procedure, which is very much just I don’t know, rational, just a reasonable thinking and a bit of research and a bit of design thinking there. And most importantly, this sort of an attempt to connect the design initiatives we do at the bottom with the business goals and interpret, interpreting those business goals correctly.
And so just one final note on that, I think, what we often as researchers and designers forget is to translate business goals into something that’s really close to us, but, and that’s important, but also verified or signed off by the business. So when a business comes to me and tells me we have a problem here, I literally sit down and I try to translate it into something that I as a designer or my colleagues as researchers understand and can deal with.
And then I go back to the business, present how we understood it, and then get a sign off on that from them. So it doesn’t feel like designers and researchers do something else over there. It must be almost like a commitment or agreement, I guess, that we are working together on the same thing, if it makes sense.
[00:17:30] Tina Ličková: Going back to what you said, because I’m listening to you, but it still pops out in my mind, the risk mitigation. We talk about using the business language, but really we can’t tell the people in the meeting room like, Oh, research is going to save your ass. But if you say. Risk mitigation. It’s exactly the business language that we should be talking.
[00:17:54] Vitaly Friedman: So I think that very often when we speak about research and design, we often speak about us having to prove our value and show the, and demonstrate the value that we’re producing, I think in many ways. To me, at least it’s a little bit different because what I ultimately try to do is to just gain a little bit of trust, just a little bit of trust to gain more trust over time.
It’s not about having this big report and placing it on a table and saying, here, see how much we are of value to your company. No, I think it’s actually much, a slightly more impactful way of doing that would be to say, or maybe we can’t speak to our users yet, but maybe I could, I don’t know, just shadow them.
Or maybe we could basically just get access to people who speak to OU’s, maybe sales, maybe customer success, and just have a bit of research done there. Because ultimately you want to eliminate the kind of situations where people get stuck. Nobody wants to get stuck. People don’t buy when they get stuck.
They don’t stay customers if they get stuck. They don’t buy or pay if they are frustrated. So we just want to increase revenue by eliminating those things. That’s great for everyone. That’s great for business. That’s great for accessibility. That’s great for usability. I guess for everything. It’s great for users, of course, in the end as well.
So I tried to say, can we just have a little pilot where I could maybe speak with one customer for 45 minutes and just see You know, talk to them how they actually do those things, how they work. Just guide me through the interface. That’s the only question I will ask. Please, could you please guide me through this interface, how you’re using this product?
That’s the only question I will ask. And then you do that, and then before you know it, you’re going to ask for a bit more, and then for a little bit more, a little bit more. And if you actually do demonstrate, like maybe one quick win or one problem to solve, that it will be solved based on your input, right?
Then all of a sudden you’ll be given more. Because I think in many ways, business sometimes just doesn’t know how much impact even small things in a particular journey can have. It can be remarkable. It can, but we shouldn’t be expecting that they know it right away. Frankly, as a designer. I don’t know either how business works in all the fine details, all the fine business decisions that need to be made, the finances, the limitations, the procurement, all those things.
I have no idea about them. So we shouldn’t be expecting business people to know exactly what we’re doing and how we’re doing it and what the impact of that can be, which is something that needs to be discussed.
[00:20:21] Tina Ličková: This is something that I’m trying to mentor some youngsters when they approach me like, Oh, they want, don’t want to do research.
And I came up with this beautiful research design with 25 people. And what you are saying is just getting the foot in the door. I’m always remembering one of my first user testing as a paid UX person was with the bank. They came in, they saw the testing. They were actually really good at the time in doing online banking and they didn’t call me for two weeks.
And after two weeks, I was trying to reach them, sending emails. And I was like, Oh, I did something really wrong. And now I can’t prove myself to the company where I was working for. And they called me like, sorry, Tina, we just needed time. We needed to breeder, but we had a big depression.
And it was just really small testing with, I don’t know, five or six people. So just, you don’t have to build this big study and do it properly. Okay. Go with the startup-ish attitude, go with the lean approach and show that we need to speak to people and step by step show them how, what to get out of it.
[00:21:32] Vitaly Friedman: Absolutely. I think that one, one really significant part that we shouldn’t be forgetting is that we need to produce impact, but I don’t pick about it, like again, proving our value. One, there is nothing more impactful than cutting five user interviews and creating video clips and then telling a story to people about how people, actual people who happen to be paying for a particular product or service, or even if they’re public citizens, right?
If it’s a public service, then they are struggling. If it’s one person, they can always dismiss it like, Oh, okay. This is outlier, whatever. Right. If it’s two people that started getting interesting, if it’s three people, it will be very difficult to dismiss. And if it’s almost every single person after five people who are representative for the customer base that you are.
operating with, or operating for, working for, all of a sudden they start thinking. Even if it’s just a little seed that you’re going to plant at this point, you might be surprised that they will be getting, or will be giving you rather, a little bit of freedom to explore your next steps, right? However, once you actually define or find some problems, one important significant part is to also identify some solutions.
And once you get both of them, Done and solutions are in place. Yeah. You might be expecting a few more requests coming your way all of a sudden, a few knocks on the door, which is exactly what you’re looking for. Right.
[00:22:55] Tina Ličková: You are a strong advocate for accessibility. And here I would like to hear directly from you, how do you get the buy-in for accessibility from the customers, from your clients?
[00:23:08] Vitaly Friedman: Yeah. So I think most of the time, I never speak about accessibility as something I need to get a buy in for. So instead I do it differently.
I sprinkle accessibility, little accessibility details, the usual suspects, the contrast ratios and the font size and things like that. Just, it’s all just sprinkled across design teams. This is what I do. So I never go into a big meeting and say, “Oh, we need to do accessibility work”, because very often this will, kind of from the business perspective, it’s really the same story as with research.
I think, I think in many ways, when we’re speaking about inclusive design, business just doesn’t understand what we mean by that, actually. Aren’t we all inclusive anyway? And also, what do you mean? What do you mean by that? One of the weirdest things that ever happened in my life, when I realized that I actually should be really careful when I’m speaking the UX language in a non UX environment, is when I was speaking about pain points.
by presenting pain points that customers have. And then somebody put me aside during the break and said, you spoke just now about pain points. Do you mean like medication? And I was like, okay, so maybe I should be really a bit more careful when I’m speaking like this. So I try to avoid any sort of UX words that might be misunderstood or not understood at all.
And instead, I sprinkle a little bit of accessibility all over the place. Now, for design, it’s relatively straightforward, right? Because in the end, designers, most of the time, also want to create accessible products. When things get implemented, and then there are some frameworks in use, which are challenging in terms of accessibility, what I do is instead of saying we need to do accessibility, I bring as early as I can, as early as I can, I bring people with diverse needs into testing.
Just as early as I can. And I really visualize that impact that poorly implemented or poorly designed software can have. Right, because I think, again, to me, this is very, it’s very impactful. Like it really is very impactful to actually see people struggling. Because no product manager wants to be seen as somebody who is avoiding or ignoring the customers.
And because we make it so public, it’s just very difficult to dismiss it. And of course, sometimes when you have, let’s say, a product that’s inaccessible and a company comes to you and they want to improve, let’s say, some, I don’t know, checkout or e-commerce flows or something like that. I always bring up the notion of what you can do in terms of accessibility as a quick win.
Again, usually those usual suspects, the color contrast and typography and so on, and also meet terms. And so I’m still able to do it. Like the things that are really broken relatively with a relative modest budget, but then. I also try to advocate just the sheer amount of people we could be addressing if we actually got it right.
So again, for me, it’s more about framing accessibility as well as a way to help businesses reach their goals. Because in my personal experience, it has never worked to just advocate for disability. Because once you do, Everybody’s going to nod, and everybody’s going to agree, and then nothing happens. Because the budgets are tight, and because, uh, there are challenges, and because we have other priorities, and so on.
But then, if you just embed, let’s say, accessibility as a way of working, even although you might be making small steps with your design team, and with your development team, that can make a difference. And, the other thing that I would highly encourage everyone to do, is to sprinkle a little bit of accessibility skills, knowledge onto the QA team.
The same thing we do with UX as well, just a little bit. So they are very much aware, at least about some of the challenges that a particular product might have. So this would definitely be a starting point for me. And then once again, you get into the motion where people Accessibility is becoming a word that people speak of and inclusive design is something that we do, right?
You can ask more. Not necessarily have to ask for it, just do more, just dedicate a few more, a bit more time for accessibility work, you might be blocked eventually. So that’s a side effect of that. But then I think also in terms of new accessibility acts that’s coming up next year, I think that we are in a position where almost no company can actually ignore accessibility, full stop.
They might. Say, okay, we, let’s do what we need to do and stop there. But again, if I bring in this notion of accessibility testing, regular accessibility testing, it would be a little bit difficult to get there. Sometimes I fail, but sometimes I win. And every win is literally a big win for me. So I’m happy about that.
[00:27:44] Tina Ličková: Vitaly, I really like your way of thinking and way of framing things. This is a really good angle. So I’m really looking forward to the warmup day. That’s the 18th of September in Vienna.
[00:27:56] Vitaly Friedman: Excellent.
[00:27:57] Tina Ličková: That’s where you have your workshops, then your talk on the 19th. We see you there and I very much recommend everybody to see you, to go and listen to your talk or go to the workshop because you are a kind of superstar, we know you like that, but you are a great guy.
[00:28:15] Vitaly Friedman: I don’t think I’m a superstar, I just really, you know what, to be honest, like, for those of you who are wondering, like, why is this weird guy posting on LinkedIn so often? Like, I mean, for me, this is all about, I don’t know, this is how I learn really. So this is the best way I learned to learn. So just in case you’re wondering, there’s no chat GPT involved in the production of those stories or whatever? No, basically I write it all. And the thing is, because I’ve been writing for the last, like what, 20 years, I’m really fast at writing and I have some ideas and some thoughts. So actually it’s not a problem for me. It takes maybe at most 25, 30 minutes to write something.
But then of course I read a lot because I really care. I really like seeing what other wonderful people are doing. And then when I see something, somebody shares, and I feel like, Oh my God, this is incredible. Everybody should know that. Then I try my best to make sure that everybody could know that. So that’s it.
There is no magic behind it, but I would love to see you as well in Vienna. I believe you will be there.
[00:29:11] Tina Ličková: I will be there. I will be moderating the smaller stage. So if anybody feels like I want to see the lady blubbering in this podcast, then..
[00:29:19] Vitaly Friedman: I do. I do.
[00:29:21] Tina Ličková: Yeah. Yeah. We will.
[00:29:22] Vitaly Friedman: But not blubbering. Definitely not blubbering.
[00:29:25] Tina Ličková: Thank you. Spasiba. It was great.
[00:29:27] Vitaly Friedman: Thank you so much, Tina. See you soon.
[00:29:32] Tina Ličková:
Thank you for listening to UX Research Geeks. If you liked this episode, don’t forget to share it with your friends, leave a review on your favorite podcast platform, and subscribe to stay updated when a new episode comes out.
💡 This podcast was brought to you by UXtweak, an all-in-one UX research tool.
