➡️ How Stéphanie uses UXtweak in her work
Challenge & Research Goals
A UX Research & UX Design Consultant at Maltem, Stéphanie Walter was working on a client project involving migration from an old tool to a new one.
The team needed to gather baseline data and conduct research that would help mitigate risks associated with transitioning to a new design, as well as help reassure stakeholders that decommissioning the previous tool was going to be a smooth transition.
We had an interface where we were migrating from an old tool to a new one. The goal was to have a baseline - gather initial data on users’ satisfaction, to later compare it to the metrics for the new design.
Solution
The team wanted to use the UMUX-Lite survey to collect user feedback and baseline metrics.
Stéphanie explains, that with the help of the UXtweak Survey Tool they conducted 2 separate UMUX-Lite surveys – one to measure users’ satisfaction with the old version of the tool, and another after the new design was implemented.
UMUX-Lite is a very short, standardized survey. We did a survey on the old design to measure the baseline. And then we launched a new tool. We gave it a little bit of time so that people could get used to it. We didn’t want it to be biased because it's new and the users are still lost.
So we waited and did a second UMUX survey. We used 7-point Likert scales inside the survey tool for it.
Stéphanie also mentions that she added an open question at the end of the survey, in order to collect additional feedback and suggestions from the users of the tool.
Outcome
Stéphanie emphasizes that for this specific study she was interested in having the raw numbers from the surveys and analyzing them herself:
And then I got the results. I did all the analysis myself in an Excel sheet. Because the calculation here is very specific. I was more interested in having the raw numbers. I could easily export the data from UXtweak to Excel and do my own analysis.
Stéphanie and the team collected a good amount of feedback that they were able to draw data-backed conclusions from.
Stéphanie also highlights the effectiveness of combining qualitative and quantitative data for richer insights:
We collected tons of feedback. It took forever to analyze. But it was really insightful. I'm not a big fan of quantitative versus qualitative. So I always try to have both. In both these experiences, I had, quantitative data at the beginning. And a very open question at the end.
There were respondents who just wrote two lines answering the open question. And people who wrote a full paragraph telling us what they want improved in the tool.
The results of the comparison between the old design and the new one were not surprising, mentions Stéphanie.
The new version of the tool was definitely much better. And it was not surprising. The old one was dated and more complicated to use.
But still, it was nice to have data. The people were so used to the old tool, that we all were a bit afraid that the new version would backfire.
It could have been complicated to migrate to the new tool, when everyone is so used to the old one. But at the end, we had great results on the new one.
According to Stéphanie, having the data that proves that the new tool was performing better really helped the team make confident product decisions and complete the decommissioning process successfully.
The stakeholders were a bit afraid to decommission the old interface. Change is hard in enterprise UX. With the data of the survey we brought (and some change management), we were able to bring more confidence in the decommission.
We prepared a helpline, just in case, but, in the end, it was almost not used, and the transition was smooth.
➡️ How Stéphanie uses UXtweak for personal projects
Challenge & Research Goals
Apart from being a UX Research & UX Design Consultant at Maltem, Stéphanie Walter is also contributing to the UX community by running a social media blog, where she regularly shares useful UX content with her followers.
As a part of that activity, Stéphanie runs a weekly “Pixels of the Week” newsletter with a roundup of all the useful content about UX, design, accessibility & tech.
Stéphanie explains, that she was trying to explore different formats of the newsletter and understand which one readers liked more. That’s why she decided to run a preference test:
At the beginning, the newsletter was just a link to the article saying, hey, the weekly links are here. But I was wondering, if people prefer to have the whole content in the article or if they prefer to have just the link which they can click to open the article?
Because some people reached out to me saying the newsletter was nice, but they didn’t like that they had to go to the website.
So I decided to do a preference test on the newsletter.
Solution
Stéphanie started the test with a screener question and then presented the participants with the 3 newsletter design versions, asking them to choose the one they preferred:
I first asked if they were subscribed. That was my screener. I had three versions of the newsletter in the test.
The first one was just the link to the article. The second one was the whole newsletter. And the third one was a mix where I had a summary link if you want to open the roundup in the blog, and then the rest. And then I asked an open question at the end to understand their choices a bit better.
Stéphanie explains that she used an open question at the end as an opportunity to get more qualitative data on her subscribers:
I wanted to also find out what they were expecting from the newsletter. So basically, I used this preference test as an opportunity to ask an open question.
I was like, okay, since they took time to participate, I might as well ask if they want something else, if they want something new.
Outcome
Stéphanie shares that the results really helped her make a confident, data-driven decision on the right format for the newsletter. After the test, she made the changes and went for the design that the most followers preferred:
With the feedback I collected I was able to improve the newsletter quite a lot. The preference test helped me decide what was the best way to present content.
I ended up with a small summary at the top, a link to the article, and then the whole rest of the content at the bottom, for those who didn't want to be redirected to the website.
48% of participants preferred this design, 38% wanted just the link, and 14% preferred the whole newsletter without the link to the article.
She also emphasizes that the open question at the end helped her collect even more insights on how she can improve the newsletter:
There were a lot of interesting answers in the comment section at the end. People shared interesting things and gave advice on how to improve even more.
👉Why UXtweak?
Stéphanie shares that the ability to export raw data as she likes and analyze it herself is what really makes UXtweak stand out to her, compared to some other research tools:
Having flexibility to easily capture the data, but also export it is very important to me.
The ability to export in whatever format I want, like CSV, and then just do whatever I need with the data. Because I've had some tools where they do the analysis for you and they don’t show you the raw data. I have specific math to do with this data and I need to analyze it myself. I want raw data. With some tools you really struggle to export.
So I think that's very good that you can just export to CSV and put it in whatever tool you need. I use Excel, but I know some people might do high-level analysis with Python, looking for word occurrences and other staff.
