The elements of profitable research
In last week’s letter, we discussed the three pillars of value-based design: research, measurement, and experimentation. Over the next few letters, we’ll go deep on each of these pillars to talk about why they’re essential to your design practice, and how to begin working with each. First up is the most profitable component of design: research.
Research is best separated into both qualitative & quantitative components. Qualitative research tells you what customers say, and quantitative research tells you what customers do.
Quantitative research methods include:
- Analytics
- Browser & device analysis
- Heat & scroll maps
- Behavior recordings
- Heuristic evaluation
Qualitative research methods include:
- User interviews
- Usability tests
- Surveys
You can do most of these for free. You can alleviate a lot of effort for under $500. Here’s everything you can do for free:
Research methods on a $0 budget
- Analytics: Google Analytics is free. This is such a powerful tool that you have no reason not to use it extensively in your optimization efforts. In fact, I guarantee you that you’re probably already gathering considerable information about your customers without realizing it. Here’s ConversionXL’s tutorial about fine-tuning your GA install. It tells you the exact steps you need to take to gather more demographic information about your customers – and begin configuring revenue- and conversion-focused goals.
- Browser & device analysis: This warrants its own category, because it really just amounts to bug-fixing. But you can take a look at the device & browser breakdowns on Google Analytics, weigh them against your average conversion rate, and make a strong case for fixing a lot of revenue-leaking problems. Everything from “the checkout page doesn’t work on Firefox” to “our mobile site is atrocious” can be vetted through GA for exactly zero dollars.
- Heuristic evaluation: You can do this yourself, at your desk, without anybody knowing about it. It’s also one of the most powerful research activities for stores that haven’t done it before! We’re going to talk about heuristic evaluation in a little bit, so stay tuned.
- Heat maps: There are numerous free, open-source heat map tools. Here’s one. If you’re a developer, do what you can to deploy it. If you’re not, take a developer out for coffee and make the case for it. State the case that it will be turned off when you get enough data. Show examples of heat maps justifying revenue-generating design decisions for other businesses. (You probably won’t get scroll maps for $0, but if you can find a free tool for them, I’ll link it in a future Revise Weekly lesson.) Too much work? Hotjar has a free plan.
- Surveys: Wufoo has a free plan. That’s the link to register for it.
- Recruitment for usability tests & customer interviews: use Wufoo’s free plan for them. Don’t forget to get the participant’s contact info (phone, email, and Skype/hangout/etc) at this step!
Research methods on a $500 budget
Why a $500 budget? That’s the petty cash limit for most company cards. You can probably get $500 requisitioned pretty easily – especially if you have your own corporate card. If not, talk to your boss during your weekly 1-on-1 (which you have, right?)
You can do a lot with $500. Mix & match these to taste:
- Recruitment for usability tests & customer interviews can get a huge upgrade by using Ethnio. $49.
- Usability tests & customer interviews need to provide some sort of compensation for the participant’s time. I generally give out $50 Amazon gift cards. (You can probably shrink this to $20 if you’re in a big crunch, but given the amount of time someone is spending, I’d make it as big as possible – especially if the participant has to commute to your office.) For 5 participants, this means $250.
- Want to save a lot of time on your usability testing? Go to UserTesting and buy 5 tests for $495.
Heat & scroll maps can become a heck of a lot more convenient by using Hotjar’s paid plans. $29.
No organizational support
It’s one thing to know how to do all this. It’s another to actually do it. Where should you start? This will probably vary from organization to organization, but:
- Quietly get your Google Analytics in order. Most organizations don’t even look at their GA install, much less optimize them. Knowing GA is a huge professional skill that every organization wants, few people have, and nobody wants to practice. Do everything you can to get administrative access to your Google Analytics account, and then configure goals on it. You may need to update your tracking snippet on your production servers. If so, provide comprehensive directions to any developers in charge.
- Run a heuristic evaluation on your site. You’ll be using GA, various browsers, and maybe even a dummy credit card. Talk with anyone who has the power to issue refunds – support, accounts receivable, etc – in case you need to run real-world transactions.
- Run a heat map. Install Hotjar’s tracking snippet on your site and get heat maps, scroll maps, and behavior recordings.
- Write up a comprehensive report about any conversion-killing issues and send it to those in charge. Make a strong case for fixing them. If most of your office is in one room, hold a lunchtime talk to discuss your research efforts.
The outcome of this should be twofold:
- Your organization should fix – and measure the economic impact of – any major revenue leaks.
- You should get slightly more support to conduct additional research. Hooray!
If you don’t get more support, continue making the case. Spread the word. Talk to your boss. Explain to other team members why this is important to the health of the business. Do it until you get firm commitment to increased research.
A little organizational support
If you do get such a commitment, you have a couple more things to do next:
- Run a customer survey. You’ll want to blast this out to your business’s mailing list, and post it as a callout on your business’s website.
- Get some usability tests. I wrote a whole lesson about usability testing on Revise Weekly last week. Go to UserTesting and order 5 tests.
These keep your budget under $500, but they provide high-value activities that begin really testing the waters for your business. Why? Because at this point, you’ve started actually listening to your customers – not merely observing their behavior. That’s hugely important towards ensuring that future A/B tests are revenue-generating.
Keep pressing for more research
Your mindset should change from day 1. While you’re doing this research on the down-low, in conversations that end up in meaningless internal wheel-spinning, ask: “Why don’t we figure out what our customers think about this?” And no, the answer isn’t “A/B test it.” It’s usually by asking them. Trust me: A/B tests are horrible interviewers.
Start with research that can be done simply, cheaply, and quickly. Then move to slightly more ambitious efforts. Your dreams of a 10-week ethnographic fieldwork project are laudable and currently unrealistic. And all the while, you should communicate the business value of research. You need to start small, get quick wins, and share widely with the organization as you nudge towards evermore ambitious research projects.
This is how companies get invested in research. Not through somebody like myself forcing them to go whole hog, but through incrementing small things, presenting their value, and progressing slowly.
This week, for paid members
- This week’s paid lesson expands on last week’s lesson explaining the difference between category & collection pages. What are some good examples of category pages, and what represents a solid baseline design for them?
- Our design of the week shows a post-purchase upsell (hooray!) done… wrong (less hooray).
Want in? Join us now – named one of the best ecommerce communities going on the web.
Already a member? Log in here and take a look at what’s new.
Links
- Looking to bring some clarity to your hypotheses? Conversion’s ALARM protocol looks like it’ll be tremendously valuable for understanding whether to finesse MDE, sample size, and other experimentation parameters.
- When someone talks about how to share work, it’s probably a little useful. When Jan Chipchase talks about how to share work, you listen carefully.
- This universally correct post about the emotional divestment that is presently happening in the collective. We are firmly a low-trust society, and ideas are no longer publicly sharable. What does this mean for our current state of apocalypse, and how will you move to address it?
- How to share bad news in your team.