User testing is an integral part of tuning the user experience but if you are not careful it can also be the most expensive part. Just like user testing being an art and a science in itself so is knowing how and when to do it to save money.
Do you need user testing?
- Use research if possible. There is a reason there are so many universities churn out information on a regular basis. Do not reinvent the wheel. If you are trying to find out the answer to a question, do a search in ACM (http://www.acm.org/) and IEEE (http://ieeexplore.ieee.org/). The biggest group is CHI (http://www.sigchi.org/)
- Is this a question that your Marketing or Sales group already knows? These groups collect reams of information on on your company’s users and you should already be working on having a good working relationship with them anyway. If you are working on internal tools then Human Resources is the place to go. You would be amazed at the amount of information they know about employee demographics to know exactly how much they need to pay everyone.
- Integrate user testing with marketing or sales data gathering. These groups are most likely going to be collecting information on a regular basis so if it is just usage surveys then it can be pretty easy to tack it on. This is the reason why I was saying it is good to have a good working relationship with these groups.
- Whatever you choose to do make sure to do it early in the sprint or cycle so it can be put into use as part of the design.
Online user testing
This kind of testing will take extra time up front since you will need to think of all the contingency problems that might crop up and either more narrowly define the tasks or add extra pages to allow more exploring. The benefits are that large groups of people can be run through the system quickly 24 hours a day and the problem of “no shows” does not exist.
- Surveys are a great way of collecting information and like I referred to earlier can be integrated with the same channels that Sales or Marketing groups use to find participants. Just be wary to not use their questions. They are looking for the “Why”, you are looking for the “How”. An easy tool I use that has a lot of different surveys is Verify (http://verifyapp.com/).
- Automated testing tools are cropping up more and more. The two ways I’ve used the most is Solidify (http://www.solidifyapp.com/) and integrating google analytics into a wireframe.
In Person Testing
In person user testing is the expensive stuff. To make it worth the money be prepared for all the contingency plans.
- When scheduling appointments, have people arrive early and be given a survey to work on while you are finishing up the last person. That way if you have no shows you can go straight into testing. If you work it out right with scheduling people then you should be able to have a normal day if someone cancels and in the rare day that there is not a “no show” your day will be a little longer. Another way is to have employees on the ready to fill in. They might not be in the demographic and they are probably more familiar with the tool so they won’t get tripped up by badly named menus but it beats sitting around doing nothing.
- Prepare, prepare, prepare. If you are testing a something that connects to an server or cloud, download everything to make sure there is a local copy in case the network connection goes out or there is no wireless signal where the testing computer needs to be. If you have everything local make sure you have extra accounts ready to go, static screens when testing buggy software, and keep the paper prototypes around just in case everything else fails. Once everything is ready try it out on a few employees. They will help you find the areas you might not have thought of where people will ask questions or attempt to wander in the product.
- Don’t over-design your wireframes. This one isn’t as obvious since it dips into the psychology side and seems to contradict the other things I’ve said. Anyone who has shown a really polished wireframe has had this headache with user testing though. The users don’t want to insult the work since in their mind it is already done. So when you are presenting the wireframes users will just keep saying they are OK with everything even when they are confused. This is still a reason why paper prototypes work so well. If using something like Basalmiq (http://www.balsamiq.com/) or Axure (http://www.axure.com/) I leave usually leave it in black, white, and shades of grey and for Axure I crank up the Sketchiness effect.
- Record everything. Body language will tell you a lot that the person won’t. The problem is with all the things you are trying to do to keep the session running smoothly you can overlook it. The easiest way I’ve found is to use an application Silverback (http://silverbackapp.com/). It records the screen and uses the laptop cam to record the users face and voice as the test is being conducted.