On most commercial websites, success is counted through increased traffic, increased conversion rates and increased revenue.
If you can associate design changes to one of these key metrics, you can understand the return on investment for UX efforts.
Even if you can’t associate changes directly to conversions or revenue, metrics like the Net Promoter Score can be used as a proxy for revenue. You can then estimate a return on investment by taking the realized gains divided by the cost of the research.
But when the purpose of your website is not to increase traffic or convert browsers to buyers, measuring the return on investment needs different metrics.
State, local and other government websites are usually in the business of providing information or allowing “users” (usually taxpayers or residents) to accomplish transactions or providing some service online. Supporting these online services requires developers, IT infrastructure, content curators, and scores of people ready to answer the phone at help desks, in person or via email.
Before the web, many transactions were handled in person, by mail or over the phone. For example, if you wanted to incorporate a business with your state, you’d needed to obtain the necessary paperwork, fill out the forms, and then mail them or deliver them in person. The forms would then go down the line of clerks who would key that information into the system.
This process was expensive, slow and prone to errors. Now, many of these municipal services are online. You can even renew your driver’s license online now, circumventing the need for the much maligned DMV (something I did this year).
Getting functionality online was a major step for government sites. Now that the functionality exists, the problem is not utility, but usability. If users cannot find or cannot use a feature online, does it really matter that it exists?
Impediments to a Good User Experience
There are likely three major impediments to a successful government user experience (assuming a service exists and functions properly).
- Awareness: Users aren’t even aware a service exists or that the information is online (e.g. renewing your driver’s license online, incorporating, paying your property taxes, finding the hours for a pool or parking).
- Findability: Once users know a service or information is online, they have to find it. If users come to a government homepage and can’t find the promised content or the links to services, it’s as if the content or service isn’t supported.
- Usability: Once the user finds the service, such as the page for paying their sewer bill, if they have problems keying in information (like 20-digit codes with eight leading zeros) or entering payment details, there’s another opportunity for failure.
Solutions to a Good User Experience
Addressing the awareness problem requires more traditional marketing efforts and messaging across channels (emails, notes on bills, radio/TV, websites). This is probably the easiest one to fix. But once users know about something, it better be ready to support their needs.
Understand Your Visitors True Intent
Launch a true intent study to be sure you know who your users are across key attributes (e.g., geography, homeowner, retiree). If you don’t have data on who is actually coming to your websites (or should be), then don’t rely on what people think in staff meetings. True intent studies are quick and cheap to run.
Conduct a Top Task Analysis
Unfortunately, most sites, even commercial sites, fall victim to information overload. Content is created by disparate departments. The organization and taxonomy are prioritized more by byzantine internal organizational structures and funding agencies than by what makes sense to users.
There is so much information and so many features that are trying to do everything, which ends up causing confusion. While most websites or web applications can support hundreds or even thousands of tasks, there are usually only a small number that drive the bulk of the reasons users visit. Understanding these top tasks is an essential first step in knowing what content to emphasize on a homepage.
Tree Testing
Once you know who the users are and what tasks they are trying to accomplish, see if they can locate the content or functionality using a tree-test. Most users will browse for information so you can’t rely on a search engine—especially when the results likely only cover a fraction of the departments. A baseline tree test can tell you what users can and can’t find and the paths they take. Plan on iterating through several trees to get the right labels and organizational structure.
Root Cause Analysis
What is the nature of all those millions of calls to information hotlines cities support? Conduct a Pareto Analysis on any data you have. There are likely a vital few number of reasons that can be addressed with better online experiences.
Surprisingly, it’s often difficult to know, even for sophisticated for-profit companies, why customers are calling. Many call center systems have comment fields and the comments call center agents use can vary substantially—making categorization a herculean task. Or the call categories can be so vague as to be not helpful.
You don’t want call center agents spending callers’ time and their own time doing laborious data entry. But you also need to know what’s going on. You may need to survey callers, call center agents or find another way to obtain actionable data.
Usability Testing
Once users have a better shot at finding the content and you have some idea about the problem areas, have users actually try to accomplish tasks. If they need to pay property taxes, give participants in the study the same form they’d get in the mail and have them enter the necessary details and see where the experience breaks down. Use the core usability metrics of completion rates, task time and difficulty to quantify the experiences for the baseline and subsequent designs.
ROI Measures
The methods briefly described above will cost money. To get some sense of the return on investing in them, you don’t have to rely on fuzzier “hours saved” numbers. You can start with a couple of clear, hard costs that will likely be reduced.
Support or Information Calls
Each call to a help desk or city information number can usually be estimated pretty well. For example, the cost of Denver’s 311 information service was around $3 per call. With around one million calls per year, you can appreciate the savings in just reduced calls alone. For every increase in findability and completion rates, the probability of a call will decrease. If you can reduce calls by just 1% (10,000 calls), your estimated savings is $30k, and many UX research efforts will have paid off in the first year alone.
Postage & Processing
Even the simplest things, like communicating to businesses that they need to file an Annual Report, used to be done exclusively by mailing a postcard in Colorado. This cost the state millions of dollars a year in postage and processing. Switching to email notices and filing electronic documents generated savings that were easily quantified. Look for ways that usability and findability issues can reduce paperwork, postage and processing costs.
There are likely other hard and soft costs that can be tied to poor findability and usability on government websites. Most agencies report what their budgets are and what each service costs online (if you can find them!).
Just because the website isn’t in the business of driving traffic, it’s a win for the city if they can redirect (or cut costs), it’s a win for the taxpayer if costs go down or are used more efficiently, and it’s a win for the resident/user who can spend less time in lines, on hold or being frustrated by poor experiences online.
