Home page of Seattle.gov in early 2011

Seattle.gov Usability Study

Overview

Goals:

  • Asses and improve the usability of Seattle’s first homepage redesign in nearly a decade, all on a shoestring budget.
  • “Teach them to fish” so the city would be capable of running usability studies on their own in the future.

Dates: June 2010 – October 2010 (4-5 months)

Target Audiences: Seattle citizens; Seattle business owners

My Roles: Usability Researcher

Outcomes

  • Successfully improved the usability of the proposed redesign, on time and under budget. We helped simplify the IA and navigation, and impressed upon them the importance of aesthetics when it comes to user trust and perception.
  • The stakeholders – including the mayor and the city council – were delighted with our work.
  • The resulting website won a Best of the Web award for city government.
  • Broke down communication silos; departments that had not worked together in 5 or more years realized this lack of communication was an active disservice to Seattle citizens.

Process

At around the time we started the project, the Seattle city website had looked like this for almost a decade:

Not bad for circa turn-of-the-century web design, but the world had moved on. In order to run a usability study, we needed to first choose tasks, which lead me to conduct a content inventory/audit for the current site. It quickly became clear that this could not be exhaustive if we were to stay on track, but thankfully 3-4 levels deep was all that was needed to plan usability study tasks.

A snippet of a content inventory, which shows lots of words connected to other words by lines, arrows, and bubbles

 

Speaking of, I based the study on Krug’s methodology. Getting all of the website stakeholders in the same room seemed to be something of a hurdle for the city, so I thought the live observation room would be a great approach. (Having the backing of the mayor was key for making this happen.)

Cover of Steve Krug's book "Rocket Surgery Made Easy"

Recruiting was also up to us, so we made up some quick-and-dirty tear off flyers and spent the better part of a day posting them on telephone poles and on community bulletin boards around every neighborhood in Seattle. (That was a lot of work to get ~dozen participants, but it was worth it to get a nice variety of folks.)

A flyer advertising the usability study with tearoffs

The city wanted to have an understanding of how the redesign improved over the old design, so we actually ran two studies as a between subjects A/B usability test ( just in case that kinda jargon gets your juices flowin’).

A person trying to navigate around the old City of Seattle website
Testing tasks on the old site.

 

As part of the process we produced some wireframes for them as a way of communicating the sort of changes they would be able to make to their proposed redesign.

a wireframe drawing of a proposed seattle.gov website
This wasn’t produced from whole cloth, it’s a massaging of the ideas they had already put together as a way of communicating recommendations

The debriefing session (which I wish I had pictures of) after the stakeholders watched the users in real time near miraculous. The shared understanding of users’ struggles helped them build new professional relationships and made it easier for them to accept recommendations for improvement, most of which they adopted. (We also had the full backing of the mayor’s office, and that authority didn’t hurt, either.) One stakeholder even confided that the observation sessions helped clear log-jams that had been intractable for years.

Reading through one of the reports I gave the city is not something you want to do. It’s super boring and out of context for you. But hey, it’s there if you want it.