White Space Matters

So far today I’ve read two really good articles dealing with web analytics: Metrics and Dimensions and Reports — Oh My! from the Omniture blog and The Complete Beginner’s Guide to Web Analytics and Measurement from UX Booth.

But seeing these two articles open in adjacent tabs lead me to do some side by side comparisons. I noticed my gut reaction the the Omniture blog was “I don’t want to read all that!” The UX Booth article was considerably longer, but it didn’t illicit such a knee jerk reaction. I think the reason for this is white space. UX Booth gives their content a bit more room to breath.

Screen Shots

The following screen shots show the first 910-ish characters of content. I attempted to preserve the white space between this content and off screen elements. I’m using PNG-8 to avoid jpg compression making the text fuzzy. The downside is the photo from Oniture ends up looking a bit off. My fault, not theirs. Also, Blogger appears to be scaling these images to be the same width. The Omniture Blog screen shot is actually about 20px wider, for what that’s worth.

Omniture Blog

UX Booth

Counting Pixels

The textual content here are within 5 characters of each other, 912 characters for Omniture vs. 907 characters for UX Booth. Both spend some screen real estate on supplementary content, a photo or a table of contents. But I find the UX Booth content much more inviting. And by my count they use about a third more pixels to present their content.

What Do You Think?

Do you notice a difference anything like what I’m talking about? Or am I “putting the ‘anal’ in analytics” to borrow a phrase from the Omniture blog? Either way, the quality of the content in both cases is top notch. Definitely worth a read if you’re into that sort of thing.

Optimizing Site Performance

Purpose & Goal

Right now I’m running some tests through webpagetest.org to establish baselines for site performance. This week I hope to generate some CSS sprites for our common images and set up gzip and xpiration headers on the server. Those are the last 3 big things pointed out by YSlow (actually I’m using Google Page Speed, but YSlow is cool too). Hopefully soon I'll be able to run a new set of tests to compare to the baselines I'm running now.

Sampling

I'm trying to get a representative sample of of pages to test. The home page gets the most traffic, followed by the Current Students page. Pride Online has the most popular departmental home page, but it's utilitarian and not representative of other pages. That moves us down to the home page for the library.

Those 3 pages realistically represent only about 1/6 of our total pages, but it’s the most heavily visited 1/6. I could add one more test case and increase my representativeness to the majority of our pages, but we’d see diminishing returns in terms of traffic (only about 16% of total site traffic). Also, the remaining 5/6 of pages are mostly text heavy and image lite. The benefits to those pages will come from optimizing the overall site template, which will be partially reflected in the 3 chosen test cases.

Thanks to Google Analytics for the ability to access that info easily.

The Site Performance widget in Google Webmaster Tools says we gained nearly half a second in our rendering time around June 6th. That half second was enough to move us up from around the 73rd percentile to the 82nd percentile. I have no idea why we saw this jump in overall site performance but I'm glad to have it. Maybe we can even hold onto it.

The Results Are (Mostly) In

The 3rd and final test is still running as I type this. I’ll link out to the results as they become available.

• Home Page
• Current Students Page
• Library

Step 1: CSS Sprites

I'm using SpriteMe and my own best judgement to create some CSS sprites for commonly used background images. I'm assuming SpriteMe already has something like Smush.it built in.

Results

• Home Page
• Current Students page
• Library Home page

Ok, maybe I need to smush my sprites after all. I've added about 60kb to each page load while freeing up about 15 http requests. Smush.it can’t find anything to optimize, but the image optimizer in YSlow can shave about 5k total from 2 of the sprites. SpriteMe makes some less than optimal choices, even occasionally introducing bugs. But like all software it has to make certain assumptions and those don’t always pan out. I need to improve upon the default sprites while I’m at it. With these newly optimized sprites, let’s look at the trade offs.

jQuery UI Theme Icons for Play Controls

The first sprite I made was actually already made for me thanks to the jQuery UI Theme Roller. I used the blue and red versions of the UI sprite to replace the play control icons under the featured stories on the home page. The bad news is that almost doubles raw file size. In exchange, I pre-load a lot of UI elements that we're not currently using anywhere else. But I plan to make more use of the jQuery UI in the future. So I think this is a worthwhile change.

	Baseline	With Sprites	Change
HTTP Requests	8	2	-6
Total Bytes	5,310	9,724	+4,414

Stripes Sprite

Both the header and the footer have background stripes that go all the way across the page. These were tiny, 1 pixel wide repeated images. But they were easy enough to combine.

	Baseline	With Sprites	Change
HTTP Requests	2	1	-1
Total Bytes	905	122	-783

Image & Badge Drop Shadows

Some day I’ll be able to pull this off with pure CSS but in the meantime I’m stuck with background images. The one for the standard large image gets used on most pages. The one for smaller images at the same aspect ratio and the badges gets used fairly often too. The other 2 don’t see as much use. Some pages don’t use any of these, but the majority of our high traffic pages do. The image produced by SpriteMe left extra white space to the right of the 16px icons screwing up the placement. I had to edit the sprite in Photoshop. After that Smush.it actually did a better job compressing the resulting image than YSlow.

	Baseline	With Sprites	Change
HTTP Requests	4	1	-3
Total Bytes	4,018	5,092	+1,074

Icons

The image produced by SpriteMe left extra white space to the right of the 16px icons screwing up the placement. I had to edit the sprite in Photoshop. After that Smush.it actually did a better job compressing the resulting image than YSlow.

	Baseline	With Sprites	Change
HTTP Requests	18	1	-17
Total Bytes	14,589	13,677	-912

Logos

I combined our logo with the RODP logo that’s featured on every page. But that only nets us 1 less HTTP request and as you’ll see below costs us significant bytes due to the differences between .jpg and .png image formats.

	Baseline	With Sprites	Change
HTTP Requests	2	1	-1
Total Bytes	12,270	39,876	+27,606

In a Nutshell

In a best case, a single page that references all these images (which is not terribly realistic), we’ve traded 28 fewer HTTP requests for 31,399 more bytes. That’s about 22 additional HTTP packets. Based on our test cases we’re realistically saving 12 – 15 HTTP requests per page. I know we’ll see a big pay off when we start effectively caching and gzipping this stuff, but I have to admit I’m disappointed with this first round of results.

Step 1.1: Learning from Mistakes

Ok, now I understand why the RODP logo was initially flagged as not to be included in a sprite due to being a jpg. I need to roll back the Logos sprite. I also need to alter the way we’re doing the random headers. We’ve been using something based on Dan Benjamin’s method from A List Apart but in terms of site performance it’s one of our slowest resources.

Results

• Home Page
• Current Students Page
• Library Home Page

We’re starting to see some minor pay off on the repeat viewing. Start render time on the current students page is now under 1 second. That’s pretty awesome, but we’ve still got work to do.

Step 2: GZip

I followed this guide as well as some official documentation from Microsoft. This online gzip test says we’re still not using compression but my web developer tool bar begs to differ.

Results

• Home Page
• Current Students Page
• Library Home Page

I just alphabetized the properties in each CSS declaration, primarily as a way to clean up my CSS and make it easier to manage. But Google’s Page Speed documentation points out the possibility of alphabetized CSS seeing more efficient gzip compression. Checking the alphabetized, minified, and gzipped version against the just minified and gzipped version with the Web Developer Toolbar (Information —> View Document Size) shows 8k either way. It doesn’t break it down into units any smaller than that. So maybe it saved me a few bytes, but not a kilobyte. So if we saw any gzip efficiency change from alphabetizing the CSS it’s less than 12.5%. If we assume rounding, then it’s less than 6.25%.

Step 3: Browser Cache

This one will be labor intensive. Before I put in the hours to set expiration dates and last modified dates for our various static resources I want to make sure they are properly optimized. We’ve covered CSS and JavaScript, but all the images that didn’t find their way into a sprite will need to be tested. I started this entry on Monday morning. It is now 1:30 on Wednesday. I this step will probably keep me busy the rest of today and a good chunk of tomorrow.

Ok, the clean up and optimization process has saved me about 10,000,000 bytes and 400 files. Gzip has been running for a full day and server’s resources don’t seem to be at all taxed. So far so good. Time to dive into Leverage Browser Caching.

I have a problem in that IIS reports the wrong last-modified date for certain files. Images seem fine. PHP files and other plain text files are not. Maybe I’m missing something in Dreamweaver. Our server also sends E-Tag hashes, which are newer than and in theory superior to last-modified headers. But they are also redundant together. E-Tags are apparently frowned upon when using multiple servers, such as a Content Delivery Network. All our content comes from just the one server so I don’t think it’s an issue with us. At worst we’s sending a single redundant HTTP response header.

So half the browser caching equation was essentially taken care of for us by IIS’s default settings. The other end of that equation is content expiration. I did not like that IIS just gave me a single check box to turn content expiration on. I wanted to be able to set it up by file extension. No such luck. I turned it on and did some testing with the Live HTTP Headers extension for Firefox. I was able to confirm that our php pages were not caching. Which is good. The last thing I want is someone to spend the next 6 months with an unchanging copy of our home page stuck in their browser cache. CSS and JavaScript files were caching, along with images. I already use version numbering on CSS, JavaScript, and image sprites. For the most part other images will have specific names anyway, so that’s good. Then I tested PDF files and did not like what I found.

I downloaded the PDF of our campus map and saw that the response headers told the browser to cache it for the next 6 months as well. We may need to update the map in the next 6 months. Maybe an office location changes to a new building or something. Worse still, many of our PDFs are newsletters. Monthly newsletters. Some offices have an archive listing years’ worth of issues while others just link to the most recent issue. In the latter case, I just save the new issue over the old issue on the server. With browsers caching these files, that wouldn’t work anymore. I’d have to give each issue a distinct name and adjust the HTML links to match, greatly complicating that process.

I rolled back those changes and tried setting up expiration on a directory by directory basis. So far I’ve only done /images/, /css/, and /js/. That covers the vast majority of our static files. Occasionally a directory off of root will have its own images directory. For now I’m leaving those alone. The handful of child directories inside /images/ also apparently did not did inherit the expiration settings of their parent folder.

The downside to all this experimenting is I have to bounce the IIS service for changes to take effect. That only takes about 8 seconds by my count, but I’t to think of someone submitting their final form for online orientation during that tiny window.

Page Speed still says we fail the browser cache check, but that’s due to the FEMA widget currently on our site. All resources being served from volstate.edu pass. So let’s run another batch of tests.

Results

• Home Page
• Current Students Page
• Library Home Page

Conclusion

This has been less a blog entry and more 4 days of notes to myself. Tomorrow I will try to recap the high points and present my data analysis.

Trusting Second Guessing My Gut

Since December I've been blogging on the new multi-user WordPress server at work, but it's currently down. Once we get it back I need to figure out how to automate cross posting. This blog space belongs to me, Derek Pennycuff. The blog on the WordPress server technically belongs to the webmaster position at Vol State. If/When I leave Vol State this will still be mine. The other will not. Echos of the knowledge management course.

The Web as She Is Used

Anyway, right now I'm struggling with some stuff that my gut tells me is right but I have no evidence to support or refute it. I'll try to lay it out here and cite some specific examples.

Email Links

The default behavior of a mailto link is to launch your default email client. Here at work that means Outlook. On a default install of Windows it usually means Outlook Express. Our audience does not make heavy use of email. Most of them probably don't have a default email client. What email they use is probably web based. Some browsers let us set our web based email client of choice as the default, but how many users actually bother to do that? Personally, I tend to use Firefox's "copy email address" feature from the contextual menu when I right click a mailto link. But I've spent enough time working tech support to know that most people don't know much about advanced features such as right click menus.

So I prefer to use the actual email address as the link text for mailto links. In addition to all of the above, it's a nice visual cue to differentiate a mailto link from a standard hyperlink.

Telephone Numbers

Making that change introduced a new problem. The easiest way for me to make this change is to add a column at the end of the table and move the email addresses there. But that puts the telephone numbers in the middle of the table. A mailto link is designed to be used online. A telephone number is non-interactive.

(Although I confirmed over the weekend that a Nexus One phone does recognize a well formed phone number on a web page and makes it behave a bit like a mailto link. Among our users that's an edge case and I have no confirmation of how other mobile devices treat phone numbers. [Several people have pointed out this is de facto standard behavior on smart phones.])

My gut tells me most people will either dial the number from the screen or write it down. Either will involve a bit of eye darting on and off the screen. Have you ever had to look away from one specific table cell and then locate it again quickly when you look back? Working in Excel can cause this just as much as tabular data on a website. It's not easy. I think putting the phone number at the end of the table helps with this a bit. This brings us to a 3rd iteration of a staff listing page.

1. 
2. 
3. 

Aesthetics vs. Usability

The list above shows a progression of what my gut tells me is increasing usability. However, my eye disagrees. Aesthetically I see each version as a step down from the previous. And without any real usability data I'm at a loss as to how to resolve this cognitive dissonance. I'm dealing with opinion either way. And since so much of the usability issue hinges on action taken off-line, picking up a phone and making a call, I can't think of a way to A/B test it.

I need to standardize these pages. Inconsistency kills both the usability and the aesthetics. All I've got to go by is my own conflicting opinions, so if you've got any ideas please share them.

Update

A strong preference for option #2 is already apparent among the people who have chimed in on this issue. Apparently I'm on to something with the email address as link text for mailto links but I'm probably over thinking the telephone number thing. I'm going to continue to mull this over mentally so feel free to continue to comment or email me if you're reading this now and have anything to share. But in the interest of productivity I'm going to make option #2 the standard format and get to work. :)

The Culture of Technological Abundance

In one of my computer science courses the instructor asked if computing resources have reached the point where efficiency can safely be ignored. I responded with a resounding “No!” but some of the younger pups in the class disagreed. I'm begging to think I was showing my age, at least within certain contexts.

Something in my brain makes me love the idea of optimization. I actually police myself away from it, but I still find myself spending hours upon hours thinking about “better ways” to do things. I was recently debating with myself about storing telephone numbers in a database as integers vs. strings. I found the integer approach most appealing. It just felt right. But I quickly figured out that would require formatting the data to be human readable every time I needed to display it as well as a bit of trickery on the front end to get the form input (which would be in a human readable format) into a basic int structure. That consumes a lot of my time, and possibly a lot of time for the users who eventually put the collected data to use. In exchange, I save a little bit of hard drive space.

The new server has 500 gigs of RAIDed space. A formatted telephone number string, such as “(888) 555 - 1234” takes up 16 characters. That's a 17 byte VARCHAR. Even less if we default to something shorter for unrequired fields left blank by the user. But for the sake of argument let’s go with VARCHAR. Storing “8885551234” as a BIGINT requires 8 bytes, saving us 9 bytes. That’s 9 out of over 500 billion available. We’ll end up with a few hundred form fields that will see a few hundred hits per year. For the sake of argument let’s say 400 squared, or 160,000. If my attempts at optimization save an average of 9 bytes per field per record, we'll run out of space after about 350,000 years. I'm not sure what the clock cycles are involved in fetching or even comparing a 16 character string vs a 10 digit number, but probably even more negligible than the storage space. Obviously, server resources are abundant when compared to my time and the time of my users.

I’m about half way done building the forms on my to-do list and I just convinced myself to change the way I handle things. Oy vey.

If data is collected for the purpose of being later presented to humans, I will store it as a string, optimization be damned. I’ll use numeric types for data that is collected to be crunched, which in all honesty if rare right now. In situations where it could conceivably be used for both, such as dates, I’m probably better off storing both versions and fetching (or sorting by) whichever is most appropriate rather than running a timestamp through the date() function as needed or converting user input into the MySQL DATE format (which is both human readable and easily sortable).

2 years into this redesign project and I’m still not done. In hindsight, the biggest setback has been my own perfectionism. I have a hand crafted attitude towards my work. I take great pride in it. But at what cost? It feels great when I see something like Smashing Magazine’s list of current best practices in form validation and I realize I’m already doing the majority of those things simply because they “feel right”. But I spent all day yesterday working on a single form, stayed 45 minutes late, and still didn’t get it done. That felt anything but great. What’s the trade off? Where do I draw the line?

I’m even doing it now. I’m encoding my apostrophes and quote marks. Can anyone out there notice the difference between “this” and "this"? It's 12 extra key strokes for me to use the "proper" encoded characters. For all I know, Blogger auto-converts them for me anyway. (*EDIT* No, it doesn’t.) I've just developed the habit over the years of hand coding HTML. 12 keystrokes per quote pairs times 5 quotes per page times 3,000 pages at 400 characters per minute is 7.5 hours. That's a full workday over the past 2 years. Is that too high a price to pay for typographic correctness?

What's the cost of XHTML validation? Of ADA compliance? That last one could end up saving us a mint if lawsuits start getting tossed around. I know where my personal comfort level lies on most of these issues and I'm willing to re-evaluate in light of new information and fresh perspectives. As the only web guy around here I guess I get to make those judgement calls for the institution. But in a freelance situation my time is the client's money, which is definitely a scarce resource. A 0.2% markup cost for things like typographic correctness may not sit well with some clients, but there's plenty of designers out there who also don't care. Maybe they can service those clients.

The Proper Care and Feeding of VolState.edu

Most literature on user centered design (UCD) focuses on redesigning existing products or creating wholly new products. No literature I’ve found focuses on applying these ideas to established, mature products in the maintenance stage of their lifecycles. However, with an eye towards abstraction, I think we should be able to take the high-level concepts and apply them to any stage of a product’s lifecycle. My goal with this document is to establish some guidelines for making iterative improvements to VolState.edu using my current understanding of UCD practices, without having to shift into a full redesign.

There is a split among experts in the field of usability. Some strive for true science. Tests use many participants, drawn from the target audience, with tight statistical fit. Conditions are controlled and kept close to real-world environments. Participants use high fidelity prototypes that are functionally complete, or very nearly so. The data such tests produce is significant and reliable. But these tests also cost into the tens or even hundreds of thousands of dollars. Attempts have been made to codify formal systems for web site development, but these systems do little to address the practical concerns of time and money. (De Troyer & Leune, 1998)

In 1989, Jacob Nielsen introduced the concept of discount usability in a paper titled “Usability engineering at a discount”. The gist of the argument, to borrow a phrase from Steve Krug is “Testing with 1 user is 100% better than testing none.” Many people, Nielsen included, have tried to show statistically where the sweet spot for return on investment lies in conducting usability studies. Many other people, such as Jared Spool, have counter argued against such reasoning. Each side tends to rely on math supplemented with anecdotal evidence. This is all well and good as a distraction for academics. Personally, I’m much more interested in producing a better product for our users than in the math and science underlying the methodology.

I’m following Krug’s ideas most closely. He goes even beyond Nielsen’s discount usability into what he calls “lost our lease, going out of business sale usability testing”. Such testing can have value without the need for strict scientific validity. He’s even gone so far as to call himself a usability testing evangelist, actively encouraging people to integrate it into their workflow. In his book, Don’t Make Me Think, he addresses the top 5 excuses for not performing usability testing.

1. No time – Keep testing as small a deal as possible and you can fit it into one morning a month. The data produced will actually save time by pointing out obvious flaws in a way that sidesteps most possible internal politics and by catching these problems as early as possible, when they are most easily fixed.
2. No money – Testing can be performed for as little as $50-$100 per session. While that’s not free, it’s far from the $50,000+ price tag of big ticket professional testing. In terms of ROI, it’s one of the wisest investments you can make.
3. No expertise – Experts will be better at it, but virtually any human being of reasonable intelligence and patience can conduct basic testing that will produce valuable results.
4. No usability lab – You don’t need one. Since Krug first wrote his book, things have actually gotten even easier with software such as Silverback.
5. Can’t interpret results – You don’t have to use SPSS to mine for valuable data. The biggest problems will be the most obvious. Even if you only identify and fix the most obvious 3 problems, that’s 3 fewer problems in the final product.

Obviously if my goal were to get published, even as a case study, I would worry a bit more about scientific validity than Krug stresses. Although even among the academic literature some examples of successful case studies can be found where no attempt was made in recruiting beyond very broad user categories. (Bordac & Rainwater, 2008) But my goal in this document is to create a framework for usability testing that is simple and affordable enough to be actionable at Vol State. Ideally, it should be straightforward enough to persist beyond my tenure in this position. For those goals, Krug seems a perfect fit.

Involving Users Through the Development Process

Step 1: Identify a Problem or Unmet Need

Occasionally, users may volunteer information related to areas where improvements are possible or identify desired features not currently available. Anything we can do to encourage and facilitate this can only help.

Direct observation may also provide vital insight. Students may be observed unobtrusively in computer labs or in the library. Employees could be observed, with their permission, within their everyday work environment. Surveys and focus groups may also assist in identifying unmet needs at a high level of abstraction.

Problems exist within the context of users and goals. A proper definition of a problem should identify these contexts. Some possible examples:

• Students wish they could perform a task online that currently requires a visit to an office on campus
• Parents want easier access to a certain type of data
• Faculty need a way to share educational resources
• Sports fans need a better way of viewing or sorting our events listings

Understanding these contexts is vital to the success of a possible solution. If parents express a desire for easier access to student records, that may very well violate FERPA, in which case that problem is not actionable. Faculty looking for ways to share educational resources may be best served by training on Delicious. Even though we won’t stress a strict statistical fit with our test participants, understanding user groups in broad categories such as this allows for quick and easy targeting which should add value to the data collected via future testing.

Step 2: Establish Benchmark (Where Applicable)

Persistent, longitudinal data collected via survey could also apply at this step. Google Analytics is another powerful tool we have at our disposal. The type of benchmarking would depend greatly on the type of problem. For example, problems that require adding new pages to the site would have no data available in Google Analytics against which to establish benchmarking. But if those pages address an unmet need revealed via surveys that fail to turn up as an unmet need in a future survey, that could serve as a benchmark. (Comeaux, 2008) While I hope to incorporate this step as often as possible, I don’t want it to serve as justification for abandoning a project due to a lack of a method of establishing a proper benchmark.

Step 3: Wireframe Possible Solutions

Brainstorm for possible solutions to the problem. Produce low-fi wireframes such as semi-functional HTML prototypes or paper prototypes. Research seems to indicate that testing multiple prototypes produces more useful data (Tohidi et al, 2006). Test 3 or 4 users loosely recruited from the broad user category identified in step 1. Aim for 3 testable prototypes, but again don’t let this guideline derail an otherwise healthy project.

Step 4: Develop Solution Based on Wireframe Testing

Perhaps a clear winner will emerge from step 3, or perhaps the best possible solution will incorporate elements from multiple prototypes. Entering into the development phase with even a small dose of user involvement cuts down on costs and ensures that we “get the right design.” (Tohidi et al, 2006)

Step 5: Think-Aloud Usability Testing with Prototype

Construct a task list informed by the context and goal(s) identified in step 1. Recruit loosely from the broad user group as in step 3. Spend a morning walking 3 or 4 users through the task list. Record sessions using Silverback. Take quick notes between each session, recording the biggest issues while they are fresh in memory. Produce a single page list of issues, not a full report. Writing formal reports consume a lot of time better invested in fixing problems or further testing. Review session recordings with stakeholders as appropriate.

Step 6: Tweak Based on Testing

Fix the biggest 3 issues revealed in step 5. Avoid the temptation to go for the low hanging fruit until the truly catastrophic issues have been resolved. If something can be fixed with 15 minutes of work, by all means do so. But don’t count that as one of the big 3 unless it is also a major problem. After the big 3 and the easy fixes are addressed, act on any remaining issues as time permits.

Step 7: Launch & Evaluate

Put the solution into production. If benchmarks were established, collect data for a length of time allowing for a reasonable comparison up to a maximum of 1 year. If the data does not indicate improvement (and the criteria for judging this will be highly subjective and depend a great deal on context) consider returning to step 1 with an eye towards testing any assumptions in the original problem description.

Don’t Sweat the Details

Use the tools at our disposal to the best of their ability. Maybe a focus group would be the perfect tool for a given job. But we can pull that off because of finances or because it’s summer and campus is like a ghost town. Move on to a less than perfect solution, or plan to readdress that particular issue when the timing is right. But if we postpone one project, we should find something to keep us busy in the meantime. With a site as large and complex as VolState.edu, we should always be able to find something to improve.

The Tools of the Trade

In this section I will list some of the tools available to us. While I’ve made an honest effort to exhaust my knowledge on the subject, I make no claims to knowing every possible method. We should actively look for new tools and add them to this list as we become aware of them.

Card Sort Studies

These studies are often used for organization and naming conventions. There are 2 varieties. (a) Closed card sort studies involve asking participants to sort pre-determined items into pre-determined categories. (b) Open card sort studies involve asking participants to sort pre-determined items into groups which can then be labeled any way the participant chooses. Such studies are cheap and easy to conduct and provide valuable information within the limited context of language choices and organization schemes. Such problems are commonly reported in the literature involving case studies of university libraries. (Bordac & Rainwater, 2008; Proffitt, 2007; Shafikova, 2005)

Surveys

In the past we’ve had great luck with SurveyMonkey.com. Care should be taken in wording questions in such a way as to avoid biasing the answers. Surveys, particularly those delivered online, allow for a great number of participants from a wide range of our target audience. Smaller, more targeted surveys can also be conducted in person or on paper depending on the need. We also have access to data collected through marketing surveys conducted in 2003, 2008, and hopefully 2009.

Observation

Don’t underestimate the power of simply watching people using a system under normal conditions. Observation alone will often only produce hints or assumptions about possible problem areas. But that can be a great starting point from which to branch out into interviews or surveys. In the case of formal observations, such as with employees, a quick follow up interview can probe for more details immediately. (Pekkola et al, 2006)

Focus Groups

While there are dangers associated with relying too heavily on preference and opinion based data such as that produced by focus groups, they can provide data useful to our purposes. For example, it could be a great way to test assumptions arrived at through observing students using the website at the library. We’ve observed students doing X, which we assume is a way of working around problem Y. We could conduct a focus group to discover if Y really is a problem at all, or if students are engaging in behavior X for reasons we didn’t think of. Focus groups are most effective when drawn from within a single user category. For example, we probably would not want to mix students and faculty in the same focus group, nor faculty and administrators for that matter. (Kensing & Blomberg, 1998)

Interviews

Interviews could be paired with observations, helping to clarify and test assumptions much like a focus group. Or they could be used to help us construct personas. They would be most useful early on to help explain the context within which a given problem may be occurring.

Prototypes

Paper mock ups for a possible user interface can be produced quickly and cheaply. In a team environment, building such prototypes could even become a collaborative, iterative process. Some teams have reported success even with complex interactive paper prototypes. (Tohidi et al, 2006; see also i ♥ wireframes) Testing with such low-fi prototypes can lead to a greater willingness for participants to offer criticism since it’s obviously a work in progress.

HTML prototypes can range from semi-functional to fully functional. I’m personally much better at HTML wireframing than illustrating, so in the absence of outside assistance I will probably rely on HTML prototypes for both low-fi and high-fi testing.

Personas

Using demographic data, marketing research, and perhaps focus groups/interviews to construct an idealized but fictional user can help make user needs more personal and real (ironically enough) to a development team. Knowing that we have students with family and work obligations is important. But building a persona around that particular cluster of user attributes can make it easier to relate to and also give the development team a shorthand for addressing those issues.

For example, we could create a persona named Barbara who is a single mother of 2, working full time graveyard shift as a manager at a 24 hour restaurant, attending Vol State for 3-6 hours per semester working on a degree in Hospitality Management. She tries as often as possible to take classes on Tuesday/Thursday to minimize the burden of day time child care. She’s not technologically proficient, but she’s willing to learn. If given the choice between driving 45 minutes to campus or performing a task online, she prefers the online option in spite of her technological limitations.

That’s a very basic persona, but it still gives us enough detail to say “But what about Barbara?” rather than “But what about students who may have family or work obligations that prevent them from easily making an on campus visit on Mondays and Wednesdays?” Obviously personas could have applications outside of the realm of website development. Barbara could also provide insight into event planning, for example.

Think-Aloud Usability Study

This is the crown jewel in my usability toolbox. In a nutshell, it’s as simple as sitting a real human being in front of your product and watching them try to make sense of and use it. Ideally, you should have some specific tasks to ask them to perform and keep them talking about what they are looking at, what they think they should do next, what they expect to happen when they do it, their reaction to what actually happens when they do it, etc.

Full descriptions of the methodology can be found in lots of different places. I personally recommend Krug’s book. He’s even made the final 3 chapters of the first edition available on his website as a free download. Those chapters contain the bulk of the info on this sort of testing. There are other books, academic articles, and even blogs available, so pick your poison. Jakob Nielsen and Jared Spool have both written prolifically across many distribution channels.

Heuristic Study/Expert Review

My list would not be complete were I to leave this off, but personally I find this the most suspect. The general idea is you have an expert go over the interface and critique it based on a list of guidelines. This idea originated with Nielsen too and his list of guidelines can be found on his website. (Nielsen, 2005)

The highest value should be placed on methods that involve observations of real people interacting with some form of the interface in question. Preference and opinion based methods, such as surveys, focus groups, and interviews, can be quite effective for collecting marketing type data. But usability data relies too heavily on context for these methods to work alone. (Krug, 2008) However, any methods requiring direct human contact in the context of a college campus present a particular problem.

Sweat Some Details, but Document Them

One detail we have to sweat is the Institutional Review Board. All those highly valued methods that involve human interaction will require IRB approval. Even with approval, there’s no guarantee that a particular methodology will work. For example, I recently put together a proposal for a card sort study that failed to successfully recruit any participants. The good news is I’ve learned one method that does not work on this campus, so we can avoid any recruitment schemes that rely solely on face to face recruitment with no offers of compensation in the future. Eventually we will discover methods that both appease the IRB and meet with success out in the field. When that happens, we should save the IRB proposal, the information letter, the release forms, and all other deliverables. The next time the need for such a test comes up, we can build on the previous success. Many researchers, Krug among them, provide examples of their own materials that we can also integrate into our workflow.

Recommendations

• Develop an easy to use feedback mechanism into VolState.edu such as a simple for to “Report a problem with this page” or an Amazon style “Was this page helpful to you?” tool. Possibly both.
• Develop a standardized survey to be delivered via SurveyMonkey.com on an annual basis. Collecting standardized, longitudinal data will allow for some forms of benchmarking. Each iteration of the survey can include a handful of questions of a more timely nature in addition to the standard set. For example, a question about a specific change made in the past year.
• Continue to develop our use of Google Analytics for statistical and possible benchmarking data. However, do so with the understanding that Jared Spool has called analytics “voodoo data” because they lack context. (Spool, 2009)
• Integrate user testing to provide the context currently missing from our toolset as laid out in the 7 step model above.

---

Bibliography

• Bordac, S, & Rainwater, J (2008). User-Centered Design in Practice: The Brown University Experience. Journal of Web Librarianship.
• Comeaux, D.J. (2008). Usability Studies and User-Centered Design in Digital Libraries. Journal of Web Librarianship.
• De Troyer, O, & Leune, C (1998). WSDM: a user centered design method for Web sites. Computer Networks and ISDN Systems.
• Kensing, F, & Blomberg, J (1998). Participatory Design: Issues and Concerns. Computer Supported Cooperative Work.
• Krug, S (2005). Don't Make Me Think: A Common Sense Approach to Web Usability, 2nd Edition. New Riders.
• Krug, S (2008). Steve Krug on the least you can do about usability. Retrieved April 22, 2009, from blip.tv Web site: http://blip.tv/file/1557737/
• Nielsen, J (2005). Heurisitics for User Interface Design . Retrieved April 22, 2009, from useit.com: usable information technology Web site: http://www.useit.com/papers/heuristic/heuristic_list.html
• Pekkola, S, Kaarilahti, N, & Pohjola, P (2006). Towards formalised end-user participation in information systems development process: bridging the gap between participatory design and ISD methodologies. Proceedings of the ninth conference on Participatory design: Expanding boundaries in design. 1, 21-30.
• Proffitt, M (2007). How and Why of User Studies: RLG's RedLightGreen as a Case Study. Journal of Archival Organization. 4
• Spool, J Journey to the Center of Design. Retrieved April 22, 2009, from YouTube Web site: http://www.youtube.com/watch?v=WCLGnMdBeW8
• Tohidi, M, Buxton, W, Baecker, R, & Sellen, A (2006). Getting the right design and the design right. Proceedings of the SIGCHI conference on Human Factors in computing systems

An open letter to the University of Alberta

Good morning everyone. I've got a bit of an odd situation, so let me start off with some background info.

I'm an international grad student enrolled in the MACT program. Since that program is offered primarily online, I'm still living at "home" in Gallatin Tennessee. I picked this program because it seemed more reputable than competing programs I found in the US, both those available within commuting distance and those offered online. The online nature of the program seemed to soften most of the hurdles traditionally associated with seeking a degree across international borders.

For the first year of the program, all those assumptions proved true and I was very happy with my overall experience with both the MACT program and the University of Alberta as a whole.

Then you stopped accepting credit cards.

Now the entire landscape has changed. Had my first few semesters gone like this, I would not have stuck with the program. By the time things took a turn for the worse, I was heavily invested in the program and hated to turn back. I can't say for certain that this move is costing you business, but I can state with authority that it has greatly tarnished my personal experience with and perception of the U of A brand.

As of right now, I still have a balance of $20.66 CAD. But on the day I made my international wire transfer payment, I printed the exchange rates published on the U of A website along with the info sheet for international wire transfers. I took this to my bank and paid a $45 USD fee for the wire transfer. Apparently in the 4 days or so it took the transfer to clear, the exchange rate was not kind to me, so my payment fell short by about 2.8%.

But how am I supposed to deal with this? I took the most up to date info available to me at the time, provided by you. I have no more control over the markets than you do. Last semester, the same thing happened, only I ended up over paying by about $25 CAD. That semester, I sent money orders via international post. That cost about $18 USD, required me to fill out customs forms, would not allow me to get any sort of delivery confirmation, and took several weeks for delivery. I had assumed it was the length of time that resulted in the imbalance, so this term I tried the faster yet more expensive method of international wire transfer. Although I must admit the relative speed is also much easier on my mind. Having no way to track or confirm delivery of something as important as payment for my education provided a very stressful couple of weeks last semester. That stress stems from a process over which you have no control, but the fact of the matter is I experienced that stress as a direct result of a change to your systems. So that experience tarnishes your brand perception anyway. I can avoid that stress in the future by disassociating myself with your brand.

With credit card payments, Visa or Mastercard took care of all the exchange rate madness instantly, so such discrepancies did not happen. Now, international wire transfer is the fastest method available to me, and that takes close to a full business week. It will be virtually impossible for me to make a payment that is correct to the penny. If I'm short, I can either spend $18 (USD) to send you $20 (CAD), and wait for several weeks. Or I can spend $45 (USD) to send you $20 (CAD) and wait about a week. Either way we can easily end up in a Zeno's Arrow situation where I'm faced with spending $18 or $45 (USD) to send you $0.60 (CAD) a month later. If I'm over, I'm sure your institution is placed in a similar situation. Only I lack the means or authority to state and enforce exchange rates past the initial transaction. And to tell the truth, I probably wouldn't worry about it even if I could. How much hassle justifies that last 3% lost to the market?

So to avoid all this, I've asked a classmate, an Edmonton native, to settle up my account as a personal favor to me. Given my options, being indebted to a classmate is the least ugly and gets around the volatility of the market.

Chances are, I'm an edge case. I doubt you have many, if any, other international students who are not actually living in Canada. Such students would have access to Canadian bank accounts and thus would be able to pay with "electronic checks" using native Canadian currency. But I would not be surprised to learn that the parents of international students have found themselves in situations similar to my own. If you've noticed a dip in the recruitment or retention rate of international students, the added red tape introduced by removing the option to pay with a credit card would be the first place to look.

This process costs me about an extra 3% per semester anyway. There's no quantitative measure for the added stress, and I have no means to track the time lost compared to the simplicity of the "old" credit card based payment system. If your institution offered an option to pay with a credit card given a ~3% markup to cover the added costs to you, I'd accept that offer in a heartbeat. And your brand perception could shrug off the layers of stress and aggravation I've come to associate with it over the past year.

Thank you for your time.
Derek Pennycuff

Content! Third-hand content, but still, content!

One of the people I encountered in all the discussion on higher ed in the web design industry is Kurt Schmidt. (ImTheSchmidt.com, isn't that one of the greatest domain names ever?)

Since then, I've been following his blog semi-daily. Earlier today he posted Make Your Site Faster - Or Else!, which is his take on info from Geeking with Greg. I started to reply to Kurt, but about the time I started my 3rd paragraph I figured I should just turn it into a blog post here. See what I mean about blogs and peer review?

The importance of speed and performance on the user experience is something that I'm currently struggling with. In the redesign I'm working on, I've made extensive use of jQuery. Everything runs super quick on all the Mac browsers. Chrome and Safari on Windows also blaze. Firefox on Windows is noticeably slower than the rest, but still tolerable. Who does that leave to drool on his shirt and crap his pants at the browser testing party?

Both IE6 and IE7 are intolerably slow. I've considered simply filtering out IE6 from all the js goodness. Since I've designed for progressive enhancement, everything will still function, just not the same way. But that still leaves IE7 users with a clunky and ungraceful experience. As much as I hate IE, I won't kid myself about it's market share.

The best solution from the users' point of view would be to test each jQuery implementation on the site separately and selectively enable/disable for IE based on performance in context. Some of the stuff I'm doing on the redesign works fine in IE. I know because I've been doing it on the current site for more than a year.

I also need to dig deeper into my jQuery-fu and make sure my calls are as efficient as possible. But some of it is straight up plug in functionality. I'm not too keen on rewriting other people's plug ins to increase efficiency. The next version of Firefox will probably incorporate ideas currently only available in Chrome's V8 js engine, making it's tolerable performance that much better. But that will probably just widen the gap between IE users and everyone else even more!

The next current version of jQuery (which I haven't updated to yet, oops) will also have a revamped selection engine that should boost performance across the board. But unless those changes impact performance in IE a hell of a lot more than everyone else, the gap will still be huge. It's the gap in the user experience that worries me.

What little testing I've done so far would seem to indicate that IE6 is bad at say 2000 milliseconds. IE7 is barely any better and a lot less consistent, with speeds ranging from 1500-2300ms. Firefox is around 700ms. Opera is 300-400ms. Safari is 200-300ms. Chrome is about 150ms (10 times better than IE7's best run!). I guess with those numbers, even if sizzle improves performance by 20% across the board, Chrome gains 30ms and IE7 gains at least 300ms. Maybe the gap will get smaller. But still, 1.2 seconds waiting for all the js to trigger is unacceptable from a user centered outlook. But why should I be punished as a designer and the 20% or so of our users who don't run IE miss out on those enhancements simply because IE can't get it's crap together? These sorts of situations are the only part of my job I hate. I'll stop there before this turns into a rant.

Well, now that I know the new version of jQuery is out, it's pointless to flap my jaw any more about this stuff until I've updated and retested. I've got work to do.

In which I explore the depths of my own geekiness

So I've been thinking a lot lately about how video game control schemes relate to basic usability, and how that can be applied to website usability.

For example, I've been playing Zack and Wiki lately. In case you haven't played it, a major game mechanic is you can turn these various animals into various tools, then you use the tools to solve puzzles. The controls for each tool is different and not spelled out for you. You can examine the tools and look for clues as to how to use it. For example, the umbrella has a button with a 2 on it, and if you push the 2 button on the wii-mote the umbrella will open. Some of them are pretty simple, like you just make a forward / back sawing motion with the wii-mote to use the saw. I haven't made it very far in the game yet, maybe half way, but so far the controls are smooth and intuitive for just about everything.

They're so good, in fact, it makes the occasional problem very noticeable. So far I've encountered two rhythm based mini-games that are terrible. I can't do them at all. My wife managed to get through one of them. There's a skeleton with a music box and you use the wii-mote to ring a bell in time with the music he plays. It should be dead simple, but apparently I swing my wii-mote too hard, or too soft, or something because he's never happy with my performance.

Seeing what I'm talking about may work much better. Behold the power of the internets:

Do you want to know something sad? Just hearing that music right now grates on my nerves. That's how scared I am by my poor user experience.

There's another point where you have to try to catch this huge fish. The controls for the fishing pole were easy enough, but I kept thinking I was screwing up 'cos I'd get the fish about half caught and the fish would spit out the bait. It turns out this is just part of the game and you're supposed to go get another worm and try again. It generally takes 2 or 3 attempts to catch the fish (maybe it's possible to do it on the first try, but I certainly couldn't do it). The problem here was there was no feedback mechanism to let me know that my progress on my first attempt would be saved for subsequent attempts. Considering the other times Wiki interjects an inane comment to point the player in the right direction, this would have been an excellent opportunity to offer some encouragement to try it again. In the absence of any encouragement or feedback that I was at least on the right track, I found myself feeling cheated. How could this not be what I'm supposed to do? Even the first time I got a 2nd worm and gave it another try, the fish's energy bar wasn't emphasized in a way that made it obvious to me that he was still tired from my 1st attempt. I eventually figured it out, and I guess that's part of the point of the game, but for a while my experience was flipped from quite fun to very frustrating.

I can't find a video with just the fishing part, but here's the whole puzzle. The fishing bit starts around 2:30 and at around 3:45 this guy fails at his first attempt too. The little grumble and black squiggles in a thought balloon is the same feedback you get when you do something wrong.

Apparenty you can also just blow him up, which is something I tried but I was attempting to lure the frog the wrong direction.

This all just serves to remind me how important user feedback is to a positive user experience. Even the most intuitive of interfaces involves a certain level of uncertainty, particularly in complex, multi-step processes. Providing a means of letting the user know s/he is on the right track can really help. In the mini-game, the example in the top right corner of the screen could show how to swing the wii-mote rather than just how to hold it (although some may argue this gives too much away) and that would save me a lot of grief on the mini-games. Since the mini-games are option, I really don't have an interest in "discovering" the proper method through experimentation. I'll quickly get frustrated and move on to more fun elements of the game (which includes virtually everything else). When the fish spits out my first worm, Wiki could easily offer some sort of feedback along the lines of "Aww, he got away! But he looks pretty tired." Instead, we get Zack giving us the same reaction we get when we do something totally off track.

Then I found this blog entry: Ramblings of a Colorblind Gamer. And now I'm thinking about how all these things tie back into web accessibility as well as usability.