Wednesday, December 16, 2009

Badges, Banners, and Calls to Action

The Problem

The folks in Admissions are getting phone calls from people who can’t find a link to the online application.

Probable Causes

  1. Users can’t find content “below the fold”
  2. General placement of call to action does not align with relative importance
  3. Banner blindness

Data

When I have a problem, I like to kill it with fire attack it with data; Google Analytics to the rescue! As of today, we’ve got just over 6 weeks worth of data since launch. Here’s what the numbers tell us about this page:

How They Got There How Many Times They Got There Percentage of Total Traffic for This Page
Future Students Landing Page text link in content 3,314 65.2%
Home Page badge 518 10.2%
Admissions Home Page 299 5.9%
Future Students Landing Page badge 199 3.9%
External (usually Google or another search) 158 3.1%
A to Z Index 120 2.4%
Total 5,081 100.0%

That’s data pulled together from several different reports as interpreted by me. There could be some rounding on Google’s part and some fudging on mine. But the general trend we see is reliable.

What Does That Mean?

By far, the most effective way we’re sending people to this content is the link about 1/3 of the way down the Future Students landing page that clearly states “The first step is to fill out an application.” In fact, 9.66% of all the people who see this page follow that link. It’s the 3rd most popular link on that page after Programs of Study and the Current Students landing page.

Taken together, the 2 badges are the 2nd biggest traffic producer, but we have a big drop off between 65% of traffic at #1 and 14% combined for #2 and #4.

After that we get a noticeable long tail effect, but that's normal for this type of data.

Applying the Data to Possible Causes

Content Below the Fold

The problem does not seem to have anything to do with scrolling. A wealth of previous research shows that content “below the fold” doesn’t really suffer for it. But it’s nice to see confirmation of this in our own data. The most effective link to this page is only visible after a bit of scrolling even on my rather large monitor. Comparing the people who are getting to the page to those who are not getting there could be apples and oranges, right? But we know about the 2nd group because they are calling us. Both the main telephone number and the direct number to the Admissions Office require a bit of scrolling to find. These people are obviously finding that information, so scrolling isn’t presenting a significant hurdle to finding information among this population.

Sub-Optimal Placement on the Page

We could still have a misalignment of placement vs. purpose even if scrolling doesn’t enter into the equation. The purpose of the badges is to feature timely content. The application process should probably be permanently featured, but a permanent badge runs askew of the core purpose of being timely and changing often. The link to the schedule for next semester makes for an excellent badge because that content is in high demand right now but in a couple of months we should be able to safely replace it with something more timely, perhaps a badge for the Academic Calendar so that people can quickly and easily find the dates for Spring Break.

So what are some other ways we can permanently incorporate “Apply Now” into the overall site template?

The primary navigation is all user role based, so adding it there wouldn't make sense. Putting it below the primary navigation would make it look like secondary navigation. At one point the site template had a “default” secondary navigation for those pages that didn’t have such a menu. It muddied the waters as to the purpose of that area of the page and it was an early cut in the design process. The area at the top, with links for the People Finder and A – Z Index, could work, but it’s already full. We’d need to delete a link in order to make room for it and I’m not comfortable dropping anything currently there.

We could add it to the Help Center under the Registration heading. Right now everything in that category applies to students who are already admitted, but I doubt most incoming students have a clear understanding of that distinction. Ask the average senior at Gallatin High School what the difference is between applying for admissions to Vol State and registering for classes at Vol State and they’ll probably look at you like you’ve got 3 heads. I would have at 17.

We could also list it in the Help Center under the Students heading. Either of those options fail to put the link visible on the screen by default, but it is accessible from (almost) any page. The footer is also an option, although that would take a bit more work on my part. We’ve got space to spare down there, but I want each area to have a clearly defined purpose.

Banner Blindness

But I think a bigger issue has been brought to light here. I’ve just gone through the navigation summary for all 7 pages that feature badges. None of the badges appear in the list of top 10 links for those pages. That may not be a bad thing if it means people are finding what they are looking for in the actual content. But it could also indicate a bad case of banner blindness.

Or Is It?

But approaching it from the other side, 92% of the traffic coming into the Schedule of Classes page are getting there through one of the various badges. That 92% translates into about 1,225 total page views, which is so tiny in comparison to the traffic coming through the landing pages that it may not make a blip on the radar. Learning Help Centers gets about 75% of its traffic from badges. SEEK gets about 94% of its traffic from the badges. But again page views measured in the hundreds are so small in comparison to the total traffic pumping through the various landing pages it’s probably easy for those numbers to fall through the cracks.

Overall badges seem to channel a significant percentage of traffic for the sorts of things people would not otherwise be aware of, such as SEEK and the Learning Help Centers. They also seem to work well for new site content such as the dedicated page for class schedule information. Both SEEK and the Learning Help Centers are fairly recent additions to the site as well. Badges seem to preform less well for older content with high awareness and lots of paths of entry. But even in the case of the “Apply Now!” badge, 14% of total incoming traffic may not sound like a significant boost, but it still translates into 700 visits. That’s nothing to sneeze at.

Conclusions

Lots of people are eventually getting to the application page, but that doesn’t change the fact that some are not and this is driving a noticeable number of phone calls to the Admissions Office. Some of the people who are eventually getting there may not be getting their easily. Maybe 10% of them were 30 seconds away from calling us too. And for every person who calls, maybe 3 or 4 other people just give up without even calling us. We really have no way of knowing. So I’m not pulling out these numbers as a means to dismiss the problem as it was reported to me. I aim to use the data available to understand the scope and context of the problem in order to find effective solutions.

We can increase our link coverage by adding it to a few of the persistent template elements:

  • Help Center —> Registration
  • Quick Links —> Students
  • Footer, not sure exactly where yet

We can tag and measure the performance of these links over time to gauge their effectiveness. But ultimately more data is needed. And it’s the sort of data Google Analytics can’t really give us.

In the near future, I plan to do some usability testing with potential students. Locating the online application will be one of the primary tasks for that research. The results may help us arrive at a long term solution.

Thursday, December 03, 2009

First month with the new site

Yesterday was December 2nd. We launched the new site on November 2nd. So as of today, I have one solid month of data on the new site available in Google Analytics. Let’s take a look at how we’re doing compared to November 3rd through December 3rd of last year. The dates don’t match up exactly so that we start on a Monday and end on a Tuesday with both date ranges.

  • Visits up 27.53%
  • Unique Visitors up 89.16%
  • Page views up 38.51%
  • Average page views up 8.61%
  • Average time on site up 22.44%
  • Bounce rate down 46.09%
  • Percentage of new visits up 90.69%

These are all positive changes. Bounce rate is a bad thing, so seeing that number go down is good. We’re reaching more people, who are looking at more pages and spending longer stretches of time before leaving. But I’m not ready to say all this is due to the redesign. After all, we’ve seen a significant enrollment increase this semester, so all these numbers should be improved over a year ago.

So let’s also compare the first month with the new site to a similar date range the month previous; 11/02/2009 through 12/02/2009 compared to 9/28/2009 through 10/28/2009, again starting on a Monday and ending on a Tuesday.

  • Visits up 0.22%
  • Unique Visitors up 15.27%
  • Page views up 30.01%
  • Average page views up 29.72%
  • Average time on site up 41.74%
  • Bounce rate down 51.39%
  • Percentage of new visits up 35.89%

This comparison is less straight forward. The new site has Thanksgiving break in this data set where the old site has no breaks, which would seem to put the new site at a disadvantage. But the old site’s figures come before registration opened up for the Spring, so in other ways it’s at a disadvantage. In other words, don’t read too much into this comparison.

It does help make it clear that the sorts of metrics tied to raw traffic have little to do with the redesign. The percentage change in visits is virtually zero. But metrics that measure engagement, such as time on site and bounce rate, actually show more improvement against a month ago than they do against a year ago. This probably helps show the natural boost we get thanks to registration opening up this time of year. When we’re talking about aggregate data it’s important to keep in mind all the variables that have nothing to do with the design of the site.

I’m more comfortable attributing large shifts in metrics for specific sections of site content that have been significantly overhauled as part of the redesign. For example, the list of our programs of study saw an increase in visits of 502.65% and an increase in unique visitors of 290.45% compared to figures for October of this year. Compared to a year ago, the difference is 548.75% increase in visits, 333.29% increase in unique visitors. One of my primary goals with the redesign was to increase the visibility of this content because I think it’s an important part of the “shopping” process. I think we can safely call that a success.