Sunday, December 17, 2006


My apologies for not posting lately but I am here to say that I have a good reason for that. I recently signed on to do some contract work with Yahoo in their Carlsbad, CA office and have been focusing all of my attention to my new work.

As some of you may know already, Yahoo is in the early stages of releasing their new search platform to the world. From what I have seen so far, it is a fantastic upgrade to the current search product.

In an effort to give their advertisers more information about their visitors, Yahoo has also ramped up their web analytics functionality. In my new position, I'll be be supporting Yahoo's high end advertisers and will be working towards bringing additional value to these advertisers through web analytics.

These are very exciting times for both online advertising and web analytics and I'm very excited to be working with the Yahoo team. I look forward to helping them make this new platform transition a successful event.

Tuesday, November 14, 2006

Case Study: A/B Testing

During my consulting stint for an online electronics retailer, we came accross a "situation" where we needed to run an A/B test. Below is a description of the test and the results.


The online retailer had 3 web sites that related to this case.

1. Site A - A main portal site (the branded site with the most visitor traffic) with direct links to their other sites

2. Site B - The retail site where visitors could shop for and purchase the electronics

3. Site C - The educational site where visitors could learn about the products before they made a decision to purchase

In this case, the marketing team decided that they wanted to push the awareness a particular new camera product. To do this, the team created an internal banner and placed it on Site A (the high trafficked portal site). Upon clicking on this banner, visitors were sent to Site C so that they could educate themselves about the camera product in more detail.

Once on Site C, if the visitor was interested in purchasing the product, they could click on a "Buy Now" link that took them to the camera's product detail page on Site B.

The "Situation"

After the banner was running for a short time, a major question arose.

1. Were we losing revenue by sending the visitor directly to Site C (the educational page) first instead of sending them directly to Site B (the purchase page)?

The asumption was, that by sending the visitor first to Site C, the visitor had to take an extra step in their browsing path in order to reach the purchase site. Why not just send the visitor directly Site B from Site A and save the extra step?

The reason was because Site B and Site C were run by two different groups within the company. The business goal for Site C was to get as much visitor traffic to their educational pages while the business goal for Site B was to generate as much revenue as possible from their pages.

This of course led to some internal corporate headbutting. The group that ran Site C wanted a direct link from Site A because it helped them meet their traffic goals. But the group that ran Site B felt that revenue and conversions were being shorted if the visitors went to Site C first.


To resolve this problem we decided to run a test and let the data speak for itself. The two groups agreed on running a two week test. The first week the camera banner on Site A linked directly to Site C (the educational site) and second week the banner linked directly to Site B (the purchase site).

Assumption: With this test, we wanted to see whether or not total revenue was significantly affected if visitors went through Site C first (A --> C --> B) instead of going directly to Site B (A --> B).

I first measured the revenue generated from the Site C "Buy Now" link to Site B (we tagged the "Buy Now" link with tracking code). This was a little tricky to do because I had to measure the percentage of traffic that only came from the camera banner (there were other ways in which visitors could reach that camera learning page). I factored in that percentage when gathering the revenue total from the "Buy Now" link.

Site C --> Site B

The second week, I measured the revenue generated from the banner on Site A to Site B (we tagged the camera banner with a different tracking code). Again, I had to measure the percentage of traffic that just came from the camera banner (there were other ways in which visitors could reach that camera purchase page).

Site A --> Site B


With my results in hand, I found that by sending visitors through Site C first (A --> C --> B), there was a drop in overall revenue compared with sending visitors directly to Site B (A --> B). However, this drop was not the significant drop off that the managers of Site B were expecting. The assumption that, sending visitors through site C was killing conversions and sales, was false.

Not only was there not a significant drop off in revenue, but I found that the AOV (Average Order Value) was actually higher (compared A-->B test) when visitors were sent through Site C (A--> C --> B) first. Visitors bought more or bought higher priced ticket items when sent through the educational site first.


In optimizing a web site, you're going to run into certain situations where assumptions may arise from fellow employees about what currently works and what doesn't. The great thing about web analytics is that allows you to test out those assumptions and find actionable solutions to help answer these assumptions. Make sure that you constantly test and make changes based on the data results....not assumptions.

In this case, upon reviewing my findings, the company decided that sending visitors through the educational site did not have a significant negative impact on sales and conversions. We also found that by sending visitors through the educational site that it actually helped to increase average order value.

Friday, November 03, 2006

Web Analytics & Privacy Concerns

I made this post on the Yahoo Web Analytics message board, but I wanted to post it here because I feel that it's important for anybody involved with web analytics to get an understanding of what's going on out there.

There are petitions being put out by consumer advocacy groups requesting that the FTC investigate the regulation of data collection. Depending on the decision that the FTC makes, there could be a large impact on the way web analytics is used today.

With that being said, I think the FTC is smart enough to see the value in web analytics and that it's not designed to infringe on a web site visitor's personal information. We all know that web analytics is designed for helping online businesses improve their site performance and marketing efforts.

Anyway, here is my post. Feel free to read the article and leave me any comments.

FTC Asked to Stop Microsoft's AdCenter

I'll bet $1,000 that these consumer advocacy groups, who petitioned the FTC, have members that go to their local grocery stores on a daily basis and use their Vons, Ralphs, Albertsons, Safeway, etc. club cards to save money on groceries. Yet, I don't see them petitioning against grocery chains.

I bet they've swiped those discount cards dozens of times without realizing that while those club cards are great for getting discounts on an overpriced box of cereal, they are really designed for tracking every item you buy.

Whoops....did I say that out loud (sorry grocery stores)?? I guess I'm all for grocery chains making my shopping experience better. I know....I know....I'm crazy! Do I really care if they know that I bought a gallon of milk and a six pack of Guinness beer??? No. In fact, I think it's great when I get two coupons in the mail a month later for discounts on what else....milk and Guinness beer.


I say...keep swiping those discount cards and let's keep getting our beer discounts!

Friday, October 27, 2006

Traffic or Conversions?

A couple of years ago, when I worked for a search engine marketing and analytics company, I was speaking to a potential client about his paid search engine marketing efforts. He was explaining to me how excited he was that he had just increased the traffic to his e-commerce web site by about 10,000 visitors.

After listening to how he used Google AdWords and Yahoo to improve his ad rankings, I dropped a bomb on him. I explained how great it was that he was getting more traffic, but I finally asked him....

"So what's your conversion percentage for those keywords you bought?"

He said, "My what?"

I said, " Do you know which keywords work well and which ones don't ?"

There was a pause.....he didn't know the answer.

I then asked, "How do you know if the money you're spending with Google and Yahoo isn't being wasted?"

He said, "Because I'm getting a lot more traffic."

I then told him, "Do you realize that you can be spending $0.50, $1.00, or even $2.00 per click on keywords that don't lead to a sale for you? By focusing on the successful keywords (the ones that drive sales) and weeding out the non-successful keywords (the ones that don't drive sales), we can optimize your PPC advertising so that you're left with nothing but keywords that drive sales for your business. We can save you money and drive more sales at the same time."

He said, "We'll, how do I track them?"


The reason why I bring this story up is because many companies are still too focused on driving traffic to their sites and they're not putting enough energy towards improving their conversion percentages (ppc keyword conversion, internal search keyword conversion, site conversion, etc).

Rule #1: Don't assume just because you get more traffic that your leads or sales are going to increase.

The large online electronics company I consulted for had millions of visits per week but their sales conversion % was only around 0.7%. Based on the amount of traffic that this site received, I soon realized that if I could just improve their sales conversion by even 0.1%, then that increase could lead to thousands of dollars in additional earnings for the company. At that point, I began optimizing the web site to improve sales conversion.

This electronics site also had traffic that was increasing month-to-month on their home page. I soon found out that the #1 most trafficked area on the home page was their internal search box. And after digging deeper, I learned that 33% of the internal searches led to a "No Results" page. This of course meant that one third of the internal search keywords being used (by visitors) weren't very effective at converting browsers into buyers. With this information in hand, I began optimizing internal search conversion.

In the examples above, there were two totally different situations where 'conversion' needed to be optimized (sales conversion and internal search conversion). When web analysts talk about 'actionable data', this is what we mean. We're gathering data from the web analytics tool and then taking action to improve low order totals or conversion percentages for example.

Don't get me wrong, we all want more traffic because it means our site is getting visibility on the web. But remember to always keep a very close eye on how well your site is converting on all aspects (online advertising, internal search, etc.). Monitoring conversion and taking action to improve it is the key to a successful online business.

As Eric Peterson once said, " do you know you’re successful if you’re not measuring your success?" You can't put it any simpler than that.

Tuesday, October 24, 2006

Century Mark!

Today my blog surpassed the 100 visits mark. Thank you to everybody who has visited the blog, read my posts, and sent me positive emails about the blog.

I tagged it with Google analytics and have been tracking its progress. According to the Geo Location report, I've got readers from all over the world! It's great to see how quickly web analytics is growing.

Next stop....500 visits!

Friday, October 20, 2006

Comments Update

I changed my settings so that you no longer need to have a registered Blogger account to leave me a comment. Feel free to add your comments to any of my posts below.

Matt Lillig

Thursday, October 19, 2006

Click Fraud: It's More Than Just Cheating Competitors

What if I told you that I could be making $25 to several thousand dollars a month just by clicking on a bunch of online advertisements which I had no interest in?

Would you care?

What if I told you that some of these links I was clicking on were your links on which you're spending between $0.40 to $8.00 per click for?

Let me put it this way.....I could be stealing away your hard earned money and burning a hole in your ad spend budget.

I should have your complete attention by now.

Well, according to the cover story in the October 2nd 2006 issue of Business Week, an investigation has revealed "a thriving click-fraud underground that is populated by swarms of small-time players, making detection difficult."

And how easy is it these days to start making money off of click fraud? Too easy. A couple from Minnesota said they dabbled in click fraud last year and made more than $5,000. They employed a simple scheme in which they "set up dummy web sites filled with nothing but recycled Google and Yahoo advertisements. Then they paid others small amounts to visit the sites, where it was understood they would click away on the ads." They then split the resulting revenue generated from their dummy web site with the search engines.

With spending on internet ads growing faster than any other sector of the advertising industry and experts estimating that 10% to 15% of ad clicks are fake these days, you can start to see how click fraud is becoming a major issue.

But what makes up Click Fraud? Try these new terms on for size....

Parked Web Site: A site typically with little or no content except for lists of Internet ads, often supplied by Google or Yahoo; many of them are the source of false clicks.

Paid-To-Read: A PTR site pays members to look at other web sites and offers from marketers (ads); often used to generate false clicks on parked web sites.

Clickbot: Software that can be used to produce automatic clicks on ads; some versions employed in click fraud can mask the origin and timing of clicks.

Botnet: A collection of computers infected with software that allows them to be operated remotely; networks of thousands of mechanism's be used in click fraud.

Even though it sounds like some kind of sci-fi movie with all of these new high tech terms, how bad has click fraud actually become? The Business Week article talks about one Atlanta based businessman who estimates that click fraud has cost his company $100,000. On the other hand, a Russian software developer is making $10,000 a year from a clickbot he sells. And click fraud is so simple these these days that clickbots can be downloaded from the internet. The article states that "clickbots are popular among online cheats because they disguise a PC's unique numerical identification, or IP address, and can space clicks minutes apart to make them less conspicuous." It also states that "more skilled Chinese programmers modify the clickbots to outwit the American search engines (like Google and Yahoo)."

So what are the big search engines doing about click fraud? "Google and Yahoo say they can identify automated click fraud and discount advertisers bills accordingly." But advertisers have to prove that they are being cheated.

My question is, if clickbots are masking IP addresses, how is an advertiser supposed to prove they are being cheated?

I find it funny that as I'm writing this post, Google just released its earnings release. Once again, it was better than expected. Why? Because as I mentioned before, spending on internet ads are growing faster than any other sector of the advertising industry. And that means....more ads can be clicked. The more ads that are clicked, the more money search engines like Google and Yahoo make. And yet, click fraud is thriving more than ever.

But, I guess if more people are spending money to adverise with search engines, then apparently click fraud isn't a high priority issue for most advertisers.....

.....just don't tell the guy from Atlanta this!

Wednesday, October 18, 2006

Wanted: "Dedicated" Web Analyst

I recently read a post on the Yahoo! Web Analytics message board that opened my eyes again to the pain of companies trying to forge ahead with finding a web analyst and getting involved with web analytics. In this case, the poster talked about how he had suddenly found himself as "the guy responsible for driving the web analytics strategy".

Reading that quote immediately brought back memories from when I used to be an account manager at one of the web analytics vendors. I can't tell you how many times I heard war stories from employees who had received the news from their managers that they were now in charge of web analytics for the company. These clients would tell me that they were now expected to run the web analytics strategy on top of their current job descriptions. As an account manager, the first thing that came to my mind was "sympathy". Knowing what I knew about the time and dedication it takes to run a successful analytics strategy, I just couldn't fathom how these employees were going to manage web analytics on top of their current work loads.

Now I can understand how somebody could implement Google Analytics and only check the reports once a month. Why? Because it's free and by not looking at the reports, you're not burning a hole in your wallet. But somebody please tell me why any company would ever spend over $10,000 (or more) on an analytics package and not have a dedicated person in place to read and interpret the data? It just doesn't make sense.

I'm here to say that being a "Web Analyst" is definitely a full time job. A successful web analytics strategy goes hand in hand with a dedicated web analyst. I've heard the experts out there say this time and time again, yet companies still try and take the shortcut route. Rather than moving in a dedicated web analyst, they end up dumping the analytics into the lap of one of their employees who already have too much to do. And note the word "dedicated". I'm not saying that a company has to go out and spend the money to hire an experienced web analyst (though it helps to ensure faster results), but at a minimum they should move a current employee into a dedicated web analyst role when jumping into web analytics. This way the employee can put their full attention into improving the online business and not be distracted by other job requirements.

So if you're a manager reading this post and you're trying to figure out why you're not getting a good ROI from your web analytics package, then you may want to take a step back and ask yourself this one simple question, "Are we taking full advantage of our web analytics investment by having a dedicated web analyst on staff?"

Monday, October 02, 2006

How many stats would a stats analyst track if a stats analyst could track stats?

Does this scenario sound familiar???

You've got 3-5 traffic dashboards to update each week....
One major traffic presentation due each month....
2-4 random employees asking you to pull data for their group and build a dasboard for them each week (by the way..."Make sure those reports are sent automatically")....
and finally train different managers on the value of using web analytics.

Sounds like a pretty busy job in itself, right? Wrong. I forgot one small still need to OPTIMIZE THE WEB SITE!

But how in the world is a web analyst supposed to find the time to optimize the web site when they're spending most of their time pulling data, building dashboards and reports, and training different groups of people within the company? Well, based on my experience as a web analyst, I believe one needs to work closely with their manager, prioritize their work, and be stern. No, not Jim Sterne (although, I'm sure that would be helpful in a web analyst position) but stern as in putting their foot down.

A web analyst should work closely with their manager and keep them informed of their workload. Many times, managers (as busy as they are) don't realize the many different types of reports and dashboards that the analyst has going on at one time. Nor do they realize that the analyst is constantly being bombarded by other managers and employees in seek of a personalized dashboard or report. Prioritization should be a big part of the conversation between the manager and the analyst. If the analyst has a clear picture about high priority goals that need to be met, he/she can effectively use their time to build out the proper reports and dashboards.

While it's very important to build out reports and dashboards, that's only one aspect of being a successful web analyst. The other part comes in being able to optimize the web site. A web analyst needs time to, well, analyze! But he/she can't do that if they are spending all of their time building reports and dashboards for people. This is where the stern part comes in. There comes a time in every web analyst's life where they have to put their foot down and say, "I'm sorry (place a coworker's or manager's name here), I simply cannot build that custom report for you right now because I have some high priority issues to take care of."

A web analyst needs to put their foot down at times, otherwise they'll be up to their ears in building reports. In my case, I wanted to make everybody happy by pulling data and customizing reports for them. However, I soon realized that I needed to be spending more of my time analyzing and optimizing the web site. And we all know that this is where the real fun is at. Running A/B tests or multivariate testing, tracking the success of internal search, tracking the success of promotions, using data to help the development team with page layout and navigation, etc., are all things that make make the job exciting. Why? Because you finally get a chance to see if your analytical assumptions are correct!

Making sure you have time to analyze the site and come up with actionable ideas is extremely important to being a web analyst. Setting aside time to come up with new ideas is what made 3M Corporation so successful. They give their employees the option to spend 15% of their work week pursuing individual projects of their choice. Most of the time, these individual projects result in brand new products for 3M. In 2000, 3M made more than 60,000 products. Nearly 35 percent of its total sales, or about $5.6 billion, came from products that had been introduced during the prior four years, and another $1.5 billion came from products introduced during 2000.

Knowing that little bit of information, think about the success that a web analyst can make if he/she has the time to optimize the web site and come up with actionable ideas. Who knows, maybe you can help your company become the online version of 3M!

Sunday, September 24, 2006

Internal Search: The Value In Monitoring Your "No Results" Keywords

We all know that search engines (like Google, Yahoo! and MSN Search) are very helpful in bringing visitors to your site. But what about internal search engines? How valuable are they when it comes to the success of your online business? Well, they can be just as valuable (if not more) because they actually help bring visitors to your shopping cart.

Whenever a visitor types in a keyword search into your homepage's internal search box, that visitor is essentially telling you, "I am interested in finding this product, I want to learn more about it, and I may even want to buy it!" But if your internal search tool can't provide the visitor with what they are looking for, then you may have just lost a sale. Beyond losing a sale, you may have also lost a potential loyal customer who (with a good first time experience to your site) might have purchased from you again in the future.

So how does all of this internal search stuff relate to web analytics?

Well, in order to build a successful internal search program for your visitors, you need to understand whether or not your visitors are able to find what they were searching for.

I recently did some contract work for a large online electronics business and discovered some disturbing information regarding their internal search results. When using the company's overlay tool from their web analytics vendor, I found that the "go" button on their internal search box was the #1 most clicked link when visitors reached their home page. That's the great news. The bad news was that nobody really paid attention as to what happened after the visitor typed in a search keyword and clicked "go".

After digging into their reports a bit deeper, I found that of all the visitors who attempted to run an internal search, 33% of them reached a "No Results" page. I probably don't have to tell you this, but 1 out of every 3 searches resulting in a "No Results" page just doesn't cut it when trying to build a positive visitor web experience. This percentage is simply too high to ignore.....especially when one of the company's key KPIs (key performance indicators) is to increase their sales conversion percentage.

So how could this be???

After looking at the top search keywords that led to a "No Results" page (one of the reports in their analytics tool), I noticed some surprises. High up in the list of "No Results" were keywords for very popular products that the company definitely had in stock. But for some reason when the visitor searched for those particular products, they were directed to a "No Results" page. What we discovered was that the visitor would only reach a "No Results" page when they typed in the keyword and hit the "Enter" or the "Return" button on their keyboard. However, if they typed in the keyword and hit the "go" button, they were directed to the proper product page. Luckily for us, were able to trace this occurrence to a problem within the way the search tool was directing visitors and we were able to knock those popular keywords off of the "No Results" list.

A second problem we encountered when looking at the top keywords that led to a "No Results" page, was that visitors were searching for products that we no longer carried or sold. To resolve this issue, we had our web developers set up special pages (called splash pages) that provided a simple explanation to the visitor. We also provided additional links so that the client could find similar products or even reach our support team if they had additional questions. By sending the visitor to a splash page instead of a "No Results" page, we were able to dramatically cut down our chances of losing the visitor entirely. At the same time, we increased our chances of a visitor viewing other products and possibly buying.

A third problem we found from looking at the "No Results" keywords report, was that visitors were searching for products that they read about in recent articles (press releases, etc). This was great for our marketing department but bad for the visitor because we realized that some of those product pages hadn't been launched yet on our web site. The solution, obviously, was to get those product pages launched as quickly as possible so that visitors could gather the information they needed. This problem turned out to be a blessing in disguise because it helped to bring a closer relationship between the marketing PR department and the web development group.

The final problem I noticed was that we had an enterprise online company using a non enterprise internal search product. Many of the great features that one could find in a more expensive product were difficult to get at using the company's current solution. It took more time to get issues resolved from this smaller internal search vendor. And as we all know, time is money. With the amount of unique visitor traffic that this online business received each month, it would of made sense to spend the extra money with an internal search vendor that mainly deals with enterprise sized web sites.

In the end, if it wasn't for web analytics, these problems would have never been discovered. With this valuable information in hand, I was able to work with a team and turn this data into actionable information that led to positive results. We successfully "knocked out" the top 5 internal search keywords that led to a "No results" page. These 5 search keywords alone made up 80% of all the unsuccessful searched terms. With a web site that generates between 4 and 6 million unique visits a month, you can see how valuable it was to optimize our internal search results. Not only did we create a more enjoyable web experience for the visitor by "cleaning up" our internal search, but we were able to turn more browsers into buyers and save the company thousands of dollars in potential lost revenue!

If you have internal search on your web site, make sure that somebody is constantly monitoring the percentage of your internal searches that lead to a "No Results" page. Monitoring and tweaking your internal search results can earn you immediate rewards!