I applied for this about a month ago and promptly forgot about it, but I got accepted today into the beta for Google’s split-testing product, Website Optimizer. I don’t have the time to invest this second into working on it, but it will be part of my next big round of business improvements (half of my Christmas break will be spent on my lonesome in Japan, so I’ll have a few days to crank out a new version and start some New Year’s marketing efforts, including split testing the heck out of stuff).
Split testing, for those that don’t know the term, is taking a page you have (for example, www.bingocardcreator.com, the place folks are most likely to enter my site from) and making an A version and a B version. A and B differ by some minor respect — for example, in A I might have the current download button and in B I might nix that download button and go for something green. Then, here’s the magic, you randomly flip for every prospect that comes to your website for a certain period (say, 2 weeks) and hand them A or B. You then measure the effect this produces on conversion. For example, if B converts 2% better than A then I make that change permanent. Then I start optimizing the text, the title, the … you get the general idea.
I haven’t ever performed true split testing before (unless you count AdWords, I suppose) — generally, when I do an experiment it means hoping that my website is stable for a week other than the thing I’m experimenting on. That is a dubious proposition in the best of times — a sudden surge of crackers can throw your conversion way off, for example, which means you either get to throw out the data for the week and start over or just make a “best guess” that the new version “feels” like it converts better or worse than the old one. I hate guessing and love data. Thus I’m hoping Google delivers a useable and useful product here. I’ll post impressions when I have them.
