A/B testing has limitations. The biggest thing A/B testing doesn’t account for is visitor behavior.

Let’s say 60% of people prefer ad set A and only 40% of people prefer ad set B. You then choose ad set A because it had the majority preference of 60%. How can you account for the other 40% of visitors who preferred something different?

Numbers lie. An example of this is that ad CPM/CTR is an average, which reveals very little about the underlying data and just how your impressions are priced.

Test groups must be random. When running A/B test, how do you segment your traffic? Who is really in each group? Does each user get assigned to a group? Each session? Each pageview?

The math is not easy or straightforward. How long to run for? What is the underlying distribution of data? What level of significance do I need? How big is the effect? How do I control for random bot traffic?

Not responsive to changes as they happen. You run your test in June, find a statistically significant effect, you’re confident in your results… but how confident are you that everything will be the same in December?

A.I. has many advantages over A/B testing:

• Reduces data mining
• Eliminates hard coding
• Robust changes in data patterns over time
• Outsources the math

Take this example. How would you traditionally tell a computer system how to define what variables make a dog a dog?

• Animal
• Mammal
• Barks
• Walks on 4 legs

This is great until you get an outlier that still technically is a dog, but doesn’t fit all the variables you gave the computer - a dog with three legs that lost one in an accident.

What AI does is it flips traditional programming on its head: rather than defining the steps to get to an outcome (“take A, B, C, D… do this to them, make a decision), we can send in some data, and let the neural network learn what’s important. We could even add a completely irrelevant descriptor, like has red ball in mouth. It’ll learn quickly that this doesn’t describe dogs at all.

A.I. increases the value of ad inventory.

When it comes to ad placements, publishers are often met with dilemmas. Which ad do you show: top of page or sidebar?

Let’s say you look at your ad data and the CPM for “top of page” and “sidebar” are both $3.02. But is the amount the same for those two ad placements when more visitor behavior variables are introduced?

Instead of doing all of this investigating yourself, you can outsource this work to Ezoic’s artificial intelligence, powered by TensorFlow.

Next, we tell TensorFlow what features in our data we want it to learn from.

After that, we tell TensorFlow what type of model to use: here, we’re using a deep neural network regressor (which is a good model to predict numbers, like CPMs).

Now, we tell TensorFlow how to load and read the data.

And lastly, we initiate training. 8 lines of code in total! It’s that easy. So what does it find?

TensorFlow figured out how to segment the data automatically, in a scalable way. This neural network found an extra 12% of ad revenue hidden in the data.

Summary statistics are uninformative, and you’re leaving money on the table by not digging deeper into your data.