MetaMuse

Behavioral Profiling: Online vs. Offline

Despite continued sensationalistic reporting on cookies, behavioral profiling and lack of online privacy, I've long held the belief that my online behavior affords me far more privacy than my offline behavior. It is in this context that I took interest to the recent Businessweek article "Digital Snoops Have Nothing on Joe Salesclerk", which contrasts Web-based behavioral profiling with offline behavioral profiling. The punch line? Yes computers are tracking your online behavior, but in terms of surveillance sophistication you can't beat a good salesperson at a brick-and-mortar store.

Clearly, online behavior is valuable data -- which many companies (such as my own) is analyzing.

We may be just browsing or comparison shopping online, but our Web wanderings paint a picture of who we are, what we desire, where we go. In all, we create a vast laboratory of human behavior for the data miners at the e-merchants, portals, and search engines.

However, this "behavioral data mining" is unavoidable, offline or online. Every purchase we make via credit/debit card, every building we enter, everyone we talk to over the phone, even our homes and the streets we drive down -- are recorded in some way, increasingly so. And there are companies that aggregate/store all this data in a way that's personally identifiable. In many ways, our offline behavior is far more open and publicly available than our online behavior. (It's amazing to me why people get so upset over browser cookies.)

Computers can search for correlations between the Web pages we look at, the articles we read, the ads we click, even the size of our e-mail communities and our instant-chat habits. This may feel intrusive, but remember: All of this observation and analysis is done by machines. Most of these analysts—the people I call the numerati—recognize us only as patterns. Their computers see us as dots in a universe of millions. [edited for brevity]

Now compare that with offline "behavioral profiling", in stores and neighborhoods.

Walk through the holly-bedecked doorways, and store owners immediately note your race, your clothes, the way you walk. Maybe they can see your car out the window. Within seconds and without conscious thought, most will draw conclusions about your social status, your income, maybe even your religion, drinking habits, and sexual orientation. (This is a level of analysis eons beyond the power of the brainiest computers.) These insightful humans, their brains busily linking you to other people they have known, heard of, read about, or even smelled, will then guide you toward the kitchen appliances, books, or tools they suspect you'll like. This attention, which on the Web would be called "targeting," is known in the physical marketplace as the "personal touch."

What do you think? Do you feel more anonymous online than offline, or are you much more concerned about your privacy online in comparison?

Actual vs. Perceived Progress

When you spend all your time at a "low altitude" elbow-deep in code (or similar deep effort), it's easy to lose site of the difference between actual progress and perceived progress. External stakeholders are often only  concerned with perceiving progress, not measuring ever little detail of actual progress.  And in many cases, it doesn't matter how much actual progress you've made if the critical stakeholders can't SEE it -- even if what they see is completely trivial and/or simplified. In fact, oftentimes the more trivial the indicator the better off you are. Simple visual indicators are the best indicators of progress, especially when the individual components to progress are complex.

For instance, check out this simple visual of an evolutionary algorithm evolving the design of a car (in Flash) to traverse the landscape without the red circles touching the ground. It's a simple visual demonstrating what I'd guess to be a complex set of code. (Evolutionary algorithms are optimization algorithms designed to evolve/mutate on their own, using mechanisms inspired by biological evolution: reproduction, mutation, recombination, and selection.)
In business, one thing you can count on is that everyone is extremely busy. Therefore, it's important to apply the fruits of your efforts to visual metaphor, so that stakeholders (investors, customers, partners, etc.) can empirically see their progress without even caring about how it was done. For board meetings, they're called KPIs (key performance indicators). It gets harder though when software engineers and business people need to communicate. This is why I think it's so important for engineers to learn to SHOW their progress, instead of simply achieve progress. No one else is going to care what you did or how you did it, until/unless you can demonstrate it. The perception of progress, therefore, is much more important than the actual progress.

Online Advertising Market Growth Still?

I saw the eMarketer online ad spending forecast just a couple days ago, along with John's note that online advertising was "taking a hit" (since when does continued growth mean taking a hit?). Looking at the forecast, obviously revised in light of the current economy, I wondered how these numbers differed from their online ad market forecast from December 2007 -- one year ago.

Here is eMarketer's online ad spending predictions from 2007-2011 side by side:

Clearly a bit different outlook, don't you think? I do think while this is currently more realistic, the actual numbers will beat these.  Here are their key takeaways combined with mine:

• [eMarketer] Continued growth. [Me] Yup, I agree. Online ad spend per household still hasn't even exceeded newspaper, direct mail and telemarketing ad spend per household, has it?
• [eMarketer] Display is suffering, but slight growth will come as the impression volume increase  outbalances the  decrease in CPM rates. [Me] Maybe. I would guess that display will be flat, unless ad networks and major publishers finally adopt more precise targeting (for improved advertiser ROI). Comparing search with display ads, rates are already so cheap and volume more plentiful -- lots of room for innovation here to the benefit of all advertisers.
• [eMarketer] Search will grow from 40% of the market to 50%, because it's much more trackable. [Me] Yes, it's much more trackable, but I'm not sure search volume is growing dramatically and I think that's the key to growing the market. Page view growth is in other areas of the Web, where display ads are currently. What if you could make that inventory much more trackable, with even better targeting than search? 
• [Me] 2009 will be the year when behavioral targeting will no longer be NEW, it will be offered by every ad network and will be a component in a majority of campaigns, and it will drive a rebirth in display advertising.

To that last point, in the last month I've spoken or met with 25% of the 50 top US ad networks, and 92% of them have already executed on a behavioral data strategy (more than retargeting) or will roll it out in the 1st half of 2009.

Let me just close with a forecast of US online advertising spending by format:

Largest Ad Networks in US

Just in case any of you ever wondered (like me) who were the top ad networks in the US, ranked by number of unique visitors and reach, click on the link you just passed or the image to the right.

Economist: Internet advertising will be relatively unscathed in the downturn

Ok, so yeah it's natural that every entrepreneur likes to point out the research and media articles that makes them feel good. The recent article in the Economist regarding the online advertising market is no exception. Key takeaways:

• Between 2000 and 2002, during the dotcom recession, online ad spending in the US fell by 27%.
• Today online advertising is much less speculative -- given the shift to SEM, performance-based pricing, and measurability -- so that it starts to be treated as a cost of sales.
• Online advertising will continue to expand during the recession. eMarketer predicts that the market will grow by 8.9% in 2009.
• Industry is cautiously excited about two areas which to date have proved ineffective and therefore cheap: video and social media.
• As eyeballs continue to migrate towards the sites where advertising hasn't quite succeeded, that clearly represents an opportunity for innovation and arbitrage.