The dispute between Google and its “bad apples” involves technical details but, fundamentally, it’s about crummy products designed to force feed ads.

In the case of AVG, it worked like this: when people downloaded its free security software, they automatically received a “safe search” product (unless they were alert enough to uncheck a box during the download process). The unwanted product then installed itself on their browser toolbars as a default search engine. It also — and this is the critical part — served as a platform for AVG to collect money by showing ads, and proved about as easy to uninstall as resigning from the French Foreign Legion.

Veteran tech writer Emil Protalinski was first to report AVG’s hijinks last summer, calling it the “worst foistware I’ve ever seen.” Since then, investor site Seeking Alpha warned that AVG’s aggressive tactics to get ad revenue would lead Google, which supplies ads to AVG through its AdSense program, to take action. The search giant apparently decided it could do without AVG and, in its new policy announcement, explained:

For example, in the last 90 days, we have seen over 100,000 complaints about software that changed users’ browser settings or about toolbars that they couldn’t uninstall.  We want to avoid these kinds of bad user experiences.

Seeking Alpha also warned that if Google jilted AVG, the site would be forced to live on Yahoo ad revenue — which would bring in less money. This prospect appears to have had its intended effect on AVG.

The outgoing CEO of AVG, JR Smith, explained in a Monday phone interview that its “safe search” product was no longer bundled in a way that forced consumers to opt out. He added that AVG recently signed a new two-year deal with Google and said it complied with all the company’s conditions.

“They’ll send you default letters and kick you off the network,” said Smith, adding that Google takes tough lines to promote a clean ad ecosystem.

Google, in response to a request for comment, only repeated its policy announcement. Yahoo did not respond at all.

Tough line or just tip of the iceberg?

Google’s new policy aims to reign in AVG-style tactics by forcing software that uses its ad services to provide “one-click uninstall” and to go through a Google approval process. In the bigger picture, the policy appears to be part of a bigger effort by Google to clean up scammy ad practices involving toolbars.

While toolbars have legitimate uses, they can also be a vehicle for mischief. In addition to AVG, other public companies like Babylon have acquired a reputation for malware; the latter offers a translation program but the installation process can also lead to browser hijacking. Some techniques are even more nasty.

Certain scams typically invite users to download a program like “Find out who unfriended you on Facebook” but really serve to inject unauthorized ads. One example is Sambreel, a notorious ad outfit that forced its way onto the webpages of the New York Times and other prominent publishers last year; the hijacked ad space likely cost the Times and others millions in lost revenues. Even Google itself has been a victim through its YouTube video site.

An executive at a major publisher forwarded new screenshots this month like the one below which shows how a “Browse to Save” toolbar device (which claims to find deals for shoppers as they search the web) has used a Sambreel product to take over YouTube’s ads:

According to the executive, who did not want to be named, Google is doing the right thing but faces an uphill battle. While it can whip sites like AVG into line, unlike companies, sleazy actors will simply shop around for another ad exchange to do the dirty work. He said that some exchanges — which act as trading houses for digital ad inventory — often turn a blind eye to bad advertisers so long as they bring in money. In this bigger picture, Google appears to be trying to raise the bar in the industry in order to prevent a crisis of confidence in the online ads that are its lifeblood.

The toolbar policy comes at a critical juncture for the online display ad industry. On one hand, the industry recently suffered another black eye from a botnet scandal and may also be losing ground to the current mania for native advertising. On the other hand, Google, AOL, Facebook and others are developing a sophisticated suite of programmatic ad tools that could make the industry more efficient than ever before.

(Image by  BMJ via Shutterstock)

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• Content monetization: News licensing and syndication still need marketplaces and infrastructure
• Who and what to watch in the new era of the living room
• Facebook’s IPO filing: ideas and implications

Witness the ad industry’s reaction to claims it is dragging its feet on “Do Not Track,” a system to let consumers tell companies not to harvest their online personal information.

“We keep getting demagogued by the FTC,” Stu Ingis, the head of the Digital Advertising Alliance, told AdWeek. The remark came in response to a speech this week in which the head of the Federal Trade Commission suggested new rules might be on the way.

Does Ingis have reason to worry? Perhaps. Consider that after a year of relative silence, the dreaded (in the ad industry, anyway) “Do Not Track” phrase is back in the news on a regular basis. Next week, for instance, U.S. Senator Jay Rockefeller (D-W.Va)  is holding a public hearing to trumpet his do-not-track legislation. Meanwhile, Microsoft and Mozilla are setting their Internet Explorer and Firefox browsers to block third-party cookies — small, ubiquitous computer programs that record the web pages you visit.

For advertisers, these developments threaten to throw a wrench into an ad system that puts a high value on “retargeted” ads. Indeed, there is already evidence that fewer cookies means lower ad prices.

On the other hand, any type of sweeping law is unlikely. While politicians like beating the privacy drum, the details — and even the definition — of “Do Not Track” are hopelessly complicated. Companies like Google and Facebook (which are giant advertising firms) claim the term is confusing to consumers and, in some cases, have chosen to ignore browser instructions about cookies.

At the same time, these companies are doubling down on ad products that target individual users. Facebook, for instance, now lets customers buy ads based on third party data and is even partnering with retail outlets like drug stores to collect users’ offline shopping information. The social network and other tech companies, which have growing clout in Washington, will lobby hard to prevent any serious change to the current ad system.

Finally, the focus on cookies could ultimately prove to be a red herring in the”Do Not Track” debate. This is because more consumers are using mobile devices to access the internet — and these devices don’t rely on cookies for advertising in the first place.  In response, Apple has developed a sophisticated way for advertisers to target users’ devices while companies like TapAd and Adelphic have found ways to identify users no matter what device they use. In other words, any new “Do Not Track” law that targeted cookies would have a limited effect in a world where marketers follow us on our mobile devices.

The bottom line is there has been more sound and fury than usual about “Do Not Track” but that comprehensive privacy reforms are nowhere in sight.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• Consumer privacy in the mobile advertising era
• Controversy, courtrooms and the cloud in Q1
• Connected world: the consumer technology revolution

The stunt-driving video Wojicki cited, in which disguised racing star Jeff Gordon takes a car salesman for a harrowing ride, was produced for Pepsi and has earned the company a ton of free online attention. (The clip is entertaining but also fake, fake, fake).

“We’re moving to a model where the user is choosing to see the ads,” said Wojicki on Tuesday at Ad Tech, an industry conference in San Francisco. She added that tools like YouTube’s TrueView, in which marketers pay only if someone watches their video, are compelling to both advertisers and viewers.

According to Wojcicki, 70 percent of the ads in YouTube videos are now the TrueView type.

This vision of an online ad economy defined by user actions (such as clicks) is consistent with AdWords, Google’s lucrative search ad product that prices ads according to their relevance.

But while the TrueView experience bodes well for Google’s success in video advertising, the company’s prospects for mobile and social advertising are less rosy. So far, newer companies like Twitter and Facebook are dominating the space.

Wojcicki said that, for mobile, “a lot stuff doesn’t exist yet” and that there is a need to create more location based “block we’re on” ad experiences.

“The users first adopt (a technology), then advertisers figure it out.”

On the social media front, Neal Mohan, Google’s $100 million ad VP, noted that “social is an important part of all ads,” and pointed to recent initiatives like Google+ pages for brands and a tool that lets advertisers connect Google+ to their AdWords account.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• Who and what to watch in the new era of the living room
• Controversy, courtrooms and the cloud in Q1
• Content monetization: News licensing and syndication still need marketplaces and infrastructure

News of the latest tune-up came on Tuesday as Facebook announced it will let marketers buy “sponsored stories” in a user’s News Feed on the basis of websites that the user has previously visited. Until now, Facebook only let brands buy stories based on a user’s profile — which is created from information the user told Facebook (age, location, “Likes” and so on).

The opportunity to use so-called “retargeting” is likely to be a hit with advertisers who regard ads based on a person’s browsing history to be especially effective and who consider Facebook’s news feed to be prime real estate. One industry executive told AdExchanger that News Feed response rates are 10 to 50 times higher than ads on the right side of the page.

Facebook also said it will be selling the News Feed through its FBX exchange, which is like an automated real-time auction house where advertisers bid to appear on your Facebook page. Until now, the tool was only available for Facebook’s right-hand ads.

Building an ad juggernaut

Opening up the News Feed for retargeting is likely to yield a nice cash boost for Facebook but, in the bigger picture, the move is part of a larger story of the company’s efforts to use different forms of data to build an all-knowing ad juggernaut.

Facebook is also, for instance, combining its own data with offline marketing information to help companies hone in on customers. As the New York Times reported, the clothing company JackThreads matched its database of two million customer emails against Facebook’s own email records — and found that two-thirds of them were on the social network. For Facebook, such opportunities are just the tip of the iceberg; the company is also working to tap into offline data on a large scale by partnering with loyalty card programs that collect drugstores and supermarket information.

All of this means that Facebook’s advertisers will be able to draw on three powerful sources of data (browsing history, offline data and Facebook records) in order to blast ads into one of the prime locations on the web — users’ News Feeds. For Facebook investors, this prospect is especially enticing given that this model can transfer nicely to mobile devices where users are spending more and more of their time. Facebook isn’t allowing marketers to buy mobile ads on its exchange just yet, but it’s a safe bet this will happen soon.

In short, Facebook appears well on its way to create a marketers’ paradise and a torrent of ad revenues. But there are still two factors that could scuttle these plans. The first is the familiar spectre of increased privacy regulation – but that is a threat Facebook and others like Google  have so far swatted away successfully. Instead, the larger peril may be the prospect of too much advertising undermining Facebook’s user experience and its vaunted design. As this all-ads screenshot from today shows, Facebook still has a ways to go in cleaning up an ad experience that too often remains irrelevant and ugly:

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• NewNet Q4: Platform mania and social commerce shakeout
• Flash analysis: the future of Yahoo
• Sector RoadMap: Social customer service in 2013

Key highlights in this report include:

• For publishers, the first step to monetizing something is to be able to measure it. The analytics tools now available make it possible to track the spread of content on social platforms closely.

• Until now there has not been a marketplace where those potential buyers and sellers of content could meet. Nor were there adequate tools to enable verifiable transactions between them. Tools like Cascade and Ricochet are helping put the foundations of such a market in place.

• Both publishers and licensees will need to seize the sort of ad hoc syndication opportunities that arise online and on social media networks. One of the major tasks facing publishers over the next three to five years will be to structure their repositories of content so they can be searched, sorted, customized, repackaged, and accessed in real time via standardized APIs.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• The future of mobile: a segment analysis by GigaOM Pro
• Frenemy mine: The pros and cons of social partnerships for online media companies
• Social media in Q1: commerce and discovery dominated

It has also resulted in “toothbrushing.net,” a little-known site with oral hygiene news, likely racking up more visitors than famous publishers like The Economist.

Here are some more details, based on sources close to the investigation and other reports, about who is affected  and the scale of the botnet.

Millions of “readers” for a toothbrush news site

The “about” section of Toothbrushing.net says the site is dedicated to “toothbrush enthusiasts” and promises the “latest on dental news.” According to a source, the site is displaying 20 million to 25 million ad impressions a month. Since the site shows four ad slots on every webpage, this loosely translates to at least 5 million visitors. Nearly all of these visitors were bots not people but, for marketers, the effect is the same — they pay either way. (The ad slots were empty when I checked today  – see the screenshot at right).

To put the traffic of the toothbrush news site in perspective, consider that a site like the Economist had 1.7 million unique visitors in December and the New Yorker had 3.1 million. These figures refer to unique visitors so it’s not an apples-to-apples comparison but, using this crude calculation, there’s a good chance Toothbrushing.net did better than both of them.

And it’s not just the toothbrush site. Other obscure sites touched by bot networks, including Sodabottle.com and Techrockstar.com, likewise served up 20 million to 25 million ads in a month. According to Spider, the analytics firm that discovered the scam, there are at least 202 such sites tied to one bot network called Chameleon. Nearly all consist of little more than a smattering of cheap content you could pay a high-school student to write.

There is also the egregious example of Directorslive.com, an obscure movie site that AdWeek reports enjoys 326 million monthly pageviews. According to a source, the only site on the web to sell more ad impressions is Facebook.

Who is affected and who is responsible

The list of advertisers that paid to appear on botnet sites include dozens of major brands, and cover a wide range of sectors such as: retail (Snickers, Ziploc, Petco); finance (Citi, Chase, Amex); telecom (AT&T, Time Warner, Sprint); automotive: (Dodge, Ford, Jaguar); services (Zipcar, Seamless).

While advertisers are the direct victims of the botnet, major web publishers are also harmed because marketers lose confidence in the integrity of display advertising and prices drop accordingly.

So who is to blame? An advertising source provided six ad networks it regarded as among those it believed to be “problematic” because their sites receive suspicious traffic. Here are their names along with an example of a suspect websites they control:  Alphabird (Driverswhoknow.com); Digimogul (USBuildingDigest.com); Forward Health (Womenshealthbase.com); Precision Media (Toothbrushing.net); HiFi network (Dailyfreshies.com); Relevad Corporation (FFog.net).

(Clarification: the CEO of Spider.io says not all of these networks received visits from the Chameleon botnet but that they may be receiving traffic from other botnets or other “nefarious” sources).

The CEO of Digimogul told AdWeek that allegations of a connection to the bot network were “silly” and that “everything is by the book.” Meanwhile, executives from Alphabird told me that they were surprised by the discovery of the botnet and that they’re working with the London analytics firm to get to the bottom of it. Spider.io’s CEO says that AlphaBird has reached out “via LinkedIn” but that the companies are not working together.

I asked the COO of Alphabird, Justin Manes, how a company staffed by a sophisticated technology and marketing team could possibly remain unaware of the suspicious traffic — especially when the traffic delivered a direct financial benefit to them. Manes responded that Alpahbird works with numerous aggregators to buy website visitors and that the corrupted traffic must have slipped in this way; he also declined to say where the company bought the traffic. As for the company’s prospects in light of the botnet scandal, Manes said he hopes that people will come to see Alphaird was duped too; he also hopes the experience will strengthen the company’s ability to sell tagging and tracking tools in the future.

Fixing the problem: better tools or law enforcement?

The existence of a bot network that hijacks thousands of American computers to perpetrate millions of dollars in fraud appears to be a serious matter. But is it serious enough for a criminal investigation?

I called the Cyber-Crimes division of the FBI to ask if it is investigated this type of activity. A spokesperson provided this response:

“While I cannot comment about the botnet you’re writing about, the FBI’s Cyber Division does investigate botnets. We have had operational successes disrupting botnets used by individuals as well as groups that use malicious advertising as part of their schemes.” The spokesperson also referred to a 2011 investigation known as “Operation Ghostclick” in which the FBI arrested six people for using computers to manipulate the online ad industry.

It’s unclear at this point if the discovery of the new botnet will lead to any criminal investigations. In the meantime, brands and publishers will likely look to self-help solutions to weed out the fraud. There is some encouraging news on this front, as metrics companies like comScore are developing measurement tools to identify and screen-out what the company calls “non-human traffic.”

This story was updated on March 20 at 10pm and March 21 at 11am to provide clarifications from Spider.io’s CEO Douglas de Jager.

(Image by qvist via Shutterstock)

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• Content monetization: News licensing and syndication still need marketplaces and infrastructure
• Social fourth-quarter 2012 analysis
• How tomorrow’s mobile-centric data centers will look

To spur interest in new video ad formats, the Interactive Advertising Bureau on Monday announced five winners in a contest called “Rising Stars” that is intended to create standards for the emerging industry. The idea of the contest is also to facilitate large-scale video ad-buying through the creation of interactive formats that are “built to work on a range of video players across multiple devices.”

Here’s a short summary of the contest winners along with my quick thoughts:

Ad Control Bar: an interactive tool bar that sits in the ad, inviting viewers to interact (good idea — provided ad makers can come up with reasons for people to interact with an ad in the first place)

Filmstrip (shown at right): a “scrollable, multipanel, horizontal” ad unit that can be stuffed with ad content (is this content overload for someone already trying to watch a video?)

Extender: invites viewers to watch a longer version of the ad (not a bad idea but sounds like wishful thinking that viewers will use it)

TimeSync: a way to insert rich media at the right time in order to invite “interaction at the most appropriate moments”; a visual display of TimeSync shows an American Apparel ad appearing when hands clasp in a Bruno Mars video (this idea of context sensitive ads within a video is impressive but can it work in practice?)

FullScreen: a viewer who clicks will see screen taken over with “a full canvas of interaction possibilities, including more video, social and catalogs” (this sounds intrusive but could be valuable to an advertiser who gets it right).

The IAB only provided short descriptions of the winners so it’s unclear when (or if) these tools will get adopted by ad buyers. The potential seems enormous, however, as a recent report says online ad impressions grew 52 percent in the last quarter of 2012.

The contest winners included a broad spectrum of media and ad tech partners, including CBS Interactive, Microsoft, DoubleClick and Tremor Video.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• Controversy, courtrooms and the cloud in Q1
• Connected consumer third-quarter 2012
• Who and what to watch in the new era of the living room

Google’s Chief Business Officer Nikesh Arora, however, indicated at the Dive into Media conference in southern California that the company has no intention of changing course. Arora brushed off a question I posed about whether Google would reinstate the more granular ad options, and emphasized that the new AdWords system meant advertisers would no longer have to run hundreds of different ad campaigns to target all demographics and devices.

The changes to AdWords are important since Google has such an out-sized footprint in online advertising and because everyone is watching how the company is responding to internet users’ large-scale migration to mobile devices. So far, ads on smaller screens have proved much less lucrative than desktop ads — frustrating publishers and worrying Google investors.

Google announced its response last week in the form of a blog post describing the new “Enhanced Campaigns.” The new system means mobile ad purchases will be a default option when people sign up to buy Google AdWords (though they can turn the mobile part off). Another major change is Google’s decision to treat tablets and desktops as the same device for ad purposes. This goes against the philosophy of advertisers and publishers who consider the experience tablet a distinct, more immersive experience (though the distinction may be less when, as here, it concerns internet searching).

Ad industry blogs like 360i and AdExchanger have noted that Google is moving away from more granular forms of marketing while advertisers have complained about a loss of control.

On the flip side, some Google watchers have praised Enhanced Campaigns as a much-needed way to consolidate ad campaigns, and to make it easier for small ad buyers to join the mobile ad-buying landscape. At the All Things D event, Arora described how Google is also making it easier for advertisers to take advantage of distinct, new ad options associated with mobile — such as overlaying time and geographic location onto search queries.

In this context, the simplified options make sense, especially as advertisers right now confront the prospect of having to run hundreds of separate AdWords campaigns to account for all demographics and devices. And the Enhanced Campaigns will no doubt juice Google’s ad prices as more people join the mobile auction market, creating more competition for local searches such as “pizza Brooklyn Park Slope.” 

But overall, the new system still seems to shortchange publishers and advertisers. Tablets provide a unique user experience and could be a fount of advertising innovation; Google’s decision eliminates some of this potential. A better option would have been to unveil the Enhanced Campaign system but to also make the older, more granular options available to those who ask for it.

(Note: We’re going to be talking about alternative monetization strategies at our paidContent Live conference in New York on April 17).

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• Content monetization: News licensing and syndication still need marketplaces and infrastructure
• Controversy, courtrooms and the cloud in Q1
• Flash analysis: future opportunities for Pinterest

Then something happened. The company’s revenues grew, its share price soared and CEO Tim Armstrong revealed a strategy to make AOL a media and advertising powerhouse. The company’s winning streak continued Friday morning as Wall Street greeted AOL’s latest earnings report with glee; the stock shot up another 12 percent when markets opened.

“We’ve walked through the the value of the turnaround and got to growth,” Armstrong said on a morning call with investors.

It’s too soon to say the company’s back on top but, for now, the results look like the real deal. Here’s why: as analysts fussed over AOL’s debacle with hyper-local site Patch and its dwindling dial-up business, the company quietly invested in state-of-the-art ad technology and rejigged AOL to inject new revenue streams. The most important of these are inside the AOL Networks group — a business unit that offers ad tech tools to publishers and advertising agencies that are still learning to navigate the world of automated ad buying. The Networks group grew 37 percent year-over-year and posted revenue of $183.5 million in Q4. (Total revenue for AOL in the quarter was up 4% from a year ago to $599 million; adjusted OIBDA income was down 7% to $123 million).

During this time, AOL has also become number two in online video thanks to products like HuffPo Live; this is significant because video is one of the most lucrative forms of online advertising. AOL now plans to draw on its fancy ad tools to create automated buying for its own video inventory while, at the same time, offering those tools to other companies who are still catching up on the video front.

Meanwhile, AOL’s media properties don’t look as dysfunctional as they did a year ago. Armstrong appears to have figured out how to manage the mercurial Arianna Huffington and, as for his pet project Patch, the hyper-local site is still losing money but he promises it will be profitable by the end of  the year.

The bottom line is that AOL has three major revenue streams, all of which look viable. There are still danger signs, of course: AOL’s display ad business looks shaky and, as Henry Blodget points out, the company’s revenues may come from three streams but nearly all of the profit is still coming from the legacy subscriber businesses.

But, for now, investors are right to like what they see. People looking for 2013′s turnaround story should stop fussing over Yahoo — it’s AOL that is poised to be this year’s comeback kid.

Correction: an earlier version misstated the sale of AOL’s patents; this has been corrected to say the patents were sold to Microsoft (which in turn sold them to Facebook).

Disclosure: GigaOM distributes some video content through AOL.

(Image by Rob Hainer via Shutterstock)

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• Connected world: the consumer technology revolution
• NewNet Q3: Facebook remakes headlines in social media
• NewNet Q1: Content Farms and Niche Networks on the Rise

The offering, which comes at a time when paywalls are finally getting more traction, shows how publishers are also persuading visitors to pay not just with money but with their time.

As my colleague Mathew Ingram explained yesterday, we’re seeing growth in so-called “survey walls” in which websites ask visitors to fill out a survey before seeing a piece of content. The option is a good way to earn revenue from the large number of people who can’t or won’t pay for content. It also presents an attractive option for mobile platforms where more and more people are consuming content.

Selectable Media, for instance, works with both advertisers and publishers to drop a selection of short video ads in front of a story, game or video. The idea is that you have to have to watch the ad till the end before you get to see what you want; viewers get to choose which ad they want to see. While this resembles what happens when a user sees a pre-roll video on YouTube, the difference is that Selectable can plunk the clips into non-video fare like games or text articles. It also offers several different ads at once, in the hopes of increasing user engagement in the ad.

On mobile devices, it looks like this:

CEO Matt Minoff said by phone that, for now, Selectable’s primary customer for mobile is online game maker 50 Cubes which asks players to watch Google and Adobe ads in exchange for game goodies. But he says the company is negotiating deals with news sites and other traditional web publishers.

The company, which has clients like HBO and Kraft and is backed by Microsoft’s Bing fund, says nearly all users watch the ads till the end. It also claims that click-through rates and brand recall is far higher than for standard pre-roll videos. Asked whether the possibility of slow-loading videos on mobile devices might be a turn-off for publishers and users, Minoff says video optimization tools and fast phone networks mean this isn’t a big issue.

So will all this work as an alternative to paywalls? It’s too soon to say but the concept seems sound — especially as publishers keep looking for a mobile advertising strategy that doesn’t involve low-priced display ads. Users too will likely get more familiar with the option of trading their time for content; the concept is now being echoed not just with “survey walls” but with projects like star director Roland Emmerich’s idea of rewarding viewers for watching movie trailers.

(Image by Eduard Stelmakh via Shutterstock)

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• NewNet Q4: Platform mania and social commerce shakeout
• Content monetization: News licensing and syndication still need marketplaces and infrastructure
• The state of cross-platform media measurement