Comments on: 80legs Cares About Your Bandwidth Cap

By: davequick

davequick — Fri, 13 Nov 2009 20:22:09 +0000

“If they charge $2/million urls -but only pay users $2.60 a month – that doesn’t quite seem right. It costs more than $2.60 a month to pay for the power to run the computer. ”

you’re mixing units.

Amazon:
- small linux instances are $227/year a piece = $0.03 per machine hour
- Then they are charging $2 per million pages and since the average web page is ~25k of stuff if you ignore the images and scripts and focus on core html and don’t pull in images and style sheets.
- Even if they were running this out of AWS – (25,000 bytes per page * 1,000,000 pages) bytes / $2 for this million pages / 1,000,000,000 bytes per GB = 12.5GB bytes per dollar of ingress that they charge…. Amazon’s 10 cents per GB = 10GB per dollar amazon charges – so it’s basically costing them a little bit if they have only one customer paying their $2/million pages otherwise if they are dual purposing the single crawl for more than one customer they are printing money
- Given that they are usikng 50k “nodes” bandwidth and compute tmie for a bunch of this you’re really likely to have profitability even if they have one taker and small overhead (i.e. small staff)

Or am I missing something?

just my swag.

By: Matt Ellsworth

Matt Ellsworth — Wed, 30 Sep 2009 15:48:18 +0000

interesting concept. Found their sit because their bot hit our site and I had never heard of it before. Then when I saw that it is a web crawl for hire, kind of tough to decide what the bot is doing on the site and whether or not its a good thing or a bad one.

If they charge $2/million urls -but only pay users $2.60 a month – that doesn’t quite seem right. It costs more than $2.60 a month to pay for the power to run the computer.

When we used to participate in the MJ12 node program, the comptuer could easily crawl a million urls a day – the problem actually became that home networking equipment isn’t setup to handle it as easy as the computer is.

By: 80Legs Is Where SETI@home Meets Google

80Legs Is Where SETI@home Meets Google — Wed, 23 Sep 2009 01:01:47 +0000

[...] By Stacey Higginbotham | Tuesday, September 22, 2009 | 6:00 PM PT | 0 comments 80Legs, a startup that aggregates CPU cycles and bandwidth to offer companies a Google-like web crawling service on demand, today launched its public beta at the DEMO conference in San Diego. The Houston-based company has raised $400,000 from Creeris Ventures to provide its web crawling service. 80Legs contracts with another Houston-based company funded by its investors to get access to volunteer’s CPU cycles, much like the SETI@home project, in which PC users offer up computer power to search for signals from extraterrestrials. 80Legs partner Plura Processing, which aggregates the cycles, pays affiliates to sign up users to volunteer their idle processing time in exchange for services like virtual gifts. Users can also give up bandwidth, which we wrote about earlier this year. [...]