Web 2.0 & Death of the Network Engineer

Internet business is highly layered today. And Web 2.0 is qualified to be an application layer and really don’t need to worry more about the infrastructure. They can start from commodity hosting plan as “electricity”. However, if they dig into depth with growing number of users and joint social connections between those users, they have to pay more attention on the peak of networking. Like the problem with Friendster two years ago.

Interesting post. I tend to agree with the CTO … if they’re plugging into something like Amazon’s server services.

If not, however, they better know something about the mechanics of how their solution will be delivered. Even a trucking company cares about the road conditions, which routes are faster, and so on.

If the company is at the stage of concept building, the CTO may more focus on the application technology like J2EE or PHP or Rails. Until they want they have beta users and really concern about operation, then networks knowledge is must.

The internet is like electricity – you need to understand it well once your needs scale. The same is true for heating, hardware failure rates and disk throughput. Most companies will never get to the critical mass where these are even an issue (it’s amazing what you can do with even just one server these days). However, from my experience once you do you find good network engineers hard to find.

I tend to agree with you about what lies outside the rack being largely irrelevant to most upstart web application/service providers, but I wouldn’t go so far as to throw load balancers, switches, and other technologies clearly within the control of the company into that mix.

Do I care about the internals of how my co-lo communicates with the outside world? Probably not. But do I care how my Netscaler distributes traffic between all of my own servers? Hell yes. Some people may do without a traditional load balancer in favor of robin-robin DNS (WordPress.com and Bloglines do this) but even in that case, you need to know what it does and how it works.

The other option, of course, is running everything on one machine, which a low-volume company can get away with, but you almost don’t even need a CTO for that sort of company. Or you can use managed hosting, which is extremely expensive.

I guess my point is, I would separate pools of knowledge into “outside the rack” and “inside the rack”. Does outside the rack knowledge matter a lot less now? Sure. Inside the rack? Probably not.

The problem with electricity is that maybe we take it a little too much for granted. Lack of choice, ever increasing costs, rolling blackouts anyone? Is this really what we want in terms of Internet?

Has the internet reached a level of abstraction comparable to plugging into an electrical socket?

If one where a “CTO”, I would hope the answer is no. A CTO really should be aware of not only the application his/her company is providing but the pipes that connect that application to the outside world. Does this mean knowledge of things down at the packet level? No. But, if something goes wrong I would hope that the CTO has the knowledge on where to start troubleshooting. This includes attacks (security), performance (servers/bandwidth), and overall availability (all of the above).

Title are thrown around a little too easily in this day and age. However, for large scale deployments, knowing what is going on from application down to IP packets is critical to making sure one’s application is on a stable infrastructure.

It all depends on what scale they’re operating at. I had a small Web 2.0 startup that was acquired by a much larger company. When we were operating with ~1M pageviews of basic web stuff, it was all a service.

Now that I’m working on services that are constantly pumping out many Gb/s of data, it’s entirely different. I still mostly deal with the software architecture, but I certainly do talk to our ops folks on a regular basis. I make sure I know what’s up with everything from our various POPs and fiber to Netscalers and Netapps.

Beyond a few special deals we have, do I care about routing tables and the like? Not really, our providers handle most of that. It’s still useful to know so we can make back-of-the-envelope guesses when designing new services.

Is this CTO implicitly saying he doesn’t think he’ll get significant usage?

These days you can add month’s use of a dedicated server for the cost of an hour worth of labor. Focus on what differentiates your product. In some business that’s the network infrastructure. But most startups are not in that business these days. That doesn’t mean you should ignore the design of your network, and not have a plan to scale. But doing so at the expense of developing your core product is premature optimization. Make it work first, have a plan to scale, and focus on that plan when the time comes, not before. There’s a tipping point when it’s smart to optimize your application instead of doubling the number of servers needed to run it. Both extreme denial of the need to understand network optimization and extreme focus on network optimization are silly moves.

For a brand-new startup that doesn’t know if it has a hit or a dud on its hands, they can certainly find out first and worry about scaling later.

But ask any so-called Web 2.0 company with scale, and they’ll tell you figuring out the server- and network-level scaling is both vital and difficult. Leaving it to someone else who doesn’t intimately know the details of your app is very risky and error-prone.

I’d say that a scenario like you described sounds like nirvana. I’d love to be there. But currently, it’s just a dream.

Oh, and you’ll also find that “electricity” is no longer a faceless utility but a very real problem for a popular web app. Power and heat densities (which is really another way of saying power and power :) are something we spend a lot of time thinking and dealing with.

Interesting. I don’t think internet is there yet. If a CTO of a web 2.0 startup doesn’t know what OSI stack is and how her/his network works, that should be fine as far as s/he is very well aware of how important her/his network is to her/his business. Also, s/he better have a strong VP or director of operations who has a good network engineer in their IT team if the network matters.

Allan,

I hope you didn’t invest in that company.

How could a CTO not have network and serverinfrastructure plans? He or she must not have been around in the 90s when boat loads of $$ were invested in the MSP/ASPs of web1.0 So what, even if Amazon, Salesforce.com, hosted app providers and/or SaaS providers and their customers now actually have access to the infrastructure which supports their requirements, you can’t just solve on of the biggest problem of the last ten years, network and infrastructure scaling, and walk away. Amazon doesn’t have 100% uptime. It and everyone else will go down. Don’t you need at least n+1 redundancy to be legit these days? What, is the CFO of that company developing it’s redudancy plans?

Even if you can plug in to electricity, network connections, computing/processing on demand, etc you still need to know what your plugging in to and if its right for you.

Great post. The net has delivered a backbone of utility that has been amazing for startups. My prediction for successful web 2.0 companies: the innovation will come from mastery of network theory.

Coming from a company that hosts many web 2.0 and SaaS applications, I know all to well the CTO that thinks hosting is a utility like electricity.

I believe that statement is true at the low end – but not when hosting a serious application. An example that comes to mind is when a client wanted full redundancy in their architecture and it all fully load balanced. Sounds standard – we will spec and build. Of course our network engineering team asked all the usual questions – but when it came down to going live – we discovered the application needed layer 7 load balancing, not layer 4 – which our devices provide. A layer 7 load balancing is at least 4 times more expensive.

This is clearly a case where the client’s lack of understanding is going to have an impact on their deployment time as well as a financial impact.

very cool post.

i guess it depends on who you are. if you are walmart you care about the details of your utility network in the same way that if you are youtube or flickr you care about the details of your network infrastructure.

however, if you just open a new retail boutique you would be crazy to care about details of your utility providers, b/c you have more important things to care about that will make or brake your business.

when things go well and you open, say, your tenth store, you may start looking into how you can save on utilities and make sure everything is reliable.

but all that said, i do think that generally speaking in-depth knowledge of network infrastructure matters a lot less today than it mattered 5 years ago for a web start-up. but i may be biased b/c we have been running a web start-up on a t-mobile hotspot network for almost a year already :)

Allan great post with a real curve ball at the end.

I think you’ve seen from the responses that a web2.0 startup company can survive with a very simple internet provision but will certainly need to have a plan to deal with growth – wasn’t twitter down most of last week due to traffic loads?

As for you end paragraph this battle is a very different one and is aimed squarely at Cisco’s enterprise market. They have a virtual monopoly over the enterprise router market because 10-15 years ago they built a box that integrated all the different protocols such as IP/IPX/Decnet/Appletalk/DLSW and simplified the network for an enterprise.

Time has moved on and I think the enterprise market is looking for a new router that integrates all the latest developments such as VoiP, QoS, Security, VLANs etc. Cisco may say they have a product but I think there is a real opportunity for a startup to develop such a router – especially now as companies are looking to refresh and move to IP VPN networks.

This is definitely an interesting and topical subject. I’m currently running a small startup and we treat our network, databases, and servers as a combined service when doing low-usage prototypes, some parts of beta-testing, and some low-usage production services. This allows us to test and design functionality independently of deployment design, and in the beginning stages of a project it allows for very quick iterations through feature-sets. It also keeps us from wasting time on premature optimization of services before their functionality is fleshed out.

However, once we start looking at production servers, appropriate network and server design still matters because it significantly affects our operational cost/user. Having come from a network management company (Micromuse), I’ve seen the operational impact of network, database, and server design (good and bad) when scaling. Interestingly, some services such as Amazon S3 and some types of hosted servers are configurable and scalable enough to give a bit more leeway when initially deploying a product and I look forward to taking advantage of these services in the future.

I have to agree with what most people have already said – if you are a CTO, and you are taking the hosting for your web2.0 app as that much of a non-issue, either you have no ambition for your product to become widely used, or you really don’t understand enough to calling yourself a CTO.

Right now, I’m working as lead dev for a stealthed web2.0 startup (ohgnoes! not another one!), and the one of the biggest things on my mind is “how do we write this so that its going to scale well, and what is the architecture (in terms of hardware, software and networking) that’s going to help us in doing that?”. If you aren’t at least asking questions (and know what they are), your startup isn’t going to go far in the long run.

I think this is like the popular question: “What was first? The Chick or the Egg?” I’m working as COO for the Network Department of my Company. What I see is, that WEb2.0 wirh it’s Applications is THE NET. When you take a look at Cisco and some other Network Vendors, you can see that the Good old Boxes are no Routers anymore. Today we have Boxes wirh a lot of Functions/Applications. Virtualization (NAS/SAN)etc. are only some further Examples. Take a look on Cisco, what they are working on!? What would you say is Google? Is it a Network Application, a ISP, an ASP? So I would say (as a proud Networker) that the next Generation CEO’s will be from the Network Department. ;-)

CTO 2.0?

“I don’t need to know any of the details, it just works!” How many times did we hear this kind of nonsense during the dot.com boom?

As many of the comments above have pointed out, any CTO worth his salary would understand why this stuff is not simply plug & play. The network is not quite that simple yet (although apparently investors are).

Interesting post and conversation. I’m also biased by an engineering background, but just founded a startup and we’ve been considering scalability and performance since day one, even though our architecture does include Amazon S3 and EC2. True it is some degree of wasted time if we never have to scale, but it would be much more costly and inefficient to try to redesign/rebuild for scale/performance down the road w/ live customers to support in the meantime than to understand what it will take from the start. This doesn’t mean initially building a massive scaleable architecure, but it does mean putting a foundation in place that can eventually scale. I agree this isn’t nearly as hard as it was even 5 years ago, but I don’t think we can yet plug in and hope for the best.

I’m in the same place as the CTO. I was most recently the most knowledgeable developer in a small agency, and was promised an ultimate Director role. I’m a developer and a strategy guy who inherited a hosting environment in a nearby coloc. I hated it. I can master and do a lot of things, including server admin, but my eyes would glaze over every time our network consultant started talking hardware. If the path wasn’t pre-selected, I would’ve recommended farming everything out to RackSpace.

I’ve got reliable hardware at home, reliable macs, a dedicated virtual at MediaTemple, and know enough about scalability to make smart recommendations to my clients. What more do I need?

It’s like Jeff Bezos saying: “At Amazon, no, we don’t care about shipping”. At the end of the day, if you’re a web start-up you ship packets. And if packets get lost, fail to route properly, and you get ongoing re-tries that clog your que (be it db, tomcat or whatever) than your users will have a poor experience. This is especially true on the web, where you can get bursts very easily. Besides, current Internet research projects (e.g. http://www.netdimes.org) which constantly monitor the Internet teach us that we’re the low level infrastructure is still miles away from being as reliable as the electric network or the water supply network. At the end of the day, god is always in the details.

We see this every day, but that is how we come in place many times. The programmers / Ceo or CTO doesn’t know what they need and how much better something can run if run with the proper network setup.

We have the privilege to help a few Web2.0 come to the success. Most of them started in the same situation the CTO say. Everything changed when they started to grow and they needed to scale. They started to move some big amount of internal traffic they upgraded there network. Same happened when you want to deploy a complex database solution like a cluster.

Like always, this is very related to the size of the web2.0 company, but at some point you will always need a network engineer.

Really interesting both- I think the CTO you talked to is a result of what Web 2.0 companies are- true Web 2.0 companies, I think, are media companies, not technology companies. Conde Nast worries about the paper they are printing on, but they worry about what they are printing more.

Hi folks,

Thanks for all of the great comments! I’m currently on business travel and have not had as much time to reply here as I would have liked.

I truly appreciate both sides of this discussion, although my network engineering roots still makes me lean towards wanting a CTO that understands details about their network infrastructure. There still seems to be some science in tuning your router, load-balancer, firewall, and so forth to make web2.0 applications perform and scale well. That being said, I know startups are using Amazon’s EC2 and S3 with some good success. And I know of more than a few web2.0 companies that plug their rack of servers into a xSP provided Ethernet port and that’s it.

And no, tomo, I did not invest in the company that sparked this discussion. Maybe if the CTO changed roles to VP of Marketing, I might change my mind :)

I’ll try to get back here and respond to more comments after my cross-country flight.

John Furrier – you said “the innovation will come from mastery of network theory” for web2.0 companies. Interesting thought! If you’re working on a web2.0 company that leverages network theory, that would be very interesting to me….

I think part of this is the emergence of application hosting companies, which you’ve written about recently.

http://gigaom.com/2007/02/26/engineyard/

Many of our customers DO know about this stuff from past ventures, but are satisfied that we know as much or more than they do and are happy to pass the load onto someone else.

For the small to medium company, a competent ISP and hosting company is probably all that’s necessary. But as video and other high-bandwidth applications become more prevalent, design and optimization become more important. The point at which it becomes necessary and/or cost effective to optimize has moved much higher on the complexity scale, as the networking devices have gotten more intelligent and the bandwidth costs have gotten cheaper.

That CTO needs to experience the Digg Effect (http://en.wikipedia.org/wiki/Digg_effect). I bet he will be singing an entirely different tune after that!

Everytime I visit Digg even for a few minutes, I come across websites which are slow or down due to the Digg Effect. That should tell us all something.

The CTO should have a fundamental understanding of technology concepts, not the operations part. Do I want a CTO doing server stuff? Not really. That’s why he’s got a server staff. The CTO should be working with the CIO to align itself with business goals.

Web 2.0 companies are slightly different, but not really. A good CTO will understand how the core technologies work and know enough to be dangerous, but be smart enough to let the folks who are server/web/desktop folks do their thing.

I’d say this CTO has had little experience of scaling issues – doesn’t know what he/she doesn’t know. Its not uncommon with startups in our experience, we’ve commented on this before

Also imho there is still quite a gap between ASP/Hosters, the Amazon Grid etc, and what you need to do to run an industrial grade service.

To a certain extent he is correct in that the internet has be “utilitized” to the average consumer, they can just plug in and expect a level of service, security and reliability. Just like you plug an appliance into the electrical outlet in your home.

However, using the same analogy, a large factory doesn’t just plug its machines into the wall and count on the electric company. They plan and engineer to make sure their needs are meet, and often gain some competitive advantage in the process.

I would say the CTO is a bit out of his depth, which is a common problem (second only to how do they actually plan to make and money) amongst the Web 2.0 bubble companies.

This is on point to some discussions from Fall Web2.0. Many companies are emerging with tools and platforms that do the “heavy lifting.” Amazon is certainly one, Salesforce.com another, there are many others listed at Programable Web.

The network engineering/server management aspects and such – I think Marc Andressen and crew had this vision back in 1999 with Loudcloud (now Opsware) and certainly other companies like Opsware have since emerged in this business area.

Likewise the new company Bianchini and Andressen founded – Ning, sort of goes to the next level. More heavy lifting is done for you and it is just the customization that needs to occur. Ning is just one example, there are several startups out there creating all aspects from the Network engineering level, host management, source control, project planning, etc.

Essentially the size of team you need to start and maintain a business has scaled down. Your capital expenses have been lowered, your upkeep costs are lowered, etc.

Look at Craigslist as one example of a company operating on a small staff and doing very well. Not sure if they have or do not have their own network engineer, but it is much cheaper to “time share” a network engineer then it is to have one on staff. Good NE’s have lots of value – if you can first find top notch NE’s who are already not employed and second afford to utilize them 100% of the time. I think most startups would have trouble with both 1 and 2 and therefore could better utilize their $$ for other areas of the business, relying instead on a shared NE.

I think that is an aspirational statement -if it is not a Utility yet it ought to be and it is a shame that it is not yet.

Ever since web hosting became decoupled from a company’s internet access pipe back in the mid 90s, the industry has been evolving in the utility direction from the bottom of the stack up: place, power, pipe, ping, processor, petabytes…

As has been pointed out, you can already do that at low end of the scale. At high end, you still have to piece the high end networking together and you do want one neck to wring when things go wrong. Good luck trying to build it in house, there are only so many clue-full network architects – much better to find a trusted partner who can do it for you and stay focused on what differentiates you.

You should only attempt complete vertical integration if you have scale that demands it or if losing control of your complete supply chain will be the death of your business. Then you might even have to build your own power plants, hedge on oil price, and dig your own trenches for fiber!

That said, it does not absolve the CTO/CIO/VP Ops of the ultimate responsibility for delivering reliable infrastructure for the company.

I hope you didn’t invest in these cats if that’s the CTO’s take on his product.

As the CTO of a startup with a zillion dollar marketing budget, I wake up at night in cold sweats worrying about waves of Digg and Slashdot traffic pummeling our product to kingdom-come. To hear a fellow “CTO” be so flippant about optimization/scale is embarrassing to say the least. These issues should be at the forefront of their day to day work life, they are doing their investors a disservice by being so “bold” about it. In fact, I don’t think his statement was bold at all, I think it was straight up stupid.

Let’s look at Friendster for an example of what happens when you don’t plan for load and scale and expecting that you can throw metal at the problem to fix it. Their inability to scale took them out of the equation and now they’re fighting to reclaim space that they once ruled.

My first month on the job was spent making sure the site we inherited from the agency building it was built to withstand hurricane Katrina levels of traffic. I have eight different contingency plan documents, scores of consulting companies on retainer to help fire fight when the need arises, as well as exhaustive audits of our codebase for every ounce of optimization we can squeeze out and documentation of problem areas we don’t have the man power to cover as we march towards launch.

This CTO is a moron, regardless of credentials. I wish him the best of luck when he’s on the phone bitching at his service provider to get more web servers up while traffic rains down like hell fire.

Maybe I’m old school though.

For those that have commented on the CTO role and how that person in the organization can be more evangelistic rather than technical, I think that is a good point. A CTO with an exceptional supporting staff can be more high level visionary than the person I met with. That being said, the CTO of the web2.0 company I met with was one of three technology people in the company at the time.

Also, the points about building a network infrastructure to be able to scale a web2.0 application to handle being popular on Digg (or slashdot.org) are well taken.

Allan,

Good post. The CTO you have talked with is, of course, a quack. But, he has a point. It is dead WRONG to start a company in 2007 thinking in the terms of a decade earlier. He SHOULD look at the Internet as electricity and try to figure out how to scale with this approach.

If you want to know what’s possible, check this: in the last couple of days, Bert Armijo, who you know well, assembled a grid with 86 processors, 150GB of RAM, 28TB of usable storage and 4.4Gbps of bandwidth to the internet. He had never seen the hardware, nor have the folks in the data center seen any of our applications. He then demonstrated 33 n-tier applications running, comprised of 297 virtual appliances and 759 volumes – all of which are consuming 64 processors and 67GB of memory.

As you know, Bert is 3Tera’s VP Marketing. True, he used to write code for a living, but that was 15 years ago. If a marketeer can do this, imagine what real engineers at a Web 2.0 startup can make happen with a technology like this.

The future is closer than it appears. Utility computing is here. Virtual private datacenters are selling as fast as our partners can build them. Large telcos are gearing up to deliver them as well.

Vlad

Why would you waste the time of GigaOm’s readers asking such a DAMNED STUPID question? You are a Venture Capitalist? Really? Apparently one of the stupid ones because if you really believe that a CTO in today’s era doesn’t need to know even the OSI layer then you are really dumb.

Using GigaOm to FISH for information is at first maybe clever but GigaOm is on the verge turning away readers and I’m thinking of UNSUBSCRIBING to the RSS feed if I continue to see bull shit posts like this one. A time WASTE.

Yeah… this CTO is an ignorant executive. It reminds me of the new business folks who said “Profit is meaningless”. I agree that perhaps for the end-user the Internet should be like electricity or perhaps refrigeration. You plug in the refrigerator and it works. If you’re business is all about massive cold storage, your chief of operations had better have a better understanding of refrigeration than just ‘plugging it in and it works.’ This CTO should be fired.

This article couldn’t make the author sound more uninformed.

Horrible article.

I tend to believe that the concept of a CTO is outdated. My preference is to have people with domain specialties, Security/Operations/Change Management/DR/Development/etc, as opposed to specific technologies. Network Engineers, Server Admins, and Language Specific Developers; are only as useful as the platform currently being used. However, a good Ops guy, or Change Control, or Development expert; will work wonders in almost any situation. Additionally, a domain specialist will drive efficiencies by appropriately outsourcing, using OSS, avoiding regulatory mistakes, and identifying landmines regardless of technology. A CTO is an outdated title from the days when specific technologies differentiated companies.

If I was a venture capitalist, I’d wonder why a Web 2.0 company has any “tech” guys whatsoever. The most exciting startup that I’m working with today is a small networking company making very specialized gear. Less than 30 employees, no in-house techies, however they are making multi-million dollar sales because the product fits a unique business need.

No, the Internet isn’t like electricity. Invest in this company and you’re throwing money away, or invest, but make shafting the CTO a requirement.

I would have closed my briefcase, stood up, and walked away.

What drivel.

Of course you can deploy a web app and be largely unaware of the network topography. Of course your hosting provider is. And the speed of the pipe to the server has precisely wang to do with load balancing and other problems of scale. Whoever thinks that without support a single-server solution will scal in any fashion is plain stupid. This is especially the case with demanding webapps like a lot of “2.0″ content.

Another “Web 2.0 is the new black” article. Way to bandwagon. Well here’s one: “Web 2.0 is the new dotcom” with all the associated bs, hype and marketingese.

I was taken back by your post at first. Network Engineers not needed you say?!?! HOGWASH!!! But then I realized the compliment that you have given me, the network engineer. We are getting better at what we do. The code (router, firewall, load balancer code) is getting better, it doesn’t have the bugs that it once did. The circuits are getting faster, processors are faster, fail overs are faster. Oh sure strange things still occur. And we are still here 24/7/365, always on call, always keeping a watchful eye. But still a far cry from the Maytag repair man. The maintenance still occurs, the upgrades still go on, but the outages have diminished. At least as far as you know ;-> After all this has been the goal hasn’t it? To be an obscure part of the landscape so that real applications CAN fulfill the needs we require. Not just the latest sports and weather. But fulfill a REAL need! And all of this is transparent to the user as well as to a good developer. A requirement I have always heard from every change window, change request, change control meeting, was how this change needs to be transparent to the user. Well it is and we are. We are transparent. The ghosts of the machine ensuring the heartbeat of the network continues to not miss a beat. We’ve gotten good at this and you might think we aren’t needed. Well… That’s good!!! That means I’m doing my job. And thanks for noticing!

Internet as utility is a compelling notion that may be true someday, but if your web product runs at any kind of large scale, that day is not today or anytime in the next few years. When things go south in the infrastructure that connects you to your revenue stream, you better know how it works so it can be fixed quickly.

I think that the electricity analogy is very apt in most respects, for the context in which it was uttered.

From within the confines of a colo or hosted rack, the point at which I plug an abstract pipe to the internet or a private network is identical to the abstract pipe from which I get electricity in that I can always purchase unlimited capacity which will be flawless to a given number of 9s.

However, in both cases I have to plan for blackouts and brownouts, for which the only truly viable solution in either case is to distribute load geographically.

Distributing load geographically only requires network knowledge if seperate nodes need to communicate with each other, and even then only if can’t be achieved acceptably in layer 7.

Taking Meebo as an example, the free version requires absolutely no communications between nodes, since that function, along with storage of user credentials is entirely handled by the networks the service connects to.

Clearly credentials for Meebo accounts are a seperate matter, but one can easily argue that database technology is also commoditized — how long till we see S3 for SQL/S4?

As long as you understand the capabilities and limitations of a given abstraction layer, you then no longer need to be intimately familiar with what goes on behind that layer.

This is clearly true by definition because what is the point of an abstraction layer if not to insulate you from what goes on behind it?

I think that is all that the CTO in question was trying to say, however abrasively.

Of course, determing at what level of abstraction to deal with things is case-by-case and figuring THAT out generally does require some deeper knowledge.

Hm. If the network engineer skill set has been rendered obsolete, it’s news to me. What’s really going on here is not that no network engineer is needed, but that your CTO friend feels confident that he can safely outsource it to someone who provides a service into which a network engineer is bundled (and hopefully efficiently leveraged).

What your CTO friend isn’t talking about is security. As long as his company has no information to protect, that’s no problem, but speaking as someone who has set up sites that actually handle customer-provided financial tokens, I can tell you that I would be sweating this a bit more than your CTO friend is. You might outsource anyway, but you’d better make sure your provider is willing and able to indemnify you if a security breach at their site causes all your customers’ credit cards to be compromised.

We def cannot say that the profession of Network of the Network Engineer will disapear, it fact, I think the demand for this job will just increase over time!

Why? Don’t know, just a feeling.

Here’s my perspective. For $20/mo, you can get a really nice VPS that will serve a significant amount of traffic, done right.

Servers are so powerful that a few boxes can take care of an astounding number of users. If you’re not doing video, your bandwidth requirements are probably low relative to the number of users you have.

We’ve come to the point NOT where network engineers are outdated. Rather, we’re at a point where network engineers are required only for the most significant problems, rather than everyone’s problem. If that means many of them are not needed, then, well, so be it. Computers are fast. Bandwidth is cheap.

I get solicitations regularly from bandwidth brokers due to the usage of some of my websites. However, I always have to turn them down, as I’m amply served by my $20/mo VPS. You can get a lot for $20/mo. If you need double that, you can get it for $40/mo.

It takes an astounding number of users OR an amazingly bandwidth intensive product to use up the bandwidth you get from a relatively cheap product.

It’s hard to accept this. I ran a website on a Sparcstation 5. I’d have to go reboot it every night at 9pm (popular time = crash time) I’ve ran websites off “super fast 266 mhz machines.” And now i’m running a bunch of websites off of a VPS that shares resources with several others.

It’s just progression of technology making certain requirements unnecessary (until you hit a certain user requirement)

Fire that guy.

He obviously isn’t making money, rather he is busy spending other peoples money.

Every profitable venture eventually requires in house management of its resources in order to obtain the least cost for secure operations. Do the math on a 10 1u master slave LAMP environment hosted at rackspace, vs. the cost of DIY and right off the bat you are 1/2 the cost to DIY vs. Rackspace. Including employee cost.

Add to that the value and reduced liability afforded in retaining control of customer data, sales data, and other systems reporting information. Does he know the pedigrees of the folks handling his precious packets? I bet he would be suprised to find out some group in India is responsible for security scans of the equipment containing client VISA CARD NUMBERS. Doh!!!

I have yet to find a single CEO/Board comfortable with the idea that a 3rd party contracted service provider has a death grip on the transactional bowels of its operations. And no one who chooses the “Internet Utility” route can truly speak with any confidence about who is accessing those systems.

Does this guy really feel comfortable that someone outside his company knows the transactional bottom line. And then there is governmental and financial compliance requirements. Does his board want to risk jail time when that “Internet Utility” fails to track data REQUIRED BY THE PATRIOT ACT properly? Does he think a service providers contract shields him from that kind of liability?

Maybe I am old school. Maybe the new fangled world of Web 2.0 includes letting others be responsible for your meal ticket.

In my book this guy is NOOB. He hasn’t been burned. He has not had his job, or for that matter the companies future at stake, when that bad ass from Austria cracks the “Internet Utility” quietly on a Saturday night at 2 am.

He should have known they only monitor known exploits, and then only during normal working hours. But then thats what buying commodity services offer. They are good for prototyping, and testing only. When the ball gets rolling and profits arrive, you must bring it in house to gain the control necessary to survive in a world wide web filled with bad asses.

2 centavos deposited…

Catfish

Cheap, fast, or accurate. Pick two.

Thanks for all of the comments folks! Good to hear that many of you think that the years I’ve personally invested in learning and teaching about the OSI model and internetworking are not outdated yet.

And, once again, for the record, I did not invest in the company with the CTO that I reference.

Yes! And all you do is plug in your servers into a switch, your switches into a router, a router into a firewall and bam! you’ve got not only a multi-tiered, multi-layered, secured network, but also your webapps will suddenly, as if like magic, be secured as well! And who needs an OS??? That’s a thing of the past! It all runs on IIS with .net in the background, and Microsoft takes care of all the configuration stuff for us. Yes, I like this new Web2.0.

Oh wait.. was I dreaming? Did I say that outloud?

–thrill

The Internet is like electricity:

• if you go stripping live wires with your bare teeth, you’re going to get hurt.

• except both of your neighbors are trying to hide their pot growing operations from the police by tapping your line

• every company has it’s share of JSO (job shaped object) that equate to wall-wart phantom loads… and that could include CTO!

• if you put a laser printer behind a UPS you’re eventually going to have to hire someone to figure out why neither you laser printer or UPS works.

• sure it’s OK to chain 5 power strips to each other in sequence. You’ve got continuity, right? What more could you need.

• yeah… we just buy everyone laptops because they, like, have a miniature UPS built into them.

• you know that data networks have arrived at the reliability destination because of the newly emerged power over ethernet standard.

• because at the end of the day, trapped in every packet is a little Maslow who just wants to self actualize.

(every last one of the above is based upon stupid things I’ve heard and seen in the IT industry)

People with business degrees (and I’m pretty sure that if you did your due diligence, you would find this is where the CTO’s background lies), shouldn’t be allowed to make comments about the technical solutions the company employs to get work done; on the other hand, maybe you should just let the idiot talk since it’s good to know when you’re being glad-handed and condescended to at the very same time.

I side with the first poster. Yes, Web 2.0 makes things easier or, as so popular referred to, ubiquitous. This, however, is merely a temporary comfort as the wonderfully autonomous functionality of today’s web applications fail you once your traffic rises to serious heights. When the fun stops and your little start up is turning into a considerable business venture, you should make sure that your technical personnel knows what they are doing.

If the company is at the stage of concept building, the CTO may more focus on the application technology like J2EE or PHP or Rails. Until they want they have beta users and really concern about operation, then networks knowledge is must.

What a jerk. If this noob CTO get hit by Digg once a day, victim of his success, he’ll cringe in pain and doesn’t even know why… This vision is like the people that dont optimize their java code since memory and processing power is cheaper and cheaper, creating mammoth applications that don’t scale and use all the resources put on it. Needless to say, this kind of people are noobs, too.

Keep your work, just pick carefully your clients.

Run screaming into the street. O’Reilly has so badly brainwshed this guy he hasn’t a clue as to what he’s talking about. He should be fired immediately. Take away his IP block and router before he does any real damage and run from his company like the wind. This Web 2.0 term will do nothing but damage because of its inaccuracy. Don’t buy into it or you’ll end up working with ignoramuses like this one.

This is the most ignorant article I’ve ever read. I would fire anyone who ever walked up to me and said anything like this. Web 2.0 has nothing to do with network infrastructure.

Dude, you are a complete MORON. Who do you think sets up and maintains the network that is hosting your web 2.0 app. What a fucktard

The CTO is an idiot, let me guess he was hired by a family member or friend. Ask his support/technical staff and I guarantee they will tell you the opposite of his “bold statement”. Mr. Leinwand you should know better than to take him seriously.

Maybe this CTO is one step ahead of you, and you all don’t even know it.

Can’t you forsee a day when everybody can create a web 2.0 application, just by focusing on application? Just outsource managed hosting to a global data center service, and have your application run code and store data on thousands of shared servers across the globe. Web 2.0 is about isolating your focus.

Here’s how business evolves: Companies enter the business of serving an application to the consumer. Other companies enter the marketplace by being enablers for the application companies to focus on their core business, and not have to worry the technology. That creates a separation of application companies from technology companies.

Web 2.0 is a concept that relates to the surge of new ways that the web can become useful. Clearly, we have yet to discover an end to the list of things that we can do on the web.

My gut feeling is any company that has put someone into the position of CTO who doesn’t think it’s necessary to know the technology they are built upon, is going to fail.

There comes a point in your growth (assuming you continue to grow) where it is no longer economically feasible to grow while paying someone else to build it for you. Not only that, sooner or later you’re going to realize that a one-size-fits-all solution is not going to get the performance you’re inevitably going to need to get out of your hardware/software to continue to be profitable. This is all IP, and if you’re letting someone else do it for you, unless you have a contract wizard of a lawyer, you can expect they will be selling it to your competitors as well.

Enough rambling, no, I don’t think the internet and it’s infrastructure is like electricity, nor do I think it ever will be.

Allan:

We’re likely to publish a blog post at Network Performance Daily on Monday dealing with this story – we would have put it up today but we had some other projects that delayed us.

– Brian Boyko – Editor, Network Performance Daily

I think that CTO is headed in the right direction. I don’t think knowing the details of how routers work will be that helpful in the future. I think most of the jobs that are done by a NOC will be outsourced. I don’t see why any reasonable american would want to stay up all night responding to snmp errors.

Wow.. I guess we have reach the utopian network. I guess no need to worry about security or anything of that sort. Viruses and stuff are a thing of the past. Oh and we shouldnt worry cause the Millinium Act ( Good ol’ Bill Clinton ) made it a crime to decipher encrypted data. So that stopped all those bad guys from getting your information. Oh well how I miss the days of UUCP,MCA,ADA,Decnet, Oh yea that would be before Cisco… Have a lovily dream..

My guess is that a CTO who believes that the Internet is something that you can plug into and not worry about will not be a CTO for very long. Even if a company outsourced the support of every router, switch firewall and server in the network, the CTO better have an idea of how it all works so he/she can explain to the CEO why his company is at a total standstill because of a network outage, or why they are being hit with a lawsuit for exposing their customer’s credit card numbers to hackers.

Have you dealt with the phone company before? That is who is managing this network!!! Good luck.

I have to disagree with this CTO. I work for a small ISP and our upstream providers make mistakes when it comes to routing tables. Just last week, ATT typed in the wrong /22 block and was routing someones else IP’s to our circuit. This effectively null routed a web server farm in TX that was in now way associated with my company. It took about 4hrs to resolve and the sites this company hosted were down for those 4 hrs. Even if a company’s web server are multi-homed with BGP, a Tier1 IXC can still affect routing to their servers. I think this CTO will be in a work of hurt if their upstream provider accidentally null routes them. In that case it will be to little to late.

Saying that the web 2.0 era of the internet will be the death of network engineering, is like saying that utility electricity was the death of electricians. Last I checked, pretty good profession..still to this day.

That said, I’d bet that the first time that CTO pluggs into “bad electricity” and looses a few $ trying to “understand” what happened is the day they high a high priced network consulting engineer to pick thier provider.

Don’t buy into the hype.

I think devices build themselves by magic elves and you plug them in which activates secret pixie dust inside, which is how things work. Development and networking technologies are just a cover up for said pixie dust.