In a talk at the Surge Conference in Baltimore, Md., Pedro Canahuati, director of production engineering and site reliability at Facebook, explained how the social network keeps the site available, reliable and efficient.

Scale smart not not just fast.

Adding servers is essential to keeping Facebook available to the rapidly growing user base, but to remain reliable for the long term, Facebook needed a system that could scale up in a hurry if it needed to add tens of thousands of servers at a time. In August 2009 when Canahuati arrived at Facebook, he said the site spent seven weeks to get 10,000 servers into production once they were plugged in. Thanks to the development of Triforce, it now takes seven days to get the site running on 10,000 servers. But in Canahuati’s opinion, the rate of getting software on servers is still too slow, so there are two Facebook employees still working on making that process faster.

Building a tool itself saves Facebook the headache of dealing with open source code that might not be able to hack the strain of its infrastructure demands, while also keeping Facebook from paying a vendor a license fee that could become astronomical as it scaled. Understanding the process of building the tools to keep the site’s infrastructure up and running, and the impact that process plays on the ultimate goal of reliability, is where Facebook has carved out another advantage that other large web services could learn from.

Like his peers at Pinterest, Canahuati stressed that when you are at massive scale, you need to keep it simple.

Make it easy to see what’s wrong.

He also added a few more tenets, including “instrument your world.” He explained how Facbeook collects a lot of data across many of its systems in order to understand how the different services that comprise the site are performing and interacting. Tools such as Scuba and Claspin are examples of this effort to take complex operational data and make it easy to understand.

“Smart visualizations are often overlooked,” said Canahuati, but scanning data at a glance and then acting quickly on it is an essential ingredient for keeping systems up and running.

Automation can be a dream — or a nightmare.

One of Facebook’s secrets to scale is automation, but automation can create its own problems. At its worst, one could create a system that automatically brings down the whole site. Elsewhere, automation can mask a more systemic problem.

However, a more persistent worry is that engineers and operations people working to automate certain actions might think they’re coding themselves out of a job. To alleviate those fears, Canahuati says Facebook has implemented several strategies that boil down into keeping a lean team works on multiple jobs and problems. This ensures that when automation solves one problem, there’s another one waiting in the wings to be solved, and also sets the expectation that employees are responsible for the whole site broadly and not just one tool set.

This approach is carried all the way to the way Facebook manages and hires its employees. It expects the engineering team that builds Facebook products to be aware of the operations side of things and build tools that help the operations team out. Operations employees are expected to be able to code and work with the engineering teams. “The software guys who build the code must take ownership for it working on a big system at scale,” Canahuati said.

But these operationally aware engineering teams and engineering-aware operational teams must have buy in at the top because people who code generally cost more, so hiring operational team members who code requires a bigger budget.

The tactic seems to work for Facebook, which is clearly trying to build a culture of responsibility and effectiveness that can scale the same way its servers do. Hence its continued release of tools that others can use to monitor their own giant deployments, as well as its operational slogan: “Fix more, whine less.”

In many ways, Canahuati’s points are good policies for any corporate culture today. Traits such as communication between teams, blame avoidance and employees that think strategically instead of just about their skillsets are helpful no matter how many servers you have.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• A near-term outlook for big data
• Migrating media applications to the private cloud: best practices for businesses
• Dissecting the data: 5 issues for our digital future

It’s about more (and less) than wide-scale fraud

To get a full sense for where Charnock sees the value of her company’s software, which tracks and analyzes electronic communications carried out over corporate networks, one really needs to read a Cataphora blog post called “Getting Big Brother Right.” You’ll notice in a hurry that while Cataphora is concerned with uncovering nefarious activity such as fraud and with discovering the truth in the case of any complaints, investigations or lawsuits, it’s also quite adept at proactively tracking and highlighting employees’ personal lives if their bosses are so inclined. It’s the latter type of activity that I think toes some ethical lines, and it’s on this issue that Charnock and I began a recent conversation.

Elizabeth Charnock

To defend the tracking of employees’ personal lives, Charnock points to the scenario of someone “flipping out.” Maybe someone is going through an ugly divorce, she explained, or is otherwise very stressed out and could be a danger to himself, to others or to the bottom line by operating heavy machinery or sensitive equipment. Those issues might not be visible to the naked eye, but they might be clearer in someone’s emails. If a company gets “sued out of existence” or faces a large civil judgment or regulatory fine, everyone loses, especially if the result is a wave of layofffs or bankruptcy.

Or, she said, consider today’s distributed workforces in which employees are spread across multiple offices or even are working from home. In the past, Charnock said, managers could see their employees on a daily basis and might be able to put two and two together if, say, the guy who came in looking hungover every morning was also an unproductive employee. That’s not always possible today, but Cataphora can help replace those in-person interactions if that same guy is emailing co-workers about his wild nights or tweeting about them from his office desktop.

The aforementioned blog, written by Cataphora VP of Marketing Rick Janowski, takes a more positive tack on this type of monitoring:

At least some level of protective monitoring helps to replace the basic kinds of feedback that managers are used to having readily available, whether that is seeing an employee with a runny nose and concluding they have a cold or seeing a normally calm person pacing around their office and inferring that they are having some kind of issue. Such knowledge helps managers make better decisions in the day-to-day management of their employees.

But this is where we get into the proverbial slippery slope. Even if you can buy into what Charnock is proposing as simply being good business and really just serving as an electronic proxy for a floor manager, you have to wonder where it stops. Where is the line at which companies decide particular information isn’t important enough to warrant monitoring and/or acting upon?

What really matters?

Charnock says she doesn’t care about whether someone is looking at Facebook, but rather cares about uncovering broad and worsening problems, or acute but intensifying problems. She also suggested that sometimes it’s in companies’ best interests to know as little as possible about their employees’ personal lives. If personal details about, for example, sexual orientation were included in an employee’s email, evidence that the company knew about them might play an important role in a wrongful termination lawsuit.

The problem, of course, is that while Charnock as a third party selling software can take a seemingly prudent stance toward monitoring, petty or overly draconian employers might not be able to draw those lines. They might equate Facebook usage with being unproductive on the company dime. Maybe they’ll find it acceptable to monitor discussions about employees’ health so they can pressure them to get in shape. Hey, it’ll mean lower insurance premiums for the company and everyone else.

But maybe there’s no problem at all with employers monitoring what their employees are doing. Maybe employees should just stop conducting personal conversations using company email accounts, computers and cell phones. The law is on the employers’ side — at least within reason — and, to some degree, so is the Electronic Frontier Foundation. Charnock said she and EFF boardmember Brad Templeton are friends, and the EFF’s main concern is that employees know when they enter the workplace that they’re communications are not free from prying eyes.

And with that, the issue of workplace privacy converges with the issue of consumer privacy. Are clear disclosures of tracking practices and informed consent all that matter, after which anything is fair game? Or do we want to guarantee a minimum of rights regardless what the technology enables and the contract stipulates? That’s a question I don’t suspect we’ll have an answer to anytime soon.

Feature image courtesy of Flickr user kissmygrandmother.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• The importance of putting the U and I in visualization
• A near-term outlook for big data
• Dissecting the data: 5 issues for our digital future

Kittlaus was not only co-founder of the company behind Siri, but also its CEO when it was acquired by Apple in 2010. The reasons for his departure included wanting to be closer to family, who are based in Chicago, and an interest in pursuing more entrepreneurial plans, according to Swisher’s sources. Kittlaus’ leave-taking is said to be “amicable” and long-planned.

Despite his role in the creation of Siri, the voice-powered assistant that’s the star of the show for the new iPhone 4S, it should remain in good hands at Apple without him, as sources claim other key staff and executives from Siri will remain at Apple following Kittlaus’ departure. The other two Siri co-founders, Adam Cheyer and Tom Gruber, both remain Apple staff members, according to their LinkedIn profiles. Gruber and Cheyer have more technical background than Kittlaus and were VP of engineering and CTO, respectively, at Siri prior to the acquisition.

Kittlaus’ departure likely won’t slow down Apple’s investment in or development of Siri, but its hiring of Noteboom away from Yahoo definitely indicates it wants to beef up its cloud infrastructure, which is a big part of what powers Siri. The next step for the digital assistant is probably rolling out international localizations of features tied to maps and local search, which will undoubtedly be a huge challenge from a data center management perspective. Apple has promised additional country localizations by 2012 on its official Siri FAQ.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• How tomorrow’s mobile-centric data centers will look
• Who and what to watch in the new era of the living room
• Controversy, courtrooms and the cloud in Q1

Yet earlier this year, when Google interviewed its employees about what they valued most at work, none of these extravagant benefits made the top of the list. Neither did salary. Instead, employees cited access to “even-keeled bosses who made time for one-on-one meetings, who helped people puzzle through problems by asking questions, not dictating answers, and who took an interest in employees’ lives and careers.”

Tangibles like salary and benefits aren’t enough to guarantee that your best and brightest creatives will remain engaged. Indeed, a recent landmark study by Arnold Worldwide of 3,000 employees and 500 executive leaders across a range of communication and advertising firms found that 30 percent of the advertising workforce say they’ll be gone from their job within 12 months.

Take Jill, an outstanding, experienced copy editor whom Agency X recently recruited at considerable expense from one of its chief rivals. Despite her outward success, she’s unsure how she’s performing, where she stands in the company, and how she fits into the overall goals of the agency. Her pay is great, she loves the Friday office happy hour, but over time, she finds herself feeling demotivated by the lack of communication, and checks out.

The loss of star performers like Jill doesn’t just leave a talent vacuum to fill; it also leaves a gaping hole in the bottom line. Indeed, a recent article in the Wall Street Journal calculated that it typically costs a company about half a position’s annual salary to recruit for that job ¾ and several times that if the position requires rare skills.

So how can your company keep its stars engaged? It comes down to creating a culture of communication — one in which employees know where the organization is headed, how they fit into these plans, and what’s expected of them. Here are a few key strategies your agency can employ to make this happen.

1. Create a culture of education

The average Starbucks barista gets more training in a year than the average employee in a communications company, according to the Arnold Worldwide study.

For employees, the single most important motivational factor was the ability to learn. Yet the study found a huge disconnect when it comes to perceptions about company training. While 90 percent of employees say they learn by figuring things out on their own, only 25 percent of executives think that employees learn independently.

To keep employees motivated, agencies need to build a culture of learning, where employees leave more enriched at the end of each day.

2. Provide regular, consistent feedback

Employee feedback is a critical part of the education process, and shouldn’t just be relegated to the annual review. To be effective, feedback needs to be specific and actionable. But that’s not always how it works.

In a study by Leadership IQ, 53 percent of employees said that when their boss praises excellent performance, the feedback does not provide enough useful information to help them repeat it. And 65 percent responded that when their boss criticizes poor performance, it doesn’t provide enough useful information to help them correct the issue.

Feedback, both positive and constructive, is most effective when given right away. Negative feedback given a month after the fact can lead to a passive-aggressive environment in which an employee feels powerless to act on the advice.

Think of it this way: no one wants to go a full day knowing their price tag was hanging from the back of their shirt, or the remnants of the salad they had for lunch were still stuck in their teeth. If an employee does something well, that activity should be encouraged. And if there’s room for improvement, they should be given the opportunity to learn for their next task.

3. Set time aside for weekly 1:1 meetings

At first, most employees and managers will cringe at the idea of yet another meeting. But instituting weekly 1:1 meetings can be the most important step you take to retaining your top performers.

In its quest to build a better boss, Google discovered that its worst managers weren’t consistent in their 1:1 meetings; some focused on meeting with people who were underperforming, while others met primarily with the top performers.

Consequently, Google implemented the best practice of 1:1 meetings with all team members.

These meetings can cover anything and everything from upcoming projects to the latest client news. With each week, discussions about goals, feedback, and concerns become a lot more natural unlike the awkward, starchy conversations during annual reviews. Over time, it becomes easier for both sides to raise potential problems and deal with them early on, before they fester into something destructive.

4. Manage the grunt work properly

Not every project is going to be awesome. That’s just the way business works. And chances are your employees understand this.

However, managers need to handle such projects responsibly and that means a few things. Boring projects should always be balanced with more stimulating work. Employees should always be told how any grunt works fits into the overall needs of the company (“If we do a good job on x, we’re hoping the client will give us their cool launch next year”). And specific parameters should always be set for the boring stuff ¾ meaning employees should always see light at the end of the tunnel.

5. Publicly acknowledge good work

All too often, managers see motivation in terms of financial compensation, but money is far from the only way to effectively reward talented employees. A 2009 survey by McKinsey Quarterly asked which incentives were the most effective in motivating employees. The top two responses were: “Praise and commendation from immediate manager” (67 percent), and “Attention from leaders” (62 percent).

Praise and commendation go a long way in making employees feel noticed and valued. And the impact of a pat on the back is multiplied when it’s done publicly. Through public commendations, employees not only feel the support and respect of their manager, but the entire organization as well (including top-level executives). Creating a framework for “social recognition” will encourage a culture of appreciation throughout your firm.

Keeping your rockstar employees on board has always been important, and don’t think that economic uncertainty will keep your employees around. Your company has worked hard to recruit some bright people and great talent; make sure an opaque work environment doesn’t drive them into the arms of your competition.

Learn more about how to keep remote workers happy and your team collaborating at GigaOM’s Net:Work event on December 8, 2011.

Daniel Debow is co-founder and co-CEO of Rypple, a social performance management platform.

Image courtesy of Flickr user Esparta. 

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• Startup growth and the new recruiting ecosystem
• Personal tools lead to practical business
• NewNet Q4: Platform mania and social commerce shakeout

However, the company has a rich past that stretches back decades. Heck, its previous CEO was a former lineman who actually spliced cables for telephone wires. The striking workers, who are responsible for maintaining Verizon’s wireline telephone network as well as its fiber-to-the-home FiOS network, are being asked for the usual share of rising healthcare costs, but also that they become more flexible. As Verizon’s wireline DSL business shrinks, the company is still supporting an expensive workforce that’s responsible for a business that may soon represent a much smaller portion of Verizon’s profits.

Verizon is competing now against companies such as Amazon and Rackspace, which don’t have unionized workers, and against pay TV providers which sometimes do. As Verizon gets more digital, mobile and cloud-focused, its culture has to shift from one where a person has a single job for 40 hours a week with set time off to one where people work all out to deliver services and innovation for decent paychecks or the chance to take home a piece of the pie. There’s plenty of room for debate over whether this is a good thing, but it’s certainly the thing that Verizon has to adjust to. Will its workers let it?

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• Who and what to watch in the new era of the living room
• 12 tech leaders’ resolutions for 2012
• The Future of Pay TV Services

As Ducker says, “I don’t think there’s a whole lot of difference between motivating someone from a virtual standpoint and motivating somebody [...] where you’re working in real time with each other under the same roof. It just comes down to simple things, like being a nice guy, being understanding, being flexible, [and] wanting to spend a bit of time with them, above and beyond just giving them tasks.”

Here are a few things Chris recommends to help build stronger relationships with your virtual team:

1. Have an open door policy. “I always make time for my management,” Chris explains, “My management fundamentally run my company for me, and I’m well aware that without them, I would have to come in and start working ridiculous hours every single day. If they need to speak with me, all they have to do is pick up the phone, and I’ll make time to speak with them.”
2. Provide opportunities for career growth. Chris says, “I never want to hire externally, unless I truly have to. I’ll always look internally to try to promote people, if possible. That creates a great culture within a company.”
3. Provide opportunities for education. “I think there [are] other things you can do,” Chris says, “not only spending the time with somebody and giving them career opportunities, but also investing in them. I regularly put my management on different types of courses, training and things like that. It’s developing them as employees, and it gives them the opportunity to better themselves, and they appreciate that.”
4. Provide support resources. “We created our Live2Sell library, which now has just over a hundred books on everything from self-help to how to get over issues and problems in the workplace,” he explains, “People can come up to the HR department and borrow the books, just like in any other library.”
5. Spend time together as a team. “We do one yearly team-building weekend,” Chris says, “where we go out and stay the night at one of the resorts here, and everybody is together.” The purpose of the weekend is to build stronger connections in a fun atmosphere. He adds, “I’ve found that I have a lot in common with [team members]. We enjoy the same things. There is a lot more in common with workmates than a lot of people think.”
6. Treat your team members like family. “We have ten core values, and the first one is, ‘Treat everyone as family.’ That’s really important for any company,” Chris says, “We recently launched our company culture website, Live2Sell Family. It’s open for the public, but it’s all about us, [and] we’re now giving out little mini birthday cakes to everybody on their birthdays.” Another example, he adds, one of his assistants is a big music-lover, and every so often, he emails an iTunes gift certificate, because as Chris says, “The little things make big, big differences.”

“Above and beyond monetary gain,” he adds, “the fact of the matter is that, as long as you give opportunities to people, you’re nice, and they enjoy working for you, they’re not going to go anywhere. As long as you’re treating them well and with respect and paying them what they’re worth, you don’t need to lose people, unless you’re not managing your company properly.” By establishing these little connections, you’ll build better relationships with your virtual team and build a greater sense of unity and purpose.

How do you keep your virtual workforce happy and motivated?

Photo courtesy Flickr user cjmartin

Related content from GigaOM Pro (sub. req.):

• Are You Empowering Your Mobile Workforce?
• How to Manage Consumer-Grade Collaborative Tools in the Workplace
• Top Remote Work Trends to Watch for in 2011

Breaking Godwin’s Law at a sub-atomic level, incendiary Gizmodo cites an anonymous source describing how security finds leakers at Apple, not to mention creating a pervasive atmosphere of fear and dread, referencing “Nazi” tactics by the “Gestapo.” That is, if you believe it.

Reading like something by Fake Steve Jobs—only not nearly as entertaining—Jesus Diaz relays the experience of “Tom,” a supposed current or former employee of Apple. Tom alleges that Apple has “moles,” or informants, “working everywhere, especially in departments where leaks are suspected.” When a leak is strongly suspected, members of the Team Apple World Police “Apple Worldwide Loyalty” arrive and an “operation” takes place.

What’s described is effectively a lockdown. Employees are forced to remain at their desks. Their cellphones are collected, and anyone needing to contact the outside is monitored. Interviews are done. NDAs are signed. If security finds the suspected leaker, and “they usually do,” the person is fired after questioning. Of the questioning itself, “Tom” has no first-hand experience.

“There is a lot that goes behind doors that I don’t really know about. I do know, however, that they really interrogate people that are serious suspects, intimidating them by threatening to sue.”

Setting aside logical inconsistencies in the article like cameras being forbidden at Apple yet every employee having an iPhone, and legal questions such as confiscating personal cell phones, “Tom” asserts this type of corporate behavior is common at Apple. With 35,000 employees, it seems difficult to imagine that were such invasive tactics the norm, that it could be kept a secret, or at least made public by more than one guy through e-mail.

Looking at comments about Apple at GlassDoor.com, a website where employees can rate their employers, there’s not a lot of Nazi analogies…though from reading Gizmodo’s article you’d certainly think there should be. There are negative comments, but in aggregate the opinion is positive. In a recent survey, Apple scored 3.9 out of 5.0 for fifth place in the top 10 tech companies to work for. As CEO, Steve Jobs had an approval rating of 91 percent, highest on the list. In contrast, Dell was rated lowest among tech companies with a score of 2.8, while CEO Michael Dell’s approval rating was 28 percent.

If you are waiting for some purple prose about working Dell tech support hell in some warehouse outside of Mumbai, you’ll probably be disappointed. There’s no fame or fortune in stating the obvious.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• Mobile Q2: Smartphone growth surges; iPad’s rule continues
• Why iPad 2 Will Lead Consumers Into the Post-PC Era
• What Google’s Honeycomb Means for Apple and Microsoft

“Adobe is restructuring its business to align costs with its fiscal 2010 operating plan and budget, the company’s three-year strategic priorities and the realities of the business environment, as well as to ensure its ability to continue investing in long-term growth opportunities,” the spokesperson continued. The advisory we received also noted that at the close of its acquisition of Omniture, Adobe reduced the Web analytics firm’s work force by 9 percent.

These are tough times for some folks in the software industry. Adobe’s announcement comes right on the heels of Microsoft laying off 800 employees.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• The state of cross-platform media measurement
• Listening platforms: finding the value in social media data
• NewNet Q4: Platform mania and social commerce shakeout