Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• Connected world: the consumer technology revolution
• Social media in Q1: commerce and discovery dominated
• 12 tech leaders’ resolutions for 2012

What seems to be missing from all of these discussions, though, is a dialogue about how to steer around this bottleneck and make big data directly accessible to business leaders. We have done it before in the software industry, and we can do it again.

To accomplish this goal, it’s helpful to understand the data scientist’s role in big data. Currently, big data is a melting pot of distributed data architectures and tools like Hadoop, NoSQL, Hive and R. In this highly technical environment, data scientists serve as the gatekeepers and mediators between these systems and the people who run the business – the domain experts.

While difficult to generalize, there are three main roles served by the data scientist: data architecture, machine learning, and analytics. While these roles are important, the fact is that not every company actually needs a highly specialized data team of the sort you’d find at Google or Facebook. The solution then lies in creating fit-to-purpose products and solutions that abstract away as much of the technical complexity as possible, so that the power of big data can be put into the hands of business users.

By way of example, think back to the web content management revolution at the turn of the century. Websites were all the rage, but the domain experts were continually banging their heads against the wall – we had an IT bottleneck. Every new piece of content had to be scheduled and sometimes hard-coded by the IT elite. So how was it resolved? We generalized and abstracted the basic needs into web content management systems and made them easy for non-techies to use. As long as you didn’t need anything too crazy, the problem was solved easily, and the bottleneck averted.

Let’s dig a little deeper into the three main roles of today’s data scientist, using online commerce as a backdrop.

Data Architecture

The key to reducing complexity is to limit scope. Nearly every ecommerce business is interested in capturing user behavior – engagements, purchases, offline transactions and social data – and almost every one of them has a catalog and customer profiles.

Limiting scope to this basic functionality would allow us to create templates for the standard data inputs, making both data capture and connecting the pipes much simpler. We’d also need to find meaningful ways to package the different data architectures and tools, which currently include Hadoop, Hbase, Hive, Pig, Cassandra and Mahout. These packages should be fit for purpose. It comes down to the 80/20 rule: 80 percent of big data use cases (which is all most ecommerce businesses need), can be achieved with 20 percent of the effort and technology.

Machine Learning

Surely we need data scientists in machine learning, right? Well, if you have very customized needs, perhaps. But most of the standard challenges that require big data, like recommendation engines and personalization systems, can be abstracted out. For example, a large part of the job of a data scientist is crafting “features,” which are meaningful combinations of input data that make machine learning effective. As much as we’d like to think that all data scientists have to do is plug data into the machine and hit “go,” the reality is people need to help the machine by giving it useful ways of looking at the world.

On a per domain basis, however, feature creation could be templatized, too. Every commerce site has a notion of buy flow and user segmentation, for example. What if domain experts could directly encode their ideas and representations of their domains into the system, bypassing the data scientists as middleman and translator?

Analytics

It’s never easy to automatically surface the most valuable insights from data. There are ways to provide domain-specific lenses, however, that allow business experts to experiment – much like a data scientist. This seems to be the easiest problem to solve, as there are a variety of domain-specific analytics products already on the market.

But these products are still more constrained and less accessible to domain experts than they could be. There is definitely room for a friendlier interface. We also need to take into consideration how the machine learns from the results that analytics deliver. This is the critical feedback loop, and business experts want to provide modifications into that loop. This is another opportunity to provide a templatized interface.

As we learned in the CMS space, these solutions won’t solve every problem every time. But applying a technology solution to the broader set of data issues will relieve the data scientist bottleneck. Once domain experts are able to work directly with machine learning systems, we may enter a new age of big data where we learn from each other. Maybe then, big data will actually solve more problems than it creates.

Scott Brave is co-founder and CTO of Baynote, an e-tail and e-commerce advisory business. He is also an editor of the “International Journal of Human-Computer Studies” (Amsterdam: Elsevier) and co-author of “Wired for speech: How voice activates and advances the human-computer relationship” (Cambridge, MA: MIT Press).

Photo courtesy of Sergey Nivens/Shutterstock.com

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• A near-term outlook for big data
• 2012: The Hadoop infrastructure market booms
• Infrastructure Q1: Cloud and big data woo enterprises

The app represents a big bet on technology by parent company Gannett which, like other newspaper chains, is on borrowed time as it tries to solve the digital dilemma before its traditional print ad earnings collapse once and for all.

The company’s gamble comes in the form of a bespoke content management system that lets editors tailor content specifically for a mobile audience rather than simply slinging content from the USA Today website. In practice, this means shorter and snappier headlines and choosing hard news stories in the morning and then more visual, feature-like fare for later in the day. The move coincides with a mass reader migration to smartphones and tablet devices.

“We’re starting to think of these as products that should be actively programmed rather than taking things that were produced for another medium,” said Gannett’s Chief Digital Officer, David Payne. He added the process gives USA Today editors more levers to pull in serving mobile content and interjects a new human element to the curation process.

As for the app itself, it’s attractive but hardly a knockout compared to the mobile sites of digital natives like The Verge or the Atlantic’s Quartz:

And then there is the revenue question.

The market for online display ads is suffering from a supply glut and collapsing prices, meaning publishers like USA Today must find other revenue sources and find them soon. Payne says video ads, which pay much better, will be a cornerstone of the site’s approach to expanding its ad offerings. Specifically, the plan is to intersperse more videos among USA Today‘s text and photo content — a plan that will be easier given Gannett’s multiple broadcast properties.

Finally, Gannett’s 80 or so regional newspapers will be plugged into the new content management system, meaning it will be easier to pluck and share content from across the USA Today and other sites. The regional papers are also part of a major paywall experiment in which Gannett is testing which paywall schemes work best in various places. These opportunities for scale and curation may let Gannett unlock other business opportunities.

Overall, Gannett came into the digital era with a weak hand. But sound management (the company is posting respectable numbers) and bold bets on tech like this one could help the USA Today enjoy a second act in coming years.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• How to deliver the next-generation web experience
• The fourth quarter of 2012 in mobile
• How to manage the signaling storm in 2013

Speaking at the paidContent 2012 event on Wednesday, Mullenweg said that he had been hard at work coming up with a new interface that will eradicate some of the complexity that WordPress is usually associated with.

“One of the things I’ve been working on for the past few months is a radical simplification of the interface,” he told interviewer Anil Dash. “WordPress it’s a complex tool, it’s like the back of a digital SLR… but that doesn’t work on a phone.”

That complexity has become one of the reasons it is used by a wide range of different people, from tiny bloggers to The New York Times, and is now reported to be the system behind one in eight sites on the web. But Mullenweg admitted it would be tough to move the company in a completely new direction, given the scale and takeup that WordPress already has.

“We’re doing that, but at the same time we’ve got tens of millions of users who love the other thing,” he said. “WordPress is really on its third or fourth major pivot. It’s easy to pivot if things are going badly, but when it’s going well you have the weight of all your existing users.”

However, he dismissed the idea that the core business of blogging was something that had been undermined by the rise of Facebook, Twitter, Tumblr and other services.

“Blogging has been declared dead at least five times,” he said. “But that’s like saying creativity is dead, or personal expresion is dead. Ultimately some percentage of the people who get a taste for it through Facebook and Twitter want their own space. And for the most part, that’s a blog.”

Disclosure: Automattic, the maker of WordPress.com, is backed by True Ventures, a venture capital firm that is an investor in the parent company of this blog, Giga Omni Media. Om Malik, founder of Giga Omni Media, is also a venture partner at True.

Check out the rest of our coverage of paidContent 2012. Full archived video on livestream (registration required).

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• Facebook’s IPO filing: ideas and implications
• 12 tech leaders’ resolutions for 2012
• Connected world: the consumer technology revolution

Miro Community does not offer video hosting on its own, but relies on feeds from YouTube and other sources, which can be imported in a matter of minutes. “Our feeling is that there’s a lot of great free (or) low-cost hosting options out there,” explained PCF-co-founder Nicholas Reville when I contacted him via email yesterday, adding: “The usage we’re targeting is people and organizations that want to bring together video from various places into one well-designed experience.”

Some of the sites that are using Miro community include the public broadcaster WDET Detroit, Duke University and the auto enthusiast community SkiddPlayer, and Reville told me that the group is in talks with other public radio stations and newspapers. “Folks who submit the best videos will tend to already have a video hosting account that they post to,” he said. Miro Community would make it possible to aggregate those videos instead of forcing people to re-upload each and every clip.

I played a little bit with the platform yesterday and today, the results of which can be seen here, and importing video feeds really does work like a breeze. The Miro Community Engine gathers embed codes and all relevant metadata from the source site. Admins then have a chance to curate these feeds, rejecting or approving each and every clip — an approach that is particularly useful if one would, for example, aggregate a feed of YouTube videos tagged with a certain keyword. However, trusted feeds can also be auto-approved.

Miro Community’s hosted version is free for non-commercial use, but users that want some advanced features, including the ability to run advertising need to apply for a premium account, which costs $99 per month after a three-month test period.

The platform has been built with the help of a $589,000 grant by the Knight Foundation, which aimed to enable “a new low-cost model for community civic media.” Reville told me that the Participatory Culture Foundation is looking to supplement foundation grants like this one with earned revenue though projects like Miro Community to establish long-term sustainable funding.

Related content on GigaOm Pro: Could Activist-Style Micropayments Be a Real-Time Ad Model? (subscription required)

Perch

• Developer API — The API enables additional features to be added to Perch as apps, without adding bloat and complexity to the original lightweight product.
• Dynamic pages — One of the first apps to be made available for Perch is dynamic pages, which allows you to add new pages to a site, an oft-requested feature.
• Content filtering and reuse — Itʼs now easy to dynamically filter a multiple item region to show only items of interest, and content can be pulled from one part of a site and automatically displayed on another.

There’s a few other neat updates as well, like the ability to customize the interface with your own CSS and JavaScript, and improved image management.

Perch costs £35 ($51) per domain (you can switch between domains as required), with no ongoing fees. You can try out a live demo here.

Let us know what you think of Perch 1.5 in the comments.

Related GigaOM Pro content (sub. req.): Report: The Real-Time Enterprise

The organizers approached me to develop a web site to aggregate these posts in real-time before, during and after the event. In the process, I learned how such a site can reinforce the development of community.

I’m sure that you’ve been to events where good ideas are hatched and projects are planned, but often, despite the best of intentions, activity loses steam after the event is over, and nothing much gets done. It’s too early yet to judge the long-term effectiveness of this particular event, but I’m optimistic that it will make more of a difference than many such gatherings.

The electronic component of the event was relatively simple. We created a content-managed web site that would act as the hub for the many places that participants would post their contributions and reflections. We didn’t require participants to use a specific CMS tool (although they were welcome to use ours). Instead, before, during and after the conference, participants were encouraged to use existing technologies with which they were familiar to document their thoughts, and we then created links to their contributions.

Before the Event

The web site went live a couple of weeks before the event began. By that time, event announcements and registration had already been posted to a section on the sponsoring organization’s web site. Participants had been asked to interview another attendee as a “get-to-know-you” exercise. The results were to have been posted, but very few did so — I suspect because the CMS used by the sponsoring organization has a significant learning curve.

In the future, I would recommend creating the event-specific web site much sooner, and using a simpler CMS-, group-blog, social-network or wiki-based system for posting pre-event discussions and comments.

During the Event

The venue at our local university had limited wired connectivity, but it was sufficient for a video stream, an audio stream, and the web updates that I was doing. The video stream worked fine; the audio stream was less successful because the university had blocked the ports we needed. For the same reason, we had to use a MiFi connection to use FTP. Thankfully, this did not affect access to our CMS.

Participants were able to connect using Wi-Fi, which worked well, aside from some issues with entering passwords. I’m guessing that perhaps fifty people were connected at any one time.

The event web site included the following, all of which was updated frequently:

• An attendee list, with links to participants’ web sites and Twitter feeds.
• The event agenda. Since the event was an “un-conference” using the “open space” approach, much of the agenda was developed on the spot by attendees.
• A Twitter statuses JavaScript badge showing posts from the event. These tweets were mostly done by me on behalf of event organizers, but other committee members also tweeted through the conference account.
• A Twitter search widget showing a real-time feed of all posts using the conference hashtag. A full-screen version of this widget was projected in the room where most discussions took place, and it proved very popular.
• A link to a Twapperkeeper archive of the Twitter hashtag feed.
• A Facebook fan box linking to the event’s Facebook page.
• A Flickr badge and links to tagged photos and videos. Flipcharts and graphs were scanned or photographed, then posted to Flickr and to the web site as JPGs and PDFs. We also put up a Picasa link at the request of attendees, but it didn’t get used.
• A Ustream video feed. Video of many sessions was fed live into the site, then archived.
• A link to YouTube search results tagged with the event’s tag.
• An audio feed. Podcasts of many sessions were made available later.
• Links to blogs of those attendees who were writing about the event.
• A wiki for allowing attendees to post notes from event sessions. We chose to use a wiki rather than giving all users access to the CMS, although I think that in future we might go the other way, as some found editing the wiki difficult.
• An RSS feed for tracking changes to all of the above.

After the Event

We had not created a LinkedIn group before the event. However, participants indicated that they wanted to have an electronic venue for continuing the discussion after the conference. We surveyed the room, and discovered that almost everyone was already a LinkedIn user. Since LinkedIn groups and subgroups can be created immediately, we chose to use that service. We could have selected another group conversation service, though, and participants may move to more sophisticated collaborative tools as their discussions continue.

Planning and Setup

If you are ever called upon to provide this sort of support to an event, it can be fun and educational. Here are some tips to make things go smoothly.

• Get involved with the planning committee as soon as possible, and get an event-specific web site posted early.
• Agree on the hashtag for the event, and publicize it.
• Coordinate with the meeting venue to make sure that it has adequate power and bandwidth (both wired and Wi-Fi), and that it does not block ports.
• Plan to bring your own equipment if possible, or make sure that you have everything you’ll need.
• Set up a “tech table” in a convenient location.
• Have a group of volunteers who can cover the full event.
• Give yourself lots of time for setup and breakdown.
• Have a dedicated laptop and projector for displaying the Twitter hashtag feed — participants will love it.
• Consider what you’ll need in the way of cameras, scanners and printers. We had them all, and they were handy to have, but we could probably have done without them.

I found that attendees’ blogging, tweeting, recording and instant posting about the event reinforced what they were thinking and learning. Thus, the effectiveness of the event was increased, along with the potential for new learning and insights to cause change in the wider world.

How do you use social media and the web for events?

Images by Flickr users Choconancy1 and hajush

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• Social Media in the Enterprise
• Controversy, courtrooms and the cloud in Q1
• Flash analysis: the future of Yahoo

Fluxiom is a web-based digital asset management system for creative teams that offers a nice, visually rich interface and the ability to scale depending on your needs. It has a number of useful features, and there’s a free version available, so I wanted to see if it could function as a tool for helping people collaborate on a wide variety of media including text, photo, audio and video.

Assets

Right away, you notice that Fluxiom is clearly content-oriented. By default, you launch into your assets page, and you can upload files there either using the web interface, or using fluxUp!, an app that makes uploading multiple files easier. If you’d rather not install any software, you can always zip a number of files together and upload the whole archive via the web interface. You can also upload entire folders via the fluxUp! app, which I think gives it a huge advantage over other similar web apps for file sharing.

Once you’ve uploaded some media, it quickly becomes apparent how well-designed Fluxiom is. Interacting with your uploads feels a lot like using a native OS X desktop app like iPhoto, and because of this, there is almost no learning curve to the software. There’s a search field prominently displayed in the top left-hand corner; you can choose either thumbnail or list view; and you can filter your assets using a number of pre-set fields using convenient buttons running along the top of the asset pane.

As in iPhoto, you can also preview each asset you upload. I was pleasantly surprised to see that Fluxiom even supports PSD previewing, which is a huge benefit for graphic and web design teams hoping to use the app. To test the limits of file type support, I uploaded a RAR archive and a Flash file (.fla). Both uploaded fine, but neither could be previewed, though Fluxiom did know what the Flash file was, at least.

Tags & Stages

You can add tags to any uploaded media for easy searching and organization. That will really come in handy when you start dealing with large volumes of media. While Fluxiom does not have folders for organizing media, the “Stages” feature presents another good way to drill down and focus only on the media relevant to your specific purposes. Stages is designed to help you share a limited set of material with a particular client. So if you’re designing an ad for someone, you can create a tag for that project, assign it to all relevant media, and then create a stage using the tag. You can then email a link to that stage which, once the correct password is entered, will allow them to view and download the media you’ve designated.

Stages is also a great way to parse out media for individual assignments to different team members. This is especially useful if you’re working with a consultant or external contractor, since you can then ensure any IP not pertinent to the task at hand remains safely out of reach.

Dropbox

Fluxiom is not only a great way for you and your team to share files amongst each other and with clients, it’s also a great way for others to share files with you. Using the “Dropbox” feature, anyone can upload a file or files to your account, so long as they have the proper address and can confirm that they have a valid email address. Media uploaded via this method is then stored in a queue where you can choose to accept it, thus moving it into your main asset library, or reject it, whereupon it is deleted.

I can think of a thousand different uses for that feature, with the most appealing to me being a submission form for photographic or illustration work for contribution to a publication or web site. Fluxiom even makes it easy for you to post your Dropbox link on your web site so that, if you want to, you can open up content submissions to the general public.

Pricing & Conclusions

Because of how easy it is to share and receive files via Fluxiom, and because of custom branding and API access, this is one of the most impressive and scalable asset management systems I’ve seen in a web app, and especially one that offers a free account. Paid accounts range from €9 (about $12.50) all the way up to €169 ($235) per month, and offer a variety of storage/user limitations. Definitely give the free account a shot first, since it may be enough if you just want to use it for yourself, but I think creative teams will quickly see the value of an upgrade.

Share your opinions on Fluxiom in the comments.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• Social Media in the Enterprise
• NewNet Q3: Facebook remakes headlines in social media
• Millennials in the enterprise, part 1: strategies for supporting the new digital workforce

Installation is straightforward, although you do need to be a little tech-savvy to get it up and running. You’ll need a server running PHP5 or later. (Check if you’re using a shared host — I found mine was still running PHP4, but could flip to PHP5 easily.) Download the zip from the Perch web site, unzip, and FTP the Perch folder to your host. Run the setup script, insert your license key, and fill out your database login details. Most of the installation legwork is done for you, although you’ll need to FTP a generated configuration file back to the server.

To start using Perch to manage the content on your site, you first need to add some editable content regions into your pages using Perch tags. Once you’ve set up the regions, log in to your Perch control panel.

Perch’s interface is very clean and simple. There are just two tabs at the top of the page, “Content” and “Users”. Under “Content” you’ll find the regions that you added to your pages. Click on a region to initialize it by selecting a content template to use. Perch has templates for many types of content included out of the box (file, image, contact, text block, article and text), but you can create your own templates using simple markup.

If you’re building a site for a client, it’s now just a case of setting up user accounts for your client under the “Users” tab. (There are only two levels of access: “Admin” and “Editor”.)  They’ll then be able to log into the control panel and edit the content as required.

Because Perch has a simple, clean and easy-to-use interface, it shouldn’t be too hard to train clients to use it, which makes it a good choice for web sites that have some changeable content but don’t require a full-blown CMS. Perch should also be a great choice if you want to retrofit some editable regions into an existing static web site. However, it is very stripped-down, so if you require, say, an inbuilt WYSIWYG editor, you’re going to need a more heavyweight solution like ExpressionEngine.

Perch costs £35 ($57) per domain (you can switch between domains as required), with no ongoing fees. Unfortunately, there’s no free trial available, but you can download a Compatibility Test Suite to check that it will work on your setup before purchasing.

Have you tried Perch? Let us know what you think in the comments.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• Social Media in the Enterprise
• Personal tools lead to practical business
• Millennials in the enterprise, part 1: strategies for supporting the new digital workforce

As users shift to new modes of communicating on the open web, they are pulling these new models into the enterprise and ushering in cultural and organizational change. This report covers the underlying technological trends, the market and its drivers, and a competitive overview of the vendors.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• The Future of Work Platforms: An Overview
• Report: The Real-Time Enterprise
• NewNet Q4: Platform mania and social commerce shakeout