After holding chief technology positions with the District of Columbia and the State of Maryland, last year he was appointed Chief Technology Officer for the Department of Health and Human Services. Ahead of a trip back West, Sivak talked with me about how open data (an increasingly hot topic in government) can drive big changes in health care, where digital health is evolving and why entrepreneurs should give DC a try. Take a look at a (lightly edited) transcript of our chat.

GigaOM: You went from fast-paced Silicon Valley to bureaucratic Washington, DC, what was the biggest adjustment challenge?  

Sivak: The most disconcerting thing, at first, is that because you’re working for an entity that is essentially operating on behalf of the taxpayer, there is this constant spotlight shining on the work that you do. I don’t think that’s a bad thing at all. In fact, I think it’s great — it’s the reason that, I think, I and a lot of other people actually do it. But you very quickly get used to it — you just kind of take it in stride.

GigaOM: At the SXSW Interactive conference you talked about how government can learn “Lean Startup” principles and other Valley-style ways of thinking. What can the Valley learn from DC?

Sivak: I’m a big fan of disruption and bending or breaking the rules to do interesting things. But, at the same time, I think it’s important to realize that there are some rules that are there for a reason and, in many cases, disruption needs a partner called sustainability. I exist in this massive agency right now — 90,000 people work for HHS — and while I’d love to activate the potential of every person there, it’s important to recognize that there are people who are well-suited and who actually should be working on the sustainability aspect … keeping the trains on time and that sort of stuff.  That’s something that’s often overlooked by people who come to it strictly from the Valley mindset.

GigaOM: You’re speaking to a group of entrepreneurs, programmers and designers at the Health Refactored conference [this] week about innovation in health care. HHS has put a lot of effort into opening up health data – which is obviously an important first step – but what else needs to happen to spur innovation?

Sivak: It’s a massively complex ecosystem and environment. [And] one of the things we can do as experts in this area and the government is help by educating people, by doing a better job of describing our data sets, by doing a better job or doing a job or basically explaining the problems that we have and the problems that we want to see solved. There are millions of examples out there of things that can be worked out but people just don’t know there are problems and don’t know it’s something that should be worked on. And that’s a place we can help.

GigaOM: It’s barely a year into your tenure at HHS. But when you look at where we need to go, how far along are we on the progress bar?

Sivak: We’re just at the very tip of the iceberg here. We’ve been working on this for a few years now, long before I got there. My predecessor Todd Park, now the CTO of the United States, kicked off the idea of the data that HHS has as being critical to revolutionizing the system. A few years ago, we started the process of data liberation — changing the default setting from closed to open within the department. That’s been the big focus and I think we’ve been successful with that but there are still pockets of resistance where people just don’t really understand what the value is and there are other complications, such as privacy restrictions and things we have to take very seriously.

GigaOM: What has to happen next?

Sivak: There are two other phases we have to start working on. The first one is around dissemination of that data – we have a website called healthdata.gov and it’s the one-stop shop for HHS data. To date, we’ve probably catalogued 40 percent of the data sets that exist at HHS, about 400+ data sets, and a much smaller number, 34 or so, have APIs attached to them. There’s still work to be done of the dissemination side, and that also includes some questions we’re wrestling with now. For example, an important feature should be an area where people can come and collaborate and discuss and ask questions and get answers. And we’re trying to decide whether that kind of forum should happen on our government website or on a third-party website that’s charged with potentially building that community.

The [other] piece is data education – explaining data better, teaching people how to work with the data in a better way and connecting people with experts in a relatively regular fashion so they can get answers to their questions, understand the best ways to use the data, etc.

GigaOM: Judging by booming investment, accelerator programs, startup launches, and other activity in the sector, digital health seems to be seeing a lot of innovation, but what kinds of innovation do you think are missing?

Sivak: The beauty of it is that it’s a massive industry and there’s so much room for people to innovate it’s insane. Some areas that I’m personally interested in and I think are interesting to the department, off the top of my head: there are tons of people running around right now with devices generating personal health-related data from Jawbones, Fitbits (see disclosure), mobile phones that capture stuff, you name it. But that data, right now, is very inactionable. There’s no advice, no pro-active suggestions, none of that – that’s one big area.

And integrating that personal health-generated data with clinical data, i.e. the stuff that your doctor generates, is massively interesting. Imagine that you have some kind of chronic condition, like diabetes, wouldn’t it be interesting if you stepped on a scale every morning or took your blood glucose everyday and that got automatically transmitted to your doctor? And the system that your doctor has in his or her office alerts him or her if you have some kind of a problem or if your stats are going in the wrong direction, so they can intervene early? I think that’s a massive area.

Also, nobody has figured out patient engagement yet. And there are some people out there, myself included, who believe that the patient is the single untapped resource in healthcare right now.

GigaOM: How can open data help support health reform and Obamacare?

Sivak: Data is one of the absolutely critical components to reforming our health care system. The big challenge with our system today is we exist in this transactional, fee-for-service environment. When you see your doctor see you in [her] office or he or she performs a procedure or test, they get paid for those things. The problem with that is the incentives are somewhat backward – it incentivizes transactions. It incentivizes people to go in to be treated when they’re sick, as opposed to being kept healthy. In order to fix it, we have to move to a system where we pay for value and outcomes.

In order to do that, we need the data. If we don’t have the data in terms of what happens when somebody gets treated with a certain drug or what kind of drug interactions exist or how effective a specific treatment is, then we’re never going to be able to incentivize the providers to do the right thing, And if you’re a provide or a doctor you’re not going to be able to figure out what the right thing to do is. Liberating this data is incredibly important to fixing the system.

GigaOM: Every though there’s been a lot of buzz about the Affordable Care Act, there’s still a lot of ignorance out there. A recent survey found that 42 percent of people polled didn’t even know it’s a law. How can data help people understand health reform?

Sivak: What that [survey] indicates is that we need to do a much better job talking about it and marketing the value of it. I think the way we can do that is through interesting and creative uses of the data. People don’t remember statistics… but when you tell the stories with the data, that’s what they start to remember. I heard a great quote the other day, which is perfect for this: the singular of data is anecdote. That’s what we have to get to if we want to sell the value of this thing.

GigaOM: What are some of the most interesting or promising uses of the data you’ve seen so far? 

Sivak: For the first time [last week], we released [hospital pricing data] in an easy-to-access public format. [It’s] the actual prices that hospitals charge Medicare for the top 100 procedures across the country and [it shows] that the prices that hospitals charge, even when they’re right next to each other are wildly different. It’s crazy. But here’s a fun little statistic: in the first day that this data set was available online, we had 110,000 downloads.

Another one of my favorite examples is what [healthcare data journalist] Fred Trotter is doing. I love [it], not necessarily because the work that he’s doing to build this social graph of doctors will determine anything interesting, but the fact that he had this idea. That he’s not a medical professional or a subject matter expert and he had this idea to take these two random numbers in a claim and use them for something that could potentially be interesting is what I think is incredibly indicative of the power of the stuff and bringing people in who are not subject matter experts.

GigaOM: You’re not just trying to convince entrepreneurs in the Valley and elsewhere to work on digital health, you’re trying to recruit them to work in government (at least temporarily).  What does DC offer that the Valley can’t?

Sivak: This is actually a very simple answer: because we can give you the opportunity to solve, literally, the most pressing problem in American society today.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• GigaOM Research highs and lows from CES 2013
• How HR can make the case for workforce analytics
• Social 2013: The enterprise strikes back

Duedil is an aggregator and data visualization outfit that provides due diligence services through a freemium model. It focuses on private companies and it gets its information about them largely thanks to the U.K.’s open data policies, although it also buys data from sources such as Companies House and the Ministry of Justice (for county court judgement debt information). Quite neatly, users can also sync the service with their LinkedIn accounts, so they can perform due diligence on their contacts.

The firm focuses on making this range of complex data — asset value, intellectual property records, turnover, litigation and health-and-safety violation records, director information — easier to intepret, and its customers range from corporate lawyers and venture capitalists to small businesses checking up on their suppliers. Seventy-five of the FTSE 100 companies are clients. Duedil has also notably been used in some of The Guardian‘s (see disclosure) data journalism efforts, such as its recent exposé on companies (and individuals) that ferret their money away in tax havens.

The service is generally free to use, although company document downloads and credit reports need to be purchased through a pay-as-you-go system. In a couple of weeks, though, Duedil will launch subscription packages that come with varying quotas of these each month.

The funding round was led by Notion Capital and Oak Investment Partners, with others such as Passion Capital and Spotify investor Shakil Khan also taking part. According to Duedil CEO Damian Kimmelman, the investment will be used to expand into new regions and hire more engineers and data scientists.

Powered by open data

It seems a happy coincidence that Duedil is planning to move into other European countries just as the governments of those countries have agreed to adopt open data policies.

“The value lies in linking datasets from data providers, governments and businesses themselves into one place,” Duedil commercial officer Andrew Connolly told me. “There is quite a progressive culture here [in the UK] in terms of open data. European data is quite exciting for us – we’ll take data from wherever we can get it.”

Duedil isn’t the only British company working in this field. Another interesting example is OpenCorporates, which is pulling in data from all over the world, but the difference there is that OpenCorporates is entirely focused on open data, whereas Duedil works with a mix of open/free and closed/paid-for, both in terms of the data coming in and the services going out.

“They’ve been a great team in terms of making data easily accessible, but they have slightly different source feed,” Connolly said. “They want to give everything away for free. For us, we believe in open data but not all data should be open. A director might not be comfortable with his home address being open to the public.”

As for expansion outside Europe, Connolly certainly sounded wary of the U.S. market, which he described as a “quagmire” due to the wide variety of jurisdictional regulations around private company data.

Disclosure: The Guardian is an investor in Giga Omni Media, which publishes GigaOM.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• What’s driving the next phase of the e-commerce evolution
• Facebook’s tactical retreat on privacy
• Google and the Ghost of Silicon Valley Past

The 27 countries agreed on the rule change on Wednesday, according to the European Commission, which is behind the proposed revision of a 2003 directive on public sector information. If the European Parliament adds its stamp of approval, national governments will then transpose the changes into their laws sometime in the next 18 months or so.

According to Neelie Kroes, the digital agenda commissioner, the European Parliament will sign off on the change soon:

Breaking … #EU member states have agreed to updated #OpenData rules. Big culture change coming, #EP to sign off in coming weeks.

— Neelie Kroes (@NeelieKroesEU) April 10, 2013

The change will give developers, businesses and citizens the right to get their hands on public data at low cost or for free. They will also be able to use data from museums, libraries and archives for the first time. Public sector bodies will only be able to charge marginal costs for sharing their data, and will also have to be more transparent about their charges. They will also be encouraged to make their data available in machine-readable formats.

According to a webpage setting out the Commission’s hopes on the matter, the data in question will cover digital maps, weather data and road congestion data, as well as information on companies and court proceedings:

“Most of Public Sector Information raw data could be re-used or integrated into new products and services, which we use on a daily basis, such as car navigation systems, smartphone apps with weather forecasts, information services for companies integrating information from various sources, such as statistical data with economic forecasts, company register data and other publicly available information.”

Some European countries, such as the UK, already have established open data initiatives (and so, of course, does the U.S.).

Sources close to the negotiations tell me that agreement was reached on the basis that cultural institutions in particular could charge a bit more than originally planned for handing out their data. Some governments had apparently been hoping to be able to charge a lot more for their institutions’ data, but were convinced that they would get more money in the form of taxation from the businesses that would spring up around open data, the sources noted.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• Google and the Ghost of Silicon Valley Past
• Is The Antitrust Trap Getting Ready to Close Around Google?
• Facebook’s tactical retreat on privacy

MapR’s decision to open source parts of it code is significant, but not groundbreaking. The company is only releasing its improvements to a handful of Hadoop-related Apache projects that are included in the MapR distribution of Hadoop, but not the proprietary code that’s MapR’s real competitive advantage in the contentious Hadoop market. While it’s still not flying the all-open-source banner like Hortonworks is, the code release puts MapR more on par with competitor Cloudera, which bolsters its open source aspects with some proprietary software for managing Hadoop clusters.

MapR also took another step in the open source direction on Thursday, announcing a partnership with Canonical that integrates MapR’s M3 distribution with the Ubuntu Linux operating system. The two also have plans to ease the installation of MapR’s Hadoop software on OpenStack-based cloud infrastructure.

I wrote recently in relation to MapR’s $30 million VC investment that the company is in a tricky position when it comes to open source. The Hadoop ecosystem was built on open source and still values it immensely, but some customers are definitely willing to pay money for products that deliver the features they want, open source or not.

As for Yelp, well, it’s just following in the footsteps of many companies — Netflix and everyone doing something on Kaggle (including GigaOM) — in trying to find new ways to use its data. The data set it’s releasing is from the Phoenix, Ariz., area and include 11,537 businesses, 8,282 checkin sets, 43,873 users and 229,907 reviews. The deadline for entries is May 20, and they can be submitted in pretty much any form you can imagine.

Hopefully, for Yelp’s sake, it doesn’t step in it the way other companies — including Netflix and AOL — have when they released supposedly anonymous data sets that were later de-anonymized. Releasing data sets gives clear benefits to both the source companies and institutions or individuals accessing the data, but privacy snafus have a away sneaking up and mitigating some of the goodwill.

Feature image courtesy of Shutterstock user Jakub Krechowicz.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• A near-term outlook for big data
• How search can unlock the power of big data
• Infrastructure Q1: Cloud and big data woo enterprises

But, last year, Bryan Sivak, the CTO and entrepreneur-in-residence at the Department of Health and Human Services, was tapped to bring more Silicon Valley spirit to the massive department. (Prior to working in government, he founded a company that was acquired by Oracle). And it looks like his touch is starting to move the agency further along a startup-inspired track.

At the SXSW Interactive conference in Austin on Saturday, Sivak said he’s tried to promote a definition of innovation that gives people the “freedom to experiment.”

“I can teach you how to experiment. I can teach you how to develop a hypothesis. I can teach you how to define some tests that generate some metrics. I can teach you how to analyze those metrics to determine whether or not your test was successful and I can give you the freedom to execute some of these things,” he said. “This is something that’s critical for an entity like the federal government, which is very bureaucratic and structured and all the things we wish it wasn’t in a lot of cases.”

Sivak isn’t the first to bring lean startup theory to HHS. Sivak’s predecessor Todd Park, co-founder of health tech giants Athenahealth and Castlight and current CTO of the United States, drew on his tech chops to start opening up health data and transforming health care. But here a few of the more recent Silicon Valley-style programs at HHS.

Yammer-powered social networking

Getting 90,000 government employees to collaborate is obviously no easy task. But using Yammer, HHS employees across the department now have the opportunity to share ideas and reach out to people up and down the bureaucratic hierarchy through HHSConnect.  Since launching a few months ago, 10,000 of the department’s employees have used the platform with many using it actively, said Sivak.

Open coworking spaces

Like many startup CEOs, Sivak said he believes in the “serendipitous collisions” that happen between coworkers who work in open spaces. But in government cubicles, he said, “the only thing you’re going to collide with is air.”  To up the chances of serendipitous in-person collaboration, the department is creating “HHSLabs” – an open, modular, technologically-tricked out work space open to anyone in the agency.  It’s also opening its doors to health startup CEOs and other private sector visitors to DC who want a temporary place to work.

Internal crowdfunding for resources

To support entrepreneurial-minded people at HHS who come up with interesting ideas but need people with other skills or resources to get their projects off the ground, Sivak said they’ve created an internal crowdfunding-like site where people can solicit support. Called “HHSFairtrade,” people can post descriptions of their ideas and others across the department can commit needed resources or support. Like Kickstarter, the project only activates once it receives all of the commitments it needs to launch.

Seed funding for internal innovators

If it’s a little bit of cash that internal innovators need to test their ideas, Sivak said they can turn to “HHSIgnite.” The program gives department employees small amounts of money to try out new approaches. If the project can show returns in three to six months, he said, it can become a stronger candidate for allocated funds.

Opening the door to outside entrepreneurs

More technologists like Park and Sivak are bringing a startup mindset to the public sector, but Sivak knows that many of the country’s most innovative thinkers don’t live inside the Beltway. To tap into their ideas, he said, the department created “HHS Entrepreneurs,”a new program based on the HHS Innovation Fellows program launched last year. One track invites HHS employees to apply to be “internal entrepreneurs” who will work on special team and get extra networking, mentoring and professional development opportunities. But the other track is open to entrepreneurs around the country who would come to HHS to work with internal entrepreneurs for 6 to 12 months on “high risk, high reward” problems, Sivak said.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• GigaOM Research highs and lows from CES 2013
• Defining work in the digital age: an analysis by GigaOM Pro
• The 2013 task management tools market

On Tuesday, the company, which provides a cloud-based learning management system and social network for schools, said that it was challenging Student Information Systems (SISes) to create APIs (application programming interfaces) to promote the open exchange of data. To date, few SISes, which store all kinds of information about students — from grades and attendance records, to addresses and more — have released APIs. That means that when ed tech companies want to bring new tools to the classroom, they have to integrate with each district individually in a potentially tedious, weeks-long process. Clever and LearnSprout both address this issue by building APIs that connect SISes with developers, but while they can get data out of the SISes, they can’t yet support a two-way data exchange.

As part of its challenge, Schoology said it will issue a bounty of $25,000 for up to ten SIS vendors from now until May. If an SIS provides access to a Web API and Schoology can integrate its platform with it in 1o days or less, the vendor gets the money.

Schoology CEO Jeremy Friedman said SIS vendors have been hesitant to release their information because of time and cost issues. “We want to help offset the cost with the benefit being that… everyone wins,” he said. “The only way to get innovation in the first place is to create an environment that can encourage it.”

In addition to students and schools benefitting from more data-driven innovation, Schoology obviously wins from the new APIs as it makes it easier for more schools to embrace the product — and it makes its platform  more interesting for developers who can build on the API it released this summer. But ed tech companies across the ecosystem — including competitors like Edmodo and Moodle — could also benefit.

Schoology, which reaches about two million educators, students and administrators, said Pearson’s (PSO) PowerSchool was the first SIS to integrate with its platform — over the next few months, it will be interesting to see how much company it gets.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• What’s driving the next phase of the e-commerce evolution
• GigaOM Research highs and lows from CES 2013
• How HR can make the case for workforce analytics

The answer depends on you ask, but it raises a question that is becoming more and more relevant in this era of “big data”: namely, just because certain kinds of information are publicly available and can be filtered and aggregated in various ways, does that mean we are always justified in publishing them? It also raises a question we have written about before in a different context: Does publishing things in a newspaper give them more weight than just making the same information available online?

The paper that published the gun-ownership information is the Journal News, a Gannett-owned publication that covers the region around New York City. Editors submitted freedom-of-information requests for the names and addresses of registered gun owners in Westchester, Rockland and Putnam counties — information that is public by state law — and then plotted those addresses on a Google map.

Was there a larger issue at stake? The paper says yes

The Journal News feature pointed out that each dot on the map only represented someone who had applied for and received a permit to own a handgun, and didn’t mean they actually had one. It also noted that the information only covered pistols or revolvers, and not rifles, shotguns or the type of assault rifle that was used in the Sandy Hook school shooting — all of which can be bought without a permit. And the newspaper wrote a long story about why it published the data: in part, because it believes that such information should be more available to the public, so that residents know who in their neighborhood owns guns, as well as how many and what kind.

Within hours of being published, the story and map were circulated through Facebook and Twitter — the map has been recommended on Facebook almost 40,000 times — and were loudly criticized by conservative news outlets such as Instapundit and Fox News. Some readers protested that publishing the data in such a way opened owners up to theft or intimidation: one said that the article was designed to “sensationalize the anti-gun frenzy” and another argued that by publishing their addresses, the paper had equated them with “sex offenders and murderers,” since those are the other kinds of public information that are usually mapped by newspapers.

In retaliation for the perceived injustice of publishing gun-owners’ addresses, at least one blogger — a lawyer, real-estate agent and author — published the addresses of journalists and executives at the Journal News, including the paper’s publisher and the editor, as well as the reporter who wrote the story accompanying the map. In addition to their addresses, he also published links to their public Facebook profiles and photos, saying journalists should be prepared to have their own personal information become public if they are going to do so to others.

Should the media have its own Hippocratic oath?

There was almost as much debate among media-industry insiders about the wisdom of publishing this data as there was outside the industry. New York Times editor Patrick Laforge and Guardian writer Heidi Moore, for example, went back and forth on Twitter for some time about the propriety of publishing such a map — with Moore arguing that it served no real journalistic purpose, since there was no larger point to the data: in other words, no larger issue was being raised, none of the owners were interviewed, and the data was not used to make any kind of broader point about gun ownership.

@moorehn @palafo but if gun ownership is good, why the outrage over publicizing? Why is this worse than showing home prices, for example?

—
Barry Graubart (@graubart) December 26, 2012

Poynter Institute faculty member Al Tompkins also argued that the newspaper handled the information in the wrong way, saying the publication of the gun owners’ addresses would have been appropriate if it was related to an investigative series on gun violence or a story with substantial public benefit — but since it wasn’t, he said the paper had not justified its invasion of people’s privacy. In effect, Tompkins argued that journalists should be bound by something like the physician’s oath, which states “First, do no harm.”

But does the simple act of mapping gun permits qualify as harm? And how can publicizing that be an invasion of privacy, if the data itself is legally considered public? Those are just a couple of questions the Journal News piece raises. The first seems to have as much to do with the perceived power of the newspaper as it does anything else — just as the publication of a photo by the New York Post (of a man about to be hit by a subway train) was seen as reprehensible in part because of the way it publicized that event.

The second question goes to the heart of the “open government” or “open data” movement: there are reams of information contained in theoretically public databases that many people might still consider private, whether it’s school information or voting records — in the same way that information on Facebook or other social networks may technically be public, but is also seen by many as private (as illustrated by a recent incident involving Randi Zuckerberg, sister of Facebook’s co-founder and CEO). At what point are we justified in making that data fully public? The answers to those questions are far from clear.

Post and thumbnail images courtesy of Shutterstock/Demis Vrublevski

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• Connected world: the consumer technology revolution
• NewNet Q1: Content Farms and Niche Networks on the Rise
• Readers weigh in: future prospects for Twitter

By accessing information in government databases through FOIA (Freedom of Information Act) requests, healthcare innovators are now able to share connections between doctors that are based on millions of physician referrals — a valuable indicator of who doctors hold in esteem.

Last month, Fred Trotter, a self-identified “hacktivist,” revealed that he had obtained a dataset of Medicare physician referrals through a FOIA request and was making the initial data available to those who supported a Medstartr crowdfunding campaign meant to build out his “DocGraph” and make it freely available. This week, he announced that he not only blew past his $15,000 funding goal, but was launching a second campaign to integrate his current data with an additional dataset.

HealthTap, a Palo Alto-based startup that connects patients with an online network of 17,000 doctors, also this week launched a new feature based partly on Trotter’s data. Called “DOConnect,” it combines Trotter’s Medicare data with physician data from its own site and other sources to give patients a new window into their doctors’ networks.

“This isn’t just friendships and business connections. This is who doctors trust,” said HealthTap co-founder and CEO Ron Gutman. “If you could know who your doctor’s doctor is, if you knew who they would choose, this lets you see that for the first time.”

A HealthTap visualization of the connections between physicians on its site.

The new tool, which reflects 25 million doctor referral connections, enables patients to see how many doctors are linked to a particular doctor, as well as their locations. As patients search for new physicians and specialists, being able to see who their current doctors are linked with could help them decide who to visit.  It also gives doctors an opportunity to build online networks that reflect their offline networks, Gutman said. In a post about his “DocGraph” project, Trotter said that his data wasn’t strictly a “referral” data set because, in some cases, doctors might be linked through a patient they both happened to see at the same time, not through an active referral. But Gutman emphasized that HealthTap’s DOConnect considered more than Medicare referrals in mapping connections between doctors.

In releasing the dataset, Trotter said his main goal was to create doctor-rating algorithms that “patients find useful and doctors find fair.” But he also hoped that academics, health policy wonks, entrepreneurs and others would use it to bring more transparency to health care overall.

Todd Park, the U.S. Chief Technology Officer, has frequently talked up the value of “setting data free” and has backed hackathons, “datapaloozas” and other open data initiatives to highlight the need for innovators to use government data for the public good — this is a great example of that vision and, hopefully, points to more similar projects in the future.

“Our goal is to empower the patient, make the system transparent and accountable, and release this data to the people who can use it to revitalize our health system,” Trotter wrote.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• GigaOM Research highs and lows from CES 2013
• Listening platforms: finding the value in social media data
• The importance of putting the U and I in visualization

Think again. Not long ago, an enterprising group of open data enthusiasts calling themselves openPlanB (cheeky tagline: ‘innovation without permission’) got their hands on a CD-ROM containing station coordinates, bus and train schedules and so on. Having converted it to JSON, early this month they put all the data online in torrent form, under the Open Database License (ODbL).

Here’s an example of the neat stuff you can do with this data – just a pretty animation, but think of all the third-party apps that could benefit from the raw material:

The reaction hit on Friday, when Deutche Bahn sales chief Birgit Bohle published an open letter to openPlanB provocateur-in-chief Michael Kreil. The letter asserted that Kreil had broken the law by publishing the data under the ODbL, as he was not the copyright owner – Bohle said Kreil wouldn’t get sued this time, but would if he persisted.

The letter said Kreil had violated the rights of not only Deutsche Bahn, but also the third parties with which it has exclusive agreements. For that, read “Google”: the U.S. web giant launched its Transit service in Germany a couple of weeks ago, based on exactly the same data we’re talking about here.

Bohle also accused Kreil of giving open data a bad name, by fooling third-party developers into thinking the Deutsche Bahn data had been legally released under the ODbL, and also because that data would be out-of-date as soon as the timetables change.

Kreil, who has been working with the Sueddeutsche newspaper since early this year on an open, interactive transport map, told me today that he’s just trying to keep the public informed.

“We asked Deutsche Bahn to open up the timetables because they’re publicly available – they’re in every train station,” he said. “But they’re afraid that third-party developers will use poor routing algorithms and then the customers may blame Deutsche Bahn.”

The French and British are opening up their transport data, Kreil noted. He also pointed out that Deutsche Bahn does have its own navigation app, but “it’s really bad”, and also useless for those without an internet connection. Such as, er, people on many trains.

“Blind people also can’t use this application,” Kreil added. “We want to fix it for free for Deutsche Bahn.”

Of course, he also doesn’t want to get sued. And for that reason, he said, he’s looking into launching a crowdsourcing effort whereby people would keep the database up-to-date by simply checking their local timetables. Automatically scraping Deutsche Bahn’s website is another option, he added.

Deutsche Bahn may be correct in saying this data is legally off-limits, but this does seem an awful lot of trouble for someone who clearly just wants to help.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• Google and the Ghost of Silicon Valley Past
• What’s driving the next phase of the e-commerce evolution
• Will cloud computing push the BRIC market to the front?

This has been a good week in that quest. Yesterday the European Parliament overwhelmingly voted through a directive that allows anyone to access ‘orphan works’ – cultural works for which no copyright owner can be located. And the day before saw the digital portal Europeana invite everyone, including commercially-minded startups, to freely reuse the metadata associated with its 20 million digitized cultural objects.

The two moves are very much connected. Europeana is made up of contributions from a wide spectrum of museums, galleries and archives, with their digitized content spanning from prehistory to today. But, when it comes to adding recent orphan works to that pool, these institutions currently have their hands tied.

That’s because, in Europe, it’s illegal to digitize orphan works. The law sees anything that was authored as under copyright, with all the restrictions that entails (the situation is similar in the U.S., although libraries and archives get a limited exemption there). And if it’s copyrighted, that means getting in touch with the copyright holder if you want to use it. If no-one knows who that is, then tough luck.

So, by 531 votes to 11 with 65 abstentions, the European Parliament voted on Thursday to recognize orphan works for the first time. Content will be granted that status as long as a diligent search is conducted to try identifying the author. If the author later pitches up, they can then claim compensation for its use, but public institutions would only have to pay a small amount as long as the use was non-commercial.

That doesn’t mean museums, for example, couldn’t sell postcards of an ‘orphan’ photograph (the directive covers audiovisual and printed material) – there’s a special get-out clause for that, although the revenue would have to be fed back into the search and digitization process.

There are similar initiatives underway in E.U. countries such as the U.K. but, as with most things copyright-related, the European Commission and legislators take the lead and national governments follow.

Pooling culture

So what do those public institutions do with their cultural collections? A lot of them make it available for viewing or listening through the Europeana portal. 20 million pieces of content is a hefty haul, and all of it comes with metadata.

That metadata is now free to use without restrictions, as it’s been made available under the Creative Commons CC0 Public Domain Dedication. That means it’s open to commercial use, and that, the Commission hopes, will lead to a wave of new apps and games for smartphones, tablets and the web.

Not only that, but the metadata can also now be used in linked open data projects involving, for example, both museums and the tourism sector. In short, it should allow new business models to emerge.

“We have a huge pile of cultural assets on Europeana, so that collection should be grown as much as possible (orphan works or whatever the relevant licensing is) and be exploited as much as possible, and shifting to open data lets us do that,” Ryan Heath, spokesman for digital agenda commissioner Neelie Kroes, told me.

But it’s not just the stimulation of new sectors that’s behind this drive for the opening-up of cultural data.

That’s part of it, for sure, and so is the desire to create a level playing field for businesses and institutions across the EU. To really understand the underlying motivation, though, you need to step back and think about the ‘single market’ idea as a whole.

The European Union exists because of the continent’s fractious history. It has a lot to do with keeping Europe competitive through scale but, on a fundamental level, it is there to stop member states fighting with each other. And the European project can only work if the EU’s half-billion inhabitants feel at least some sense of unity.

And one small but meaningful way to encourage unity is to digitally pool as much of Europe’s cultural history as possible and present it through a single, unified portal. As Heath put it: “We share a heritage and heritage is so important that we should all be able to access all of it for free online.”

I know this seems incredibly trivial at the moment, what with the euro crisis pushing the EU toward banking union and maybe even political – they’re no longer scared to use the word – federation. But, even if the issue of cultural metadata pales next to that kind of seismic subject matter, it’s absolutely part of the same picture.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• Will Standardizing the Cloud Cause Clarity or Confusion?
• Google and the Ghost of Silicon Valley Past
• Social media in Q1: commerce and discovery dominated