That’s why Morgridge Institute, a non-profit biomedical researcher based in Madison, WI, used Cycle Computing’s software atop Amazon Web Services infrastructure to process and index human stem cells to build an extensive knowledge base. Morgridge won Cycle Computing’s Big Science Challenge earlier this year which gave it access to Cycle’s technology and Amazon’s cloud, and it has now completed its run, which topped out at 8,000 cores and represented about a million hours, or 115 compute years of work, according to Jason Stowe, CEO of Cycle Computing. All for $0.0175 per compute hour.

Building an encyclopedia of cells

Morgridge’s goal was to create a knowledge base, or an index of associations between genes and the types of cells those genes could turn into, Stowe explained in an interview. “They can turn into pluripotent stem cells, which are very similar to embryonic cells in that they can potentially differentiate into any sort of cell.” The problem with using adult stem cells in the past was that a liver cell stayed a liver cell — pluripotent cells allow researchers to design the cells they need.

Morgridge Institute’s Victor Ruotti.

Victor Ruotti, molecular biologist at Morgridge, was thrilled to get the resources. “We wanted to take cells and compare them to other cells. We took as many samples as we had and gathered as much information as we could. We compared every cell type to every other tissue we had and built a database to say which cell types there are.”

That accumulated knowledge can help researchers figure out how to build the types of cell structures they need for experimentation and research.

The sample size was not that large, 124 samples, but each sample had more than 20 million data points to be compared. “When you multiply that out, it’s a very complex and resource intensive problem,” Ruotti said. Netted out, that’s a total of 1,003,303 core-hours against 11,955 pairs of samples processed, Cycle said.

The work ran on a mix of Amazon EC2 spot instances, including some high-memory instances on Centos.

Goal: curing disease in a petri dish

“We’re in an exciting phase in stem cell research [with pluripotent cells]. If a doctor needs cardiovascular cells to work on vessel or artery constructions, he can get them,” Ruotti said. “Medicine will change because doctors will be able to treat diseases in a petri dish. We can take a disease and simulate it in the lab, treat it in the lab, hopefully cure it there, and then implement the same cure in patients.”

Cycle Computing is determined to show that important workloads can run efficiently on low-cost public cloud infrastructure. Last year, it helped  Schrödinger Pharmaceutical spin up a 50,000-node Amazon cluster for its computational drug design work.

“We did a large run for a Big 5 pharma last year and the most common comment we got from the articles was, ‘I wonder if I could play “Call of Duty 4″ on this massive supercomputer.’” Stowe said. “What worried me about all the glitter was people would miss what is truly gold, which is that scientists can access world-class compute infrastructure at very reasonable cost and in a short time frame.”

Feature photo courtesy of  Flickr user CodonAUG

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• What Amazon’s new Kindle line means for Apple, Netflix and online media
• Microsoft Azure: What It Is, What It Costs and Who Should Care
• Cloud and data first-quarter 2013: analysis and outlook

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• A near-term outlook for big data
• Defining Hadoop: the Players, Technologies and Challenges of 2011
• Infrastructure Q2: Big data and PaaS gain more momentum

The problem for computational chemists and biologists is there’s a trade-off between accuracy and speed. This  ”Naga” compute environment built by Cycle atop the Amazon cloud eased that tradeoff, said Ramy Farid, president of New York-based Schrödinger, which specializes in computational drug design.

“We’ve got these really accurate methods but they would take months on a normal cluster. The problem is we want to do the best possible science fast,” Farid said in an interview this week.

That’s where Cycle Computing comes in. The company has made its name building high-performance computing atop AWS infrastructure and has previously deployed 10,000 and 30,000 core clusters on the cloud. For Schrödinger, it upped the ante to 50,000 cores. The alternative in this case would be for Schrödinger to build its own 50,000-core cluster or log time on a supercomputer, said Cycle Computing CEO Jason Stowe.

HPC for rent

“Practically speaking, the latter is impractical for a for-profit company and is generally restrictive. If you’re an academic wanting time on the San Diego Super Computer, for example, you’ll have months of wait time to get approved and even then you get a limited-time window — so if something with your software is not working at that time, you’re out of luck.” And, big supercomputers don’t tend to run the kinds of software these companies want to run. “The beauty of the cloud is it runs your flavor of Linux and other software,” Farid said.

On the other hand, building a 50,000 core cluster could easily cost $20 million to $30 million, he said. The Schrödinger project, by contrast, cost about $4,850 per hour to run.

For this trial, the cluster had access to all regions of AWS and used all of them in some capacity. The application used the various EC2 APIs to provision the resources.

“All the compound data for analysis was uploaded into S3 [Amazon's Simple Storage System]. The cluster was provisioned along side it and grabbed data from S3 to run the calculations and then pushed it back into S3,” said Crowe, who will be talking about the implementation Thursday at an Amazon event in New York. The application also took advantage of some Amazon IP address and DNS capabilities.

Finding a needle in a very big haystack

Farid could not talk much about the specific research purpose other than to say the goal was to find compounds that could be developed into drugs that fight a type of cancer.  The use of the huge cluster enabled Schrödinger to use the more accurate version of its Glide software — in the past it would have had to use the less accurate screen and perhaps miss some compounds that could be extremely useful.

“The problem they’re solving is amazing. It’s like the target is a lock and the confirmation is a key — what Ramy’s software does is let you simulate 21 million keys potentially matching that lock. There are typically a number of false negatives and the reason is you can’t sample all the orientations [of key to lock] properly so it will not show a match that could actually be a match. You could miss amazing drugs that could have an impact,” Crowe said.

The use of this huge cluster enabled Schrödinger to run its much more accurate, but much more compute-intensive version of its screening software and find compound candidates the other software may have missed.

The result of the three-hour run? “We identified  a number of compounds that we will purchase and test,” Farid said.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• 2012: The Hadoop infrastructure market booms
• A near-term outlook for big data
• NewNet Q2: Google closes the quarter with a bang