Telematique, water and fire.

Attention Span 2009.05.02

What caught my attention today... actually, I tried to stay OFF the web today, and focus on writing. So, not too much:

Number Crunching Made Easy

Newsweek calls out the use of the OSS emanating from Argonne. Globus' Nimbus makes the mainstream press.

The cloud is already making high-performance computing more readily available to researchers in the developed world. The Nimbus project at the Argonne National Laboratory in Illinois has developed open-source software that can launch a virtual supercomputer within minutes on Amazon's Elastic Compute Cloud. Earlier this month, nuclear physicists at the Brookhaven National Laboratory in New York used the service to rush through a set of new simulations on data from the lab's Relativistic Heavy-Ion Collider—set up to give a glimpse into what the universe may have looked like in its first few moments—rather than waiting weeks or months for a slot to open up on the lab's big computers. The innovation could have a profound effect: simplifying and speeding up research in everything from renewable energies to drug testing.

Neo4j - the graph database

This, to me, sounds so appropriate for a number of projects I've been contemplating. I'm hoping that the skills curve is not so steep that I can't chip away at it.

Neo4j is a graph database. It is an embedded, disk-based, fully transactional Java persistence engine that stores data structured in graphs rather than in tables. A graph (mathematical lingo for a network) is a flexible data structure that allows a more agile and rapid style of development.

You can think of Neo4j as a high-performance graph engine with all the features of a mature and robust database. The programmer works with an object-oriented, flexible network structure rather than with strict and static tables — yet enjoys all the benefits of a fully transactional, enterprise-strength database.

Cloud Computing

Clouds, the Criminal Element and V12N to the Rescue

Christofer Hoff has an interesting post today that reminded me of a conversation I had with Steve Tuecke, Carl Kesselman and Ian Foster in 2004 when we were establishing Univa.  The conversation pointed out that grid computing was, with little fanfare, a fundamental basis for the botnets being implemented by the "bad guys", and that grid computing models would be the most reasonable basis on which to protect the enterprise from a significant portion of intentional security threats.

The challenge, as Hoff makes clear, is the capability of establishing appropriate (and appropriately malleable) policies that travel with the applications and data.  The malleability required is not likely to be found today in conventional provisioning and scheduling systems, nor in conventional configuration management systems. 

The industry is at the stage in the evolution of utility computing/cloud computing/grid computing where the flexibility and dynamic nature of virtualization has to be applied generously to the solution of infrastructure problems like security.  At Replicate, we've started to apply it to 21st century datacenter fault prevention and remediation, which is a challenge big enough to last us a while.  It's likely to bring us in contact with a number of the security issues Hoff raises, though from a different starting point.  I'll certainly be joining Hoff in watching companies grappling with this ... though Hoff's more likely to be on the playing field (BJJ mat?), and I'll be in the bleachers.

Rational Survivability: Cloud Computing: Invented By Criminals, Secured By ???

...
One of the obvious benefits of cloud computing is the distribution of applications, services and information. The natural by-product of this is additional resiliency from operational downtime caused by error or malicious activity.

This benefit is a also a forcing function; it will require new security methodologies and technology to allow the security (policies) to travel with the applications and data as well as enforce it.

The Missing Piece in Cloud Computing: Middleware Virtualization

This is an interesting piece about the role of middleware (the "classic" tiers and APIs) and of virtualization in attaining the real benefits of utility computing and Cloud Computing (though I'm still hardpressed to distinguish the two terms in a meaningful way). 

What caught my eye particularly is the approach they've taken at Gigaspaces to virtualization for the application container. This notion of bundling and consolidating the logic needed to enforce SLAs and simultaneously meet the requirements of the application architects is an approach for which I have great respect, and which we're employing in our efforts regarding network virtualization at Replicate Technologies.

Nati Shalom's Blog: The Missing Piece in Cloud Computing: Middleware Virtualiztion

...
In the current server-centric world we use middleware to provide common infrastructure services, such as application containers, data and messaging. To make that same model work in a cloud environment we need to virtualize all of those components. That is, we need to virtualize the container, the data and messaging. By doing so we abstract the application from the fact that it is running on a "cloud" and make the transition from a server-centric model to cloud computing relatively seamless. How do we achieve that?
...
The SLA-driven container takes an application bundle and manages the deployment of that bundle over a set of containers based on Service-Level Agreements. The SLAs define the clustering topology (e.g., partitioning, size of the application pool, scalability, fail-over policies, etc.). It is used to map the available physical compute resources to the application needs. It is also used to provide self-healing capabilities to our application. For example, we can set an SLA to ensure that at any point in time we always have primary and backup instances for each node in our environment - and that each node's primary and backup must run on separate physical machines. In case a primary fails, the system will dynamically set the backup as the new primary, and will launch another backup on another machine.

Powered by ScribeFire.

Grid Gurus

Rich Wellner, my friend and former colleague at Univa UD, has started editing a blog called Grid Gurus. He's got a very talented group of people contributing, including Ian Foster, Tim Freeman, Sinisa Veseli, and Scot Koranda.  The blog already has a nice collection of themes developing ... and I can't wait for the next 434 parts of "Better Know a VM."

Grid Gurus: Why the grid is still important

Grid computing is celebrating 11 years next month, and is poised to become increasingly mainstream in the coming years. There are a number of reasons that this is true, and most of them are the time tested ideas that have been proving themselves in your research institutions and businesses for years. The grid is about allowing your organization to run more efficiently and more effectively than can be done with more conventional technology solutions. It's about bringing many machines together in coordination around a task. It's about bringing data storage and movement to bear in a coordinated fashion with your application. It's about allowing people from different parts of your organization to work together more easily.

How Obvious is the eGenera Patent?

While I've been holed up in conference presentations on virtualization, I completely missed the fact that Egenera has been awarded a patent on an N+1 tiered disaster recovery solution. I came across the news through the following post by Joe Foran. I have to agree that this one should be reconsidered in light of the recent ruling on patents and "Obviousness". While definitely written with an unapologetic point of view, this post is a good read and worth the time.

Why the eGenera Patent is Dangerous — Server Virtualization Blog

...The short of it - You have multiple boxes on a network that are mirrors of one another. One fails, another takes over its role. There’s usually hardware or software in-between that keeps things synchronized and detects the failure. This part of the patent is worded to be host-, network- and processor-inclusive, which would be obvious because most clusters are situated on networks, don’t necessarily need to run the same processors, and are hosts. The “big” improvement is in the use of the term “site” - where the product is meant to restore an entire data center’s configuration. In the press release, this means that if you have four data centers and one disaster site, if any one data center fails, the disaster site takes on the complete configuration of the failed site (i.e., all nodes, network configurations, etc.). This is a huge step forward in disaster recovery, but it’s not patent-worthy because there are a zillion ways to do this.

[Update] Bob McNeil of Egenera, whom I met on Tuesday and the IDC Virtualization 2.0 conference in San Francisco, pointed out to me that Joe Foran had the company name wrong. It's Egenera, not eGenera. He also points out that the patent was more than six years in the making and covers quite a lot of IP.

Technorati Tags: eGenera, DR, disaster recovery

Technorati Tags: disaster recovery, DR, eGenera

The flawed analogy: The Electricity Grid & the Grid Computing Utility

Jon Udell is dead-on in his discussion of Amazon's S3 (simple storage service) and metered web services. And this raises an important point that many in the utility computing / grid utility sectors have yet to fathom: namely, that what's missing from the successful recipe is a rich metering, billing, accounting, and settlement infrastructure. What's interesting is that so few of the commercial providers of "grid computing" utilities or technology destined for such utilities, have given this any attention. It's a mystery to me, but unless something changes pretty quickly -- because these technologies are a lot harder to build and deploy than one would think -- we will all be living with re-worked versions of the metering & accounting systems from venerable systems management companies (think CA, Symantec/Veritas and BMC), telco billing suppliers (like Amdocs and Convergys), or the likes of EDS. How does that make you feel?

The other point that caught my attention in Jon's article is his clear thinking about the flawed analogy between the electricity grid and the provision of grid computing as a utility. He makes the point that, unlike the electric grid, what's delivered by a grid computing utility is hardly fungible to the point that we can go without itemization and "flavors" of service. The metrics aren't that simple because grid computing is not a single value-proposition.

I have some other issues with the analogy -- and believe me, I have to live with it WAY too much given what I do in my day job. (It's hard being co-founder of a grid infrastructure company, created with the three people most responsible for the "grid" terminology and many of the most crucial contributions to grid computing.) But, I'll let that list simmer for a while. Perhaps a post in a week or two that addresses why the analogy takes us all down the wrong path. In the meantime, I applaud Jon for raising this issue and specifically in the context of metering and metrics.

Metered Web services

More generally, I’d like to find out whether metering infrastructure services in this way will prove technically and economically viable. When we talk about a grid of Web services, we like to compare it to the power grid, but the analogy is deeply flawed in at least one way. My electric bill isn’t itemized. I don’t know what it costs me to run each of my appliances, or how long it will take to amortize the cost of replacements. Lacking this feedback, we make poor individual decisions that, collectively, add up to a tragic misallocation of resources.

Creating what’s called the “energy web” -- a marketplace where smart producers and consumers of power exchange price signals in real time -- will require a massive overhaul of our legacy power grid. There’s just no way for us to start from scratch. But in the realm of Web services, we’re just now building the grid. Given a clean slate, perhaps we can figure out how to aggregate demand, meter usage, and value services for what they do rather than just for the eyeballs they attract.

Technorati Tags: Grid, Utility computing

Informatica & DataSynapse

In a press release this week, DataSynapse and Informatica announced a partnership agreement whereby DS's GridServer technology will support the Informatica PowerCenter data integration platform.

DataSynapse has to expand its market beyond their successful foray into the financial services sector. Business intelligence, drafting behind partners like Informatica, seems a sensible approach to me.

Link: DataSynapse Helps Deliver Enhanced Grid Computing Capabilities to Informatica's Leading Data Integration Platform

The agreement will enhance the deployment of PowerCenter in service-oriented IT environments where heterogeneous system resources are pooled and allocated as needed to satisfy computing demand. Informatica's PowerCenter empowers customers to cost-effectively scale their data integration processing across such pooled resources. An interface with GridServer provides added flexibility to coordinate PowerCenter workloads with other applications on the grid. The result is a more agile, scalable and manageable IT infrastructure that speeds application processing time for improved throughput and productivity.

The grid infrastructure world

The grid infrastructure world just got more interesting. The article doesn't provide a schedule on which customers can buy hosted services from EMC Corp. But it does say that within 2 years EMC will offer a packaged version of the technology.

The questions that occur to me:

With what products -- EMC's or third party's -- will the new grid infrastructure be certified? Their Information Lifecycle Management product lines?

What's missing from the recipe? Scheduling? Accounting? Policy framework?

Link: EMC buys grid software, offers hosted services

EMC Corp. today announced that it has acquired the intellectual property for grid software developed by Acxiom Corp. for $30 million, and will work with the company to offer grid-based hosted services. Acxiom is a $1.9 billion public company based in Conway, Ark.

Specifically, EMC has acquired the grid middleware that Acxiom uses for internal purposes to run its hosted business analytics service. Acxiom's customers include Citigroup, Bank of America, BMW, Charles Schwab, Sears, Nationwide, Western Union, Sprint, Discovery Communications Inc. and Unilever that buy data mining, data hygiene and other business analytics services from Acxiom on a hosted basis. Acxiom is the only company that has deployed the grid software.
...

EMC Buys Acxiom's Grid Software

The grid infrastructure world just got more interesting. The article doesn't provide a schedule on which customers can buy hosted services from EMC Corp. But it does say that within 2 years EMC will offer a packaged version of the technology.

The questions that occur to me:

With what products -- EMC's or third party's -- will the new grid infrastructure be certified? Their Information Lifecycle Management product lines?

What's missing from the recipe? Scheduling? Accounting? Policy framework?

Link: EMC buys grid software, offers hosted services

EMC Corp. today announced that it has acquired the intellectual property for grid software developed by Acxiom Corp. for $30 million, and will work with the company to offer grid-based hosted services. Acxiom is a $1.9 billion public company based in Conway, Ark.

Specifically, EMC has acquired the grid middleware that Acxiom uses for internal purposes to run its hosted business analytics service. Acxiom's customers include Citigroup, Bank of America, BMW, Charles Schwab, Sears, Nationwide, Western Union, Sprint, Discovery Communications Inc. and Unilever that buy data mining, data hygiene and other business analytics services from Acxiom on a hosted basis. Acxiom is the only company that has deployed the grid software.

According to Ian Baird, chief technology officer of grid and utility computing at EMC, Axiom's software includes all the elements to build a complete grid, such as security tools, grid portals, directory and scheduling services, databases and other components. "It's a complete stack unlike many point solutions for grid … the Globus toolkit is a collection of different elements," Baird said.

Within two years EMC plans to sell this grid software to customers in a nonhosted scenario. "It will allow a company to provide IT services against its infrastructure on a utility basis," Baird said. The vision is for IT resources to be much more flexible, and to be available and paid for on demand, improving utilization and lowering costs. EMC said the grid software can run other applications besides business analytics, for healthcare, retail and other industries.
...

Link: EMC Unveils Grid Gameplan

An another article which sheds a bit more light on EMC's and Axciom's plans.

... EMC and Axciom have actually constructed a multi-faceted partnership. EMC is paying for Acxiom’s grid software, but the two companies will jointly develop and sell a hosted grid service that Acxiom offers. Eventually, they will integrate systems, software, and services into a non-hosted grid product. EMC will then sell the grid product, while Acxiom continues to sell the hosted services.

"We will jointly market the hosted solution while we are building out the real solution that we wish to deliver -- that’s a product that installs beyond the customer firewall," says Ian Baird, EMC's CTO of grid and utility computing and head of its new grid incubation unit. ...

S-a-a-S requirements for scalability and continuity

I'm not sure why people are surprised that outages still occur at service providers like Salesforce.com and TypePad. It does make for "good" tech journalism, in that it provides the journalist with a couple of hooks to warn the readership of both the wonders of Software as a Service (S-a-a-S) and its perils.

The point is this: scaling for the highly successful S-a-a-S provider is hard, and continuity is terribly important. What's not clear to the provider in advance of a move or a data center upgrade is the degree to which they need to prepare (and then pay for) business continuity. This is one of the areas Univa hopes to address by deploying Univa Globus Enterprise as part of a "lower cost" / higher performance approach to continuity and failover.

Link: Week of crashes highlights on-demand peril |
InfoWorld | News | 2005-12-21 | By Stacy Cowley, IDG News Service

...
Salesforce.com is being tight-lipped about the roots of Tuesday's outage. A software problem with one of Salesforce.com's database clusters caused the service to be intermittently unreachable for some customers for about six hours, according to Bruce Francis, Salesforce.com's vice present of corporate strategy. Salesforce.com does not yet know the extent of the outage, he said.

...

Salesforce.com's blackout followed similar downtime from other "software as a service" providers. Six Apart Ltd.'s TypePad blog hosting service went down for the day last Friday following a failed storage upgrade. Affected customers included Major League Baseball's MLB.com site, which hosts all of its blogs with TypePad. In addition, the del.icio.us bookmark-sharing service that Yahoo Inc. just bought suffered days of problems last week after its data center lost power. ...