Telematique, water and fire.

Wired Scenes -- Netsec and Virtualization

Greg Ness continues to develop a narrative that hits close to home.  It's clear that his lunchtime conversations with Allwyn Sequeira get into the "deep tech."

Greg and Allwyn arrive at some interesting conclusions with respect to network security and virtual machine migration.  In response to a comment on his Archimedius blog, Greg characterizes the idea of limiting VM movement to specific security domains as a short term fix.  In a sense I agree, but my consideration of the long term brings me to the conclusion that the solution lies in policy implemented by "smart" constraints.

Greg hits one key issue when he states:

There is also the issue of lock-stepping server and network policies, at a time where virtualization is enabling more responsiveness.

Right.   But, what if the independently established network policies and the independently established server requirements could be reconciled? What if we could identify a solution that satisfies the requirements of both, a workable compromise?  That's an approach we're pursuing at Replicate Technologies, the company I officially joined last month.

Consider this:  It's not only network security, but also network integrity that must be maintained when supporting the group migration of VMs.  If one wants to move an N-tier application using VMware's VMotion, one wants a management framework that permits movement only when the requirements of the VM "flock" making up the application are met by the network that underpins the secondary (destination) location.  By that, I mean:

• First, the assemblage of VMs need to arrive intact. 

If, because of a change in the underpinning network, a migration "flight plan" no longer results in a successful move by all the piece parts, that's trouble.  If disaster strikes, you don't want to find that out when invoking the data center's business continuity procedure.  All the VMs that take off from the primary location, need to land at the secondary.

• Second, the assemblage's internal connections as well as connections external to the "flock" must continue to be as resilient in their new location as they were in their original home. 

If the use of VMotion for an N-tier application results in the a new instance of the application that ostensibly runs as intended, but is susceptible to an undetected, single point of network failure in its new environment, someone in the IT group's network management team will be looking for a new job.

The "containers" I'm speaking of are both security containers and network integrity containers.  In order to be useful in a production computing environment, these containment areas must have identified permeabilities ... connections with peers, customers, and providers.   

I'll take the opportunity in the coming weeks to write more about the nature of the containers we (Replicate) foresee putting into place.  In the meantime, be assured that, like Greg and Allwyn, the last thing we want to put into place is a brittle, overly restrictive sandbox that incarcerate VMs.  In addition, we want to make best use of VMs and VLANs, while simultaneously restraining any VM under management from making VLAN connections that would compromise network security.

Virtsec in the Trenches | AlwaysOn

...

LIMIT VM MOBILITY TO WITHIN SECURITY DOMAINS

At my Archimedius blog one of the visitors shared his strategy, which involved limiting VMs to movement within specific security domains. While this is a smart short term step, over the medium and long term it still represents limiting the infrastructure agility enabled by virtualization at the heart of the business case. There is also the issue of lock-stepping server and network policies, at a time where virtualization is enabling more responsiveness.

Even if you constrain movement the VMs behind well-tuned intrusion protection systems and firewalls may still be vulnerable. Unpatched instances can appear minutes after signature tunes or vulnerability scans. As I mentioned in The Beginning of the End, the heightened flexibility of virtualization introduces change factors that hasten the obsolescence of any static security measure. You may limit VM traffic within security domains, but you still have the issue or percolating vulnerabilities within each domain microcosm. Adding more domains means more management and observation resources and less flexibility.

RE-ARCHITECT THE NETWORK

A similar notion is to architect the network around security (and other constraints). This may be tolerable in the short term, but over time you could offset some of the benefits and flexibility of virtualization by constraining traffic to cordoned off VLANs. In his NGDC deep dive presentation (which inspired this blog series about) Blue Lane CTO Allwyn Sequeira notes the eventual outcome “VLAN spaghetti”, a term used long before virtsec. Taking it a step further you get heightened complexity and constrained mobility.  ...

Powered by ScribeFire.

MAC Attacks and Disguise

When I started reading this, I thought it was going to go in a completely different direction... something akin to providing VMs with a unambiguous name/identifier that would potentially ease some of the burdens of VM management.  Whoa... was I wrong on that one.

Kutz posits that in order to defend VMs from malicious attacks, administrators might disguise the VM by establishing a disguise -- a MAC address of a type of server other than what it really is.  This, he posits, would make it less amenable to programmatic attacks.  Well... that might be the case, but it raises the other issue of VM management, administration and discovery by legitimate third parties.  It also would place a distinct burden on VM management systems (such as VMware's VirtualCenter) to support this kind of disguise without, itself, getting confused about what kind of device is sitting out on the network.

Return of the MAC — Server Virtualization Blog

... Virtualization vendors also produce Ethernet adapters — virtual network interface cards (NICs). Most VMs would be rather useless if they could not access some sort of network, so virtualization vendors must create virtual NICs in order for the VMs to get on the big wide world of Webs. And since these virtual NICs have to participate on the network just as if they were physical, they must use MAC addresses. Because the first 24 bits of these MAC addresses, the OUI, is organization-specific, there is a real potential for network administrators to detect not only if a machine on the network is virtual by its MAC address, but also what type of virtual machine it is (what vendor’s software is hosting it). While best practices dictate that you do not change the MAC address of VMs, enterprise virtualization solutions do present this as an option, and, because of this, here is the scenario I see occurring.

One way to harden the Apache Web server is to use mod_security to alter the Web server’s signature. For example, you can fool clients into thinking that the Web server hosting their favorite videos is actually a Microsoft Internet Information Systems (IIS) 5.0 server instead of Apache 2.2. Administrators do this in order to fool attackers into attempting the wrong types of attack vectors. Even though best management practices dictate that administrators NOT alter their VMs’ MAC addresses, I forsee them doing so anyway in order to fool would-be hackers into attempting the incorrect attack vectors on VMs. For example, if a VM is hosted on ESX and its MAC address has an OUI registered by Microsoft, then a would-be attacker may try known Microsoft Virtual Server or Hyper-V exploits on the VM instead of ESX exploits.

Who knows? Twelve months from now altering a VM’s MAC address to be that of another vendor may be considered a best practice, but right now, with the already complex problem of managing virtual hardware, IT administrators are best served to leave their VM MAC addresses well enough alone.

Of course, that doesn’t stop the idea from being completely and utterly cool!

Powered by ScribeFire.

Server virtualization... it's business critical

What more can I say?  Server virtualization is definitely not a flash in the pan.

Server virtualization rated as business critical by majority of large organizations, says TheInfoPro | Tekrati Research News

TheInfoPro today announced that over two-thirds of users view server virtualization -- now the second highest priority among Server professionals -- as critical to achieving their business objectives, with another 28 percent viewing it as valuable. Ninety percent of Server professionals view virtualized servers as the next enterprise IT server platform. Further, they believe that hosting applications on their own physical servers will become the exception.

“There are many factors driving this continued shift to a virtualized server environment, but the primary motivator indicated for F1000 organizations is cost-savings, followed closely by server sprawl, and consolidation,” said Bob Gill, TIP's Managing Director of Server Research. “Though cost savings will still be considered important, by 2010, as organizations catch up with the current pace of virtualizing environments, this line of thinking will give way to other drivers being considered primary benefits, such as dynamic provisioning and disaster recovery.”

According to TIP's study, this relentless expansion of virtualization and growth in new applications has given way to a continuous increase in server software spending, with 50% of organizations indicating increased spending. In fact, Server pros are indicating that, in some case, over $5M is being spent on these products yearly.

Powered by ScribeFire.

Ping and pray

This snippet from Virtual Strategy Magazine, courtesy of Tarry Singh.  The point of the post is that while migration (hot, warm or cold) is one of the blessings of server virtualization, it's not sufficient to use conventional approaches to failover.  The first point is particularly noteworthy: if clustering and failover in the virtualized data center require serious investment in manual configuration, setup, scripting and testing, we've lost a good deal of the benefit. 

Virtualization For Everyone: Marathon Technologies on Virtualization and High Availability

But protecting virtual environments from unplanned downtime is a different matter. In many cases, virtual environments employ traditional clustering and failover techniques, which use rudimentary heartbeat pings to check the status of a virtual machine. This approach suffers from several drawbacks:

• Clustering and failover add cost and complexity to the environment, requiring manual configuration, setup, scripting and testing to define the appropriate actions to take in case of failures. This additional administrative complexity can introduce errors, contributing to availability issues.
• Heartbeat pings are unable to reliably determine the health of a virtual machine and may not distinguish between I/O path failures, server failures, and lack of system resource. In some cases, these limitations may result in unnecessary or false failovers. In other cases, discrete storage or network device outages are not identified as failures and the system does not fail over.
• The failover process is far from certain; it assumes that the administrator has configured the standby system appropriately for the application and has maintained that configuration. If the target system is not configured appropriately, then when a failover does occur, the application or virtual machine is inoperable on the standby system, causing a "failed failover." Given the sense of uncertainty, some refer to this approach as "ping and pray."

Technorati Tags: virtualization, BCR,