After quite a long absence from blogging I have decided to return to discussing some of the new and (hopefully) interesting technologies that my team and I work with. I work for Sonian as the Director of DevOps, and I manage 5 very skilled individuals who assist me with the operation of our systems in the cloud. We work with lots of cutting edge technology (Such as Chef from Opscode), and where there is no software to do what we need - we create it ourselves (See Sensu for an example).

In addition - I’m working to increase my knowledge of Ruby as we use Ruby for all our tools and applications. Coming from a non-computer science background, that will be a challenge in itself. I’ve starting by reading the fantastic book written by Chris Pine - Learn to Program.

I’ll be writing more about some of the challenges of managing big data and large system at scale in the cloud - where you have no control over network and disk I/O (among other things we can’t control), and discuss how we deploy, monitor, scale and secure our systems.

Amazon recently announced a new tier of storage available within their web services cloud infrastructure.  Amazon’s current storage solution, S3, is truly the gold standard for durable cloud based storage that provides 99.999999999% durability (which if my math is right, means that for every 100 Billion objects stored in S3, Amazon “may” lose a single object every year). Amazon is listening to their customers, and now provides a lower cost (33% cheaper) S3 storage solution called Reduced Redundancy Solution (RRS).

I had started this blog initially as a way to discuss storage and virtualization solutions while working as a technology consultant.  But recently a new opportunity presented itself, and I’ve now made the transition out of consulting, and back to the start-up world.   This most recent adventure is with a company called Sonian which provides a cloud based data archiving and eDiscovery solution.  What is so wonderful about this new venture is we leverage the Amazon Web Services cloud providing us the ability to consume storage and computing by the granule.  We don’t need to make huge capital outlays in data centers, storage, servers, etc…  And since we don’t need to buy and maintain all this hardware (which will eventually be refreshed in 3-5 years), we can keep the costs low and pass on those savings to our customers.

A few months before the vSphere release VMware showed some amazing stats in regards to the increased level of I/O that can be attained in a virtual infrastructure.    They posted this info on their blog and the outcome of the testing was impressive.   They were able to achieve 350,000 I/O operations per second on a single vSphere host (ESX 4.0) and with just 3 virtual machines.  Their testing utilized the EMC Enterprise Flash Drives, which have an incredibly high throughput.  They talked about how the VMware Paravirtual SCSI (PVSCSI) adapter was able to achieve 12% more throughput with 18% less CPU cost compared to the LSI virtual adapter.

vSphere was just released to general availability today, and one of the best features of this upgrade is the addition of VMware Fault Tolerance.   From the VMware site:

VMware Fault Tolerance is leading edge technology that provides continuous availability for applications in the event of server failures,  by creating a live shadow instance of a virtual machine that is in virtual lockstep with the primary instance. By allowing instantaneous failover between the two instances in the event of hardware failure, VMware Fault Tolerance eliminates even the smallest of data loss or disruption.

I ran into an very interesting issue today with a client who is using Veeam Backup and Replication to keep their virtual machines replicated to a remote ESX server for disaster recovery.  Veeam starts a replication job and will take a snapshot of the virtual machine and then replicate the main VMDK disk file to the remote site.  When the backup job finishes Veeam will tell VMware to remove the snapshot until the next replication schedule runs.  Since we are replicating our VM’s across a slow WAN connection (600Kbps optimized with Citrix WANScalers) the replication can often timeout, or hang.  Today I noticed that the replication had not updated since last night.  So I needed to stop the replication and re-start it.  Since the Citrix WANScalers can cache as well as compress, restarting a failed replication job is usually pretty quick, as most of the data was previously cached on the Citrix boxes. Here are the details of what I found, and how I fixed it…

I am frequently asked the question about how to grow a VMware virtual disk (VMDK) and have it be recognized by the operating system.   If you are trying to simply extend a non-system volume within Windows (ie, anything other than the C:\ drive), then the process is pretty simple (refer to MS KB 325590).   But when you are trying to grow a C:\ with windows, you need to get around the limitation of extending the system partition.  This is just one more instance where VMware shows how powerful and flexible it truly is.

I just saw today that it looks like VMware ESX 3.5 Update 4 was released a couple of days ago.  I’m pretty excited about this upgrade as it includes an updated vmxnet adapter. 

From the VMware site:

Expanded Support for Enhanced vmxnet Adapter — This version of ESX Server includes an updated version of the VMXNET driver (VMXNET enhanced) for the following guest operating systems:

• Microsoft Windows Server 2003, Standard Edition (32-bit)
• Microsoft Windows Server 2003, Standard Edition (64-bit)
• Microsoft Windows Server 2003, Web Edition
• Microsoft Windows Small Business Server 2003
• Microsoft Windows XP Professional (32-bit)