SharePoint on a VM: Can Peter rescue a nonprofit in peril?

In 2005, Florida was hit hard by hurricanes — not quite as hard as New Orleans, but damage ran to several millions dollars. As a result, a nonprofit team of builders formed a disaster relief crew that’s ready to go to work should the need arise in the future. They asked if I might help get their SharePoint implementation up and running. Since I’m not that great with a hammer, this was my way of lending a hand. But what I thought was a simple request turned into many hours adding users to Active Directory, configuring SharePoint pages, and configuring SharePoint permissions.

Months passed, then the team contacted me again — it turns out they have never done a backup. What — all that data and no backup?! I started looking into solutions. After considering free Windows Backup (given that the budget is tight) as well as Mimosa NearPoint for SharePoint, which is my favorite SharePoint backup solution but overkill for this group’s needs, I settled on Microsoft’s System Center Data Protection Manager (SC DPM). It’s reasonably priced and has a bevy of features, including the ability to back up virtual machines easily.

[ Read J. Peter Bruzzese’s “Don’t be caught without a SharePoint recovery solution” | Doing server virtualization right is not so simple. InfoWorld’s expert contributors show you how to get it right in this 24-page “Server Virtualization Deep Dive” PDF guide. ]

The biggest problem was that the servers I was looking to back up were running in VMs on a Hyper-V server. The parent system was made a domain controller during the configuration (in fact, it was the only domain controller for the domain). This is a big no-no. Hyper-V parent systems should be used for backup software, antivirus software, and applications of that sort. You should not try and run heavy features like Active Directory or Exchange on the parent. My attempts to install SC DPM made that issue painfully clear: It would not install on the parent because the parent was a domain controller. Houston, we have our first of many problems.

Note: A very funny response to an FAQ section on Hyper-V that Microsoft offers has the question “Can I run applications in the Microsoft Hyper-V parent partition?” The answer: “No. The Microsoft Hyper-V Server parent partition is specifically for system management.” It’s funny because it isn’t completely accurate. You can run apps, but you shouldn’t.

So the first thing I needed to do was get the parent virtual machine to a neutral member server state. I created a new child virtual machine and joined the domain. Then I made it a domain controller in the existing domain. I moved over all FSMO roles (the five special roles that were running on the parent VM). I replicated the user accounts — all was well.

Bracing for the leap of faith And now came time for my leap of faith. I had been reading reports of things “breaking horribly” when you try to remove a domain controller running on a parent VM, with your only other domain controller being a child VM. It all works great until the reboot part, supposedly.

The smarter solution would have been to configure a separate physical system as a server and make it a domain controller, so when I dropped down the parent system and removed it from the domain, I wouldn’t have taken a 50-50 shot that everything goes black. I knew this was the smarter solution but didn’t have the .iso file with Windows Server 2008 on me to make the virtual machine on my laptop. Instead, I decided to push ahead.

As I rebooted the parent, all I could think of was the line from “Ghostbusters” where Bill Murray is told that crossing the streams is bad: “Try to imagine all life as you know it stopping instantaneously and every molecule in your body exploding at the speed of light.”

I closed my eyes and rebooted — and it all worked great! The Hyper-V parent system came up as a member server in the domain, and we were ready to move forward. Or so I thought.

As my fear of breaking Active Directory subsided, I was left with a new fear: that my child virtual machines were wrecked. Although my new domain controller came up just fine, my other two VMs, including my SharePoint server and another server running Enterprise QuickNooks, were not initiating smoothly. Urgh.

After some TechNet Forum research, it seemed the solution was to remove the parent from the domain and make it a workgroup again — another leap of faith. I did it — and it worked! Now my VMs were all up and running again. I succeeded in bringing the parent virtual machine back to a neutral state and putting all the functioning services into three child VMs.

Now I’m ready to install SC DPM. And it fails. Why? Because now the system is not a member of the domain. So I rejoined the domain as a member server (again) and rechecked to make sure all the VMs are functioning (they are — whew!), then performed all the tests to make sure everything was as it was before I started this 10-hour nightmare.

With all of this work, I still had no backup solution in place. Leaps of faith with no recovery path are just plain stupid, and I was foolish for not even taking a simple precaution with my virtual machines. I should have paused when I had three child VMs (domain controller, QuickBooks, and SharePoint) and done a simple export of the three. Or at the bare minimum, I should have brought the three servers down and copied the virtual machine .vhd files over to an external drive.

But I didn’t. Instead, I moved forward, preparing for the crash with the mindset that I was going to blame the original configuration for all the woes — which was only partially true.

I hope this in-the-trenches story teaches you two things. First, set up Hyper-V properly. Keep the parent bare, with minimal applications. Second, don’t take risks without knowing how to recover if things go horribly wrong.

So what happened? I went to install SC DPM — and it failed, even though it reported that I met all the prerequisites. But the SQL install didn’t work out and left me a message to manually install SQL 2005 with Reporting features.

Having landed short, I tried something else I decided I’d had enough. I went back to the drawing board and found a solution for using Windows Backup with the VSS service to back up my virtual machines. Of course, it had its own issue: It backed up the whole volume, not the individual VMs. But at this point I just wanted a backup, any backup. It worked; I got my backup and a somewhat happy ending.

Of course, I can’t let go of this challenge; my next move will likely be to get SC DPM up and running, even if I don’t ultimately use it to back up the virtual machines. However, there is another solution I’m looking into testing: a product called BackupAssist. I’m intrigued by the manufacturer’s claim that BackupAssist can make a VHD from the host that allows you to granularly restore VMs; also, it costs $250, but it’s available for only $150 to nonprofits. I’m going to test it out and let my faithful InfoWorld audience know the results in a future column.

What experiences have you had with Hyper-V?

This article, “SharePoint on a VM: Can Peter rescue a nonprofit in peril?,” was originally published at InfoWorld.com. Read more of J. Peter Bruzze’s Enterprise Windows blog and follow the latest developments in Windows Server technologies at InfoWorld.com.