Category Archives: Server Support

Bluescreen 0xc00002e2

Server Support

Judge a smidge of background first. I came in this morning and the Hyper-V guest of our windows server domain controller was getting paused while booting with a disk IO error of some sort inside the VM (there were no error within the cluster itself). It turned out that due to an issue I’m having with the DPM backups that I had a half dozen snapshots stacked up for the server. After merging them all the server then stopped getting paused but then did bluescreen with the error 0xc00002e2 and the Internet was less than helpful.

The error is caused by an issue with the active directory database and running a repair on it helpfully fixed it up. I first booted to the repair mode command prompt and tried to run esentutl /p against ntds.dit, but it failed with the error “Unable to find the callback library ntdsai.dll” and the fix for that was to run the utility from the Windows instance that was being repaired. In this case the repair tool dropped me off to an X:\ drive, but the Windows instance that I wanted to repair was actually the E: drive. So I changed the path to e:\windows\system32 so the command was something like:

cd e:\windows\system32
esentutl /p e:\windows\ntds\ntds.dit

I honestly figured it would fail, but it ran through successfully and the server booted up fine afterwards. Note that I have other DCs for this domain so I didn’t have to worry too-too much about the actual data and state of the DS database since it was just going to plowed over with the objects from one of the other DCs.

DPM and Hyper-V and iSCSI

Server Support, ,

It seemed like a good idea, and it still seems like a good idea when it works, but when my combination of Data Protection Manager, Hyper-V, and iSCSI gets grumpy, it gives me ulcers.

My recent issue started, as it usually does with this setup, with a power outage. The power went out, and when it came back on all the servers came back up properly.   “No Problem”, I thought, however a little bit later I noticed that DPM was having issues backing up the virtual servers on one of the virtual hosts.  I tried to kick the backup jobs off manually, but they failed with a VSS error of event ID 12305, “Volume/disk not connected or not found” and that the VSS provider was in a “bad state”.  I tried some of the easy things that I found on the ‘net but to no avail.  I figured I’d reboot the box and that it would “figure itself out “when it came back up (which is often the case when my DPM backups delightfully start bombing out on a virtual host).  This time however, the server came up and my iSCSI virtual machines were gone, unlisted in the Hyper-V manager.  Despite the fact that Hyper-V has delisted virtual machines on me several times in the past as well, it never ceases to cause me to question my career choice when it happens.

Investigating, I saw that the iSCSI drive along with the virtual disks were there so my hope was that the VHDs were okay.  When Hyper-V was trying to add the virtual machines it was kicking out an error along the lines of an OID that couldn’t be found.  Distressed, I decided to just recreate the virtual machines from the VHDs (not a first for me either), however I made the fortunate mistake of letting Hyper-V store the virtual machines in the default directory on the system partition.  The first two machines started up fine, but the third kicked out an error that it couldn’t write the memory file.  I had forgotten that Hyper-V keeps a swap file of the memory for the virtual machine and I had run out of space for such files on the system partition.  I figured that I’d have to recreate the virtual machines, again, but in their original directories on the iSCSI drive.  Before I went through that work again though, I figured I would take a shot at modifying the XML config for the servers that I had just created.

It was then in the Hyper-V ProgramData folder that noticed something peculiar – the links to the original non-operational virtual machines weren’t working.  When I pulled the directory listing I found them pointing at ‘F:’, but there was a different drive on ‘F:’ and the virtual machine iSCSI drive had been moved to ‘H:’.  It turns out that after the power outage the server had decided to snag an unassigned iSCSI drive that was on an attached Netgear ReadyNas box and assign it to ‘F:’.  The Virtual Host had worked fine through the week because the drive mapping didn’t take affect until after I rebooted the server.  After reassigning the drives my Hyper-V and DPM are happy again, but I’m sure they’ll get back at me eventually.

Resetting The Raiser’s Edge Password

Server Support,

I notice that Blackbaud now has a utility out that handles Supervisor password resets without going through the faxing rigamarole, but when I had to do this I don’t know if that would have been a big help anyway as my customer A) Needed supervisor access to Raiser’s Edge right away and B) They had fired the one employee who had both supervisor access and was the Blackbaud support user.

The two things I did have though was administrator access to the Raiser’s Edge MS SQL database and the SQL Server Management Studio.  My quick and dirty fix was to open the database (re7), open ‘tables’ and then find the ‘USERS’ table (dbo.USERS).  I then right-clicked the table and then chose ‘open table’.  There is one other thing that’s needed for this procedure at this point: a password that is known.  In this case I had the one’s below:

I’ll point out that I put no energy into checking out the hashing code used (i.e., can the PASSWORD field be blank?  Are the passwords hashed the same on every RE install?, etc.).  In this case all I did was copied the PASSWORD hash field from an account whose password was known to the Supervisor PASSWORD field.  (It goes without saying that I also backed the database up first (right click the database->tasks->Back Up…))

Migrating from 32bit SharePoint Services

Server Support,

This is my definitive guide on migrating from 32 bit Windows SharePoint Services 3 to SharePoint Foundation 2010 (64 bit only, oof!).  It’s not quite as definitive as I would like as I barely had time to work this through once, let alone multiple times:

  1. I changed the user account that runs WSS3 from ‘NETWORK SERVICE’ to a domain account (though I’m unsure if this step is required).  I originally did this when trying to restore the 32bit WSS3 onto the 64bit box and then using 64bit WSS3 to try and upgrade it.  Don’t do that by the way.  Anyway it made working with the database easier on the whole anyway.
  2. Backed up the SharePoint content database, called STS_servername_1 in my case. (This server used the ‘Windows Internal Database’, an extra crippled version of the already crippled SQL express.  I had installed the SQL 2005 tools on there for a different project so I used the ‘SQL Server Configuration Manager’ to change the service argument to add “;-T7806” and I then used the SQL management studio to do the backup).
  3. Installed 64bit Sql Server 2008 express on the new 64 bit server (I named the main instance the same as the instance on the 32bit box, not sure if that matters)
  4. Installed SharePoint Foundation on the new 64 bit server (it automatically set itself up in the SQL instance I installed).
  5. Restored the content database to the new server (again, using the management studio; I know that it can be done through the command prompt but I like the visual feedback).
  6. Within the Central Administration site on the new server I went to ‘Manage Content Databases’, clicked on the preinstalled content database and removed it in the subsequent interface.  This step removes the content linked to ‘/’ so that the restored database can go there.
  7. Added the restored database to the farm using stsadm. (in our case: stsadm -o addcontentdb -databasename STS_servername_1 -databaseserver newserver-url http://newserver/)
  8. Upgraded the content database using the ‘upgrade-spcontentdatabase’ command within the Sharepoint PowerShell.
  9. The content should now show up when you pull up http://newserver/ , but to complete the upgrade (and keep the content database from showing as ‘mostly upgraded’ or something along those lines) I had to upgrade the view under ‘Site Settings’ within the Sharepoint site and then going to (I believe) ‘Visual Upgrade’.

Obviously our setup is fairly simple with one server driving the whole ‘stack’.  Our content is also fairly static (it didn’t matter that the database backup that I was working with was a day old).  Your results, of course, may vary (but will probably do a sight better than trying to put something together from the amalgamation of websites that only convey one piece of the puzzle!).

MAS Client Issue

Server Support

Part of my motivation for patching up the system was so that we would be running a version of MAS 90 that was officially supported on 64bit Windows 7.  An odd issue that’s possibly related to compatibility cropped up today though.

When going into one of our companies, MAS would crash/stop responding when going in to “reprint journals” under the general ledger reports.  It worked for every other company AND even worked for that same company if the report-program was run from a 32 bit system.

The big clue was that the UI for the report never launched after clicking OK for the accounting date, which usually tells me that an ‘output issue’ exists.  After comparing the settings under company maintenance it turned out that the one that didn’t work didn’t have “Use Workstation Default Printer for STANDARD Report Setting” or “Form Code” checked.  After checking those two boxes and accepting the changes “reprint journals” began working for 64 bit systems.