WSUS broken after SP2 update on SBS 2008

The title says it all.
As it turns out i'm not the only one and google is scattered with people having the same issue.

I had a client who had upgraded his WSUS sp1 on a SBS 2008 box to SP2, without reading the preresuisites (found here: http://www.microsoft.com/download/en/details.aspx?id=5216)

So it messed up WSUS beyond repair.

I first tried the basics, uninstalling WSUS sp2 , delete all the files in the WSUS directory etc and then reinstall the original WSUS from the 2nd DVD of SBS 2008.

The install completed successfully however WSUS was still broken, and would throw erros like:

The SimpleAuth Webservice is not working 
The ClientWebservice is not working 

WSUS administration console was unable to connect to the server using the remote API

System.IO.IOException -- De handshake is mislukt als gevolg van een onverwachte pakketgrootte.
Source
System
Stack Trace:
   bij System.Net.Security.SslState.StartReadFrame(Byte[] buffer, Int32 readBytes, AsyncProtocolRequest asyncRequest)
   bij System.Net.Security.SslState.StartReceiveBlob(Byte[] buffer, AsyncProtocolRequest asyncRequest)
Microsoft.UpdateServices.UI.SnapIn.Scope.ServerSummaryScopeNode.OnExpandFromLoad(SyncStatus status)
  
Last one is in dutch because I'm too lazy to translate and the client installed the SBS in dutch O_O



As I said I tried uninstall and reinstall, but didn't do the trick. Then I noticed that the WSUS administration console site would not get removed from IIS sites, even though WSUS was uninstalled. So:


1 Uninstall WSUS , go to control panel --> Programs and Features --> Windows updates services --> remove --> check all the boxes to remove all of the files.

2 Delete the contents of the WSUS folder , usually located under c: , don't delete the folder itself.

3 Delete the WSUS administration Site under IIS --> Administrative Tools --> IIS Manager --> In the left pane there should be a site called 'WSUS ADMINISTRATION' --> right click --> Delete
(ONLY DO THIS WHEN HAVING SAME PROBLEM AS ME, AND ONLY THIS SITE, NOTHING ELSE!!!)

4 Install the report viewer as specified in the requirements mentioned earlier (if not done so already)

5 Download the WSUS SP2 Installer from the link specified earlier.

6 Right click installer and run as admin.

7 Now WSUS should work as before and you'll be running SP2 , but unfortunately you'll need to configure WSUS from scratch.

HP DL 165 G7 and P800 RAID Controller

I'm actually quite happy to be able to post this success :)
I have a client where they have an MSA 60(HP Storage works) in combination with the P800 Array Controller, All worked fine in an old ML350.

But now I had to try and make it work with the new HP DL 165 G7 Server they bought for running ESX.

I've encountered various problems, but I'll spare the most of them. Also I didn't note all the steps taken in detail, so just bear with me.

Problem 1:

The server comes with a p410 controller built-in, to be able to use the P800 (which is HUGE) you need to lose that controller. The P800's PCI connection is much smaller then the one on the mobo, but don't mind that and shove it in.
But then you cannot use the internal disks the server has. The workaround is to order a special SAS cable from amazon or something, to connect the disks to the P800's internal SAS Connector


Problem 2:

The BIOS does NOT give any note of the controller during the POST so there is no way to configure it. You need to download the "Offline Array Configuration Utility" from HP. You need to use the HP Usb creator, or burn it to a bootable CD.
Now you are able to use the ACU to configure the disks/volumes.
ESX DOES NOT SUPPORT LUNS OVER 2TB -512 Bytes


Actually, those were the 2 major problems...
After that , I was able to install ESX on the MSA and boot from it as well...
One thing to note: You can install the ACU and HP Management agents on the server, IF you install ESX , if you install ESXi there is no way to do this!!!

V-79-57344-34108 - An unexpected error occurred when cleaning up snapshot volumes. Confirm that all snapped volumes are correctly resynchronized with the original volumes.

Last week I had another interesting issue with backup exec.
As you can see from the title the job ended with a :

V-79-57344-34108 - An unexpected error occurred when cleaning up snapshot volumes. Confirm that all snapped volumes are correctly resynchronized with the original volumes.

and another :

V-79-57344-65033 - Directory not found. Cannot backup directory D:\HOMEDIRS\BLA\Desktop and its subdirectories.

Needless to say that the directory ofcourse was available and backup ran fine like forever until then...

But first a little scenario of the problem:

-ESX 4 Host
-Windows 2008 R2 SP1 with shares on drive "D:"
-Backup of C: has no problem what so ever
-VSS errors filling up event log during backup of D:


So to isolate the problem I had checked every goddamn Symantec KB, every MS KB... But all to no avail.
When checking out the VCenter , nothing unusual!

But then when I went clueless, and just for the fun of it connected directly to the ESX Host instead of Vcenter, I saw a Snapshot task of the said VM running and hung at 95% for like 1 week already!!(from the backup exec VMWare backup)

So I cancelled this snapshot, but still not any luck with my D: Disk.

Then my collegue spotted something I didn't see, for some reason we had 2 VSS providers:
-Backup Exec VSS Provider
-BEVSS Provider

As it turns out, when making backups of whole VMs, Backup exec generates an extra VSS Provider (BEVSS Provider) in the services console of windows, and when backups are complete removes it again. But because the snapshot hung, the service didn't get removed and somehow backupexec messed up!

Now to solve this:

There is a script that get's called after a VM Backup Job ( Post-Thawte-script.bat)
located in "c:\windows"

Run this script manually and reboot, this should fix the issue and remove the extra VSS Provider!!

more info here: http://www.symantec.com/connect/forums/backupexec-2010-bevss-provider-installs-inself

Blackberry Administration Service keeps stopping with event ID 7031

I just had a BES Server where the Administration service - Application service would keep starting and then stopping.
Turns out that the service was competing with M$ DNS Server for the ports.
To fix this open the registry editor and navigate to:
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters

-Right-click ReservedPorts, and then click Modify.
Paste these ports in the list:
48855-48858
45588-45588
49955-49955
45599-45599

BES Express Server stops working after installing outlook

Last I came across an issue which looking back at it was obvious enough, but not at the time:
I needed to export some mailboxes to pst files on exchange 2010 NOT SP1 using the EMS (Shell), so as you might (not) know you need Outlook x64 installed on the server where you want to do the export from.

No problem, I just installed outlook and proceeded with my exports. But then it turbed out that my BES Express which was on the same server stopped working. The BES Services wouldn't start anymore.
So after some digging in the event logs it turns out that outlook overwrites the mapi32.dll file that you installed as a BES Prerequisite (MAPI CDO 1.2.1 pack).

The solution:

-Uninstall outlook , then reboot the server.
-Then go to control panel --> Add/Remove programs --> select the MAPI/CDO pack from the list and remove it.
-Download the pack again from here
-Reinstall the pack
-That's it, the BES Services should start now and you should be up and running again.

Recovering PHANTOM vms

Ok this will be a post about something that had me baffled for 3 days...
We had an esxi server with 2 live vm's which were a windows 2008 DC and a 2008 Exchange 2010 server.
Someone at work somehow managed to remove both the vm's vmdk,vmx and snapshots.
So you would think this was a genuine eipc fail. The vms were no longer listed in vsphere, and in the datastore there were only Flat.vmdks(luckily!) and some delta files.

However upon investigation it turned out that the vms were still running???!!!
They were phantom vm's. the domain was still functioning and so was exchange, but there was nothing to see in esxi.

You don't need to be a rocketscientist to realize that this is both good and bad , if the vm is restarted for any reason then the vm and domain is toast. But it did give me the chance to think of the best recovery option. Cloning was not an option since it wasn't listed in vsphere, nor was VeeAm or anything else.Also there were about 4 snapshots per machine, which were also deleted as a result so was the snapshot chain.

So here's how i did it.
I've tested 2 options and both were successfull.

Option 1 = Take backup with backup exec, recreate vmdk(descriptor file) and recreate vm , then restore the backup taken with BE. (Did this with the 2008 DC)

Option 2 = do a V2V conversion of the VM resulting in a new vm, with all your settings saved. (Did this with the Exchange Server 2010)


Option 1:

TAKE A BACKUP FIRST OF ALL YOUR NEEDED FILES!!! there is no guarantee that you will have a working vm after this!!

-Log into the ESX using ssh/console
-CD into the right folder (/vmfs/volumes/yourdatastore/yourphantomVM)
-to recreate a vmdk( disk descriptor file):
first you need to have the size of the FLAT file in bytes. You can check this by issueing: "du " this will output the size in bytes.
-then enter this command to create the files:
vmkfstools -c -youramountofbytes- -a lsilogic -d thin temp.vmdk
This will create a TEMP.vmdk disk , with an lsilogic controller (this is the default one, but in your case could be buslogic or anything else, please check.)
And it will mark the disk as thin to save space. if your FLAT.vmdk is thick provisioned, then you will need to change this later.
-Now we have the needed .vmdk file (temp.vmdk) so now we can remove the temp-flat.vmdk since we don't need it:
rm temp-flat.vmdk
-Next we need to rename the new temp.vmdk to the same name as your FLAT file.
to do so issue: "mv temp.vmdk -yourfilenamereflectingtheFLATfile-.vmdk"
-Now we need to edit the descriptor file to point to the right disk.
"vi yourfile.vmdk"
-In case you're not familiar with vi editor, you need to press the "insert" button to be able to edit the file and escape to exit edit mode. to save your chanes enter ":w" to exit VI enter ":q".
-Now edit the file so it resembles:

# Disk DescriptorFile
version=1
CID=fb183c20
parentCID=ffffffff
createType="vmfs"

# Extent description
RW 8388608 VMFS "YOUR FLAT FILE.vmdk"

# The Disk Data Base
#DDB

ddb.virtualHWVersion = "4"
ddb.geometry.cylinders = "522"
ddb.geometry.heads = "255"
ddb.geometry.sectors = "63"
ddb.adapterType = "lsilogic"
ddb.thinProvisioned = "1"

-NOTE THE ddb.thinProvisioned = "1" LINE, IF THE DISK WAS THICK PROVISIONED, THEN CHANGE THIS TO 0!!

-Now we have our new disk descriptor file, we can recreate the VM.
-Knowing you have a backup of the phantom, you can power it down.
-Log into vsphere/vcenter and create a new vm, before completion choose to edit the hardware.
-Remove the disk attached to the vm and click "add" add a new hard disk and browse to the new .vmdk you made.
-Hopefully this will boot your vm without any problems and all is well.
-If you have the same issue I had , which was you realize you could only recover the Base disk and see that the VM is in a state it was before any snapshot made. This is because the snapshots are toast as well. Therefor I hope you listened and made a backup using Backup Exec or something so you can restore the AD/system state etc.
-Now you only need to clean up a bit, because now you will have a new VM with disks residing in another folder etc, so move them to the new folder (carefully!)


Option 2:

AGAIN, TAKE A FULL BACKUP !!!!!

-Remote desktop to the vm, or use another pc for the conversion
-Download and install VMware converter stand anlone client.
-Follow the wizard, choose "convert this powered on machine"
-Choose your ESX as destination
-Select the apropriate options for the VM to be created (default is fine normal), only check the disks, and mark them as "Thin" if needed.
-Then run the converter.
-When it is done, log into your esx and you will see a fresh VM.
edit the vm's settings and disconnect the network adapter(to prevent conflicts)
-Boot the VM, and check everything using the Vsphere console. When all looks swell, you can power off your Phantom VM and start using the new one!
-Don't forget to reconnect the network adapter.

Bootmgr is missing - Server 2008r2

When you come across the error "Bootmgr is missing" in windows 7/Server 2008.
the answer is:
Load a windows cd(7/2008) for the apropriate OS and boot it, then choose "reapair my computer" , then choose command promt.(in case of HW raid, load the proper drivers first)
Type: cd recovery
Then type: startrep.exe
If you're lucky this will solve your problems automaticaly.

In my case it didn't so i had to do it manually, first of all when in the commandpromt enter:
bootrec /fixmbr
if this doesn't resolve it, make sure the right partition/disk is set as "Active".
it has to be the small partition that goes before your main drive(C:)

go back to the command prompt and enter diskpart
now enter list volume
look for the said volume (it's about 100mb)
select volume
type: Active

then make sure none of the others are marked active, select them by using
"select volume ".
Then type: Inactive

now you shoud be able to boot without problems.