Thursday, July 11, 2013

Microsoft Failover Cluster event 1196, standard fixes do not work

Working at a client of mine we ran into an error 1196 Source Microsoft-Windows-Failover-clustering. this error indicates:

Cluster network name resource 'Cluster Name' failed registration of one or more associated DNS name(s) for the following reason:

The handle is invalid.
.
Ensure that the network adapters associated with dependent IP address resources are configured with at least one accessible DNS server.

There are many posts on the internet addressing this problem but none of them worked for us. take a look at them if you want:

  1. http://smtp25.blogspot.nl/2008/10/cluster-network-name-resource-name_23.html
  2. http://blog.subvertallmedia.com/2012/12/06/repairing-a-failover-cluster-in-windows-server-2012-live-migration-fails-dns-cluster-name-errors/
  3. http://haythamalex.wordpress.com/2012/10/02/cluster-name-failed-registration-of-one-or-more-associated-dns-names-for-the-following-reason/
  4. http://jaminquimby.com/joomla253/9-uncategorised/473-cluster-name-resource-failed-registeration-in-dns
We used the suggestions of these post and found out they are not sufficient for us; what is the matter?
All posts point to DNS registration, all fixes do not really touch DNS records.

I our case the DNS record in DNS (the CNO (Cluster Name Object)) A-record turned out to be faulty. when we removed the record we cleaned up our AD-integrated DNS records like this:

  1. removed A record from the DNS server on which the cluster nodes try to register themselves
  2. Update Server Data Files
  3. Clear Cache
  4. Restart DNS service
  5. Do this on all DNS servers in your domain or wait for the DNS replication to distribute these changes
Now we fixed the Cluster with the following recipe:
  1. Temporarily move the CNO account into the Computers container
  2. Log into one of the cluster nodes with a domain account that has the ResetPassword right in the domain
  3. Simulate failures for the cluster Network Name resource until it is in a permanent failed state
  4. Once the resource is in a Failed state, right-click on the resource, choose More Actions and then click Repair
Once this was done the A-record was created on the primary DNS server the cluster nodes point to. (this time with the right object rights)

till next time.

Important information on Error 1196: Microsoft has provided a Hotfix alailable for download here:
http://support.microsoft.com/kb/2838043
 
 

Help, my virtual DC has got virtual time!

Time services are very important in an Active Directory environment, this is because some required security mechanisms in AD do heavily rely on the correct time of a client; especially Kerberos. this service is responsible for proper authentication in a domain and Kerberos by default does not trust client tickets with a client time delta greater then 5 minutes.

To get an understanding about the Windows Time services please do read http://tigermatt.wordpress.com/2009/08/01/windows-time-for-active-directory/
The key to understanding time services in an Active Directory is: there is only ONE reliable time authority in the directory, this is the Domain Controller which holds the PDC FSMO role (the domain controller that emulates a Windows NT PDC). all other machines do synchronize their time to this PDC emulator. generally as a client you do not have to worry about your local windows time and possible skew to the PDC emulator.

So what's the purpose of this post then?

Look at this picture:
you can imagine, when the PDC emulator has got troubles with its own time; the entire Windows Forest has got time problems.

A computer keeps track of time in the following way: System time is measured by a system clock, which is typically implemented as a simple count of the number of ticks that have transpired since some arbitrary starting date, called the epoch :source: http://en.wikipedia.org/wiki/System_time

Now look at this:


The Host (physical machine) must divide all its processor slices between all VM's (and also reserve some slices for itself). so when your PDC emulator is a virtual machine it cannot rely on its own 'hardware' clock because it is inaccurate.
Windows will indicate problems like these with event error: ID 50, Source W32Time.



the way to tackle this problem is: use the virtualization software integration services. Hyper-V, for instance, installs Hyper-V Time Synchronization Service which will use the Host hardware clock to keep time.

So the bottom line in virtualized environments is:
  • Keep the Virtualization hosts time in sync with a reliable clock.
  • Use the integration services of the virtualization software to couple the Windows time to the hardware clock of the underlying host
  • Keep windows time services running (they are enabled by default), configure the PDC emulator to sync with the same time source the virtualization hosts use
My previous post (the one from July 10, 2013) can help identify what DC's are syncing with whom and the time skew they have.

Till next time

Wednesday, July 10, 2013

script to check synchronization of time service (NTP Offset) of the Domain Controllers in an Active Directory in a GUI

In larger Active Directory environments the synchronization of time on all domain controllers is essential for a healthy AD. AD services will run into problems if the times on domain controllers are ‘out-of-sync’ especially Kerberos (the authentication mechanism used by Windows Active Directory) is sensitive to this.

To check for the domain controllers and their delta (the deviation from their own local time entry and the time on the domain controller the machine synchronizes with) the command W32tm can be used.




The output is a bit hard to read though so i made a little script that extracts the NTP offset and its syncing DC to list it in a small GUI.

the output looks like this:



the GUI is generated with SAPIEN’s great PrimalForms Community Edition. the code is added here so you can use it for yourself. the script does not need any configuration because it uses the ActiveDirectory PowerShell module to get all domain controllers from a Directory.


















#Generated Form Function
function GenerateForm {
########################################################################
# Code Generated By: SAPIEN Technologies PrimalForms (Community Edition) v1.0.10.0
# Generated On: 9-7-2013 14:38
# Generated By: Bas Huygen
########################################################################

#region Import the Assemblies
[reflection.assembly]::loadwithpartialname("System.Drawing") | Out-Null
[reflection.assembly]::loadwithpartialname("System.Windows.Forms") | Out-Null
#endregion

#region Generated Form Objects
$form1 = New-Object System.Windows.Forms.Form
$label3 = New-Object System.Windows.Forms.Label
$tbNTskew = New-Object System.Windows.Forms.TextBox
$btnext = New-Object System.Windows.Forms.Button
$btPrevious = New-Object System.Windows.Forms.Button
$tbMaster = New-Object System.Windows.Forms.TextBox
$label2 = New-Object System.Windows.Forms.Label
$label1 = New-Object System.Windows.Forms.Label
$tbDC = New-Object System.Windows.Forms.TextBox
$InitialFormWindowState = New-Object System.Windows.Forms.FormWindowState
#endregion Generated Form Objects

#----------------------------------------------
#Generated Event Script Blocks
#----------------------------------------------
#Provide Custom Code for events specified in PrimalForms.

#the ActiveDirectory module is used to list all domain controllers into an array called $ArrayDC
Import-Module ActiveDirectory
$index=0
$ArrayDC=(Get-ADDomainController -Filter *|select name)

#Function Get-DCTime runs w32tm /monitor on the tested DC
#all subsequent sentences extract the values and list them in the textboxes
Function Get-DCTime($index){
$testit=$ArrayDC[$index].name
$tbDC.appendtext($testit)
$substr=(w32tm /monitor /computers:$testit|where{$_ -like "*NTP:*"})
$NTPvalue=$substr.substring(9,20)
$tbNTskew.appendtext($NTPvalue)
$RefID=$substr.substring(36)
$tbMaster.appendtext($RefID)}

#the previous and next buttons invoke Get-DCTime on a previous- or next indexID
$btPrevious_OnClick=
{
$tbDC.clear()
$tbNTskew.clear()
$tbMaster.clear()
$index=$index - 1
Get-DCTime $index
}


$btnext_OnClick=
{
$tbDC.clear()
$tbNTskew.clear()
$tbMaster.clear()
$index=$index + 1
Get-DCTime $index
}

$OnLoadForm_StateCorrection=
{#Correct the initial state of the form to prevent the .Net maximized form issue
    $form1.WindowState = $InitialFormWindowState
}

#----------------------------------------------
#region Generated Form Code
$System_Drawing_Size = New-Object System.Drawing.Size
$System_Drawing_Size.Height = 232
$System_Drawing_Size.Width = 407
$form1.ClientSize = $System_Drawing_Size
$form1.DataBindings.DefaultDataSourceUpdateMode = 0
$form1.Name = "form1"
$form1.Text = "AD Time services checker"

$label3.DataBindings.DefaultDataSourceUpdateMode = 0

$System_Drawing_Point = New-Object System.Drawing.Point
$System_Drawing_Point.X = 41
$System_Drawing_Point.Y = 115
$label3.Location = $System_Drawing_Point
$label3.Name = "label3"
$System_Drawing_Size = New-Object System.Drawing.Size
$System_Drawing_Size.Height = 23
$System_Drawing_Size.Width = 125
$label3.Size = $System_Drawing_Size
$label3.TabIndex = 7
$label3.Text = "NTP Offset relative to"

$form1.Controls.Add($label3)

$tbNTskew.DataBindings.DefaultDataSourceUpdateMode = 0
$System_Drawing_Point = New-Object System.Drawing.Point
$System_Drawing_Point.X = 190
$System_Drawing_Point.Y = 112
$tbNTskew.Location = $System_Drawing_Point
$tbNTskew.Name = "tbNTskew"
$System_Drawing_Size = New-Object System.Drawing.Size
$System_Drawing_Size.Height = 20
$System_Drawing_Size.Width = 177
$tbNTskew.Size = $System_Drawing_Size
$tbNTskew.TabIndex = 6

$form1.Controls.Add($tbNTskew)


$btnext.DataBindings.DefaultDataSourceUpdateMode = 0

$System_Drawing_Point = New-Object System.Drawing.Point
$System_Drawing_Point.X = 191
$System_Drawing_Point.Y = 164
$btnext.Location = $System_Drawing_Point
$btnext.Name = "btnext"
$System_Drawing_Size = New-Object System.Drawing.Size
$System_Drawing_Size.Height = 23
$System_Drawing_Size.Width = 75
$btnext.Size = $System_Drawing_Size
$btnext.TabIndex = 5
$btnext.Text = "Next"
$btnext.UseVisualStyleBackColor = $True
$btnext.add_Click($btnext_OnClick)

$form1.Controls.Add($btnext)


$btPrevious.DataBindings.DefaultDataSourceUpdateMode = 0

$System_Drawing_Point = New-Object System.Drawing.Point
$System_Drawing_Point.X = 109
$System_Drawing_Point.Y = 164
$btPrevious.Location = $System_Drawing_Point
$btPrevious.Name = "btPrevious"
$System_Drawing_Size = New-Object System.Drawing.Size
$System_Drawing_Size.Height = 23
$System_Drawing_Size.Width = 75
$btPrevious.Size = $System_Drawing_Size
$btPrevious.TabIndex = 4
$btPrevious.Text = "Previous"
$btPrevious.UseVisualStyleBackColor = $True
$btPrevious.add_Click($btPrevious_OnClick)

$form1.Controls.Add($btPrevious)

$tbMaster.DataBindings.DefaultDataSourceUpdateMode = 0
$System_Drawing_Point = New-Object System.Drawing.Point
$System_Drawing_Point.X = 190
$System_Drawing_Point.Y = 72
$tbMaster.Location = $System_Drawing_Point
$tbMaster.Name = "tbMaster"
$System_Drawing_Size = New-Object System.Drawing.Size
$System_Drawing_Size.Height = 20
$System_Drawing_Size.Width = 177
$tbMaster.Size = $System_Drawing_Size
$tbMaster.TabIndex = 3

$form1.Controls.Add($tbMaster)

$label2.DataBindings.DefaultDataSourceUpdateMode = 0

$System_Drawing_Point = New-Object System.Drawing.Point
$System_Drawing_Point.X = 41
$System_Drawing_Point.Y = 75
$label2.Location = $System_Drawing_Point
$label2.Name = "label2"
$System_Drawing_Size = New-Object System.Drawing.Size
$System_Drawing_Size.Height = 23
$System_Drawing_Size.Width = 100
$label2.Size = $System_Drawing_Size
$label2.TabIndex = 2
$label2.Text = "Synchs with"
$label2.add_Click($handler_label2_Click)

$form1.Controls.Add($label2)

$label1.DataBindings.DefaultDataSourceUpdateMode = 0

$System_Drawing_Point = New-Object System.Drawing.Point
$System_Drawing_Point.X = 41
$System_Drawing_Point.Y = 37
$label1.Location = $System_Drawing_Point
$label1.Name = "label1"
$System_Drawing_Size = New-Object System.Drawing.Size
$System_Drawing_Size.Height = 23
$System_Drawing_Size.Width = 143
$label1.Size = $System_Drawing_Size
$label1.TabIndex = 1
$label1.Text = "Domain Controller Name: "

$form1.Controls.Add($label1)

$tbDC.DataBindings.DefaultDataSourceUpdateMode = 0
$System_Drawing_Point = New-Object System.Drawing.Point
$System_Drawing_Point.X = 190
$System_Drawing_Point.Y = 34
$tbDC.Location = $System_Drawing_Point
$tbDC.Name = "tbDC"
$System_Drawing_Size = New-Object System.Drawing.Size
$System_Drawing_Size.Height = 20
$System_Drawing_Size.Width = 177
$tbDC.Size = $System_Drawing_Size
$tbDC.TabIndex = 0

$form1.Controls.Add($tbDC)

#endregion Generated Form Code

#Save the initial state of the form
$InitialFormWindowState = $form1.WindowState
#Init the OnLoad event to correct the initial state of the form
$form1.add_Load($OnLoadForm_StateCorrection)
#Show the Form
$form1.ShowDialog()| Out-Null

} #End Function

#Call the Function
GenerateForm