ExchMailboxDatabase resource and WaitForFirstDAGMember script issues
kevindd992002 opened this issue · 0 comments
Problem description
We've been using the xExchangeDsc (and now ExchangeDsc) module for years now for our Exchange 2016 servers without any issues. We're using State Configuration in Azure Automation as our pull server. We deploy updated DSC conf files once, remove them as nodes in Azure Automation, and just let the drift check take care of any drift on the onprem servers.
Recently, we saw DSC errors in both the first DAG member and additional DAG member/s in both our lab and prod environments (same exact issue). Let's concentrate on the lab for now as I can do whatever there. There are only two Exchange servers there with the one DAG, one mailbox database (two copies, one on each node). So pretty straightforward and basic setup.
On the first dag member, I get this error:
Database must have only a single copy for the DB path or log path to be moved
I checked the psm1 script being used y the ExchMailboxDatabase resource and saw this block for when a database exists already:
# First check if a DB or log move is required
if (($PSBoundParameters.ContainsKey('EdbFilePath') -and (Compare-StringToString -String1 $db.EdbFilePath.PathName -String2 $EdbFilePath -IgnoreCase) -eq $false) -or
($PSBoundParameters.ContainsKey('LogFolderPath') -and (Compare-StringToString -String1 $db.LogFolderPath.PathName -String2 $LogFolderPath -IgnoreCase) -eq $false))
{
if ($db.DatabaseCopies.Count -le 1)
{
Write-Verbose -Message 'Moving database and/or log path'
Move-DatabaseOrLogPath @PSBoundParameters
}
else
{
throw 'Database must have only a single copy for the DB path or log path to be moved'
}
}
Here's a snippet of what I have in the DSC conf file:
$dagName = 'USCHIZWDAG4002'
$dbName = "$($dagName)-DB1"
# Create primary database
ExchMailboxDatabase "MDB_$dbName"
{
Name = $dbName
Credential = $Credential
EdbFilePath = "C:\ExchangeDatabases\$($dbName)\$($dbName).db\$($dbName).edb"
LogFolderPath = "C:\ExchangeDatabases\$($dbName)\$($dbName).log"
Server = $server.Name
CircularLoggingEnabled = $true
DatabaseCopyCount = 2
IssueWarningQuota = '1946MB'
ProhibitSendQuota = '2048MB'
ProhibitSendReceiveQuota = '2253MB'
AllowServiceRestart = $true
DependsOn = '[ExchMailboxServer]MailboxServer'
}
And here's the EdbFilePath and LogFolderPath of the actual database in the Exchange server:
[PS] C:\Windows\system32>(Get-MailboxDatabase).EdbFilePath.PathName
C:\ExchangeDatabases\USCHIZWDAG4002-DB1\USCHIZWDAG4002-DB1.db\USCHIZWDAG4002-DB1.edb
[PS] C:\Windows\system32>(Get-MailboxDatabase).LogFolderPath.PathName
C:\ExchangeDatabases\USCHIZWDAG4002-DB1\USCHIZWDAG4002-DB1.log
As you can see, this SHOULD NOT require a move yet the resource still wants to move it (which is why it entered the DB/log move block). I can't really see anything wrong anywhere so I'm asking for support here.
Now on the second/additional DAG member, I get this error:
Database Availability Group does not have at least one member after waiting the specified amount of time.
And this is the WaitForFirstDAGMember
block:
# Wait for DAG to have at least one member before adding additional members (to avoid cluster creation race condition issues)
Script WaitForFirstDAGMember
{
GetScript = {
# Establish remote PowerShell session
Get-RemoteExchangeSession -Credential $Credential -CommandsToLoad 'Get-DatabaseAvailabilityGroup' -Verbose:$VerbosePreference
$dagProperties = Get-DatabaseAvailabilityGroup -Identity $dagName
$result = @{'CurrentDAGMembers' = $dagProperties.Servers.Name}
return $result
}
TestScript = {
# Establish remote PowerShell session
Get-RemoteExchangeSession -Credential $Credential -CommandsToLoad 'Get-DatabaseAvailabilityGroup' -Verbose:$VerbosePreference
$dagProperties = Get-DatabaseAvailabilityGroup -Identity $dagName
if ($dagProperties.Servers.Name.Count -gt 0)
{
return $true
}
else
{
return $false
}
}
SetScript = {
# Establish remote PowerShell session
Get-RemoteExchangeSession -Credential $Credential -CommandsToLoad 'Get-DatabaseAvailabilityGroup' -Verbose:$VerbosePreference
$foundFirstDAGMember = $false
for ($i = 0; $i -lt 5; $i++)
{
$dagProperties = Get-DatabaseAvailabilityGroup -Identity $dagName
if ($dagProperties.Servers.Name.Count -eq 0)
{
Write-Warning "DAG $dagName object has no members yet. Sleeping for 60 seconds."
Start-Sleep -Seconds 60
}
else
{
$foundFirstDAGMember = $true
break
}
}
if (!$foundFirstDAGMember)
{
throw 'Database Availability Group does not have at least one member after waiting the specified amount of time.'
}
}
DependsOn = '[ExchWaitForDAG]WaitForDAG'
}
Manually getting the DAG members count, I get this:
[PS] C:\Windows\system32>$dagProperties = Get-DatabaseAvailabilityGroup -Identity "USCHIZWDAG4002"
[PS] C:\Windows\system32>$dagProperties.Servers.Name.Count
2
So again, the script should see that the DAG has "at least one member" and should continue. Any clues?
Verbose logs
First DAG member:
VERBOSE: [USCHIZWXCH4101]: LCM: [ Start Resource ] [[ExchMailboxDatabase]MDB_USCHIZWDAG4002-DB1]
VERBOSE: [USCHIZWXCH4101]: LCM: [ Start Test ] [[ExchMailboxDatabase]MDB_USCHIZWDAG4002-DB1]
VERBOSE: [USCHIZWXCH4101]: [[ExchMailboxDatabase]MDB_USCHIZWDAG4002-DB1] Importing the module DSC_ExchMailboxDatabase in force mode.
VERBOSE: [USCHIZWXCH4101]: [[ExchMailboxDatabase]MDB_USCHIZWDAG4002-DB1] Entering function 'Test-TargetResource'. Notable parameters: Name =
'USCHIZWDAG4002-DB1'
VERBOSE: [USCHIZWXCH4101]: [[ExchMailboxDatabase]MDB_USCHIZWDAG4002-DB1] Reusing existing Remote PowerShell Session to Exchange
VERBOSE: [USCHIZWXCH4101]: [[ExchMailboxDatabase]MDB_USCHIZWDAG4002-DB1] Importing the ExchangeDsc Remote PowerShell Module.
VERBOSE: [USCHIZWXCH4101]: [[ExchMailboxDatabase]MDB_USCHIZWDAG4002-DB1] Removing the imported "Add-DatabaseAvailabilityGroupServer" function.
VERBOSE: [USCHIZWXCH4101]: [[ExchMailboxDatabase]MDB_USCHIZWDAG4002-DB1] Loading module from path
'C:\Windows\TEMP\DSCExchangeModule\DSCExchangeModule.psm1'.
VERBOSE: [USCHIZWXCH4101]: [[ExchMailboxDatabase]MDB_USCHIZWDAG4002-DB1] Importing function 'Get-MailboxDatabase'.
VERBOSE: [USCHIZWXCH4101]: [[ExchMailboxDatabase]MDB_USCHIZWDAG4002-DB1] Importing function 'Get-Recipient'.
VERBOSE: [USCHIZWXCH4101]: [[ExchMailboxDatabase]MDB_USCHIZWDAG4002-DB1] Importing function 'Set-ADServerSettings'.
VERBOSE: [USCHIZWXCH4101]: [[ExchMailboxDatabase]MDB_USCHIZWDAG4002-DB1] Invalid setting 'EdbFilePath'. Expected value:
'C:\ExchangeDatabases\USCHIZWDAG4002-DB1\USCHIZWDAG4002-DB1.db\USCHIZWDAG4002-DB1.edb'. Actual value: ''
VERBOSE: [USCHIZWXCH4101]: [[ExchMailboxDatabase]MDB_USCHIZWDAG4002-DB1] Invalid setting 'LogFolderPath'. Expected value:
'C:\ExchangeDatabases\USCHIZWDAG4002-DB1\USCHIZWDAG4002-DB1.log'. Actual value: ''
VERBOSE: [USCHIZWXCH4101]: LCM: [ End Test ] [[ExchMailboxDatabase]MDB_USCHIZWDAG4002-DB1] in 0.6920 seconds.
VERBOSE: [USCHIZWXCH4101]: LCM: [ Start Set ] [[ExchMailboxDatabase]MDB_USCHIZWDAG4002-DB1]
VERBOSE: [USCHIZWXCH4101]: [[ExchMailboxDatabase]MDB_USCHIZWDAG4002-DB1] Importing the module DSC_ExchMailboxDatabase in force mode.
VERBOSE: [USCHIZWXCH4101]: [[ExchMailboxDatabase]MDB_USCHIZWDAG4002-DB1] Entering function 'Set-TargetResource'. Notable parameters: Name =
'USCHIZWDAG4002-DB1'
VERBOSE: [USCHIZWXCH4101]: [[ExchMailboxDatabase]MDB_USCHIZWDAG4002-DB1] Reusing existing Remote PowerShell Session to Exchange
VERBOSE: [USCHIZWXCH4101]: [[ExchMailboxDatabase]MDB_USCHIZWDAG4002-DB1] Importing the ExchangeDsc Remote PowerShell Module.
VERBOSE: [USCHIZWXCH4101]: [[ExchMailboxDatabase]MDB_USCHIZWDAG4002-DB1] Removing the imported "Set-ADServerSettings" function.
VERBOSE: [USCHIZWXCH4101]: [[ExchMailboxDatabase]MDB_USCHIZWDAG4002-DB1] Removing the imported "Get-Recipient" function.
VERBOSE: [USCHIZWXCH4101]: [[ExchMailboxDatabase]MDB_USCHIZWDAG4002-DB1] Removing the imported "Get-MailboxDatabase" function.
VERBOSE: [USCHIZWXCH4101]: [[ExchMailboxDatabase]MDB_USCHIZWDAG4002-DB1] Loading module from path
'C:\Windows\TEMP\DSCExchangeModule\DSCExchangeModule.psm1'.
VERBOSE: [USCHIZWXCH4101]: [[ExchMailboxDatabase]MDB_USCHIZWDAG4002-DB1] Importing function 'Get-MailboxDatabase'.
VERBOSE: [USCHIZWXCH4101]: [[ExchMailboxDatabase]MDB_USCHIZWDAG4002-DB1] Importing function 'Mount-Database'.
VERBOSE: [USCHIZWXCH4101]: [[ExchMailboxDatabase]MDB_USCHIZWDAG4002-DB1] Importing function 'Move-ActiveMailboxDatabase'.
VERBOSE: [USCHIZWXCH4101]: [[ExchMailboxDatabase]MDB_USCHIZWDAG4002-DB1] Importing function 'Move-DatabasePath'.
VERBOSE: [USCHIZWXCH4101]: [[ExchMailboxDatabase]MDB_USCHIZWDAG4002-DB1] Importing function 'New-MailboxDatabase'.
VERBOSE: [USCHIZWXCH4101]: [[ExchMailboxDatabase]MDB_USCHIZWDAG4002-DB1] Importing function 'Remove-MailboxDatabase'.
VERBOSE: [USCHIZWXCH4101]: [[ExchMailboxDatabase]MDB_USCHIZWDAG4002-DB1] Importing function 'Set-ADServerSettings'.
VERBOSE: [USCHIZWXCH4101]: [[ExchMailboxDatabase]MDB_USCHIZWDAG4002-DB1] Importing function 'Set-MailboxDatabase'.
VERBOSE: [USCHIZWXCH4101]: LCM: [ End Set ] [[ExchMailboxDatabase]MDB_USCHIZWDAG4002-DB1] in 0.6770 seconds.
PowerShell DSC resource DSC_ExchMailboxDatabase failed to execute Set-TargetResource functionality with error message: Database must have only a single copy for the DB path or log path to be moved
+ CategoryInfo : InvalidOperation: (:) [], CimException
+ FullyQualifiedErrorId : ProviderOperationExecutionFailure
+ PSComputerName : localhost
Second/Additional DAG member:
VERBOSE: [USCHIZWXCH4102]: LCM: [ Start Resource ] [[Script]WaitForFirstDAGMember]
VERBOSE: [USCHIZWXCH4102]: LCM: [ Start Test ] [[Script]WaitForFirstDAGMember]
VERBOSE: [USCHIZWXCH4102]: [[Script]WaitForFirstDAGMember] Importing the module MSFT_ScriptResource in force mode.
VERBOSE: [USCHIZWXCH4102]: [[Script]WaitForFirstDAGMember] Reusing existing Remote PowerShell Session to Exchange
VERBOSE: [USCHIZWXCH4102]: [[Script]WaitForFirstDAGMember] Importing the xExchange Remote PowerShell Module.
VERBOSE: [USCHIZWXCH4102]: [[Script]WaitForFirstDAGMember] Removing the imported "Get-DatabaseAvailabilityGroup" function.
VERBOSE: [USCHIZWXCH4102]: [[Script]WaitForFirstDAGMember] Loading module from path 'C:\Windows\TEMP\DSCExchangeModule\DSCExchangeModule.psm1'.
VERBOSE: [USCHIZWXCH4102]: [[Script]WaitForFirstDAGMember] Importing function 'Get-DatabaseAvailabilityGroup'.
VERBOSE: [USCHIZWXCH4102]: LCM: [ End Test ] [[Script]WaitForFirstDAGMember] in 0.4760 seconds.
VERBOSE: [USCHIZWXCH4102]: LCM: [ Start Set ] [[Script]WaitForFirstDAGMember]
VERBOSE: [USCHIZWXCH4102]: [[Script]WaitForFirstDAGMember] Importing the module MSFT_ScriptResource in force mode.
VERBOSE: [USCHIZWXCH4102]: [[Script]WaitForFirstDAGMember] Performing the operation "Set-TargetResource" on target "Executing the SetScript with the user supplied credential".
VERBOSE: [USCHIZWXCH4102]: [[Script]WaitForFirstDAGMember] Reusing existing Remote PowerShell Session to Exchange
VERBOSE: [USCHIZWXCH4102]: [[Script]WaitForFirstDAGMember] Importing the xExchange Remote PowerShell Module.
VERBOSE: [USCHIZWXCH4102]: [[Script]WaitForFirstDAGMember] Removing the imported "Get-DatabaseAvailabilityGroup" function.
VERBOSE: [USCHIZWXCH4102]: [[Script]WaitForFirstDAGMember] Loading module from path 'C:\Windows\TEMP\DSCExchangeModule\DSCExchangeModule.psm1'.
VERBOSE: [USCHIZWXCH4102]: [[Script]WaitForFirstDAGMember] Importing function 'Get-DatabaseAvailabilityGroup'.
WARNING: [USCHIZWXCH4102]: [[Script]WaitForFirstDAGMember] DAG object has no members yet. Sleeping for 60 seconds.
WARNING: [USCHIZWXCH4102]: [[Script]WaitForFirstDAGMember] DAG object has no members yet. Sleeping for 60 seconds.
WARNING: [USCHIZWXCH4102]: [[Script]WaitForFirstDAGMember] DAG object has no members yet. Sleeping for 60 seconds.
WARNING: [USCHIZWXCH4102]: [[Script]WaitForFirstDAGMember] DAG object has no members yet. Sleeping for 60 seconds.
WARNING: [USCHIZWXCH4102]: [[Script]WaitForFirstDAGMember] DAG object has no members yet. Sleeping for 60 seconds.
VERBOSE: [USCHIZWXCH4102]: LCM: [ End Set ] [[Script]WaitForFirstDAGMember] in 300.7700 seconds.
PowerShell DSC resource MSFT_ScriptResource failed to execute Set-TargetResource functionality with error message: Database Availability Group does not have at least one member after waiting the specified amount of time.
+ CategoryInfo : InvalidOperation: (:) [], CimException
+ FullyQualifiedErrorId : ProviderOperationExecutionFailure
+ PSComputerName : localhost
DSC configuration
First DAG member:
$dagName = 'USCHIZWDAG4002'
$dbName = "$($dagName)-DB1"
# Create primary database
ExchMailboxDatabase "MDB_$dbName"
{
Name = $dbName
Credential = $Credential
EdbFilePath = "C:\ExchangeDatabases\$($dbName)\$($dbName).db\$($dbName).edb"
LogFolderPath = "C:\ExchangeDatabases\$($dbName)\$($dbName).log"
Server = $server.Name
CircularLoggingEnabled = $true
DatabaseCopyCount = 2
IssueWarningQuota = '1946MB'
ProhibitSendQuota = '2048MB'
ProhibitSendReceiveQuota = '2253MB'
AllowServiceRestart = $true
DependsOn = '[ExchMailboxServer]MailboxServer'
}
Second DAG member:
# Wait for DAG to have at least one member before adding additional members (to avoid cluster creation race condition issues)
Script WaitForFirstDAGMember
{
GetScript = {
# Establish remote PowerShell session
Get-RemoteExchangeSession -Credential $Credential -CommandsToLoad 'Get-DatabaseAvailabilityGroup' -Verbose:$VerbosePreference
$dagProperties = Get-DatabaseAvailabilityGroup -Identity $dagName
$result = @{'CurrentDAGMembers' = $dagProperties.Servers.Name}
return $result
}
TestScript = {
# Establish remote PowerShell session
Get-RemoteExchangeSession -Credential $Credential -CommandsToLoad 'Get-DatabaseAvailabilityGroup' -Verbose:$VerbosePreference
$dagProperties = Get-DatabaseAvailabilityGroup -Identity $dagName
if ($dagProperties.Servers.Name.Count -gt 0)
{
return $true
}
else
{
return $false
}
}
SetScript = {
# Establish remote PowerShell session
Get-RemoteExchangeSession -Credential $Credential -CommandsToLoad 'Get-DatabaseAvailabilityGroup' -Verbose:$VerbosePreference
$foundFirstDAGMember = $false
for ($i = 0; $i -lt 5; $i++)
{
$dagProperties = Get-DatabaseAvailabilityGroup -Identity $dagName
if ($dagProperties.Servers.Name.Count -eq 0)
{
Write-Warning "DAG $dagName object has no members yet. Sleeping for 60 seconds."
Start-Sleep -Seconds 60
}
else
{
$foundFirstDAGMember = $true
break
}
}
if (!$foundFirstDAGMember)
{
throw 'Database Availability Group does not have at least one member after waiting the specified amount of time.'
}
}
DependsOn = '[ExchWaitForDAG]WaitForDAG'
}
Suggested solution
No because I can't find any error in the script unless this is a bug
Operating system the target node is running
OsName : Microsoft Windows Server 2016 Standard
OsOperatingSystemSKU : StandardServerEdition
OsArchitecture : 64-bit
WindowsBuildLabEx : 14393.6167.amd64fre.rs1_release.230802-0927
OsLanguage : en-US
OsMuiLanguages : {en-US}
Exchange Server edition and version the target node is running
ServerRole Edition AdminDisplayVersion
---------- ------- -------------------
Mailbox Enterprise Version 15.1 (Build 2507.6)
Mailbox Enterprise Version 15.1 (Build 2507.6)
PowerShell version and build the target node is running
Name Value
---- -----
PSVersion 5.1.14393.5582
PSEdition Desktop
PSCompatibleVersions {1.0, 2.0, 3.0, 4.0...}
BuildVersion 10.0.14393.5582
CLRVersion 4.0.30319.42000
WSManStackVersion 3.0
PSRemotingProtocolVersion 2.3
SerializationVersion 1.1.0.1
ExchangeDsc version
Name Version Path
---- ------- ----
ExchangeDsc 2.0.0 C:\Program Files\WindowsPowerShell\Modules\ExchangeDsc\2.0.0\ExchangeDsc.psd1