Function with parallel runspaces shows strange behaviour

by megamorf at 2013-01-16 03:31:47

Hey guys,

I’ve written an advanced function over the past few days and I think it’s useful in its current state. But I can’t wrap my head around what’s happening when the function is run multiple times in a row.

Here’s the function:

Function Get-DateOfLastUpdate {

[CmdletBinding()]
param(
[Parameter(Position = 0)]
[String]
$Computername,

[Parameter(Position = 1)]
[String]
$Filepath,

[Parameter(Position = 2)]
[int]
$ThrottleLimit = 10,

[switch]$invoke
)


Write-Verbose "[$($myinvocation.mycommand)] :: Checking if AD module is loaded…"
if(-not(Get-Module -Name ActiveDirectory))
{
try { Import-Module -Name ActiveDirectory }
catch { return; Write-Error -message "Error loading ActiveDirectory module" }
}

if($computername)
{
Write-Verbose "[$($myinvocation.mycommand)] :: Computername parameter is used...&quot;<br> Write-Verbose &quot;[$&#40;$myinvocation.mycommand&#41;] :: $Computername contains [$($computername.count)] servers…"
}
else
{
Write-Verbose "[$($myinvocation.mycommand)] :: Retrieving servers from AD…"
$computername = Get-ADComputer -filter "" -searchbase "OU=Server,OU=Systemkonten,DC=ihkberlin,DC=intern" -properties "name","operatingsystem" |
where{$_.operatingsystem -match "windows" } |
Sort Name |
Select -expandproperty name
Write-Verbose "[$($myinvocation.mycommand)] :: Query returned [$($computername.count)] servers…"
}

Write-Verbose "[$($myinvocation.mycommand)] :: Preparing parallel processing of update timestamp…"
$SessionState = [system.management.automation.runspaces.initialsessionstate]::CreateDefault()
$Pool = [runspacefactory]::CreateRunspacePool(1, $throttleLimit, $SessionState, $Host)
$Pool.Open()

<# Return results from async runspaces http://msdn.microsoft.com/en-us/library … 0;v=vs.85&#41;.aspx #>
$ScriptBlock = {

param($computer)

$obj = "" | select computername, date
$obj.computername = $computer

if(test-connection $computer -count 1 -quiet)
{
$reg = [Microsoft.Win32.RegistryKey]::OpenRemoteBaseKey('LocalMachine', $computer)
$date = [DateTime]( $reg.OpenSubKey("SOFTWARE\Microsoft\Windows\CurrentVersion\WindowsUpdate\Auto Update\Results\Install") ).getvalue("LastSuccessTime")
}
else
{
$date = "—"
}

$obj.date = $date
return $obj
}

$threads = @()
$result = @()

$handles = foreach($computer in $computername) {
$powershell = ::Create().AddScript($ScriptBlock).AddArgument($Computer)
$powershell.RunspacePool = $Pool
$powershell.BeginInvoke()
$threads += $powershell
}

do {
$i = 0
$done = $true
foreach ($handle in $handles) {
if ($handle -ne $null) {
if ($handle.IsCompleted) {
$result+= $threads[$i].EndInvoke($handle)
$threads[$i].Dispose()
$handles[$i] = $null
} else {
$done = $false
}
}
$i++
}
if (-not $done) { Start-Sleep -Milliseconds 500 }
} until ($done)

if($Filepath)
{
$result | export-csv -notypeinformation -path $filepath -force
if($invoke){ invoke-item $filepath }
}

return $result

}



When run a few times in a row it works fine but when I continue invoking the function my powershell suddenly tells me the following which roughly translates to 'Exception when calling EndInvoke with 1 argument. The thread was not started" and the rest of the lines say "The thread is running or was cancelled. Restart not possible". Ultimately, using normal cmdlets like clear-screen and get-childitem gives me an error in my powershell prompt after the function seemingly screwed up in the middle of closing the runspaces.
I checked the scopes but everything the function does is contained in its local scope. The function shouldn’t affect the parent scope, right?


PS H:&gt; Get-DateOfLastUpdate -Verbose -Computername $comp
VERBOSE: [Get-DateOfLastUpdate] :: Checking if AD module is loaded...
VERBOSE: [Get-DateOfLastUpdate] :: Computername parameter is used...
VERBOSE: [Get-DateOfLastUpdate] :: $Computername contains [25] servers...
VERBOSE: [Get-DateOfLastUpdate] :: Preparing parallel processing of update timestamp...
Ausnahme beim Aufrufen von "EndInvoke" mit 1 Argument(en): "Der Thread wurde nicht gestartet."
Bei Zeile:70 Zeichen:45
+ $result+= $threads[$i].EndInvoke <<<< ($handle)
+ CategoryInfo : NotSpecified: (:slight_smile: [], MethodInvocationException
+ FullyQualifiedErrorId : DotNetMethodException

Ausnahme beim Aufrufen von "EndInvoke" mit 1 Argument(en): "Der Thread wird ausgeführt oder wurde abgebrochen. Neustart nicht möglich."
Bei Zeile:70 Zeichen:45
+ $result+= $threads[$i].EndInvoke <<<< ($handle)
+ CategoryInfo : NotSpecified: (:slight_smile: [], MethodInvocationException
+ FullyQualifiedErrorId : DotNetMethodException

Ausnahme beim Aufrufen von "EndInvoke" mit 1 Argument(en): "Der Thread wird ausgeführt oder wurde abgebrochen. Neustart nicht möglich."
Bei Zeile:70 Zeichen:45
+ $result+= $threads[$i].EndInvoke <<<< ($handle)
+ CategoryInfo : NotSpecified: (:slight_smile: [], MethodInvocationException
+ FullyQualifiedErrorId : DotNetMethodException

Ausnahme beim Aufrufen von "EndInvoke" mit 1 Argument(en): "Der Thread wird ausgeführt oder wurde abgebrochen. Neustart nicht möglich."
Bei Zeile:70 Zeichen:45
[... many more lines with the same error]

computername : srvwts101
date : 04.01.2013 22:04:24

PS H:&gt; cls
Der Thread wurde nicht gestartet.
PS>cls
Der Thread wurde nicht gestartet.
PS>
PS>
PS>
PS>gci
Der Thread wurde nicht gestartet.
PS>
by nohandle at 2013-01-16 05:27:19
It failing after multiple invocations of the command are run implies you hit some kind of limit. Check the task manager if there is enough memory etc.
Also showing us the whole exception expanded ($error[index] | fl * -force) and resolved (all inner exceptions expanded), would probably help.

btw:
#does not work the cmdlet raises non-terminating error
try { Import-Module -Name ActiveDirectory }
catch { return; Write-Error -message "Error loading ActiveDirectory module" }

#why the back tick before the C? you missed $?
Write-Verbose "[$($myinvocation.mycommand)] :: Computername parameter is used...&quot;<br><br>#why not including the operating system in the ldap filter?<br>$computername = Get-ADComputer -filter &quot;*&quot; -searchbase &quot;OU=Server,OU=Systemkonten,DC=ihkberlin,DC=intern&quot; -properties &quot;name&quot;,&quot;operatingsystem&quot; | <br>where{$_.operatingsystem -match &quot;windows&quot; }</code></blockquote>by megamorf at 2013-01-16 06:09:59<blockquote>[quote=&quot;nohandle&quot;]It failing after multiple invocations of the command are run implies you hit some kind of limit. Check the task manager if there is enough memory etc. <br>Also showing us the whole exception expanded ($error[index] | fl * -force) and resolved (all inner exceptions expanded), would probably help.<br>[/quote]<br><br>Okay, here are the full errors.<br><br>This doesn't say much:<br><code><br>PSMessageDetails &#58;<br>Exception &#58; System&#46;Management&#46;Automation&#46;MethodInvocationException&#58; Ausnahme beim Aufrufen von &quot;EndInvoke&quot; mit 1 Argument(en)&#58; &quot;Der Thread wird ausgeführt oder wurde<br> abgebrochen&#46; Neustart nicht möglich&#46;&quot; ---&gt; System&#46;Threading&#46;ThreadStateException&#58; Der Thread wird ausgeführt oder wurde abgebrochen&#46; Neustart nicht möglich<br> &#46;<br> bei System&#46;Management&#46;Automation&#46;Runspaces&#46;AsyncResult&#46;EndInvoke()<br> bei System&#46;Management&#46;Automation&#46;PowerShell&#46;EndInvoke(IAsyncResult asyncResult)<br> bei EndInvoke(Object , Object&#91;&#93; )<br> bei System&#46;Management&#46;Automation&#46;DotNetAdapter&#46;AuxiliaryMethodInvoke(Object target, Object&#91;&#93; arguments, MethodInformation methodInformation, Object&#91;&#93; or<br> iginalArguments)<br> --- Ende der internen Ausnahmestapelüberwachung ---<br> bei System&#46;Management&#46;Automation&#46;DotNetAdapter&#46;AuxiliaryMethodInvoke(Object target, Object&#91;&#93; arguments, MethodInformation methodInformation, Object&#91;&#93; or<br> iginalArguments)<br> bei System&#46;Management&#46;Automation&#46;ParserOps&#46;CallMethod(Token token, Object target, String methodName, Object&#91;&#93; paramArray, Boolean callStatic, Object val<br> ueToSet)<br> bei System&#46;Management&#46;Automation&#46;MethodCallNode&#46;InvokeMethod(Object target, Object&#91;&#93; arguments, Object value)<br> bei System&#46;Management&#46;Automation&#46;MethodCallNode&#46;Execute(Array input, Pipe outputPipe, ExecutionContext context)<br> bei System&#46;Management&#46;Automation&#46;AssignmentStatementNode&#46;Execute(Array input, Pipe outputPipe, ExecutionContext context)<br> bei System&#46;Management&#46;Automation&#46;StatementListNode&#46;ExecuteStatement(ParseTreeNode statement, Array input, Pipe outputPipe, ArrayList&amp; resultList, Execut<br> ionContext context)<br>TargetObject &#58;<br>CategoryInfo &#58; NotSpecified&#58; (&#58;) &#91;&#93;, MethodInvocationException<br>FullyQualifiedErrorId &#58; DotNetMethodException<br>ErrorDetails &#58;<br>InvocationInfo &#58; System&#46;Management&#46;Automation&#46;InvocationInfo<br>PipelineIterationInfo &#58; {}<br></code><br><br>Here's an interesting one<br><code><br>PSMessageDetails &#58;<br>Exception &#58; System&#46;Management&#46;Automation&#46;MethodInvocationException&#58; Ausnahme beim Aufrufen von &quot;EndInvoke&quot; mit 1 Argument(en)&#58; &quot;Der Thread wurde nicht gestartet&#46;&quot; ---<br> &gt; System&#46;Threading&#46;ThreadStartException&#58; Der Thread wurde nicht gestartet&#46; ---&gt; System&#46;OutOfMemoryException&#58; Eine Ausnahme vom Typ &quot;System&#46;OutOfMemoryExcep<br> tion&quot; wurde ausgelöst&#46;<br> --- Ende der internen Ausnahmestapelüberwachung ---<br> bei System&#46;Management&#46;Automation&#46;Runspaces&#46;AsyncResult&#46;EndInvoke()<br> bei System&#46;Management&#46;Automation&#46;PowerShell&#46;EndInvoke(IAsyncResult asyncResult)<br> bei EndInvoke(Object , Object&#91;&#93; )<br> bei System&#46;Management&#46;Automation&#46;DotNetAdapter&#46;AuxiliaryMethodInvoke(Object target, Object&#91;&#93; arguments, MethodInformation methodInformation, Object&#91;&#93; or<br> iginalArguments)<br> --- Ende der internen Ausnahmestapelüberwachung ---<br> bei System&#46;Management&#46;Automation&#46;DotNetAdapter&#46;AuxiliaryMethodInvoke(Object target, Object&#91;&#93; arguments, MethodInformation methodInformation, Object&#91;&#93; or<br> iginalArguments)<br> bei System&#46;Management&#46;Automation&#46;ParserOps&#46;CallMethod(Token token, Object target, String methodName, Object&#91;&#93; paramArray, Boolean callStatic, Object val<br> ueToSet)<br> bei System&#46;Management&#46;Automation&#46;MethodCallNode&#46;InvokeMethod(Object target, Object&#91;&#93; arguments, Object value)<br> bei System&#46;Management&#46;Automation&#46;MethodCallNode&#46;Execute(Array input, Pipe outputPipe, ExecutionContext context)<br> bei System&#46;Management&#46;Automation&#46;AssignmentStatementNode&#46;Execute(Array input, Pipe outputPipe, ExecutionContext context)<br> bei System&#46;Management&#46;Automation&#46;StatementListNode&#46;ExecuteStatement(ParseTreeNode statement, Array input, Pipe outputPipe, ArrayList&amp; resultList, Execut<br> ionContext context)<br>TargetObject &#58;<br>CategoryInfo &#58; NotSpecified&#58; (&#58;) &#91;&#93;, MethodInvocationException<br>FullyQualifiedErrorId &#58; DotNetMethodException<br>ErrorDetails &#58;<br>InvocationInfo &#58; System&#46;Management&#46;Automation&#46;InvocationInfo<br>PipelineIterationInfo &#58; {}<br></code><br><br>Followed by Parsing errors that probably occur due to memory contraints<br><code><br><br>ErrorRecord &#58; Fehlende schließende &quot;}&quot; im Anweisungsblock&#46;<br>StackTrace &#58; bei System&#46;Management&#46;Automation&#46;Parser&#46;ReportException(Object targetObject, Type exceptionType, Token errToken, String resourceIdAndErrorId, Obje<br> ct&#91;&#93; args)<br> bei System&#46;Management&#46;Automation&#46;Tokenizer&#46;Require(TokenId tokenId, String resourceIdAndErrorId, Object&#91;&#93; args)<br> bei System&#46;Management&#46;Automation&#46;Parser&#46;ScriptBlockRule(String name, Boolean requireBrace, Boolean isFilter, ParameterDeclarationNode parameterDec<br> laration, List1 functionComments, List1 parameterComments)<br> bei System&#46;Management&#46;Automation&#46;Parser&#46;FunctionDeclarationRule()<br> bei System&#46;Management&#46;Automation&#46;Parser&#46;StatementRule()<br> bei System&#46;Management&#46;Automation&#46;Parser&#46;StatementListRule(Token start)<br> bei System&#46;Management&#46;Automation&#46;Parser&#46;ScriptBlockRule(String name, Boolean requireBrace, Boolean isFilter, ParameterDeclarationNode parameterDec<br> laration, List1 functionComments, List1 parameterComments)<br> bei System&#46;Management&#46;Automation&#46;Parser&#46;ParseScriptBlock(String input, Boolean interactiveInput)<br> bei System&#46;Management&#46;Automation&#46;AutomationEngine&#46;ParseScriptBlock(String script, Boolean interactiveCommand)<br> bei System&#46;Management&#46;Automation&#46;ScriptCommandProcessor&#46;&#46;ctor(String script, ExecutionContext context, Boolean isFilter, Boolean useLocalScope, Bo<br> olean interactiveCommand, CommandOrigin origin)<br> bei System&#46;Management&#46;Automation&#46;Runspaces&#46;Command&#46;CreateCommandProcessor(ExecutionContext executionContext, CommandFactory commandFactory, Boolea<br> n addToHistory)<br> bei System&#46;Management&#46;Automation&#46;Runspaces&#46;LocalPipeline&#46;CreatePipelineProcessor()<br> bei System&#46;Management&#46;Automation&#46;Runspaces&#46;LocalPipeline&#46;InvokeHelper()<br> bei System&#46;Management&#46;Automation&#46;Runspaces&#46;LocalPipeline&#46;InvokeThreadProc()<br>WasThrownFromThrowStatement &#58; False<br>Message &#58; Fehlende schließende &quot;}&quot; im Anweisungsblock&#46;<br>Data &#58; {}<br>InnerException &#58;<br>TargetSite &#58; System&#46;Collections&#46;ObjectModel&#46;Collection1[System.Management.Automation.PSObject] Invoke(System.Collections.IEnumerable)
HelpLink :
Source : System.Management.Automation


[quote="nohandle"]
btw:
#does not work the cmdlet raises non-terminating error
try { Import-Module -Name ActiveDirectory }
catch { return; Write-Error -message "Error loading ActiveDirectory module" }

#why the back tick before the C? you missed $?
Write-Verbose "[$($myinvocation.mycommand)] :: `Computername parameter is used…"

#why not including the operating system in the ldap filter?
$computername = Get-ADComputer -filter "
" -searchbase "OU=Server,OU=Systemkonten,DC=ihkberlin,DC=intern" -properties "name","operatingsystem" |
where{$_.operatingsystem -match "windows" }
[/quote]

Thanks for your input
1) I’ll rewrite that then
2) The $-symbol is not there, I dunno why. thanks for spotting the typo :slight_smile:
3) I’m just getting into the AD module and LDAP filters. I’ll include the OS in the LDAP filter instead

PS: I tried changing the culture to en-US but i just wouldn’t change (PSv2, WS 2008 R2 SP1)
by megamorf at 2013-01-16 07:20:18
I added a bit of logic to track the memory consumption but it doesn’t seem like a lot to me - barely 3MB. Here’s the modified part of the script (lines 80-96 of original function):

$initial = (Get-Process -Id $PID).WorkingSet

do {
$memory = ((Get-Process -Id $PID).WorkingSet - $initial)
[system.Console]::Title = ("Current Memory Load: {0:0.00} MB." -f ($memory/1MB))

$i = 0
$done = $true
foreach ($handle in $handles) {
if ($handle -ne $null) {
if ($handle.IsCompleted) {
$result+= $threads[$i].EndInvoke($handle)
$threads[$i].Dispose()
$handles[$i] = $null
} else {
$done = $false
}
}
$i++
}
if (-not $done) { Start-Sleep -Milliseconds 500 }
} until ($done)


Also, my observations tell me that up until a throttlelimit of 5 the problem doesn’t occur, so it really seems like a memory problem.