Invoke-Command Question on Remote File Server

PowerShell Help
1

Hi Gang,

I’m attempting to get a list of files on some of our file servers that are newer than a given date range. I’ve been through a couple different methodologies of gathering the objects, but ultimately, running the script block in an invoke-command proved substantially faster.

My question is, once I invoke the command on the remote machine, I see it running through the remote systems task manager just fine. But then, I want to return those objects I’ve stored in the invoke-command variable back to the local system and run a foreach loop on it locally. I’m getting hung up, and my output isn’t being returned to the host I’m running the script from and going to excel like planned.


Invoke-Command -Session $sessionFS13 {$colFS13Files= Get-ChildItem d:\ -Recurse | where {$_.psiscontainer -eq $false -and $_.CreationTime -gt $date} | Select-Object fullname,length,creationtime}
		Invoke-Command -Session $sessionFS13 {$colFS13Files}
		foreach ($file in $colFS13Files) {
						$filepath = $file.FullName 
						$excelSheet4.Cells.Item($intRow,1) = $filepath
						$size = $file.Length
						$excelSheet4.Cells.Item($intRow,2) = "{0:N2}" -f ($size / 1MB)
						$creationTime = $file.CreationTime
						$excelSheet4.Cells.Item($intRow,3) = $creationTime
						$owner = ((Get-Acl $filepath).owner) #$filepath | Get-Acl | select owner
						$excelSheet4.Cells.Item($intRow,4) = $owner
						$intRow ++					
}#foreach
		
# add up length column (column B)
			$range = $excelSheet4.usedRange
			$row = $range.rows.count # Takes you to the last used row
			$Sumrow = $row + 1 #last used row + 1
			$r = $excelSheet4.Range("B2:B$row") # select the column to Add up
			$functions = $appExcel.WorkSheetfunction #setup variable for excel to run a built-in function (next line)
			$excelSheet4.cells.item($Sumrow,2) = $functions.sum($r) # this uses the Excel sum function defined above
			$rangeString = $r.address().tostring() -replace "\$",'' # convert formula to Text
			$excelSheet4.cells.item($Sumrow,1) = "Total Size (MB)" # Print string "Total Size" in column A & last row + 1
			$excelSheet4.cells.item($Sumrow,2).Select() #Print calculation
			$excelSheet4.range("a${Sumrow}:b$Sumrow").font.bold = "true" # makes text in last row bold
			$excelSheet4.range("a${Sumrow}:b$Sumrow").font.size=12 # Changes the font size in last row to 12 points

Thanks!

Karson

2

Try it like this:

$colFS13Files = Invoke-Command -Session $sessionFS13 { Get-ChildItem d:\ -Recurse | where {$_.psiscontainer -eq $false -and $_.CreationTime -gt $date} | Select-Object fullname,length,creationtime}

foreach ($file in $colFS13Files) {
    # .. etc
}

In your original code, the $colFS13Files variable only existed in the remote session. By assigning the results of Invoke-Command to a local variable, then you can enumerate over them with a foreach loop.

3

Ahh - perfect. So simple, can’t believe I missed that.

Thanks, Dave.

4

Well, I’m back at it. This time, trying to optimize this as best as I can from a timeframe aspect. It seems that waiting on each Invoke-Command to finish on the current file server, output the results to Excel line-by-line, then proceed on to the remaining four servers one-by-one made the script finish in ~5 days.

Brainstorming a little bit, I thought of trying to setup an array to run the invoke-commands in parallel on each server as a separate job. But, I’m still back to my original question of how to store the job’s objects as a local variable.

Maybe I’m way off in left field on this - is there something from an optimization standpoint that would work better?

#region Setting Array of jobs and checking on their status
$arrayJobs = @()
$arrayJobs += $colFS10Files = Invoke-Command -ComputerName fileserver10 -AsJob -JobName fileserver10_Job -ScriptBlock {Get-ChildItem d:\home -Recurse -File | where {$_.CreationTime -gt ((Get-Date).AddDays(-7))} | Select-Object fullname,length,creationtime,@{n="owner";e={(Get-Acl $_.fullname).owner}}}

$arrayJobs += $colFS11Files = Invoke-Command -ComputerName fileserver11 -AsJob -JobName fileserver11_Job -ScriptBlock {Get-ChildItem d:\ -Recurse -File | where {$_.CreationTime -gt ((Get-Date).AddDays(-7))} | Select-Object fullname,length,creationtime,@{n="owner";e={(Get-Acl $_.fullname).owner}}}

$arrayJobs += $colFS12Files = Invoke-Command -ComputerName fileserver12 -AsJob -JobName fileserver12_Job -ScriptBlock {Get-ChildItem e:\home -Recurse -File | where {$_.CreationTime -gt ((Get-Date).AddDays(-7))} | Select-Object fullname,length,creationtime,@{n="owner";e={(Get-Acl $_.fullname).owner}}}

$arrayJobs += $colFS13Files = Invoke-Command -ComputerName fileserver13 -AsJob -JobName fileserver13_Job -ScriptBlock {Get-ChildItem d:\ -Recurse -File | where {$_.CreationTime -gt ((Get-Date).AddDays(-7))} | Select-Object fullname,length,creationtime,@{n="owner";e={(Get-Acl $_.fullname).owner}}}

$arrayJobs += $colFS14Files = Invoke-Command -ComputerName fileserver14 -AsJob -JobName fileserver14_Job -ScriptBlock {Get-ChildItem e:\ -Recurse -File | where {$_.CreationTime -gt ((Get-Date).AddDays(-7))} | Select-Object fullname,length,creationtime,@{n="owner";e={(Get-Acl $_.fullname).owner}}}

#monitor to see when all jobs are completed
$Complete = $false #$complete will be used to determine when all the jobs are completed
while (-not $complete) { #checks array to see if the job status is 'running'
    $arrayJobsInProgress = $arrayJobs | 
        Where-Object { $_.State -match 'running' }
    if (-not $arrayJobsInProgress) { "All Jobs Have Completed" ; $complete = $true } 
}

$arrayJobs = Receive-Job #return all jobs in their respective local variables containing the collection of new files
5

You’re on the right track. You do want to use the -AsJob switch to Invoke-Command, and save the resulting Job object to a variable. However, you don’t assign the results to (for example) $colFS10Files until you call Receive-Job. Whether you keep track of these in an array, hashtable or whatever is up to you. Here’s an example using hashtables to map computer names to job objects (and then later to the jobs’ results):

$computerNames = 'Server01','Server02','Server03'

$jobs = @{}
$results = @{}

foreach ($computer in $computerNames)
{
    $jobs[$computer] = Invoke-Command -ComputerName $computer -AsJob -ScriptBlock { Do-Something }
}

Wait-Job -Job $jobs.Values

foreach ($entry in $jobs.GetEnumerator())
{
    $results[$entry.Key] = Receive-Job -Job $entry.Value
}

$results

# At this point you can loop over $results and do things with the data.
6

Thanks for that example, Dave. It’s getting me on the right path to eventually get each server stored in its own respective $colFS10files, $colFS11files, etc… variable.

I think I’m getting hung up on the first foreach. I’d loop the creation of the jobs, except the invoke-command is different on several of them making the need to break them out into individual jobs, manually, the easiest way (for me) ;o)

I “borrowed” the while (-not $complete) { chunk from Ed Wilson to replace your Wait-Job since I couldn’t understand how to associate the -Job parameter for my entire custom/manual $arrayJobs and wait for them all, as a whole, to be completed.

I’ll try and nail down where my thought process is:

  1. after the last of my manual $arrayJobs entry is finished, I should have a wsmprovhost.exe process using a fair amount of CPU on each remote system as it runs the invoke-command as a job, right? I’ve estimated it should take at least few hours, as the biggest file server should be recursing through ~4TB of data
    a) If wsmprovhost.exe is indicative of the remote job running, then my array is bonky because that process never kicks off like it normally would on any of the servers. And, the job shows “completed” if I copy/paste a single job from the array into the console in a matter of 10-15 seconds.

  2. if I can get the jobs to stay in “running” status for a respectable amount of time, can I ditch the second “foreach” and manually change the $results variable into, for my example,:

     $colFS10files = Receive-Job -Job fileserver10_job
     $colFS11files = Receive-Job -Job fileserver11_job

…then process data into Excel spreadsheet?
In my head, I think I can. Once I get objects stored in my array, I’ll know for sure.

Thanks again, Dave.

Edit, here’s my new array without the incorrect variable usage:

#region Setting Array of jobs and checking on their status
$arrayJobs = @()
$arrayJobs += Invoke-Command -ComputerName fileserver10 -AsJob -JobName fileserver10_Job -ScriptBlock {Get-ChildItem d:\home -Recurse -File | where {$_.CreationTime -gt ((Get-Date).AddDays(-7))} | Select-Object fullname,length,creationtime,@{n="owner";e={(Get-Acl $_.fullname).owner}}}

$arrayJobs += Invoke-Command -ComputerName fileserver11 -AsJob -JobName fileserver11_Job -ScriptBlock {Get-ChildItem d:\ -Recurse -File | where {$_.CreationTime -gt ((Get-Date).AddDays(-7))} | Select-Object fullname,length,creationtime,@{n="owner";e={(Get-Acl $_.fullname).owner}}}

$arrayJobs += Invoke-Command -ComputerName fileserver12 -AsJob -JobName fileserver12_Job -ScriptBlock {Get-ChildItem e:\home -Recurse -File | where {$_.CreationTime -gt ((Get-Date).AddDays(-7))} | Select-Object fullname,length,creationtime,@{n="owner";e={(Get-Acl $_.fullname).owner}}}

$arrayJobs += Invoke-Command -ComputerName fileserver13 -AsJob -JobName fileserver13_Job -ScriptBlock {Get-ChildItem d:\ -Recurse -File | where {$_.CreationTime -gt ((Get-Date).AddDays(-7))} | Select-Object fullname,length,creationtime,@{n="owner";e={(Get-Acl $_.fullname).owner}}}

$arrayJobs += Invoke-Command -ComputerName fileserver14 -AsJob -JobName fileserver14_Job -ScriptBlock {Get-ChildItem e:\ -Recurse -File | where {$_.CreationTime -gt ((Get-Date).AddDays(-7))} | Select-Object fullname,length,creationtime,@{n="owner";e={(Get-Acl $_.fullname).owner}}}
7

So, the remote file servers are only on v2, so the -File parameter was my issue there. I’m sacrificing some efficiency by having to include $_.psiscontainer -eq $false in the pipeline, but until I get v3 or upgrade them to 2012 R2 and get PoSH v4, this’ll have to do.

That being said, I’ve fixed the array issues. Now I just need to figure out the collection of the receive-job to my variables…

Thinking out loud, so ignore the rambling :slight_smile: