Parallel usage of imported PSSession - Runspaces/AsJob/Invoke-Parallel

Hi

I’ve been having this headache for a long time now so now i’m asking for assistance.

-Short issue description: Need to run in parallel using Imported PSsession to performance optimize code.

First is the extremely simplyfied version of my script where i connect and get all mailboxes from exchange online and using those mailboxes, gather further information about each mailbox. Issue here being since this is exchangeonline, stuff takes time. Also some commands are time consuming. In average each query takes 5-15 sec. Which off-course is quite a lot when you have a lot of mailboxes.

Function QueryExchangeOnline{
    Param($mailbox)
    
    #Do multiple heavy query's like:
    # -Get Calendar permissions
    # -Get-RecipientPermission
    # -Get-MailboxPermission
    # -Get-MailboxFolderStatistics
    # -Get-MailboxFolderPermission
    # -Get-Casmailbox

    $mbx = get-mailbox -Identity $mailbox.Identity

    $obj = @{}
    $obj.identity = $mbx.identity
    #emulating commands described above
    $randomnumber = 5..15 | Get-Random
    Sleep -Seconds $randomnumber

    $obj
}

#Get credentials to connect to exchangeonline
    $creds = Get-Credential

#Create and import Exchangeonline PSSession
    $session = New-PSSession -ConfigurationName Microsoft.Exchange -ConnectionUri "https://outlook.office365.com/PowerShell-LiveID?PSVersion=4.0" -Credential $creds -Authentication Basic -AllowRedirection
    $null = Import-PSSession $session -AllowClobber -DisableNameChecking

#Get all mailboxes from ExchangeOnline
    $allmailboxes = Get-Mailbox -ResultSize Unlimited

#Do various querying and return hashtable to pipeline
    foreach($m in $allmailboxes){
        QueryExchangeOnline -mailbox $m
    }

I’ve tried lots of stuff and even have something working in paralel using powershell runspaces. Problem is, it creates a nye pssession for exchange online for each query, which just makes it even slower.

#Get credentials to connect to exchangeonline
    $creds = Get-Credential

#Create and import Exchangeonline PSSession
    $session = New-PSSession -ConfigurationName Microsoft.Exchange -ConnectionUri "https://outlook.office365.com/PowerShell-LiveID?PSVersion=4.0" -Credential $creds -Authentication Basic -AllowRedirection
    $null = Import-PSSession $session -AllowClobber -DisableNameChecking

#Get all mailboxes from ExchangeOnline
    $Array = Get-Mailbox -ResultSize Unlimited

#Create empty Array
    $RunspaceCollection = @()

#Open Define the number of processes the pool should allow.  minimum and maximum values (1 and 5)
    $RunspacePool = [RunspaceFactory]::CreateRunspacePool(1,20)
    $RunspacePool.Open()

#Define the actual work each runspace will perform.
    $ScriptBlock = {
        Param($creds,$mailbox)

            $so = New-PSSessionOption -SkipRevocationCheck
	        $session = New-PSSession -ConfigurationName Microsoft.Exchange -ConnectionUri "https://outlook.office365.com/PowerShell-LiveID?PSVersion=4.0" -Credential $creds -Authentication Basic -AllowRedirection -SessionOption $so
	        $null = Import-PSSession $session -AllowClobber -DisableNameChecking

        #Do multiple heavy query's like:
        # -Get Calendar permissions
        # -Get-RecipientPermission
        # -Get-MailboxPermission
        # -Get-MailboxFolderStatistics
        # -Get-MailboxFolderPermission
        # -Get-Casmailbox

        $mbx = get-mailbox -Identity $mailbox.Identity

        $obj = @{}
        $obj.identity = $mbx.identity
        #emulating commands described above
        $randomnumber = 5..15 | Get-Random
        Sleep -Seconds $randomnumber

        $obj
    }

Foreach($mailbox in $Array){

   #Create a PowerShell object to run add the script and argument.

   $Powershell = [PowerShell]::Create().AddScript($ScriptBlock).AddArgument($creds).AddArgument($mailbox)

   
   #Specify runspace to use
   $Powershell.RunspacePool = $RunspacePool

   #Create Runspace collection
   [Collections.Arraylist]$RunspaceCollection += New-Object -TypeName PSObject -Property @{
       Runspace   = $PowerShell.BeginInvoke()
       PowerShell = $PowerShell
       Item       = $mailbox
   }
 }


#Check the RunSpace and validate that all of the tasks in the pool have completed.  Once completed, send 'EndInvoke' to each one.
#     Once all of the tasks are completed, remove the runspace.

 While($RunspaceCollection){

 Foreach($Runspace in $RunspaceCollection.ToArray()){

  If($Runspace.Runspace.IsCompleted){
   $Runspace.PowerShell.EndInvoke($Runspace.Runspace)
   $Runspace.PowerShell.Dispose()
   $RunspaceCollection.Remove($Runspace)
  }
 }
}

My script in it’s current form takes about 15-20 hours, why running in parallel would be a very welcome optimization that could optimize by factor <20.

Has someone done something like this or does someone have any pointers, it would be great! :slight_smile:

I’m found only one way - direct remote session without importing

$session = New-PSSession -ConfigurationName Microsoft.Exchange -ConnectionUri "https://outlook.office365.com/PowerShell-LiveID?PSVersion=4.0" -Credential $creds -Authentication Basic -AllowRedirection

'user1','user2' | Start-RSJob { $mb = $_; Invoke-Command -Session $using:session { get-mailbox $args[0] } -ArgumentList $mb }


Hmm, forget it, with many users I get “session busy”
I think you may only pre-open several sessions and use it with some hand-made throttling