# Multiple PowerShell Scripts processing one input file

**URL:** <https://forums.powershell.org/t/multiple-powershell-scripts-processing-one-input-file/21794>\
**Category:** PowerShell Help\
**Created:** [March 2, 2023, 3:54pm UTC](https://forums.powershell.org/t/multiple-powershell-scripts-processing-one-input-file/21794 "2023-03-02T15:54:06Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Phil\_Worsley](https://avatars.discourse-cdn.com/v4/letter/p/e480ec/32.png) [@Phil\_Worsley](https://forums.powershell.org/u/Phil_Worsley)\
**Post date:** [March 2, 2023, 3:54pm UTC](https://forums.powershell.org/t/multiple-powershell-scripts-processing-one-input-file/21794/1 "2023-03-02T15:54:06Z")

</div>

I need to clear around 75k legacy user profile folders spread across my SAN. So far I’ve got a script that will process the deletion of the folder… albeit slowly. (Robocopy is lethal!)

So to try and speed things up I run 10 powershell scripts in parallel. So far I’ve split the input file into batches of 1,000 (using Powershell) and target each powershell script to one of the smaller files. Challenge is that some user profiles take hours to process and some seconds so some scripts take days and others hours. It’s a constant challenge to keep these working.

It would be wonderful if I could have one input file and have all my scripts grabbing the next folder from the ‘queue’ and processing that.

The alternative would be to split the big file into very small batches (10?) and then have each Powershell script moving the next file in the queue into it’s own folder and process it there.

Each approach has a challenge with contention - just wondered if anyone else had done something similar and how they approached it.

Kind regards,  
Phil Worsley (University of Leeds)

---

<div class="post-metadata">

**Author:** ![Olaf](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.powershell.org/olaf/32/992_2.png) [@Olaf](https://forums.powershell.org/u/Olaf)\
**Post date:** [March 2, 2023, 4:32pm UTC](https://forums.powershell.org/t/multiple-powershell-scripts-processing-one-input-file/21794/2 "2023-03-02T16:32:01Z")

</div>

Phil,  
Welcome to the forum. 👋🏽

When you say …

> [@Phil\_Worsley](#):
>
> albeit slowly. (Robocopy is lethal!)

… you’re doing something wrong. robocopy is actually the fastest (built in) tool when it comes to file system operations.

In general … depending on the actual task you have to do and the approach you use you can use the built in functionality of `Foreach-Object -Parallel` of the PowerShell version 7.x to speed up your task.

> **[ForEach-Object (Microsoft.PowerShell.Core) - PowerShell](https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.core/foreach-object?view=powershell-7.3#-parallel)**
>
> The ForEach-Object cmdlet performs an operation on each item in a collection of input objects. The input objects can be piped to the cmdlet or specified using the InputObject parameter. Starting in Windows PowerShell 3.0, there are two different ways...

With the parameter `-ThrottleLimit` you can limit or increase the amont of parallel running script blocks.

Please read the help completely including the examples to learn ho to use it.

---

<div class="post-metadata">

**Author:** ![dotnVo](https://avatars.discourse-cdn.com/v4/letter/d/4af34b/32.png) [@dotnVo](https://forums.powershell.org/u/dotnVo)\
**Post date:** [May 16, 2024, 8:15pm UTC](https://forums.powershell.org/t/multiple-powershell-scripts-processing-one-input-file/21794/3 "2024-05-16T20:15:05Z")

</div>