Powershell Core Linux - multi-core execution

Hi,

I am writing some powershell stuff (on linux) for Windows users. I am reading and parsing large text files and inserting data into into SQL, etc…

On linux, it only utilizes a single CPU core, and sometimes takes a rather long time. I cannot seem to find any info on utilizing more than a single core to run the scripts.

Does anyone know of a way to accomplish it using powershell core on Linux? I have not fired up a windows virtual machine to see if the same scripts utilize more than a single core either, but I would definitely assume so considering the effort to make powershell multi-platform by MS. If the answer there is no as well, I will need to convert all of this to Python, but that makes more setup work for me with the end users :slight_smile:

 

TiA,

 

Greg

PowerShell core scripts runs under pwsh process and AFAIK, there is no way where we can execute a script by utilizing multiple cores, I mean I don’t know if there a setting like that exist. I may be wrong here :slight_smile:

start-job will use multi processes (or put & at the end of a line), start-threadjob multi threads, powershell 7 also has foreach-object -parallel. There’s runspaces too but that’s more difficult. “get-content -readcount 0” is faster.

Start-Job is not native in pwsh linux…

 

[greg@dellxps SQLite]$ pwsh Start-Job -ScriptBlock { Get-Process -Name pwsh }
The argument ‘Start-Job’ is not recognized as the name of a script file. Check the spelling of the name, or if a path was included, verify that the path is correct and try again.

pwsh runs -file by default, not -command.

pwsh -c "start-job {'hi'} | receive-job -wait -auto"

hi

I guess I am too newb at pwsh, but when I try this:

pwsh -c “Start-Job { ‘routeDownload.ps1 importDat’ } | Receive-Job -wait -auto”

 

I see it spawn, but there is no process running afterward. I have a 14k line file which sleeps after each line for this test, pwsh immediately exits. ImportDat is a function btw…

 

When I run it native (pwsh routeDownload.ps1 importDat), it runs fine and remains in the process list until complete.

You need to learn the basics. Plus the current directory of start-job is the home directory.

pwsh -c “Start-Job { /users/me/routeDownload.ps1 importDat } | Receive-Job -wait -auto”

No offense, but I am just trying to make a Windows user happy.

 

Your latest comment did work when I put $PWD in front of the command, but it did still just utilize a single core (the origin of the post).

Thanks to you both for the comments and help, but I will just proceed with python since I have never been a Windows user and will probably need a steep learning curve which is probably a lot longer than setting up the user’s machine with the python requirements.

 

Thanks again though :slight_smile:

 

Well, I do know that in Windows, you can add -MTA to PowerShell.EXE for a “multithreaded apartment”. Not sure how you would accomplish this on Linux and or if it will use additional cores or not.

I tried this option when I was having issues with a windows form calling functions and being locked up until the function completed, and by god, it worked. I did not however validate multiple cores being used. Your mileage will of course vary.

Usually you would do multiple jobs, or a job while staying in the shell doing other things.