I am writing some powershell stuff (on linux) for Windows users. I am reading and parsing large text files and inserting data into into SQL, etc…
On linux, it only utilizes a single CPU core, and sometimes takes a rather long time. I cannot seem to find any info on utilizing more than a single core to run the scripts.
Does anyone know of a way to accomplish it using powershell core on Linux? I have not fired up a windows virtual machine to see if the same scripts utilize more than a single core either, but I would definitely assume so considering the effort to make powershell multi-platform by MS. If the answer there is no as well, I will need to convert all of this to Python, but that makes more setup work for me with the end users
PowerShell core scripts runs under pwsh process and AFAIK, there is no way where we can execute a script by utilizing multiple cores, I mean I don’t know if there a setting like that exist. I may be wrong here
start-job will use multi processes (or put & at the end of a line), start-threadjob multi threads, powershell 7 also has foreach-object -parallel. There’s runspaces too but that’s more difficult. “get-content -readcount 0” is faster.
[greg@dellxps SQLite]$ pwsh Start-Job -ScriptBlock { Get-Process -Name pwsh }
The argument ‘Start-Job’ is not recognized as the name of a script file. Check the spelling of the name, or if a path was included, verify that the path is correct and try again.
I see it spawn, but there is no process running afterward. I have a 14k line file which sleeps after each line for this test, pwsh immediately exits. ImportDat is a function btw…
When I run it native (pwsh routeDownload.ps1 importDat), it runs fine and remains in the process list until complete.
No offense, but I am just trying to make a Windows user happy.
Your latest comment did work when I put $PWD in front of the command, but it did still just utilize a single core (the origin of the post).
Thanks to you both for the comments and help, but I will just proceed with python since I have never been a Windows user and will probably need a steep learning curve which is probably a lot longer than setting up the user’s machine with the python requirements.
Well, I do know that in Windows, you can add -MTA to PowerShell.EXE for a “multithreaded apartment”. Not sure how you would accomplish this on Linux and or if it will use additional cores or not.
I tried this option when I was having issues with a windows form calling functions and being locked up until the function completed, and by god, it worked. I did not however validate multiple cores being used. Your mileage will of course vary.