Launching Powershell and its script(*.ps1) from Python produces different output

PowerShell Help
1

I have this command set which i want to run to get all installed apps and their versions. The commands are saved to a file name psGetAppVersions.ps1:

Content of psGetAppVersions.ps1:

powershell Set-ExecutionPolicy -ExecutionPolicy RemoteSigned -Scope CurrentUser -Force
powershell -Command “gp HKLM:\Software\Microsoft\Windows\CurrentVersion\Uninstall\* | select DisplayName, DisplayVersion, Publisher | Format-Table -AutoSize | Out-String -Width 4096 > C:\temp\testOut.txt”

When i launch it via terminal, i get the proper output ~ 17KB in size.

powershell .\psGetAppVersions.ps1

First few lines of the correct output are:

DisplayName DisplayVersion Publisher

Windows Driver Package - Plantronics, Inc. (usbser.ntamd64) Ports (04/21/2009 5.1) 04/21/2009 5.1 Plantronics, Inc.
Windows Driver Package - Cambridge Silicon Radio (CSRBC) USB (10/26/2012 2.4.0.0) 10/26/2012 2.4.0.0 Cambridge Silicon Radio

 

However, when i tried subprocess or pipe or open or os.system in both Python or Perl and invoke the command, i get a totally different output, something like ~41KB in size:

The first few lines on this incorrect output are:

DisplayName DisplayVersion Publisher

Cisco Webex Meetings 40.2.8 Cisco Webex LLC
Carbon Black Sensor 6.2.2.90503 Carbon Black, Inc.

It seems, whenever you invoke Powershell through Python or another interpreter like Perl, calling whatever method they support for command line execution, you will get the wrong output. I am completely shocked such bad results can be the case, especially when we want to try to automate such task and to verify the correct apps and their versions. Please help suggest a solution for this critical issue. This was run on Python 3.7.3 on Windows 10 as well as Strawberry Perl 5.30.2.1

Thanks in advance!

 

2

What version of powershell? Windows powershell does all sorts of odd things with encodings. Try powershell 7 or specify an encoding.
I hope this helps.

3

Thanks Doug!

My Powershell version is:

PS C:\Users\jnguyen19> Get-Host | Select-Object Version

Version

5.1.17763.1007

 

Here is a snipplet of the python script, using Popen, but it doesnt matter any method in Python would yield same bad output:

cmd2 = r"powershell.exe C:\\Users\\jnguyen19\\scratchPad\\psGetAppVersions.ps1;"

process = subprocess.Popen(cmd2, stdout=subprocess.PIPE, shell=True).communicate()[0]


Can you please help provide where and how to encode it? an example?  encode to what?
I know i had to decode('ascii') to print out the output:
print(process.decode('ascii'))

I tried setting default encoding in the *.ps1 script like so:
$PSDefaultParameterValues = @{ '*:Encoding' = 'Windows-1252' }

it didn't help.
4

I also didn’t notice the format-table. I get nothing different from using it on my system versus cutting that part out. Format- cmdlets already output string instead of objects, so I’d just stick with out-string. I also don’t know what width 4096 is supposed to do vs just default, for me it made no difference. You can use powershell to output instead of console redirector > using several different cmdlets. You can pipe to set-content instead and it offers -encoding parameter. There is also out-file and it offers -encoding parameter as well. Either of the following may help with your issue, maybe not. I’m assuming the data is just getting skewed.

powershell -Command “gp HKLM:\\Software\\Microsoft\\Windows\\CurrentVersion\\Uninstall\\* |
    select DisplayName, DisplayVersion, Publisher | Out-String -Width 4096 | set-content C:\\temp\\testOut.txt -encoding ascii”

powershell -Command “gp HKLM:\\Software\\Microsoft\\Windows\\CurrentVersion\\Uninstall\\* |
    select DisplayName, DisplayVersion, Publisher | Out-String -Width 4096 | out-file C:\\temp\\testOut.txt -encoding ascii”

You may need a different encoding, I only put ascii as an example. Hope this is helpful.

5

Thanks Doug!

You’ve been super helpful. It seems I need to go with PowerShell 7.0. PS7.0 has a slightly altered cmdlet command set. I used -Width 4096 as the third column were being truncated when i redirected to the text file.

With PS 7.0, the output is now ~16K. I think the output are smaller now, but consistent.

The one thing i see is that it hangs and does not write the output, until i type into the console exit. But the PS process is still running?

6

Yeah powershell 7.0 they’ve corrected some mistakes and improved in several areas. I think the best thing for powershell (.net) was going cross platform. Seems this in conjunction with opensource has forced some standards that Windows didn’t always adhere to. Anyways, what do you mean? You launch this via your application and it opens a console window and leaves it open? You can always add exit to the end of a script if you want to be sure it closes the console. You shouldn’t have to though if you’re calling powershell externally.

7

I added ‘exit’ as the last line of my script. it didnt help. When i ran on the console, manually:

pwsh -File “.\psGetAppVersions.ps1”, it returned back to command prompt, but the file is not written until i type ‘exit’ into the console?

This command doesn’t always work in the console and no output?: pwsh -Command ."psGetAppVersions.ps1"

Should i be using -File or -Command to specify the file? i was using -Command in the last version, but this version seems flaky for the -Command, so i tried -File and it worked.

 

8

Which console are you running from? Why would you expect a script started from a console to close that console? It seems you had it open already if you’re running it from there, but still the exit I figured would work. I just tested and exit does not close the console. If you want to manually close the window you can type exit. What is the end goal for how you’ll execute this? I thought it was from python? Also, I usually don’t specify the parameter and let positional take care of it, which appears to be -file with pwsh.exe. I think you may want to also look at -encodedcommand if you will be calling this from another application.

-EncodedCommand | -e | -ec

Accepts a base64-encoded string version of a command. Use this parameter to submit commands to PowerShell that require complex quotation marks or curly braces. The string must be formatted using UTF-16 character encoding.

For example:

$command = 'dir "c:\program files"
$bytes = [System.Text.Encoding]::Unicode.GetBytes($command)
$encodedCommand = [Convert]::ToBase64String($bytes)
pwsh -encodedcommand $encodedCommand

From https://docs.microsoft.com/en-us/powershell/module/microsoft.powershell.core/about/about_pwsh?view=powershell-7

Finally, when I run -file, -command, -encodedcommand, or just pwsh script.ps1 they all immediately output the file identically. I’m running from within powershell 7 and the contents of script.ps1 is the following

gp HKLM:\\Software\\Microsoft\\Windows\\CurrentVersion\\Uninstall\\* |
select DisplayName, DisplayVersion, Publisher | Out-String -Width 4096 | set-content C:\\temp\\testOut.txt -encoding ascii

The same test with | out-file worked the same for me. Not sure why you would experience that.

9

Oh no, i was just running from the windows Terminal console to test the output when i run manually versus via Python. I want to see if the output matches. It does now, unlike PowerShell 5.0. It seems the Windows installer packages didn’t show up, but both running via Terminal and via Python script matches 100% now, so i can continue to automate. Thank you so very very much for all your help!

Outstanding support. The best support i’ve gotten in over a week. Was trying to get help on StackOverflow, it seems it was beyond their knowledge. Thanks again Doug!

10

I’m just glad I was able to help. Take good care.

11

Hi Doug, thanks for the help and well wish. I still have a blocker issue…

If I may bother you again. It seems the list matches from the command line execution to the python execution. UNFORTUNATELY, it left out lots of key applications that I must verify for. So, I will need another set of query that would pull out all the apps, not just the ones that have uninstall folders. Do you have such query?

This one left out a lot on PowerShell 7.0, but have them listed in Powershell 5.0 in terminal mode:

powershell -Command "gp HKLM:\Software\Microsoft\Windows\CurrentVersion\Uninstall\* | select DisplayName, DisplayVersion, Publisher | Format-Table -AutoSize | Out-String -Width 4096

I googled and came up with this one, the list is longer but still left out the critical ones I was looking for, such as Avatier:

powershell -Command "gp HKLM:\Software\Wow6432Node\Microsoft\Windows\CurrentVersion\Uninstall\* | select DisplayName, DisplayVersion, Publisher | Format-Table -AutoSize | Out-String -Width 4096

I am tested them both on Powershell 5.0 first before trying on PowerShell 7.0. I’ll continue to search for the right query that would list them all on Powershell 7.0.

Let me know if you have a better query. You can verify it against the Apps & Features under control panel. Thanks.

 

I went into the Registry and see that I can see Avatier package under:

Computer\HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Uninstall{5AFBB52D-2BAF-443C-8822-41602A6A7D53}

So, that translate to:

gp HKLM:\Software\Microsoft\Windows\CurrentVersion\Uninstall\*

but Avatier is not showing up? Any ideas how to force this?

12

Powershell 7 only lists packages installed by packagemanagement. You’ll need to use PS 5.1 for sure. Now the question you are asking is a long, ongoing discussion. There is no absolute way to list all the installed programs. If you’re querying both the 64 bit and 32 bit uninstall keys, and it’s not listed, then you may have to find another way to detect the program. There is also the Win32_product wmi class, but it is problematic in several ways, including being extremely slow. Please use your favorite search engine to look at all the discussions about this particular topic.

13

Thanks Doug! It seems I couldn’t create a full-end-2-end one-button solution via my Python script. I had to create a separate windows console script to invoke PowerShell to query the apps under PowerShell 5.0, and then feed that output to a file, which will be later use by my Python script. I think its a little bit less headache. Thanks again for all your help. This is my first exposure to the dark side of PowerShell, wasn’t as smooth as i was hoping :frowning: