How to start a process as a separate process tree?

PowerShell Help
1

Greetings,

I’m trying to figure out how to write a script that starts a process on the local computer, but in a separate process tree. I need the process’s standard out and error to still stream to the console, and I need the script to wait for and exit with the process’s exit code. I need the script to not wait for or terminate any descendants the process properly backgrounds. It would be nice if the process inherited the script’s working directory and environment, but I could specify the required working directory and the few required environment variables as arguments if need be.

I’ve tried Start-Process. That seems like the closest thing to what I need, particularly when used with Wait-Process instead of -Wait (as the former doesn’t wait on descendant processes). The only problem I’m having with Start-Process is that I don’t see a way to specify that the process not be a descendant of my script’s process tree. I don’t have control over the system that calls my script, and it does seem to “-Wait” on all my script’s descendants before proceeding. That’s what I need to overcome.

I’ve tried Start-Job, but that also seems to start the process as a descendant.

It does appear that Invoke-Process would start a separate process tree, but it seems to require elevated privileges (that I don’t have and wouldn’t be able to obtain), even when running a local command:

> Invoke-Command -ComputerName . -ScriptBlock { echo "Testing" }
[localhost] Connecting to remote server localhost failed with the following error message : Access is denied. For more
information, see the about_Remote_Troubleshooting Help topic.
    + CategoryInfo          : OpenError: (localhost:String) [], PSRemotingTransportException
    + FullyQualifiedErrorId : AccessDenied,PSSessionStateBroken

I’ve also looked at Invoke-CimMethod and Win32_Process.Create, as suggested in a StackOverflow post. This seems like it might be my best bet, but how do I redirect the process’s standard out and error back to the script, and exit the script with the process’s exit code?

My context is a Bamboo continuous integration server PowerShell script task that invokes my script, which in turn invokes the Gradle build tool via its Gradle Wrapper .bat script. How exactly Bamboo invokes script tasks is a black box, but it seems to “-Wait” instead of “Wait-Process”. The Gradle Wrapper typically backgrounds a daemon process that outlives a single build. Even when I use Wait-Process in my script, and return an exit code as specified in Bamboo’s knowledge base article on this issue, my jobs still hang until the Gradle Daemon exits after its 3-hour idle timeout. They suggest an alternative workaround of disabling the Gradle Daemon, but that’s not how Gradle is primarily designed to be used, and I keep running into reliability issues with that approach. It also has negative performance implications.

If I can start the Gradle Wrapper from my script as a separate process tree, so Bamboo doesn’t wait on the Gradle Daemon; still log the Gradle Wrapper’s output, so Bamboo will include it in the logs for the build job; and exit my script with the Wrapper’s exit code, so Bamboo knows whether the build succeeded or failed; I believe that’ll solve my problem.

2

I don’t believe you can both detach a process and still get console I/O. An option that comes to mind might be

  • A background job with Start-Job with a script block
  • Call the process with Start-Process with -Wait and -NoNewWindow and the -Redirect IO parameters
  • Write your IO to a file and poll data from the file to get it in your console
  • Stop polling when Job status becomes Completed
3

Just throwing Spaghetti against the wall here … If you do a “PowerShell -Help” … is there anything there that might help your cause like -MTA ??

4

Thanks for the reply, @neemobeer.

Perhaps I’ve not fully understood your suggestion, but from what I can tell, processes started by Start-Job and Start-Process are still descendants of the script’s process tree.

I have a test script where I’m using Start-Process with -Wait to simulate how Bamboo seems to invoke the user-supplied script for a job’s script task:

‘Mock-Bamboo-Server.ps1’

$startProcessArguments = @{
    FilePath = "powershell.exe"
    ArgumentList = ".\User-Task-Script.ps1"
    Passthru = $true
    NoNewWindow = $true
    Wait = $true
}

$process = Start-Process @startProcessArguments

exit $process.ExitCode

I then have my user-supplied script that is to somehow invoke the Gradle Wrapper:

‘My-Task-Script.ps1’

$job = Start-Job -ScriptBlock { Set-Location $using:PWD; .\gradlew.bat } -ArgumentList "help"

# Stream the job's output
Register-ObjectEvent -InputObject $job.ChildJobs[0].Output -EventName 'DataAdded'
Register-ObjectEvent -InputObject $job -EventName StateChanged
$job | Receive-Job -Wait

# Not yet sure how to get the last exit code of the job
$exitCode = 0

Write-Host "Exiting " $exitCode
exit $exitCode

With this setup, if the Gradle Wrapper (‘gradlew.bat’) ends up forking a Gradle Daemon process (i.e. there isn’t already one running the in the background from a prior build), the ‘Mock-Bamboo-Server.ps1’ hangs on its -Wait even after ‘My-Task-Script.ps1’ exits.

If there was already a daemon running in the background, or if I disable the daemon by adding the --no-daemon argument to ‘gradlew.bat’, ‘Mock-Bamboo-Server.ps1’ exits immediately after ‘My-Task-Script.ps1’ exits.

If ‘My-Task-Script.ps1’ somehow ran ‘gradlew.bat’ in a separate process tree, and I could still get its output and also get its exit code, I believe ‘Mock-Bamboo-Server.ps1’ would never hang.

5

Thanks, @tonyd. I’ll read up on MTA.

6

Does gradle.bat output what you need and exit despite what it does with the daemon?

7

That it does. It logs the build output and any errors to standard out and error respectively, and exits right when the build completes with an appropriate exit code.

8

I’d try this then.

$stdOutTempFile = "$env:TEMP\$((New-Guid).Guid)"
$stdErrTempFile = "$env:TEMP\$((New-Guid).Guid)"

$startProcessParams = @{
    FilePath               = 'cmd.exe'
    ArgumentList           = '/c .\gradlew.bat'
    RedirectStandardError  = $stdErrTempFile
    RedirectStandardOutput = $stdOutTempFile
    PassThru               = $true
    NoNewWindow            = $true
    WorkingDirectory       = $PWD
}

$gradleprocess = Start-Process @startProcessParams
$gradleprocess | Wait-Process

$gradleoutput = Get-Content -Path $stdOutTempFile -Raw
$gradleerror = Get-Content -Path $stdErrTempFile -Raw
Remove-Item $stdOutTempFile, $stdErrTempFile -ErrorAction SilentlyContinue

Write-Host Gradle output: $gradleoutput
Write-Host Gradle error output: $gradleerror
Write-Host Gradle exitcode: $gradleprocess.ExitCode

As you discovered, Wait-Process doesn’t wait on the descendants. Here we simply wait for the gradlew.bat to complete, grab the output/error from the temp files, and then delete the files. I used a test batch file that calls another script with start "" powershell.exe -ex bypass -noprofile -file c:\temp\second.ps1 /b which will sleep 30 seconds to simulate the “daemon” process. Then the batch sleeps for 5 seconds before completing. I’m able to see the output of the batch (gradle) along with the exit code and then 25 seconds later the “daemon” script completes.

9

Thanks, @krzydoug. Unfortunately, in my situation the Bamboo server is an additional caller that does seem to “-Wait” on all descendants of my script. They offer no options to control this behavior. Even if the script I supply uses Wait-Process to return right when the Gradle Wrapper does—with the Gradle Daemon process disowned and still running in the background—the Daemon is still a descendant process of my script, and so Bamboo continues to “-Wait” after my script returns. That’s why I believe I need to somehow start the Gradle Wrapper (‘gradlew.bat’) as another process owned by the current user, but not as a child process of the script itself.

I additionally need the script to stream the ‘gradlew.bat’ output while it runs (e.g. via Get-Content -Wait or similar) for observability, as Bamboo shows job output in its console in real time. This is essential for observability, so developers can follow the progress of and troubleshoot long-running jobs.

10

I’ve finally cooked up something that seems to work. I’m relatively new to PowerShell, so any feedback on how to make this more idiomatic or robust would be welcomed.

‘Start-UserProcess.ps1’:

<#
    .SYNOPSIS
    Starts a process as a new top-level process for the current user.

    .DESCRIPTION
    Starts and waits for a process. Similar to
    `Start-Process ... | Wait-Process`, except uses `Invoke-CimMethod` and
    `Win32_Process.Create` to start the process as a top-level process of the
    current user instead of as a child process of the caller.

    `Wait-Process` is used to wait on the process to complete. Any descendants
    of the process are not waited on, and will continue or be killed after the
    process completes (depending on how they were started).

    The process's standard output and error are redirected to temporary files,
    and streamed to the standard output and error of the host until the process
    completes. The temporary files are then deleted. Standard output and error
    lines may be interleaved out-of-order relative to how they were output by
    the process due to the nature of streaming the separate files
    asynchronously.

    .PARAMETER FilePath
    Passed to `Start-Process`. See the corresponding documentation.

    .PARAMETER ArgumentList
    Passed to `Start-Process`. See the corresponding documentation.

    .PARAMETER WorkingDirectory
    Passed to `Start-Process`. See the corresponding documentation. Defaults to
    $PWD.

    .INPUTS
    None. You can't pipe objects to Start-User-Process.

    .NOTES
    Sets `$LASTEXITCODE` to that of the user process.

    .EXAMPLE
    Start-User-Process -FilePath notepad.exe
    Starts Notepad as a new top-level process.
#>
function Start-UserProcess
{
    [CmdletBinding()]
    param (
        [Parameter(
                Mandatory = $true,
                HelpMessage = "The filename (with optional path) " +
                        "to background as a process."
        )]
        [string]$FilePath,

        [Parameter(
                HelpMessage = "The arguments to use when backgrounding " +
                        "the process."
        )]
        [string[]]$ArgumentList,

        [Parameter(
                HelpMessage = "The working directory (defaults to `$PWD)."
        )]
        [string]$WorkingDirectory = $PWD
    )

    # Stop the script on uncaught errors.
    $ErrorActionPreference = "Stop"

    Set-Variable -Name HIDDEN_WINDOW -Value 0 -Option Constant

    $stdOutTempFile = "$env:TEMP\$( (New-Guid).Guid )"

    Write-Debug "Creating standard output temp file: $stdOutTempFile"
    New-Item -Path $stdOutTempFile -ItemType File | Out-Null

    $stdErrTempFile = "$env:TEMP\$( (New-Guid).Guid )"

    Write-Debug "Creating standard error temp file: $stdErrTempFile"
    New-Item -Path $stdErrTempFile -ItemType File | Out-Null

    $envVars = Get-ChildItem env: | ForEach-Object {
        "$( $_.Name )=$( $_.Value )"
    }
    Write-Debug "Including environment variables:`n$( $envVars | Out-String )`n"

    $processStartupClass = Get-CimClass -ClassName Win32_ProcessStartup

    $processStartupProperties = @{
        EnvironmentVariables = $envVars
        ShowWindow = $HIDDEN_WINDOW
    }

    $processStartupInformation = New-CimInstance -CimClass $processStartupClass `
    -Property $processStartupProperties -ClientOnly

    $commandLine = "powershell.exe -Command & { " +
            "`$process = Start-Process -FilePath '$FilePath' "

    if ( $PSBoundParameters.ContainsKey('ArgumentList'))
    {
        $commandLine += "-ArgumentList '$ArgumentList' "
    }

    $commandLine += "-RedirectStandardOutput $stdOutTempFile " +
            "-RedirectStandardError $stdErrTempFile " +
            "-WorkingDirectory $WorkingDirectory " +
            "-NoNewWindow -PassThru; " +
            "`$process | Wait-Process; " +
            "exit `$process.ExitCode " +
            "}"

    $processClass = Get-CimClass -ClassName Win32_Process
    $createProcessParams = @{
        CimClass = $processClass
        MethodName = "Create"
        Arguments = @{
            CommandLine = $commandLine
            ProcessStartupInformation = [CimInstance]$processStartupInformation
        }
    }

    Write-Debug "Invoking command: '$commandLine'..."
    $userProcessId = $( Invoke-CimMethod @createProcessParams ).ProcessId

    $getStdOutProcessParams = @{
        FilePath = "powershell.exe"
        ArgumentList = @(
            "-Command",
            "Get-Content -Path $stdOutTempFile -Wait " +
                    "-ErrorAction SilentlyContinue"
        )
        NoNewWindow = $true
        PassThru = $true
    }

    $getStdErrProcessParams = @{
        FilePath = "powershell.exe"
        ArgumentList = @(
            "-Command",
            "Get-Content -Path $stdErrTempFile -Wait " +
                    "-ErrorAction SilentlyContinue " +
                    "| ForEach-Object { `$host.ui.WriteErrorLine(`$_) }"
        )
        NoNewWindow = $true
        PassThru = $true
    }

    Write-Debug "Tailing standard output and error..."
    $getOutputProcesses = @(
        Start-Process @getStdOutProcessParams
        Start-Process @getStdErrProcessParams
    )

    $userProcess = Get-Process -Id $userProcessId

    Write-Debug "Waiting on user process $userProcessId..."
    $userProcess | Wait-Process

    Write-Debug "Removing standard out and error temp files..."
    Remove-Item $stdOutTempFile, $stdErrTempFile

    Write-Debug "Waiting on standard output and error tails..."
    $getOutputProcesses | Wait-Process

    $exitCode = $userProcess.ExitCode

    Write-Debug "Setting `$LASTEXITCODE to $exitCode"
    $global:LASTEXITCODE = $exitCode
}

My task script becomes just:

‘My-Task-Script.ps1’:

. .\Start-UserProcess.ps1

Start-UserProcess -FilePath 'cmd.exe' -ArgumentList '/c .\gradlew.bat build'

exit $LASTEXITCODE

When I start that via the ‘Mock-Bamboo-Server.ps1’ script I shared above—which uses Start-Process -Wait—the script waits for the Gradle Wrapper to return, shows its standard out and error in the meantime, receives its exit code, and doesn’t hang on the descendant Gradle Daemon process afterwards (if launched). The Daemon does continue running in the background.

11

The above approach is live on my CI server and seems to be working. The only issue is stopping the script doesn’t stop the forked process, but I’d say that’s more of a separate issue that could be solved via a forked watchdog process.

I do wonder if the process management, error handling, and redirection couldn’t be improved upon by writing the above as a C# binary cmdlet instead of as a script-based cmdlet, but I’ll leave that as a possible future effort.

12

See in my opinion you should start a process as a separate process tree in PowerShell, you can use the Start-Process cmdlet with the -NoNewWindow parameter. This parameter ensures that the new process is created as a separate process tree.

Like: Start-Process -FilePath “yourprogram.exe” -NoNewWindow
This will start “yourprogram.exe” as a separate process tree

Hope this will help you in resolving your query.