I was tasked with implementing a log copy solution in our environment. For reasons outside my control, it had to be a homegrown solution, so I utilized Powershell and put together a nice little tool, seen at the end of my text.
The objective was to use an off-domain local account with read-only access to the source share, grab the relevant files, copy them to the destination share (a pull from the destination server) AS an on-domain service account (separate from the off-domain local account on the remote server). The script runs successfully using this code, with an encrypted credential file. I know this because the logs are retrieved and my destination logs are updated with the relevant counts. However, on the source server for some off-domain machines, I’m encountering Security log Event ID 4625, indicating that my on-domain account is failing to log on.
Because the account is not used in the New-PSDrive command, I’m having difficulty seeing where in my code that this account is trying to log in to the remote server. I’m wondering if anyone can provide me with enough breadcrumbs to figure this out. I know the script is querying what it needs to, cleaning out the source directory properly and that only the off-domain account has access to the off-domain share it is reaching, and yet these errors persist. I should also mention that this script is running successfully without errors for all on-domain machines, as well as a litany of other off-domain machines, it’s just a select few that are throwing this error consistently.
$UserName = 'Service-Account'$Password = Get-Content 'C:\FilePath\CredFile.txt' | ConvertTo-SecureString$ServiceCred = New-Object System.Management.Automation.PSCredential($UserName, $Password)$SourceIP = 'x.x.x.x'$SourceShare1 = 'ShareName1'$SourceShare2 = 'ShareName2'$DestPath1 = 'C:\Directory\AppLogs'$DestPath2 = 'C:\Directory\IISLogs'$Log = 'C:\Directory\CopyLog.log'$DaysBack = "-21"$CurrentDate = Get-Date$DateToDelete = $CurrentDate.AddDays($Daysback)Try{New-PSDrive -Name "AppLogs" -PSProvider "FileSystem" -Root \\$SourceIP\$SourceShare1 -Credential $ServiceCred$Files1 = Get-ChildItem AppLogs:\* | Where-Object {$_.LastWriteTime -gt (Get-Date).AddDays(-14)} -VerboseCopy-Item $Files1 -Destination $DestPath1 -ErrorAction Stop -Force -VerboseGet-ChildItem -Path \\$SourceIP\$SourceShare1\* | Where-Object {$_.LastWriteTime -lt $DateToDelete } | Remove-Item -Recurse -Verbose$Files1Count = $Files1.Count"$Files1Count files were written to $DestPath1 on $CurrentDate." | Out-File -FilePath $Log -Append}Catch{"No files were written to $DestPath1 on $CurrentDate." | Out-File -FilePath $Log -Append}Try{New-PSDrive -Name "IISLogs" -PSProvider "FileSystem" -Root \\$SourceIP\$SourceShare2 -Credential $ServiceCred$Files2 = Get-ChildItem IISLogs:\* | Where-Object {$_.LastWriteTime -gt (Get-Date).AddDays(-14)}Copy-Item $Files2 -Destination $DestPath2 -ErrorAction Stop -Force -VerboseGet-ChildItem -Path \\$SourceIP\$SourceShare2\* | Where-Object {$_.LastWriteTime -lt $DateToDelete } | Remove-Item -Recurse -Verbose$Files2Count = $Files2.Count"$Files2Count files were written to $DestPath2 on $CurrentDate." | Out-File -FilePath $Log -Append}Catch{"No files were written to $DestPath2 on $CurrentDate." | Out-File -FilePath $Log -Append}Remove-PSDrive -Name AppLogs -VerboseRemove-PSDrive -Name IISLogs -Verbose
What user context is the script being run in. You created the drive mapping using the credentials of ‘Service-Account’ but the script is accessing the files in the context of the account running the script.
...I'm having difficulty seeing where in my code that this account is trying to log in to the remote server.
Suggestions to better understand what's happening with your code:
Sometimes using a little Write-Output here and there helps you better understand what's going on when, and lead you to the 'aha' moment.
Try/catch should be handling as little as possible - like one command. Because there is so much going on within one try/catch, you can't isolate/remediate errors efficiently.
Change your catch blocks from a custom string to using Write-Error $_ to output the actual error, at least for testing. This might get you to the issue much quicker when it hits a problematic system.
In addition to Sean's post, does the account being used have an restrictions on it or lack of privileges/permissions?
“What user context is the script being run in. You created the drive mapping using the credentials of ‘Service-Account’ but the script is accessing the files in the context of the account running the script.”
The drive mapping is created using the Service-Account credentials, let’s rename that Service-AccountLocal, which is the off-domain account with read-only access to the source directory on the off-domain machine. The script context runs as Domain\Service-Account, a separate account that is domain-joined. That account runs all the scripts and has modify access to the destination directory on the local machine.
I think I know what you’re getting at here, but the error occurs when you run New-PSDrive, not during other command runs. We have tested and confirmed this already. The problem we are experiencing is related to the New-PSDrive. That command uses only the remote machine’s local account with read-access to the desired source directory. The error occurs before we even attempt a Copy-Item, or a Get-ChildItem, in the script.
We have a theory, as we have seen this behavior in other applications in our environment, that for some reason Powershell is not using NTLM credentials to connect, but rather going through this process instead:
Checking for Kerberos credentials
Failing with Kerberos credentials
Trying NTLM creds and succeeding
Again, these scripts are working consistently and are copying data correctly, etc etc, but something about the New-PSDrive command is triggering these errors. However, based on our testing and troubleshooting, at this time we believe the issue to be larger than just a Powershell bug.
I have implemented a solution for the time being, and would like to post it here, but I would also like to acknowledge the fact that we have confidently determined that New-PSDrive is still attempting to use incorrect credentials to connect to an off-domain share BEFORE utilizing the credentials expressed in the -Credential Parameter. Command seen below:
As for the solution, I tested making a mirrored local account on the destination server that matches the name and password of the source server. I granted the account local logon rights and log on as a batch job as well. When doing this, I modified the script I’m using to not pass any credentials at all and instead run using its own access. This allowed the script to run successfully removed all of my errors.
After all this troubleshooting, I can confirm that despite the New-PSDrive command being piped a specific credential that it still tries to reach out to the off-domain machine using the credentials that are running the code. It is my understanding that New-PSDrive should not do this and should only use the credentials that I give it, but unfortunately the answer to whether or not New-PSDrive is acting as intended eludes me. If anyone has additional input on this aspect of the execution I would love to learn more. As far as this same problem applying to other systems we have in place, the common denominators seem to be whenever we have a domain-joined service account utilizing off-domain credentials to run a service or task. Our server monitoring solution and our patching software have both been the culprit of similar issues in the past, and all of these applications are always accessing off-domain machines using an on-domain service account.
Finished script (runs as a mirrored local account on the destination server):
$SourceIP = 'x.x.x.x'
$SourceShare1 = 'ShareName1'
$SourceShare2 = 'ShareName2'
$DestPath1 = 'C:\Directory\AppLogs'
$DestPath2 = 'C:\Directory\IISLogs'
$Log = 'C:\Directory\CopyLog.log'
$DaysBack = "-21"
$CurrentDate = Get-Date
$DateToDelete = $CurrentDate.AddDays($Daysback)
Try
{
New-PSDrive -Name "AppLogs" -PSProvider "FileSystem" -Root \\$SourceIP\$SourceShare1
$Files1 = Get-ChildItem AppLogs:\* | Where-Object {$_.LastWriteTime -gt (Get-Date).AddDays(-14)} -Verbose
Copy-Item $Files1 -Destination $DestPath1 -ErrorAction Stop -Force -Verbose
Get-ChildItem -Path \\$SourceIP\$SourceShare1\* | Where-Object {$_.LastWriteTime -lt $DateToDelete } | Remove-Item -Recurse -Verbose
$Files1Count = $Files1.Count
"$Files1Count files were written to $DestPath1 on $CurrentDate." | Out-File -FilePath $Log -Append
}
Catch
{
"No files were written to $DestPath1 on $CurrentDate." | Out-File -FilePath $Log -Append
}
Try
{
New-PSDrive -Name "IISLogs" -PSProvider "FileSystem" -Root \\$SourceIP\$SourceShare2
$Files2 = Get-ChildItem IISLogs:\* | Where-Object {$_.LastWriteTime -gt (Get-Date).AddDays(-14)}
Copy-Item $Files2 -Destination $DestPath2 -ErrorAction Stop -Force -Verbose
Get-ChildItem -Path \\$SourceIP\$SourceShare2\* | Where-Object {$_.LastWriteTime -lt $DateToDelete } | Remove-Item -Recurse -Verbose
$Files2Count = $Files2.Count
"$Files2Count files were written to $DestPath2 on $CurrentDate." | Out-File -FilePath $Log -Append
}
Catch
{
"No files were written to $DestPath2 on $CurrentDate." | Out-File -FilePath $Log -Append
}
Remove-PSDrive -Name AppLogs -Verbose
Remove-PSDrive -Name IISLogs -Verbose