Old code stopped working in the past week

I’m using Powershell to conduct migrations of mass files from a network drive to a user’s OneDrive. My Powershell has been working for the past several months, until this past week. My code steps through all the user’s files and produces a report that lists all the files older than 3 years. A user that has a HUNDREDS of files going back to 2012 called me, asking why his report of old files was empty. Looking through the code I found that the following code fails to pull any files, except for the files in the root folder.

$dir = <the user's root folder of their network drive>

# The following is a list of folders in the root folder that we don't want to move to OneDrive.
$ExcludeDir = "Adobe|CSF|Favorites|IBM|Outlook|Remote|Windows|Remote Assistance Logs|Google|My Data Sources|Datair|Downloads|My Shapes|My Videos|My Pictures|My Music|Recycler|cache"

Get-ChildItem -File -Path $dir -Recurse  | Where-Object {$_.PSChildName -notmatch $ExcludeDir -and $_.PSParentPath -eq "Microsoft.PowerShell.Core\FileSystem::" + $dir}

the above only pulls up the files in the user’s root folder.

Get-ChildItem -File -Path $dir -Recurse | Where-Object {$_.PSChildName -notmatch $ExcludeDir}

The above pulls a near full list of files. None of the files from the folders listed in $ExcludeDir

Get-ChildItem -File -Path $dir -Recurse  

The above pulls a nearly the same list of files. The test user I’m using, the list only includes 10 extra lines in the output… There should be dozens, if not over a hundred lines. None of the additional files are from the “ExcludeDir” list. They are from the list of folders I had previously pulled.

The difference in files also doesn’t make sense.

Example file name: Interest Rate Outlook.xls

I’m completely perplexed how this stopped working all of a sudden. Thoughts?

Personally, I would have done things differently, but I actually have a question on your declaration of:

$ExcludeDir

I am not at all familiar with that syntax. You have spaces, pipes and no quotes at all. I had no idea PowerShell could deal with that. Can you explain for my curiosity? Thanks.

I’m pretty sure the code is incomplete/broken. … happened probably while pasting here.
$ExcludeDir is most likely a regex pattern string and should actually be wrapped in quotes. :wink:
And there is a pipe missing right in front of Where-Object

Good to know, thanks Olaf :slight_smile:

You’re completely correct on both observations. There is a missing pipe, I’ll add that back in.

With this code you check if the file name - and it’s really just the file name without the path - does not match to any of the partly strings you have in your regex pattern. If you want to explude folders that does not make that much sense.

I’d expect you check against either the fullname of the file or the directory name.

Since you obviously want to make sure to have only files contained in the folder the user specified why don’t you start your Get-ChildItem command in this folder?

1 Like

I’m now seeing all the causes of the issues. This code could never have worked… that’s not good for the previous 4 groups I migrated…

I’m sorry for you. :flushed:

That’s something positive I think. :+1:t4: :slightly_smiling_face:

No you can start over and do it right. :wink:

Issue is I don’t have time to start over. =)

Also I checked my log files from earlier in the year and the script was working… somehow…

Any recommendations on how I could adjust the Get-ChildItem call to exclude all files from the list of folders? That’s my main issue.

I’m still confused how the raw Get-ChildItem call doesn’t appear to be pulling files from those folders… Going to go back and look closer.

I was going to suggest filtering the folders first to make it quicker but using an exclude list is problematic because when -recurse is used it still includes files in subfolders of excluded folders and it excludes folders with the same name, in folders that you want to include. e.g. if you exclude folder1 then
E:\Temp\psFiles\folder1\subfolder1\file1.txt gets included but
E:\Temp\psFiles\folder2\folder1\file1.txt is excluded.

I think I would take the approach of getting all the files then splitting on the path to ensure that the folder you’re excluding is in the root folder:

$dir = 'E:\Temp\psFiles'
$excludeList = 'folder1','folder5'
Get-ChildItem -File -Recurse | Where-Object {($_.FullName -split '\\')[3] -notin $excludeList} | Select-Object FullName

Output

FullName
--------
E:\Temp\psFiles\folder2\folder2-file1.txt
E:\Temp\psFiles\folder2\folder2-file2.txt
E:\Temp\psFiles\folder2\folder1\folder2-folder1-file1.txt
E:\Temp\psFiles\folder3\folder3-file1.txt
E:\Temp\psFiles\folder3\folder3-file2.txt
E:\Temp\psFiles\folder4\folder4-file1.txt
E:\Temp\psFiles\folder4\folder4-file2.txt
E:\Temp\psFiles\folder6\folder6-file1.txt
E:\Temp\psFiles\folder6\folder6-file2.txt

I’m still getting files within those folders.

$ExcludeDir = 'Adobe','CSF','Favorites','IBM','Outlook','Remote','Windows','Remote Assistance Logs','Google','My Data Sources','Datair','Downloads','My Shapes','My Videos','My Pictures','My Music','Recycler','cache'
$files = Get-ChildItem -File -Path $dir -Recurse | Where-Object {($_.FullName -split '\\')[3] -notin $ExcludeDir}  | Select-Object FullName

||Line 3707: \egfiler1\user_sata<username>\Favorites\Bing.url
||Line 3708: \egfiler1\user_sata<username>\Favorites\FabVirtualHome.url

I’ll look more into how -split works to figure out why it isn’t doing what you thought it would.

The edits are from me swapping in and out double quotes and single quotes for the code I ran, to see if there was a difference. Mirroring your suggestion, using single quotes, found no difference.

Ah I see the issue! Need to change the [3] to the correct value in the split out array!

You do. Sorry, I should have mentioned that. Well done for figuring it out :+1:. Are you getting the expected results now?