Script to clean backup files

I am working on creating a small script that will help me clean up a backup folder. Currently the backups are done nightly at 10 pm.

I would like to trim files somewhat like this;
-Keep each day for 4 weeks
-Week 5 to lets say 26 keep only first day of week backups (Mondays)
Week 27 to week 52 keep only first day of the month. (i might not keep this last clean as weekly might leave me enough room).

One small issue we have is some days the backup is doubled. This is an issue from the vendor and cannot be fixed at this time, so I would like to delete backups that have more than 1 backup in day. I guess this can be performed before the other checks and trimming.

I was wondering if there is anything similar already out there that can give me a starting point or if anyone can help me with small bits of this?

Simon, welcome to Powershell.org. Please take a moment and read the very first post on top of the list of this forum: Read Me Before Posting! You’ll be Glad You Did!.

This forum is for scripting questions rather than script requests. We do not write customized and ready to use scripts or solutions on request.

Did you try to search for it. Usually there are already scripts available you could adapt to your special needs. You could search PowershellGallery or StackOverflow or simply use your favorite internet search engine.

I did read the rules and not asking to write a script. I was asking if anyone knew of something similar and I have used my favorite search engine and cannot seem to find anything.

My first attack is to get a file list using a filter, but the filter doesnt seem to be working, The bellow code doesnt work when I am including the Where-Object, almost like its being ignored. When i just use the Get-ChildItem $path, it lists all my files.

$path = “o:\test”

Get-ChildItem $path -recurse |
Where-Object { $.CreationTime -ge “03/01/2020” -and $.CreationTime -le “04/22/2020” }

 

Your code worked fine for me. You’re certain there are files in o:\test?

Yes when i remove the line

Where-Object { $.CreationTime -ge “03/01/2020” -and $.CreationTime -le “04/22/2020” }

it displays all my files in that test folder. The files i have are from 4/13/2020 to 4/21/2020

[quote quote=221514]Your code worked fine for me. You’re certain there are files in o:\test?

[/quote]
ok when i copied the files for test filtering it changed the createdtime for all of them to today’s date, only noticed after forcing powershell to display the creationtime field.

Guess part one of my solution is completed.

Google -> powershell delete files older than

I have been using scripts from this search for years, but my original post is a more advanced than just a simple remove old files.

OK, try this as a starter …

$root = 'D:\someBackupRootFolder'
Get-ChildItem -Path $root -Recurse -File |
    ForEach-Object {
        $fileAge = New-TimeSpan -Start ($_.LastWriteTime).Date -End (Get-Date).Date
        if(
            $fileAge.Days -gt 28 -and
            $fileAge.Days -lt 182 -and
            $_.LastWriteTime.DayOfWeek -ne 'Monday'
        ){
            $_
        }
    }

That covers your first 2 conditions. Try to add the last one yourself, ok? :wink:

After allot of testing i achieved pretty much what i was looking for. This will also be handy for other projects for backup manipulation. Thank you all for the help and bits of code.

$path = 'c:\Backup'
$archivePath = 'c:\Backup Archive'
$fileExtension   = '*.zip'

$weeklyStartDay = 29
$weeklyEndDay = 182
$archiveDays = $weeklyEndDay + 1        # Set days for archiving backups to different folder

# delete duplicate files for daily backups
echo 'Clean duplicates'
Get-ChildItem -Path $path -Filter $fileExtension -File |                # get files that start with the prefix and have the extension '.zip'
    Where-Object { $_.BaseName -match '_\d{14}$' } |                    # that end with an underscore followed by 14 digits
    Group-Object -Property @{Expression = { $_.LastWriteTime.Date }} |  # create groups based on the date part without time part
        ForEach-Object {
            $_.Group | 
            Sort-Object -Property LastWriteTime -Descending |           # sort on the LastWriteTime property
            Select-Object -Skip 1 |                                     # select them all except for the first (most recent) one
            Remove-Item -Force                                   # delete these files
        }

# after X days, remove all backups except Mondays
echo 'Clean older bacckup files, keep Mondays'
Get-ChildItem -Path $path -File |
    ForEach-Object {
        $fileAge = New-TimeSpan -Start $_.LastWriteTime -End (Get-Date).Date
        if(
            $fileAge.Days -ge $weeklyStartDay -and
            #$fileAge.Days -le $weeklyEndDay -and
            $_.LastWriteTime.DayOfWeek -ne 'Monday'
        ){
            #$_
            Remove-Item -Path $_.FullName -Force 
        }
    }

# archive files older then specified date
echo "Archive older backups"
If(!(test-path $archivePath)) { New-Item -ItemType Directory -Force -Path $archivePath | Out-Null }
Get-ChildItem -Path $path -File |
    ForEach-Object {
        $fileAge = New-TimeSpan -Start $_.LastWriteTime -End (Get-Date).Date
        if(
            $fileAge.Days -ge $archiveDays
        ){
            #$_
            Move-Item -Path $_.FullName -Destination $archivePath 
        }
    }

Simon,
Great. Thanks for sharing.

May I suggest some little tweaks anyway? :wink:

You use Get-ChildItem for almost the same files 3 times. That’s going to take a lot of time, which you can avoid. Simply create the list of files to process once and use it as often as you need.
Something like this …

$path = 'c:\Backup'
$archivePath = 'c:\Backup Archive'
$fileExtension = '*.zip'

$weeklyStartDay = 29
$weeklyEndDay = 182
$archiveDays = $weeklyEndDay - 1

$CompleteBakupFileList = Get-ChildItem -Path $path -Filter $fileExtension -File |
                        Select-Object -Property *, @{Name = 'fileAgeDays'; Expression = { (New-TimeSpan -Start $_.LastWriteTime -End (Get-Date).Date).Days } },
                                                    @{Name = 'DayOfWeek'; Expression = { $_.LastWriteTime.DayOfWeek } }
If (-not (test-path $archivePath)) { New-Item -ItemType Directory -Force -Path $archivePath | Out-Null }

$CompleteBakupFileList |
Where-Object { $_.BaseName -match '_\d{14}$' } |
Group-Object -Property @{Expression = { $_.LastWriteTime.Date } } |
ForEach-Object {
    $_.Group | 
    Sort-Object -Property LastWriteTime -Descending |
    Select-Object -Skip 1 |
    Remove-Item -Force
}
foreach ($BackupFile in $CompleteBakupFileList) {
    if (
        $BackupFile.fileAgeDays -le $archiveDays
    ) {
        Move-Item -Path  $BackupFile.FullName -Destination $archivePath 
    }
    elseif (
        $BackupFile.fileAgeDays -le $weeklyStartDay -and
        $BackupFile.fileAgeDays -ge $weeklyEndDay -and
        $BackupFile.DayOfWeek -ne 'Monday'
    ) {
        Remove-Item -Path $BackupFile.FullName -Force
    }
}

Olaf I received an email you had suggested a few modification of the code but cant see it here, was it removed?

The SPAM detection of the forum is quite picky. It got on hold for moderation. I already asked to release it. It might taky some time. :wink:

Its released from spam now :slight_smile:

Great. Thanks. :smiley:

Small bug with your suggestion Olaf, line 30 and 31 should be reversed for the logic of -ge and -le

$BackupFile.fileAgeDays -le $weeklyStartDay -and
$BackupFile.fileAgeDays -ge $weeklyEndDay -and

should be

 $BackupFile.fileAgeDays -ge $weeklyStartDay -and
$BackupFile.fileAgeDays -le $weeklyEndDay -and

I hope you catched that before you rolled it out to production. :wink: But as you’re not complaining about more than that the rest seems to work as expected!? :wink:

hahaha thats where the -WhatIf comes in handy. Yes everything else works perfectly thank you again for the help.