Compare and move files to S3

How to compare files in s3 object and local drive and move only missing files

Welcome to the forum. :wave:t3:

This forum is for scripting questions rather than script requests. We do not write customized and ready to use scripts or solutions on request.

We actually expect you to make an own attempt at the first place to get your task done or to solve your problem. If you have done so already please document here what exactly you have done and show your code. Then we probably might be able to help you step further.

1 Like

I have built this script but I am not able to compare and move only missing files also moreover the $diff and $missingfiles variables are not giving the correct output. This is still not complete script, I still have to write a command to move the missing files from local drive to S3.

$Extensions = “.diff”,“.bak”,“.trn”
$AV_Zone = (Get-EC2AvailabilityZone).RegionName
$Bucket_Name = ‘******************+$AV_Zone[0]
$HOSTNAME = $env:computername

$ID = (Get-EC2Tag | Where-Object {$.Key -eq ‘Hostname’ -and $.Value -eq $HOSTNAME})
$App = (Get-EC2Tag | Where-Object {$.ResourceId -eq $ID.ResourceId -and $.Key -eq ‘Application’})
$tag = $App.Value
$sourceloc = (Get-ItemProperty ‘HKLM:\SOFTWARE\Microsoft\Microsoft SQL Server\MSSQL15.MSSQLSERVER\MSSQLServer’).BackupDirectory
Set-AWSCredential -AccessKey ****************-SecretKey *************** -StoreAs default

List the objects in the S3 bucket

$s3Objects = Get-S3Object -Bucket $Bucket_Name -KeyPrefix $HOSTNAME/Backup

#List the files in local drive
$FilesinLocaldrive = Get-ChildItem -Path $sourceloc -Recurse

Compare the two lists of files and objects

$diff = Compare-Object $FilesinLocaldrive $s3Objects

$MissingFile = $FilesinLocaldrive| Where-Object {!$s3Objects.contains($_.Name)}

Your Compare-Object line will never work since you’re trying to compare different object types. The etag on an s3object is the file hash (minus the -## part if it exists, this is versioning). You should only need to compare the local file hash to the etag to determine if the file changed.

The missing files line also won’t work since you’re checking of the s3object array contains a string (name) when it is an array of objects. You need to compare each against name. If there is a match skip, if you iterate through the array and no match found it is a missing file


I am new to PowerShell and it’s really lil tough for me to understand this though I tried since yesterday but wasn’t able to get a fix. It would be a great help if you could help me to get the exact fix please

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.