Using ROBOCOPY with powershell script

Hi guys, need your guidance in sorting the following out.

Scenario: I have a file server where new folders with files are created everyday. These folders size may range from few hundred MBs to GBs. For archiving purposes, my task is to copy yesterday’s records (Folders and the files within) along with file permissions to another location. Here is my script:

Define the variables

$today = get-date -Hour 0 -Minute 0 -Second 0 -Millisecond 0
$yesterday = get-date $today.adddays(-1) -Hour 0 -Minute 0 -Second 0 -Millisecond 0

Getting output and Copy to another location with timestamps

get-childitem -Path D:\data\SOURCEFOLDER -recurse | where {$.CreationTime -gt (get-date $Yesterday) -and $.CreationTime -lt $today} | ForEach-Object {Robocopy D:\data\SOURCEFOLDER ‘\servername\destination’ /E /COPY:DATSOU $_.Name}

Issue: Everything works well until ForEach-Object, somehow ROBOCOPY doesn’t copy the extracted content. Robocopy creates the whole folder structure from the SOURCE directory. I tried COPY-ITEM, but it has some limitations, so had to use ROBOCOPY. Any ideas what I am doing wrong here?

I don’t think it is powershell. It’s the /e switch from robocopy

Hi Frank, I have tried /S switch too, it hasn’t worked either. What i can see, robocopy is going through each and every folder (thousands) but not picking up the one delivered by powershell.

Wrap the robocopy command into Invoke-Command and then run through each object.

ForEach-Object {Invoke-Command -Command "Robocopy D:\data\SOURCEFOLDER '\\servername\destination' /E /COPY:DATSOU $_.Name"}

Thank you.

Thanks Kiran, will try it on Monday and let you know.

don’t use the /s switch either… you code copied just the files for me with just removing the /e switch

Thanks Frank. I think the U (auditing) switch in COPY /DATSOU could be at fault too. Anyways, will try and let you know.

Hi Kiran,
It fails with the following error:

Invoke-Command : Cannot bind parameter ‘ScriptBlock’. Cannot convert the “Robocopy D:\data\SOURCEFOLDER ‘C:\temp\testcopy’ /E /COPY:DATSOU FOLDER1.Name” value of type “System.String” to type
“System.Management.Automation.ScriptBlock”.
At line:4 char:176

  • … mmand -Command "Robocopy D:\data\SOURCEFOLDER ‘C:\temp\testcopy’ /E /COPY:DATSOU …
  •                ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    
    • CategoryInfo : InvalidArgument: (:slight_smile: [Invoke-Command], ParameterBindingException
    • FullyQualifiedErrorId : CannotConvertArgumentNoMessage,Microsoft.PowerShell.Commands.InvokeCommandCommand

Sorry, use Invoke-Expression, it’s not Invoke-Command

It creates the entire folder structure from SOURCE at the destination, and then goes through each and every file in the subfolders which i don’t want. Imagine going through hundreds and thousands of folders looking for content. What i want Robocopy to do is just pick the results produced by powershell and not to look into every folder.

Tried this too, no luck. I did a test run on files, it works great, but when folders and subfolders get involved, it doesn’t do the job.
The interface and logs says it is copying files, but the destination has got nothing. Checked the disk space to see if some change is taking place somewhere else, but no. Any other suggestions?

Then remove -recurse flag from from Get-ChildItem in your script.

why you don’t use robocopy’s /MAXAGE /MINAGE switches and play with get-childitem ?
you can generate values for these switches with PS and let ROBOCOPY filter files.

and, for better parameter handling try to launch robocopy this way

$parameters = @($source, $destination)
if (...) { $parameters += '/MaxAge:{0}' -f $age }
# [...]
& robocopy $parameters

Hi Max, can you please elaborate it bit further, sorry I don’t have scripting background. What is IF (…) statement doing? Thanks.

“If” intend the non-binding of adding some parameters to robocopy :slight_smile:
if you will be satisfied not creation but modification time, you need something like “robocopy source destination /minage:1 /maxage:2” without powershell at all
or look at other examples https://social.technet.microsoft.com/wiki/contents/articles/1073.robocopy-and-a-few-examples.aspx

the main message: be best way to call robocopy: use “& robocopy $parameterarray” syntax

because robocopy doesn’t support creationtime nor file-with-path selection,
for exactly your variant I can suggest this:

$source= 'd:\1'
$destination= 'd:\2'
$today = (Get-Date).Date
$yesterday = $today.AddDays(-1)
get-childitem $source -Recurse | where-object { $_.CreationTime -gt $yesterday -and $_.CreationTime -lt $today } | Foreach-Object { robocopy (Split-Path $_.FullName) $destination $_.Name }

It copies file WITHOUT folder structure into destination
for maintaining folder structure there is need some magic for destination folder tree creation

and the magic is:

get-childitem $source -Recurse |
where-object {
    $_.CreationTime -gt $yesterday -and $_.CreationTime -lt $today
} | Foreach-Object {
    $s = (Split-Path $_.FullName)
    $d = $destination + $s.Substring($source.Length)
    robocopy $s $d $_.Name
}

the additional switches for robocopy at you taste

I’ve been away from the office today, will give it a go tomorrow and let you know the outcome. Hopefully, it works!

Hi Max,

The 2nd script worked, but i think we are missing something from it. It took 3 hrs 20 mins to copy just 4.56 GB of data from 1 SATA volume to another. Normal copy & paste takes only approx 2 mins.
I started the transcript to get some logs and found the following:

Host Application: C:\Windows\system32\WindowsPowerShell\v1.0\PowerShell_ISE.exe
Process ID: 3420


Transcript started, output file is C:\temp\logs.txt
Transcript started, output file is C:\temp\logs.txt
Transcript started, output file is C:\temp\logs.txt
Transcript started, output file is C:\temp\logs.txt




ROBOCOPY :: Robust File Copy for Windows
ROBOCOPY :: Robust File Copy for Windows
ROBOCOPY :: Robust File Copy for Windows
ROBOCOPY :: Robust File Copy for Windows




Started : Thursday, 22 June 2017 9:54:20 AM
Started : Thursday, 22 June 2017 9:54:20 AM
Started : Thursday, 22 June 2017 9:54:20 AM
Started : Thursday, 22 June 2017 9:54:20 AM
Source : D:\data\SOURCEFOLDER
Source : D:\data\SOURCEFOLDER
Source : D:\data\SOURCEFOLDER
Source : D:\data\SOURCEFOLDER
Dest : C:\temp\testcopy
Dest : C:\temp\testcopy
Dest : C:\temp\testcopy
Dest : C:\temp\testcopy
Every line in log file is x4. Does that mean, it is copying multiple (x4) times? I thought it could be the recurse parameter, however after removing the parameter the script would not copy any folder. Thoughts?

Don’t know why it repeat many times but this code really call robocopy one time per one file, this is the bottleneck.
Seems you start-transcrpt somewhere inside cycle but nut before script run

So, as I’m already said, you should see if you can use Modification time or step away from robocopy.

what problems you have with copy-item ?
If it is lfn sopport look on PSAlphaFS module

I started transcript at the beginning of the script. The problem i had with Copy-item was, it wouldn’t copy the timestamps; unless you have any suggestions.