Download all specific files and create structure for sites and subsites in a SC

Hi Guy’s,

I have found a script that is useful for what i am trying to achieve but i have been unable to amend it to get the desired result. The current script only iterates through the top site and libraries within and creates the corresponding folders and downloads the relevant files to these folders. However, what i would like to do is instead of the top site, i would like to iterate through all sites and subsites and libraries and download and create the structure accordingly. So far when i have tried i am able to create the folders for the sites but this puts them at the top level so end up with a url 1000’s of characters long, not good. So am trying achieve that if there are 7 sites under the top level then they are created under the TL folder accordingly and if each site has a subsite then they are placed into their relevant site folder and library within etc…
I’m still learning PS so at the mo i’m butchering big hefty scripts to get the results i need. Below is the current script i have made slight tweaks but reverted to the working copy that downloads just the top level site and libraries.
Granted this script looks at the site, but even when i try and change to use the sutecollection allwebs it’s still not working.
Any help greatly appreciated. Thanks

This script will extract all of the documents and their versions from a site. It will also

download all of the list data and document library metadata as a CSV file.

Add-PSSnapin Microsoft.SharePoint.PowerShell -erroraction SilentlyContinue

$destination: Where the files will be downloaded to

$webUrl: The URL of the website containing the document library for download

$listUrl: The URL of the document library to download

#Where to Download the files to. Sub-folders will be created for the documents and lists, respectively.
$destination = “D:\Export”

#The site to extract from. Make sure there is no trailing slash.
$site = “http://yoursitecollection/yoursite

Function: HTTPDownloadFile

Description: Downloads a file using webclient


$ServerFileLocation: Where the source file is located on the web

$DownloadPath: The destination to download to

function HTTPDownloadFile($ServerFileLocation, $DownloadPath)
$webclient = New-Object System.Net.WebClient
$webClient.UseDefaultCredentials = $true

function DownloadMetadata($sourceweb, $metadatadestination)
Write-Host “Creating Lists and Metadata”
$sourceSPweb = Get-SPWeb -Identity $sourceweb
$metadataFolder = $destination+""+$sourceSPweb.Title+" Lists and Metadata"
$createMetaDataFolder = New-Item $metadataFolder -type directory
$metadatadestination = $metadataFolder

   foreach($list in $sourceSPweb.Lists)
          Write-Host "Exporting List MetaData: " $list.Title
          $ListItems = $list.Items 
          $Listlocation = $metadatadestination+"\"+$list.Title+".csv"
          $ListItems | Select * | Export-Csv $Listlocation  -Force


Function: GetFileVersions

Description: Downloads all versions of every file in a document library


$WebURL: The URL of the website that contains the document library

$DocLibURL: The location of the document Library in the site

$DownloadLocation: The path to download the files to

function GetFileVersions($file)
foreach($version in $file.Versions)
#Add version label to file in format: [Filename]_v[version#].[extension]
$filesplit = $file.Name.split(".")
$fullname = $filesplit[0]
$fileext = $filesplit[1]
$FullFileName = $fullname+"_v"+$version.VersionLabel+"."+$fileext

          #Can't create an SPFile object from historical versions, but CAN download via HTTP
          #Create the full File URL using the Website URL and version's URL
          $fileURL = $webUrl+"/"+$version.Url

          #Full Download path including filename
          $DownloadPath = $destinationfolder+"\"+$FullFileName

          #Download the file from the version's URL, download to the $DownloadPath location
          HTTPDownloadFile "$fileURL" "$DownloadPath"


Function: DownloadDocLib

Description: Downloads a document library’s files; called GetGileVersions to download versions.


$folderUrl: The Document Library to Download

$DownloadPath: The destination to download to

function DownloadDocLib($folderUrl)
$folder = $web.GetFolder($folderUrl)
foreach ($file in $folder.Files | ?{ $_.Item[“Colour”] -eq ‘Red’ })
#Ensure destination directory
$destinationfolder = $destination + “” + $folder.Url
if (!(Test-Path -path $destinationfolder))
$dest = New-Item $destinationfolder -type directory

    #Download file
    $binary = $file.OpenBinary()
    $stream = New-Object System.IO.FileStream($destinationfolder + "\" + $file.Name), Create
    $writer = New-Object System.IO.BinaryWriter($stream)

          #Download file versions. If you don't need versions, comment the line below.
          GetFileVersions $file


Function: DownloadSite

Description: Calls DownloadDocLib recursiveley to download all document libraries in a site.


$webUrl: The URL of the site to download all document libraries

function DownloadSite($webUrl)
$web = Get-SPWeb -Identity $webUrl

   #Create a folder using the site's name
   $siteFolder = $destination + "\" +$web.Title+" Documents"
   $createSiteFolder = New-Item $siteFolder -type directory 
   $destination = $siteFolder

   foreach($list in $web.Lists)
          if($list.BaseType -eq "DocumentLibrary")
                 Write-Host "Downloading Document Library: " $list.Title
                 $listUrl = $web.Url +"/"+ $list.RootFolder.Url
                 #Download root files
                 DownloadDocLib $list.RootFolder.Url
                 #Download files in folders
                 foreach ($folder in $list.Folders) 
                 DownloadDocLib $folder.Url


#Download Site Documents + Versions
DownloadSite “$site”

#Download Site Lists and Document Library Metadata
DownloadMetadata $site $destination

Have a look at the IIS WebAdministration module…

… to list all the sites and their site UNC path(s), or other info, that you’d put into a collection and feed to your script.

Something like this should get you started on such a use case. Using that Path property to create your directory structure.

Get-Website | select name,id,state,physicalpath, 
@{n="Bindings"; e= { ($_.bindings | select -expa collection) -join ';' }} ,
@{n="LogFile";e={ $_.logfile | select -expa directory}}, 
@{n="attributes"; e={($_.attributes | % { $ + "=" + $_.value }) -join ';' }}

name         : TestSite
id           : 2
state        : Started
physicalPath : C:\inetpub\TestSite
Bindings     : http
LogFile      : %SystemDrive%\inetpub\logs\LogFiles
attributes   : name=TestSite;id=2;serverAutoStart=True;state=1