[SOVLED] Crawl a webfolder for a specific file type [Complete N00b warning]

PowerShell Help
1

Hi I need a little help. (complete n00b)

I need to get all the CSV files from a homepage (specific folder) and download them to a local folder.

This first part works and I get a nice list of all the *.CSV files:

$WebResponse = Invoke-WebRequest "https://somepage.dot/"
$links2 = $WebResponse.links | select href | where {$_.href -like '*csv*'}

The next part, not so much I have serveral issues:

  1. I need to trim the filename and I know it’s because I’m doing some thing wrong, but I just can not figure out what.

  2. Invoke-webrequest also has problems with the trim caracters, so it throws an error: Invoke-WebRequest : Cannot bind parameter ‘Uri’. Cannot convert the “@{href=https://somepage.dot/somefile.csv}” value of type “Selected.System.Management.Automation.PSCustomObject” to type “System.Uri”.

ForEach ($href in $links2)
{
    $FileName = Split-Path $href -leaf
    $filelocation = "C:\Temp\" + $FileName.Trim("}")
    Invoke-WebRequest $href -OutFile $filelocation
}

I know it must be something basic I’m missing.

Kind regards Jgaard.

2

jgaard,
Welcome to the forum. :wave:t4:

In PowerShell we work with objects and properties. So there’s barely the need to do string acrobatics like .Trim().

When you output $links2.href to the console you will see that there are the plain link paths. So instead of

ForEach ($href in $links2)

you could do

ForEach ($href in $links2.href)

Or inside your loop instead of using only $href you use $href.href.

1 Like
3

Perfect I knew it was a “small” thing like objects and properties :wink:

Thank you for your quick help.

4

Great. I’m glad it helped. :+1:t4: :slightly_smiling_face: