Matching on multiple lines with -contains or -like

PowerShell Help
1

Hello,

I would like to be able to match on multiple lines and I am having troubles trying to figure out how to do this. I have a feeling I am going to need regex but I am not sure. Any help would be greatly appreciated.

$url = “http://contoso.com/robots.txt”

$disallowall = {
User-agent: *
Disallow: /
}

$robots = (Invoke-WebRequest -Uri $url -ErrorAction Stop).content | Out-String
if ($robots -contains $disallowall){

Write-Host “yes” -ForegroundColor Green

}

The issue I am having is “$robots -contains $disallowall” says its false even though $disallowall definitely contains these two lines. I have tried to use “$disallow” and other variations but cannot get this to match on the multiple lines in $disallowall variable with -contains or -like. Obviously the URL in this example wouldnt match but in my working example it does.

2

It is unclear if {} are required in your match string. A beginning and ending pair of {} denotes a script block in PowerShell. When comparing strings, it is likely unnecessary. Using quotes, here-strings, and other string formatting mechanisms will be the way to go.

If the downloaded content has line breaks that match your [console]::NewLine configuration, you could simply just use a here-string with the -like operator:

$disallowall = @'
User-agent: *
Disallow: /
'@

$robots = (Invoke-WebRequest -Uri $url -ErrorAction Stop).content | Out-String

if ($robots -like "*$disallowall*") {

    Write-Host "yes" -ForegroundColor Green

}

If the line break characters are unknown, you can test with -match, which uses regex. That will allow for a specific newline character pattern match. This will match 0 or 1 carriage returns and a line feed. It also escapes the literal *.

$disallowall = 'User-agent: \*\r?\nDisallow: /'

$robots = (Invoke-WebRequest -Uri $url -ErrorAction Stop).content | Out-String

if ($robots -match $disallowall) {

    Write-Host "yes" -ForegroundColor Green

}
3

It appears your code blocks took on some formatting. Can you please re-post your code suggestions?

4

There are several issues. First, are you sure that Invoke-RestMethod is returning what you expect as it’s odd to refer directly to a txt file. The $disallowall is defined as a script block. The contains operator is an array comparison:

PS C:\WINDOWS\system32> "red","white","blue" -contains "white"
True

PS C:\WINDOWS\system32> "red","white","blue" -contains "purple"
False

This most likely would not work unless the content is array and a line matches exactly. What exactly does the $robots contain?

5

Rob,

Using my real URL (not contoso.com), the $robots contains the following.

#robots.txt file

User-agent: *
Disallow: /

I am checking robots.txt to confirm that file contains the following two lines consecutively.

User-agent: *
Disallow: /

Also, I have tried to match on this using regex and I cannot get that working either.

$robots -match [regex]“User-agent: *[\r\n]Disallow: /”)

6

When using the -contains operator on an array, it will look at/match elements.

$array = @(
    "one",
    "two",
    "three"
)

$array -contains 'three' # true

$array -contains 'hree' # false

String types have a .Contains method you can use to find text. Something like this works

($array | Out-String).Contains('hree') # true

7

Thanks Adminofthings45! This seemed to be the part I needed.

$disallowall = @‘
User-agent: *
Disallow: /
’@

Can you tell me why you wrap the variable value in @? I understand that leading with @ forces it to be an array (even if it only contains a single value) however why is it necessary to end with @ also?

8

@‘’@ is a here-string. It is used to help preserve formatting for strings that span multiple lines.

9

Perfect! Thank you AdminOfThings45! I really appreciate your guidance.