Downloading an HTTP table to an array and insert to PRTG

PowerShell Help
1

Downloading an HTTP table to an array
https://community.spiceworks.com/topic/543224-downloading-an-http-table-to-an-array

HI all, I would be very grateful for your help with a script I based upon the above article.

I have managed to get some parts of it it to work, extracting information from
innerHTML, innerText, outerHTML and outerText (below) but
I have trouble with presenting results and feeding this into a variable so result can be uploaded to the PRTG monitoring server.

Much appreciating your help,

Thanks!

$ie = New-Object -ComObject "InternetExplorer.Application"
#$ie.visible = $true
$ie.silent = $true
$ie.Navigate("<a href="http://download.webpage.com/webpage/test.html"">http://download.webpage.com/webpage/test.html"</a>)
while($ie.ReadyState -ne 4) {start-sleep -m 100}
#while ($ie.Busy) {Start-Sleep -Milliseconds 50}

Write-Warning "Retrieving information from web"
$body = $ie.Document.getElementsByTagName("table")

Write-Warning "Tables : "
$tables = $body
$tables

$tables2 = $tables | select innerText
Write-Warning "Tables 2: "
$tables2

$RawTables = $body

Write-Verbose "Found $($RawTables.Count) tables in HTML"
$TableNum = 1

ForEach ($RawTable in $RawTables)
{ Write-Verbose "Parsing headers for Table #$TableNum"

$InnerHTML = ($RawTable | select innerText).innerText -split "`n"

Write-Warning "InnerHTML: "
$InnerHTML

$Search = $InnerHTML
Write-Warning "Search: "

$Search

$Fields = [ordered]@{}
ForEach ($Line in $Search)
{ ForEach ($Field in $Line.Matches)
{ $Fields.Add($Field.Groups[1].Value,$null)
}
}

Write-Warning "Parsing table data for Table #$TableNum"
<strong>If ($Fields.Count -gt 0)</strong>

{ $Search = $InnerHTML | Select-String -Pattern "<t[hd] ?.*?>(.*?)</t[hd]>" -AllMatches

Write-Warning "Search2: "
$Search
$Row = [PSCustomObject]$Fields
$Index = 0
$Table = ForEach ($Match in $Search.Matches)
{ $Field = $Fields.GetEnumerator().Name[$Index]
$Row.$Field = ($Match.Groups[1].Value -replace "<.*?>","").Trim()
If ($Index -eq ($Fields.Count - 1))
{ $Index = 0
Write-Output $Row
$Row = [PSCustomObject]$Fields
}
Else
{ $Index ++
}
}
[PSCustomObject]@{
Raw = $InnerHTML
Data = $Table
Fields = $Fields
}
}
<strong>Else</strong>
{ Write-Warning "Unable to determine headers for Table #$TableNum, skipping it"
}
$TableNum ++
}

This code will need to be inserted as well (not finished)

Sends results to PRTG for monitoring:

#“<prtg>”

“<result>”

“<channel>Javascript enabled</channel>”

“<value>$yesno</value>”

“</result>”

“<result>”

“<channel>Client IP address</channel>”

“<value>$clientip</value>”

“</result>”

“<text>$returndescription</text>”

“<result>”

“<channel>Real connecting IP address</channel>”

“<value>$realip</value>”

“</result>”

“<result>”

“<channel>Server IP address</channel>”

“<value>$serverip</value>”

“</result>”

“<text>$returndescription</text>”

“<result>”

“<channel>M_IP1</channel>”

“<value>$m_ip1</value>”

“</result>”

“<result>”

“<channel>M_IP2</channel>”

“<value>$m_ip2</value>”

“</result>”

“<text>$returndescription</text>”

“<result>”

“<channel>M_IP1</channel>”

“<value>$m_ip1</value>”

“</result>”

“<result>”

“<channel>IP mapp DNS mapp</channel>”

“<value>$imdm</value>”

“</result>”

“<text>$returndescription</text>”

“<result>”

“<channel>Serial</channel>”

“<value>$serial</value>”

“</result>”

“<result>”

“<channel>RTT Small hreq</channel>”

“<value>$rttsmall</value>”

“</result>”

“<result>”

“<channel>BW DL Small</channel>”

“<value>$bwdlsmalldl</value>”

“</result>”

“<result>”

“<channel>BW UL Med</channel>”

“<value>$bwmedul</value>”

“</result>”

“<result>”

“<channel>BW DL Large</channel>”

“<value>$bwlargedl</value>”

“</result>”

“<result>”

“<channel>BW UL Large</channel>”

“<value>$bwdllargeul</value>”

“</result>”

#“</prtg>”

[System.Runtime.Interopservices.Marshal]::ReleaseComObject($ie)
Remove-Variable ie

End of Script

================================================================
This is the innerHTML, innerText, outerHTML and outerText I retrieve the information out from.

innerHTML :

JavaScript enabled<script type=“text/javascript”>
document.write(‘Yes’);
</script>Yes
<noscript>No</noscript>Client IP address11.11.11.111Real connecting IP address11.11.11.111Server IP address33.333.333.33 (machine ip = 44.444.444.44)M_IP1:66.666.666.66M_IP2:55.555.555.55IP mapping matches DNS mapping:NoNetwork name:Network NameSerial:1234RTT for small http request17 msBW for DL small kbyte to browser55183.6 kbps (HIT)BW for UL medium kbyte from browser29681.1 kbpsBW for DL large Mbyte to browserCalculating…BW for UL Large kbyte from browserCalculating…

innerText :
JavaScript enabled
document.write(‘Yes’); Yes No

Client IP address
11.11.11.111

Real connecting IP address
11.11.11.111

Server IP address
33.333.333.33 (machine ip = 44.444.444.44)

M_IP1:
66.666.666.66

M_IP2:
55.555.555.55

IP mapping matches DNS mapping:
No

Network name:
Network Name

Serial:
1234

RTT for small http request
17 ms

BW for download small kbyte to browser
1001.1 kbps (HIT)

BW for UL medium kbyte from browser
1001.1 kbps

BW for DL large Mbyte to browser
Calculating…

BW for UL Large kbyte from browser
Calculating…
outerHTML :

JavaScript enabled <script type="text/javascript"> document.write('Yes'); </script>Yes <noscript>No</noscript>
Client IP address 11.11.11.111
Real connecting IP address 11.11.11.111
Server IP address 33.333.333.33 (machine ip = 44.444.444.44)
M_IP1: 66.666.666.66
M_IP2: 55.555.555.55
IP mapping matches DNS mapping: No
Network name: Network Name
Serial: 1234
RTT for small http request 17 ms
BW for DL small kbyte to browser 1001.1 kbps (HIT)
BW for UL medium kbyte from browser 1001.1 kbps
BW for DL large Mbyte to browser Calculating...
BW for UL Large kbyte from browser Calculating...
outerText : JavaScript enabled document.write('Yes'); Yes No

Client IP address
11.11.11.111

Real connecting IP address
11.11.11.111

Server IP address
33.333.333.33 (machine ip = 44.444.444.44)

M_IP1:
66.666.666.66

M_IP2:
55.555.555.55

IP mapping matches DNS mapping:
No

Network name:
Network Name

Serial:
1234

RTT for small http request
17 ms

BW for DL small kbyte to browser
1001.1 kbps (HIT)

BW for UL medium kbyte from browser
1001.1 kbps

BW for DL large Mbyte to browser
Calculating…

BW for UL Large kbyte from browser
Calculating…

2

Hey! Welcome!

First up, can I recommend you chuck all of the code within that post into a [`pre`] block (see the notes above the editor). It’s quite hard to read, as is.

Secondly, (and this might be an issue with the formatting, as above, where I’ve not seen the question for the code) - what are you actually trying to achieve here? You say you’re having difficulty formatting the output?

3

Hi again, thank you for your message and help!

I have trouble with presenting results and feeding this into a variable so result can be uploaded to the PRTG monitoring server.
The error I get when running code is:
"
WARNING: Parsing table data for Table #1
WARNING: Unable to determine headers for Table #1, skipping it
"

It seems I can extract the lines with this code:

$InnerHTML = ($RawTable | select innerText).innerText -split "`n"

But then when ordering ($Fields) I get the error message since the IF clause doesnt work and ELSE clause kicks in:

$Fields = [ordered]@{}
ForEach ($Line in $Search)
{ ForEach ($Field in $Line.Matches)
{ $Fields.Add($Field.Groups[1].Value,$null)
}
}

Write-Warning "Parsing table data for Table #$TableNum"
If ($Fields.Count -gt 0)

{ $Search = $InnerHTML | Select-String -Pattern "(.*?)" -AllMatches

Write-Warning "Search2: "
$Search
$Row = [PSCustomObject]$Fields
$Index = 0
$Table = ForEach ($Match in $Search.Matches)
{ $Field = $Fields.GetEnumerator().Name[$Index]
$Row.$Field = ($Match.Groups[1].Value -replace "< .*?>","").Trim()
If ($Index -eq ($Fields.Count – 1))
{ $Index = 0
Write-Output $Row
$Row = [PSCustomObject]$Fields
}
Else { $Index ++
}
}
[PSCustomObject]@{
Raw = $InnerHTML
Data = $Table
Fields = $Fields
}
}
Else
{ Write-Warning "Unable to determine headers for Table #$TableNum, skipping it"
}

I need also to correct
[pre]
$Row.$Field = ($Match.Groups[1].Value -replace “< .*?>”,“”).Trim()
[/pre]
but that is probably the next issue when I get the IF clause to work.