Ok! This is where I should have looked at the output of Invoke-WebRequest before responding… I was just looking at the code presented and focused on the loop. So, the second loop I posted should still be the one you want to go with, but what needs to change is how you are accessing the fields you want in the CSV. Instead of using the XML format that you see in the outfile, you’re going to need to traverse what is returned by Invoke-WebRequest.
After running the real version of “$SearchResult = $SearchTermNumber[1…20] | ForEach-Object -Process {Invoke-WebRequest -outfile C:\temp\Phones_Results_$_.xml -PassThru -URI “[Search tool URI” }”, take a look at the output from $SearchResult[0]. That will give you the HtmlWebResponseObject returned by the first Invoke-WebRequest. You’ll see that there are several different attributes related to the HTML response. (The code below is using on the examples in help for Invoke-WebRequest):
PS C:\> $r = Invoke-WebRequest -URI http://www.bing.com?q=how+many+feet+in+a+mile -OutFile c:\temp\iwrtest.xml -PassThru
PS C:> $r
StatusCode : 200
StatusDescription : OK
Content : //<h...
RawContent : HTTP/1.1 200 OK
Pragma: no-cache
Transfer-Encoding: chunked
Vary: Accept-Encoding
Cache-Control: no-cache, no-store, must-revalidate
Content-Type: text/html; charset=utf-8
Expires: -1
P3P: CP="...
Forms : {sb_form}
Headers : {[Pragma, no-cache], [Transfer-Encoding, chunked], [Vary, Accept-Encoding], [Cache-Control,
no-cache, no-store, must-revalidate]...}
Images : {@{innerHTML=; innerText=; outerHTML=; outerText=;
tagName=IMG; id=id_p; class=b_icon id_avatar; style=DISPLAY: none;
src=data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAEALAAAAAABAAEAAAIBTAA7}}
InputFields : {@{innerHTML=; innerText=; outerHTML=; outerText=; tagName=INPUT; spellcheck=false;
onfocus=_ge('b_header').className='b_focus';; id=sb_form_q; title=Enter your search term;
class=b_searchbox; maxLength=1000; value=how many feet in a mile; name=q; autocomplete=off;
autocorrect=off; autocapitalize=off}, @{innerHTML=; innerText=; outerHTML=;
outerText=; tagName=INPUT; tabIndex=0; id=sb_form_go; title=Search; class=b_searchboxSubmit;
type=submit; value=Submit Query; name=go}, @{innerHTML=; innerText=; outerHTML=; outerText=; tagName=INPUT; id=sa_qs; type=hidden; value=ds;
name=qs}, @{innerHTML=; innerText=; outerHTML=;
outerText=; tagName=INPUT; type=hidden; value=QBRE; name=form}...}
Links : {@{innerHTML=Web; innerText=Web; outerHTML=Web; outerText=Web; tagName=A; href=/?scope=web&FORM=HDRSC1;
h=ID=SERP,5032.1}, @{innerHTML=Images; innerText=Images; outerHTML=Images;
outerText=Images; tagName=A; href=/images/search?q=how+many+feet+in+a+mile&FORM=HDRSC2;
h=ID=SERP,5033.1}, @{innerHTML=Videos; innerText=Videos; outerHTML=Videos;
outerText=Videos; tagName=A; href=/videos/search?q=how+many+feet+in+a+mile&FORM=HDRSC3;
h=ID=SERP,5034.1}, @{innerHTML=Maps; innerText=Maps; outerHTML=Maps; outerText=Maps; tagName=A;
href=/maps/default.aspx?q=how+many+feet+in+a+mile&mkt=en&FORM=HDRSC4; h=ID=SERP,5035.1}...}
ParsedHtml : mshtml.HTMLDocumentClass
RawContentLength : 75540
PS C:> $r | gm
TypeName: Microsoft.PowerShell.Commands.HtmlWebResponseObject
Name MemberType Definition
---- ---------- ----------
Equals Method bool Equals(System.Object obj)
GetHashCode Method int GetHashCode()
GetType Method type GetType()
ToString Method string ToString()
AllElements Property Microsoft.PowerShell.Commands.WebCmdletElementCollection AllElements {get;}
BaseResponse Property System.Net.WebResponse BaseResponse {get;set;}
Content Property string Content {get;}
Forms Property Microsoft.PowerShell.Commands.FormObjectCollection Forms {get;}
Headers Property System.Collections.Generic.Dictionary[string,string] Headers {get;}
Images Property Microsoft.PowerShell.Commands.WebCmdletElementCollection Images {get;}
InputFields Property Microsoft.PowerShell.Commands.WebCmdletElementCollection InputFields {get;}
Links Property Microsoft.PowerShell.Commands.WebCmdletElementCollection Links {get;}
ParsedHtml Property mshtml.IHTMLDocument2 ParsedHtml {get;}
RawContent Property string RawContent {get;}
RawContentLength Property long RawContentLength {get;}
RawContentStream Property System.IO.MemoryStream RawContentStream {get;}
Scripts Property Microsoft.PowerShell.Commands.WebCmdletElementCollection Scripts {get;}
StatusCode Property int StatusCode {get;}
StatusDescription Property string StatusDescription {get;}
You’re going to have to figure out what part of that response you actually want to include in your CSV. That data is probably going to come from the innerHTML property of the objects in the AllElements property of response. That may not be too bad, depending on the complexity of the html returned by your search tool…
Someone here may have a better solution than using Invoke-WebRequest…