by karmista at 2013-01-05 06:01:49
Curious Case of Windows Forms Authentication via PowerShell -by DonJ at 2013-01-08 08:23:57
How to login to a webpage via powershell and download a million pdf documents?
Hi Powershell Gurus,
Need your assistance with this interesting problem in this regard.
We need to login to a webpage via powershell and download nearly a million pdf documents. There are 2 levels of authentication - Negotiate (via a standard pop-up & Forms (via the UN/Pwd field in the login.aspx)
Last week, I had a post on this wonderful forum & got a brilliant idea to pursue a programmatic approach.
The web site is hosted on IIS 6. It has "Integrated Windows Authentication" check box checked (IIS 6).
web.config Authentication & Authorization info looks like this -
"
<authentication mode="Forms">
<forms loginUrl="login.aspx" name="MyApp" timeout="720" />
</authentication>
<authorization>
<allow users="?" />
</authorization>
<sessionState mode="StateServer"
stateConnectionString="tcpip=127.0.0.1:42424" sqlConnectionString="data
source=127.0.0.1;user id=sa;password=" cookieless="false" timeout="720"
/>
"
Below Power Shell code authenticates & gets to the login.aspx page. However, I am unable to perform the "Forms" authentication from the login.aspx page.
If I attempt to directly download URL2 i.e. $webclient.DownloadFile($url2,$path), I get a HTTP 500 Internal Server Error message.
Need your assistance in getting through these authentication via tweaks to the below code.
$url = "http://10.20.30.40/myApp/login.aspx"
$path = "D:\temp\myfile1.pdf"
#$url2 = "http://10.20.30.40/myApp/servpdf.aspx?docno=1234567&allToTiff=true"
$webclient = new-object System.Net.WebClient
$credCache = new-object System.Net.CredentialCache
$creds = new-object System.Net.NetworkCredential("username","password", "domain")
$credCache.Add($url, "Negotiate", $creds)
$webclient.Credentials = $credCache
$webclient.DownloadFile($url,$path)
#$webclient.DownloadFile($url2,$path)
This is going to be pretty complicated, and you’re getting pretty deep into .NET programming. For a PowerShell forum, I don’t think you’re necessarily going to get a lot of answers. You might consider asking at StackOverflow.com, which has a developer following and folks who’ve probably done this.
Forms authentication doesn’t accept a credential in the way you’re providing. With Forms auth, you’re not responding to an HTTP authentication prompt, you’re entering a username and password into form fields and pressing a Submit button. The Web server authenticates that on the back-end, and then issues you one or more cookies in the response to show you’ve been authenticated. You have to store those cookies and re-present them in every new request. A Web browser does all of this for you; given that you’re essentially writing you’re own Web browser, it’s on you.
I can’t help but wonder if some Download Manager-style browser plugin might not be a better way for you to do this. I’ve seen tools that can crawl a site and download files in bulk. Otherwise, you’re going to be doing quite a lot of coding to make this work.