Automating website task with powershell

PowerShell Help
1

Every now and then I have to perform an audit on user details on multiple websites and would really like to automate part of it. At the moment I log into the website, navigate to the user section, export the users, run my powershell script that finds new and old users and I then upload those to the website to make changes, the problem is that I have to this 18 time to the same website with 18 different logins. If I could find a way to automate the login and download process that would be wonderful.

2

There are multiple steps you have outlined here, but you are not showing any code you tried. We need to see it in order to help you.

As for you use case psudo code it out then code for each, then put it all together:

1 - log into the website
Do a search for ‘powershell login to a website’, you’ll get plenty of examples.
Get this working to how you like it and move to your next point

2 - navigate to the user section
Do a search for ‘powershell website navigation’, you’ll get plenty of examples.
Get this working to how you like it and move to your next point

3 - export the users
Do a search for ‘powershell website scraping’, you’ll get plenty of examples.

Now, put it all together, test it, if you have issues post back here and we’ll see what we can do.

For multiple sites, it just passing in a list and loop to run the same code on each site, that is if each site is identical if design. If not, then you have bigger issues, and at that point, time to hire someone to do this for you.

3

You can have a look at below cmdlet as well.

Get-Help Invoke-RestMethod -Online

4

Yes there are a few steps and I took your advice and did a fresh search on this topic as I tried this a year ago with no success. I am stuck on the first part login in, code below. What part of the $r should I be looking at to see if the login is successful or not?

[pre]

$r=Invoke-WebRequest https://app.readingeggs.com/login -SessionVariable fb

$form = $r.Forms[0]

$form.Fields[“username”]=“MyName”
$form.Fields[“password”]=“Password”

$r=Invoke-WebRequest -Uri (“https://app.readingeggs.com” + $form.Action) -WebSession $fb -Method POST -Body $form.Fields

[/pre]

5

This all of course depends on the site you are hitting.
Single page site, or a mult-page site, plain html or JavaScript. Each have their own idiosyncrasies.

So, for a simple page site…

$url = 'https://pwpush.com'
($FormElements = Invoke-WebRequest -Uri $url -SessionVariable fe)  
($Form = $FormElements.Forms[0]) | Format-List -Force
$Form | Get-Member
$Form.Fields

# so you end up here

$password = '1234'
$loginUrl = 'https://pwpush.com'

$ie = New-Object -com internetexplorer.application
$ie.visible = $true
$ie.navigate($loginUrl)

while ($ie.Busy -eq $true) 
{ Start-Sleep -Seconds 1 }

($ie.document.getElementById('password_payload') | 
select -first 1).value = $password
Start-Sleep -Seconds 1 

$ie.Document.getElementsByName('commit').Item().Click();
Start-Sleep -Seconds 1

Let’s say, I am landing on a internal ADFS site. After of course scraping the page to field all needed fields and actionable object to deal with, would be this…

$username = 'user@contoso.com'
$password = 'SomeSuperPassword'
$loginUrl = 'https://adfs.contoso.com/adfs/ls/idpinitiatedsignon' 

$ie = New-Object -com internetexplorer.application
$ie.visible = $true
$ie.navigate($loginUrl)

while ($ie.Busy -eq $true) { Start-Sleep -Seconds 1 }

($ie.document.getElementById('idp_GoButton') | 
select -first 1).click()
Start-Sleep -Seconds 1 

($ie.document.getElementById('userNameInput') | 
select -first 1).value = $username
Start-Sleep -Seconds 1 

($ie.document.getElementById('passwordInput') | 
select -first 1).value = $password
Start-Sleep -Seconds 1 

($ie.document.getElementById('submitButton') | 
select -first 1).click()
Start-Sleep -Seconds 1 

($ie.document.getElementById("idp_SignOutButton") | 
select -first 1).click()
Start-Sleep -Seconds 1

A more complicated site, AirCanada in this case, that often changes, so, code I wrote here, may not even work as smoothly as before. It’s a multiple page site.

$url = 'https://www.aircanada.com/ca/en/ado/profile/sign-in.html'
($FormElements = Invoke-WebRequest -Uri $url -SessionVariable fe)  

($Form0 = $FormElements.Forms[0]) | Format-List -Force
$Form0.Fields

($Form1 = $FormElements.Forms[1]) | Format-List -Force
$Form1.Fields

($Form2 = $FormElements.Forms[2]) | Format-List -Force
$Form2.Fields

$SiteSource = Invoke-WebRequest -Uri $url
$SiteSource.AllElements | Where{$_.TagName -eq "Button"} `
| Select-Object -Expand InnerText

$SiteSource.AllElements | Where{$_.TagName -eq "Button"} `
| Select-Object -Property * | Where innerText -eq 'SIGN IN'

# So armed with the above, you get to do this.

$url = 'https://www.aircanada.com/ca/en/ado/profile/sign-in.html'

$ie = New-Object -com internetexplorer.application
$ie.visible = $true
$ie.navigate($url)

while ($ie.Busy -eq $true) { Start-Sleep -Seconds 1 }

($ie.document.getElementById('agencyIATA') | 
select -first 1).value = '1234567'

($ie.document.getElementById('agencyID ') | 
select -first 1).value = '789'

($ie.document.getElementById('bookingAgent') | 
select -first 1).value = '159'

($ie.document.getElementById('password') | 
select -first 1).value = '1234'

($ie.document.getElementById('rememberAgencyInfo') | 
select -first 1).value = $true

Start-Sleep -Seconds 2

($ie.Document.IHTMLDocument3_getElementsByTagName('button') `
|  Where-Object innerText -eq 'SIGN IN').Click()

So, often, you will find on some sites, you literally have to loop elements to get to what you are after.

So, the question becomes, do you really need to login to the site interactively to get the data or can you get at it with an Invoke-* directly? So, this you will need to literally investigate / scrape every site you plan to hit to be sure what you need to pass.