Schedule rolling reboots with service check & email?

Ive been banging my head on the desk trying to take a batch script and get the functionality in powershell.

I need to schedule reboots of servers and have the servers reboot one at a time according to the order of the file the server is in. I need to check a service or two, and then capture a log file for all servers in that specific scheduled reboot time, then email it out.

I am basically lost on this one entirely as I’ve tried all the options and I can schedule a single reboot, but nothing in a rolling manner, I can’t get the logs or service checks to go, or the email.

Can anyone help me out? The batch file is old and it isn’t great, but I’m also trying to learn more. All servers don’t have powershell on them, and it’s a mix of 2003-2012.

Do the services on the 1st server listen on a specific TCP port? You could have the second server do an while loop or do until those ports are back up and listening from the first server being rebooted and then trigger each dependent server in that manner.

emailing will only work if the SMTP server you’re using alows the IP Address of the server sending the email to relay.
http://www.microsoft.com/technet/prodtechnol/WindowsServer2003/Library/IIS/58f05ef9-55a3-42b3-9f57-27fdc8723b8a.mspx?mfr=true

Also if the batch files work maybe you can post the batch file code here so we can look at it and show how it might be done with PowerShell.

-VERN

Howdy -

Thank you for the response, I’m still working at this. The service I’m checking now is the RPC server and I found a better “batch file script” online than what I had so I am trying that, but it’s still the same process basically and I think it could be better. I’d like the emails to be formatted so it’s the server name - reboot start time - time server came back online, and anything else useful which I could add. I’m still trying to figure out how to do this in powershell and not having luck at all. One issue I have with this script is that I often get the “hang on reboot” alert emails and if I increase the time limit then it takes a long time to reboot the list of servers as some reboot quicker, I know I can’t fix that as it’s based on the server, but maybe a way to not have the hung reboot alerts unless it’s really hung? I can send email from the system that I run this on.

echo off

REM ********************************
REM Set the time limits for the items below

REM Set the max time here for the time for the server to shutdown
set ST=1200
REM Set the max time in SECONDS for a server to complete POST and start booting
set BT=200
REM Set the max time in MINUTES for a server to start after POST
set CT=10
REM Set the name of the mail server:
set MLSVR=mail.testdomain.local
REM Set the TO: email address:
set TOEML=Jake@testdomain.local
REM Set the From: email address:
set FRMEML=jake@testdomain.local

REM End of Tunable Parameters
REM ********************************

REM Finding the Local Timezone
for /f “tokens=1,2*” %%K in ( ’ reg query HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\TimeZoneInformation ’ ) do if %%K==StandardName set timezone=%%M

REM Creating the temporary Log file
echo ***Logging Started on %DATE% %TIME% %TIMEZONE% *** > log.txt
echo %USERNAME% on %COMPUTERNAME% has initiated the script at %CD% >> log.txt

REM Checking if the list of servers for reboots is available.
if NOT EXIST computers.txt goto nocomputers

REM Listing the servers that will be rebooted.
echo The following Servers will be rebooted: >> log.txt
echo The following Servers will be rebooted:
type computers.txt >> log.txt
type computers.txt
echo. >> log.txt
echo.

:routinestart
REM Parse file and call the reboot subroutine with the server name as argument.
for /F %%i in (computers.txt) do (call :restartscript %%i)

REM Subroutine to rename the temporary log file and then email the final log.
:cleanup
for /f “tokens=1-5 delims=/ " %%d in (”%date%") do (
echo. >>%USERDOMAIN%-%%g-%%f-%%e.txt
type rebootlog.txt >> %USERDOMAIN%-%%g-%%f-%%e.txt
del /F rebootlog.txt
bmail -s %MLSVR% -t %TOEML% -f %FRMEML% -h -a “%USERDOMAIN% Server Reboot Log” -m %USERDOMAIN%-%%g-%%f-%%e.txt
)
goto eof

REM The actual reboot script.
:restartscript
REM Initializing the Log file.
set COMP=%1
echo. >> log.txt
echo.
echo %DATE% %TIME% *** Log for %COMP% >> log.txt
echo %DATE% %TIME% *** Log for %COMP%
echo %DATE% %TIME% Checking if %COMP% is up >> log.txt
echo %DATE% %TIME% Checking if %COMP% is up

REM Checking if server is up if it is not up, skip this server and go to the next one.
ping %COMP% > nul
if %ERRORLEVEL% NEQ 0 goto offline
goto startreboots

REM Rebooting the server.
:startreboots
echo %DATE% %TIME% %COMP% is Up >> log.txt
echo %DATE% %TIME% %COMP% is Up
echo %DATE% %TIME% Rebooting %COMP% >> log.txt
echo %DATE% %TIME% Rebooting %COMP%
shutdown -r -f -m \%COMP% -t 0
echo %DATE% %TIME% %COMP% is rebooting… please wait… >> log.txt
echo %DATE% %TIME% %COMP% is rebooting… please wait…

for /L %%a in (1,1,%ST%) do (
ping -n 2 -w 1 %COMP% > nul
if ERRORLEVEL 1 goto down
)
set CAUSE=has hung on shutdown
goto computerhung

:down
echo %DATE% %TIME% %COMP% is down, waiting for server to come up >> log.txt
echo %DATE% %TIME% %COMP% is down, waiting for server to come up

for /L %%a in (1,1,%BT%) do (
ping -n 2 -w 1 %COMP% > nul
if NOT ERRORLEVEL 1 goto start
)
set CAUSE=has hung during or after POST.
goto computerhung

:start
echo %DATE% %TIME% %COMP% is starting up… please wait for a minute… >> log.txt
echo %DATE% %TIME% %COMP% is starting up… please wait for a minute…

for /L %%a in (1,1,%CT%) do (
ping -n 40 -w 1 %COMP% > nul
echo %DATE% %TIME% Checking the RPC Service status… Pass %%a of %CT% >> log.txt
echo %DATE% %TIME% Checking the RPC Service status… Pass %%a of %CT%
sc “\%COMP%” query RpcSs | FIND “RUNNING” > nul
if NOT ERRORLEVEL 1 goto online
echo %DATE% %TIME% The RPC Service has not started… >> log.txt
echo %DATE% %TIME% The RPC Service has not started…
)
set CAUSE=has hung on Boot
goto computerhung

:online
echo %DATE% %TIME% %COMP% is up - RPC Service is running. >> log.txt
echo %DATE% %TIME% %COMP% is up - RPC Service is running.
goto eof

:computerhung
bmail -s %MLSVR% -t %TOEML% -f %FRMEML% -b “%USERDOMAIN% - %DATE% %TIME% %TIMEZONE% %COMP% %CAUSE%” -h -a “%USERDOMAIN% - %COMP% %CAUSE%”
echo %DATE% %TIME% %COMP% %CAUSE%. Skipping to next server. >> rebootlog.txt
echo %DATE% %TIME% %COMP% %CAUSE%. Skipping to next server.
goto eof

:offline
echo %DATE% %TIME% %COMP% not reachable… skipping >> log.txt
echo %DATE% %TIME% %COMP% not reachable… skipping
goto eof

:nocomputers
echo Make sure the file “%CD%\computers.txt” exists and is readable >> log.txt
echo Make sure the file “%CD%\computers.txt” exists and is readable
echo Stopping Script execution now. >> log.txt
echo Stopping Script execution now.
goto cleanup

:eof

I’ll need some time to analyze the whole batch file but this first response I will just try to give you the PowerShell equivalents to some of your commands.

TimeZone

Get-WMIObject Win32_TimeZone | Format-Wide caption

append to a file instead of ECHO >

“The Following Servers will be rebooted” | Out-File log.txt -Append

The for loop with content from a file

Keep in mind this assumes your currently logged on user is admin of those remote machines

$computers = Get-Content .\computers.txt
foreach ($computer in $computers) { Restart-Computer -ComputerName $computer }

echo the date and time into your log file

Get-Date | Out-File .\log.txt -Append

Sending email

Whatever bmail is may still work but if you know your SMTP relay server info

help Send-MailMessage -Full # actually best to read this help doc (since I’ve never done this in production)

Checking is server is up # PING still works in PS

Test-Connection is the CMDLET name ping is an alias

help Test-Connection -Full # it has some cool stuff so you should read the entire help doc #

I showed rebooting remote servers above

interesting for /L # I am not familiar with what is in your %%a parameter

I showed PowerShell foreach loop above

GOTO computerhung

In PowerShell you create functions and then call the function at that point in the script instead of saying GOTO

Functions once defined stay in memory and behave like an actual command, so you just run it at that point in your script but define it prior to

ECHO text to user

$datetime = [datetime]::now
“$datetime $computer is starting up. . . please wait a minute. . .”

Query a service # tragically not as powerful as SC

Get-Service RpcSs

look at using restart-computer with the -wait parameter - - that makes the process wait until the machine has rebooted.
If you add the -For parameter as well you can tell the reboot process to check for specifics

Hi Vern,
I’m trying to get back to this, but if you can help me I would appreciate it. I did try to work on this using the equivalent commands, but I’m not making any progress.

hi,

you can start with this?

$servers = @("server1","server2") $Body = "server was rebooted at" $mailParam = @{ To = "Dude@contoso.com" Subject = "Server rebooted" From = "noreply@contoso.com" smtpserver = "mail.contoso.com" } $servers | foreach { # Wait 60 seconds maximum for server to complete the reboot Restart-Computer -ComputerName $_ -Wait -for WinRM -Timeout 60 if (get-service spooler -ComputerName $_).status -eq "running" { # good to continue, send email? $Mailbody = "$_ $body $([datatime]::now)" Send-MailMessage @mailParam -Body $mailbody } else { Write-error "Bail out, server is not up" break # this will exit the foreach loop } }

You could even throw in some test-connection statements etc

Cheers

Tore