formating a hash table report

by suave9987 at 2012-08-27 14:19:19

Hello All!

I have a Hash table with keys and values that correspond to names of an Application ID and a file name resepctively. I want to display a report so that when a Application ID is not found in a list of Master Application IDs then it will be sent to an out-file and the report should show this:

-----------------------------------------------------------------------------------------------------------
From ‘AID List - Third party .txt’ AID listed are NOT found in the NSM_All_AID’s.txt result
A0000000030000
A00000000317
A0000000035351

Instead right now it is repeating this over and over again:

-----------------------------------------------------------------------------------------------------------
From ‘AID List - Third party .txt’ AID listed are NOT found in the NSM_All_AID’s.txt result
A0000000030000
A00000000317
A0000000035351
-----------------------------------------------------------------------------------------------------------
From ‘AID List - Third party .txt’ AID listed are NOT found in the NSM_All_AID’s.txt result
A0000000030000
A00000000317
A0000000035351
-----------------------------------------------------------------------------------------------------------
From ‘AID List - Third party .txt’ AID listed are NOT found in the NSM_All_AID’s.txt result
A0000000030000
A00000000317
A0000000035351
…etc…

In the Hash table the key looks like so: A0000000030000 or some number in this format.
The Value is a file name something like this: AID List - Third party .txt

Here is the scirpt I am running to generate the report:


$pathOfG = "C:\Users\user\Desktop\Project1-Phase 2"
$getAIDFolder = "C:\Users\user\Desktop\Project1-Phase 2\AID"

$linespace = ""
$nextComp = "In complement, the following AIDs are present in some AID Lists and missing from the Get Status result"
$space = ""
$TOTcomp2 = "$linespace","$linespace","$nextComp", "$space"
$TOTcomp2
cd $getAIDFolder
$ht = @{}

$ListFiles = gci "$getAIDFolder\Extracted AID*.txt" | select -expand fullname
$ListFileNames = $Listfiles | foreach {$_ | Split-Path -Leaf}
#get all the App ID’s from the single line files
gci "$pathOfG\Extracted AID.txt" |
get-content |
foreach {
#Create a hash table entry for each App ID. Set the value to a collection of all the listfile names
$ht[$] = {$ListFileNames}.invoke()
}

Foreach ($ListFile in $ListFiles){
#Save the filename for this list file
$Filename = $ListFile | Split-Path -Leaf
#Read the App IDs from this list file
Get-Content $ListFile |
foreach{
$lines = "-----------------------------------------------------------------------------------------------------------"
$message = "From ‘$fileName’ AID listed are NOT found in the $nameOnly1 result"
$resultString = "$lines","$message"
if ($ht[$
]){
#If an App ID is found, remove this listfile from the collection of listfile names in the hash table for that key.
$ht[$].remove($Filename) | out-null
}
#Message if it can’t find that App ID in the hash table
else{

"$lines rn$message rn"
$ht.keys | sort |? {$ht[$
] -contains $Filename}}
}
}



The lines where this is displayed:

foreach{
$lines = "-----------------------------------------------------------------------------------------------------------"
$message = "From ‘$fileName’ AID listed are NOT found in the $nameOnly1 result"
$resultString = "$lines","$message"
if ($ht[$]){
#If an App ID is found, remove this listfile from the collection of listfile names in the hash table for that key.
$ht[$
].remove($Filename) | out-null
}
#Message if it can’t find that App ID in the hash table
else{

"$lines rn$message rn"
$ht.keys | sort |? {$ht[$] -contains $Filename}}


will do the comparing and will generate the report output.

I am just having trouble because it keeps repeating in the above format.

Any suggestions?

Thanks!
by DonJ at 2012-08-27 14:25:07
It’s a little tough to follow all of this in my head - so I’ll tell you what I’d do to troubleshoot this. Every single time you put a value in a variable, output that variable using Write-Verbose.

Write-Verbose "$filename now contains $filename&quot;<br><br>At the top of your script&#058;<br><br>$VerbosePreference = 'Continue'<br><br>And then run it again. I suspect the output will tell you exactly where you've got the logic wrong. Bottom line, I think you've probably got your ForEach loops nested incorrectly, but I'm having trouble seeing it without that verbose output.</blockquote>by suave9987 at 2012-08-27 14:41:38<blockquote>Ok I ran this:<br><br><code><br>$VerbosePreference = 'Continue'<br><br><br><br>$pathOfG = &quot;C&#58;\Users\user\Desktop\Project1-Phase 2&quot;<br>$getAIDFolder = &quot;C&#58;\Users\user\Desktop\Project1-Phase 2\AID&quot;<br><br>$linespace = &quot;*****************************************************************************************************&quot;<br>$nextComp = &quot;In complement, the following AIDs are present in some AID Lists and missing from the Get Status result&quot;<br>$space = &quot;&quot;<br>$TOTcomp2 = &quot;$linespace&quot;,&quot;$linespace&quot;,&quot;$nextComp&quot;, &quot;$space&quot;<br>$TOTcomp2<br>cd $getAIDFolder<br>$ht = @{}<br><br>$ListFiles = gci &quot;$getAIDFolder\Extracted AID\*&#46;txt&quot; | select -expand fullname<br>$ListFileNames = $Listfiles | foreach {$_ | Split-Path -Leaf}<br>#get all the App ID's from the single line files<br>gci &quot;$pathOfG\Extracted AID&#46;txt&quot; |<br> get-content | <br> foreach {<br> #Create a hash table entry for each App ID&#46; Set the value to a collection of all the listfile names<br> $ht&#91;$_&#93; = {$ListFileNames}&#46;invoke()<br> }<br> <br> Foreach ($ListFile in $ListFiles){<br> #Save the filename for this list file<br> $Filename = $ListFile | Split-Path -Leaf<br> Write-Verbose &quot;$filename now contains $filename"
#Read the App IDs from this list file
Get-Content $ListFile |
foreach{
$lines = "-----------------------------------------------------------------------------------------------------------"
$message = "From ‘$fileName’ AID listed are NOT found in the $nameOnly1 result"
$resultString = "$lines","$message"
if ($ht[$
]){
#If an App ID is found, remove this listfile from the collection of listfile names in the hash table for that key.
$ht[$].remove($Filename) | out-null
}
#Message if it can’t find that App ID in the hash table
else{
$ht[$
]
$lines
"$lines rn$message rn"
$ht.keys | sort |? {$ht[$_] -contains $Filename}}
}
}



and it does seem like what you are saying is happening. the out put looks like this:

VERBOSE: $filename now contains AID List - Third party .txt
-----------------------------------------------------------------------------------------------------
From ‘AID List - Third party .txt’ AID listed are NOT found in the result

A0000000030000
A00000000317
A0000000035351
-----------------------------------------------------------------------------------------------------
From ‘AID List - Third party .txt’ AID listed are NOT found in the result

A0000000030000
A00000000317
A0000000035351
…etc for 14 times then we see the next verbose.
VERBOSE: $filename now contains AID List refrence Product.txt
---------------------------------------------------------------------------------------
From ‘AID List refrence Product.txt’ AID listed are NOT found in the result

A00000000317
A0000000035574696C
A0000000090002FFF1F41089A0000140
---------------------------------------------------------------------------------------
From ‘AID List refrence Product.txt’ AID listed are NOT found in the result

A00000000317
A0000000035574696C
A0000000090002FFF1F41089A0000140
etc…for 51 more times then the final verbose is stated:
VERBOSE: $filename now contains AIDList1.txt
----------------------------------------------------------------------------------------------------
From ‘AIDList1.txt’ AID listed are NOT found in the result

A00000000317
A0000000035574696C
A0000000090002FFF1F41089A0000140
A0000000090002FFF1F41089A0000250
A0000000090002FFF1F41089A0000740
A0000000090002FFF1F41089A0000810
A0000000090005FFFFFFFF8911010000
etc…and it shows this one twice.

So yes I think my loops are off now.

Did I put the verbose in the correct areas?

Thanks for your help and any more will be greatly appreciated!
by DonJ at 2012-08-27 14:52:53
Yeah, that helps a bit. So… what output would you expect to see, here? If it were working correctly?
by suave9987 at 2012-08-27 16:09:50
I would expect to see the same output but no repeats the way that I am seeing things. I wouldn’t want a repeat 51 times like above and would just simply want every block to be displayed once. I am having issues with the logic behind how you say my for loops are reacting. I tried other ways but still all roads lead to this one…

I appreciate your help on this!
by DonJ at 2012-08-27 17:52:34
So, that means you’ve got a variable - something which contains your output - that isn’t getting blanked out, but instead is just being appended to, each time through your loop. Again - I can’t actually run your code since I don’t have the input files (and I’m a bit nervy at running code from forums), but that’s likely the problem.

You could add a lot more Write-Verbose. I literally put one after every single time I change a variable. Any variable. All variables. Don’t pick and choose. What you need to do is follow the logic of your script, line by line, and extremely verbose output will help you "see" what your script is doing internally.
by poshoholic at 2012-08-27 18:34:27
Just adding something I saw when I looked at this briefly this morning but didn’t have time to reply. Based on a quick scan I think the issue is in how you process the content when you do this:

[script=powershell]#Read the App IDs from this list file
Get-Content $ListFile |
foreach{[/script]

Get-Content reads the entire contents of a file, returning one line at a time. Each line is then sent to the ForEach-Object script block (where you use the foreach alias). Then those lines are processed, and you have the logic that outputs the information you’re seeing repeatedly if the application id isn’t found in the hash table. It seems that this happens 14 times in ‘AID List - Third Party.txt’, 51 times in ‘AID List refrence Product.txt’, and so on, which is why you’re seeing those counts. So the looping logic seems to be your problem here. If you don’t want it to output 14 times, then you need to change how you are processing the contents of those files.
by DonJ at 2012-08-28 06:24:29
Great catch, Kirk - I blipped over the pipe at the end of that.
by suave9987 at 2012-08-28 08:17:13
Thank you everyone. I will go back and try to fix the logic of my loops. I do admit that I became lost along the way because of the personal understanding towards hash tables vs collections seemed to be slowing me down. I see what you are saying Kirk. This makes perfect sense on why I was getting the repeats.

Don, once again I need to think like the shell and it seems that my short absence from it has cost me to keep my concetration. I will definitely be using verbose now to see the variable changes.

Thanks everyone.
by suave9987 at 2012-08-30 08:12:17
Ok everyone so I have made some changes and believe I have a way to format the way I want except I am having some conditional logic problems. I have written this:

Function COMPAREreport2{
$linespace = "
"
$nextComp = "In complement, the following AIDs are present in some AID Lists and missing from the Get Status result"
$space = ""
$TOTcomp2 = "$linespace","$linespace","$nextComp", "$space"
$TOTcomp2|out-file "$pathOfG\AID Integrity Check Report – $nameOnly1" -append
cd $getAIDFolder
$ht = @{}
$ListFiles = gci "$getAIDFolder\Extracted AID*.txt" | select -expand fullname
$ListFileNames = $Listfiles | foreach {$_ | Split-Path -Leaf}
#get all the App ID’s from the single line files
gci "$pathOfG\Extracted AID’s.txt" |
get-content |
foreach {
#Create a hash table entry for each App ID. Set the value to a collection of all the listfile names
$ht[$] = {$ListFileNames}.invoke()
}
$current = ""
Foreach ($ListFile in $ListFiles){
#Save the filename for this list file
$Filename = $ListFile | Split-Path -Leaf
#Read the App IDs from this list file
Get-Content $ListFile |
foreach {
$next = $Filename
$lines = "-----------------------------------------------------------------------------------------------------------"
$message = "From ‘$fileName’ AID listed are NOT found in the $nameOnly1 result"
$resultString = "$lines","$message"

if ($ht[$
]){
#If an App ID is found, remove this listfile from the collection of listfile names in the hash table for that key.
$ht[$].remove($Filename) | out-null
}

#Message if it can’t find that App ID in the hash table
else{
if($current -eq $next){
write-verbose "we are now looking at $Filename inside the nexted IF"
$

}
else{
write-verbose "we are now looking at $Filename inside the nexted ELSE inside ELSE"
"$lines rn$message rn$_ "
}

}
$current = $next
}|out-file "$pathOfG\AID Integrity Check Report – $nameOnly1" -append
Write-verbose "out of the if statments"
}
}
COMPAREreport2



I have the new variables $next and $current that follow the $Filename 's. If my $current = $next then only the AppID will be displayed. Else it will just display the normal message with the name of the file and that it is not listed in the GetStatusFile like so:

-----------------------------------------------------------------------------------------------------------
From ‘AID List - Third party .txt’ AID listed are NOT found in the GetStatusfile.txt result
A000000045463200


But in the first case if the $current = $next then only the AppID should be tacked on like so:

-----------------------------------------------------------------------------------------------------------
From ‘AID List - Third party .txt’ AID listed are NOT found in the GetStatusfile.txt result
A000000045463200
A000000046372299
A000000028394422

I am having a little bit of issues with this because it never goes into the sub-else statement. So it jumps right away into the sub-IF statment and only displays the AppID. I dont understand this because the first time around the $current cannot possibly be equal to the $next but apparently it says it is. I cannot get the logic of it down.

So I know I have been taxing on this but please let me know if you can help on what I am doing wrong.

Any help would be greatly appreciated!!


Thanks so much again!
by suave9987 at 2012-08-30 08:33:55
I have found the bug and fixed the problem I believe with the help of Write-Verbose! Thank you DONNN!

This is what I did wrong:

foreach {


$lines = "-----------------------------------------------------------------------------------------------------------"
$message = "From ‘$fileName’ AID listed are NOT found in the $nameOnly1 result"
$resultString = "$lines","$message"

if ($ht[$]){
#If an App ID is found, remove this listfile from the collection of listfile names in the hash table for that key.
$ht[$
].remove($Filename) | out-null
}

#Message if it can’t find that App ID in the hash table
else{$next = $Filename
write-verbose "current is $current and next is now $next"
if($current -eq $next){
write-verbose "we are now looking at $Filename inside the nexted IF"
$
}
else{
write-verbose "we are now looking at $Filename inside the nexted ELSE inside ELSE"
"$lines rn$message rn$
"
}

}
$current = $next
Write-verbose "Current is now $current"
}|out-file "$pathOfG\AID Integrity Check Report – $nameOnly1" -append
Write-verbose "out of the if statments"


At the point where I originally set $next equal to $Filename this was out side of the IF-ELSE Statments. I wrote alot of Verbose remarks and I found that $next and $current are always the same then because at first I never went into the SUB-IF-ELSE statements. So I inserted the $next = $Filename like so:

if ($ht[$]){
#If an App ID is found, remove this listfile from the collection of listfile names in the hash table for that key.
$ht[$
].remove($Filename) | out-null
}

#Message if it can’t find that App ID in the hash table
else{$next = $Filename
write-verbose "current is $current and next is now $next"
if($current -eq $next){
write-verbose "we are now looking at $Filename inside the nexted IF"
$
}
else{
write-verbose "we are now looking at $Filename inside the nexted ELSE inside ELSE"
"$lines rn$message rn$
"
}

}
$current = $next


At now it works the way I want!

Thank you to Don because if I had not had the sense of a Wite-verbose command pounded into my head then I wouldn’t of ever found the problem!

Thanks to all for the help!