Search for text then return text inbetween braces

vegarjb · August 21, 2021, 5:09pm

Hello everyone,

I am trying to search for text in files and dynamically return the lines in the files based on the location of the found text. This is how I am finding the files with the string in it:

$found = Get-ChildItem -Path C:\Git\organization\organization-service-7 -Include . -Recurse | Select-String -Pattern locals

This is how I am getting the line number of the found text:

$filename, $linenumber, $searchword = $finds -split “:”, 3

I am having a really hard time figuring out the rest. Here is the logic and an example…

Search a folder and all subdirectories for
1. Certain file name (i.e *.log)
2. Certain text in the in scope files (i.e locals)
3. For this eaxmple the directory is c:\bin\day\logs
Every time the text locals is found in *.log files , search for the first occurance of { (open brace) starting on the line where locals was found.
Once { is found then find the closing } bracket. (Note: There could be other { and } encountered before the } you are looking for is found)
Once the proper } is found.
a. Output to a text file
1. File name
2. The text searched for
3. The line where the first { was found
4. All line inbetween the { and }
5. The line where the } (closing brace) was found

Example:
Folder : c:\bin\day\logs
Files found : downtime.log and uptime.log
Files are in c:\bin\day\logs
Search starts in c:\bin

Content of downtime.log:

locals
{
location_down = {
{country = $(current)
{state = $(state)
{city = $(city)
}
}
}
}

Content of uptime.log:

locals
{
location_up = {
{country = $(current)
{state = $(state)
{city = $(city)
}
}
}
}

text file output:

c:\bin\day\logs\downtime.log
locals
{
location_down = {
{country = $(current)
{state = $(state)
{city = $(city)
}
}
}
}
c:\bin\day\logs\uptime.log
locals
{
location_up = {
{country = $(current)
{state = $(state)
{city = $(city)
}
}
}
}

Any help would be appreciated,

Ray

Olaf · August 21, 2021, 7:31pm

Ramon,
Welcome to the forum.

Maybe we can find a solution togehter. The structure of the search pattern you’re looking for seems pretty regular. Sounds for me like “find the word “local” and the following 9 lines” …

This could be achieved with Select-String and the parameter -Context. Try to run the following line of code for one of your target files to see what I mean:

$Path = 'D:\sample\LogFile.log'
Select-String -Path $Path -Pattern locals -Context 0, 9 |
ForEach-Object {
    [PSCustomObject]@{
        FileName    = $_.Filename
        LineNumber  = $_.LineNumber
        Path        = $_.Path
        Match       = $_.Matches[0]
        PostContext = $_.Context.PostContext
    }
}

Now you have all you wanted - The file name, the path, the line number, the actual match and the following 9 lines. Is it that what you was looking for?

Of course you can provide Select-String a path with a wildcard in it to catch all files of a certain type in one directory. To search even subdirectories you would need to use a loop because Select-String does not recurse.

Regardless of all that could you please format your code or sample data as code using the preformatted text button ( </> ) next time. That’l make it much easier to copy your code and sample data and to play around with it to be able to reproduce your challenge.

Thanks in advance.

vegarjb · August 22, 2021, 3:16am

Thanks for the help you are giving me some ideas. The problem is not getting 9 lines of code, it is determining the n lines of code.

For example

Find locals on line 40 ( locals has 6 characters), so start to search for { on line forty position 7
not found on 40 but found on 41. Continue searching for }. The next } is on line 46 but that belongs to line 45 {. This keeps going until line 50 where the closing brace is found. So lines 41 - 50 would be outputted

40 locals 
41 {
42   location_down = { 
43       {country = $(current)
44         {state = $(state)
45           {city = $(city)
46            }
47          }
48        } 
49     }   
50  }


Taking it to the extreme... if all of this was put on 1 line, 1 like would be returned becaue the opening and closing braces are on the same line

locals {location_down = { {country = $(current) {state = $(state {city = $(city) } } } } }

Thanks,

Ray

krzydoug · August 22, 2021, 6:11am

Hi Ray,

I must say this is confusing with some contradictions. First, when you post the sample files, you show the text with no indentation. However your last reply shows indentation. Your sample files also only contain the exact text your after, I’ll assume this file has other text you’re trying to pull this out of?

Next, when you say “The line where the first { was found” it really makes me think you want the line number where it was found. It appears Olaf may have thought the same thing. However, your sample output text file just shows the file name followed by the entire pattern you’re trying to match. Perhaps we can rephrase bullet point 4 to the following.

4. Output the file name followed by the entire matched pattern, all text/lines from the word locals to the appropriate closing curly brace.

Also, your original post (please format code as code with the </> button) had a missing closing curly brace. You did correct this in your last reply.
Ok, so I added some other text to the sample files for this demonstration.

Push-Location $env:temp

Set-Content -Path .\uptime.log -Value @'

some random text
some random textsome random text
some random textsome random textsome random text

locals
{
location_up = {
{country = $(current)
{state = $(state)
{city = $(city)
}
}
}
}
}
some random text
some random textsome random text
some random textsome random textsome random text
'@

Set-Content -Path .\downtime.log -Value @'
some random text
some random textsome random text
some random textsome random textsome random text

locals
{
  location_down = {
    {country = $(current)
      {state = $(state)
        {city = $(city)
        }
      }
    }
  }
}


some random text
some random textsome random text
some random textsome random textsome random text
locals
{
  location_down = {
    {country = $(current)
      {state = $(state)
        {city = $(city)
        }
      }
    }
  }
}
some random text
some random textsome random text
some random textsome random textsome random text
'@

And now that I have a couple of sample files, here is one possible way to achieve (what I believe is) your goal.

Set-Content -Path .\output.txt -Value $(
    foreach($logfile in Get-ChildItem -Filter *.log) {
        $output = $false
        $braces = 0
    
        switch -Regex (Get-Content -Path $logfile.FullName){
            'locals' {
                $logfile.FullName
                $output = $true
                $_
            }

            '{' {
                if($output){
                    $braces++
                    $_
                }
            }

            '}' {
                if($output){
                    $braces--
                    $_
                }
            
                if($braces -eq 0){
                    $output = $false
                }
            }

            default {if($output){$_}}
        }
    }
) -PassThru

Pop-Location

Now if we could always depend on the final closing curly brace to be at the beginning of a line and the rest will have one or more spaces before them, we could use something as simple as this.

Push-Location $env:temp

[regex]$pattern = '(?s)locals.+?{.+?(?<! )}'

Get-ChildItem -Filter *.log | ForEach-Object {
    foreach($match in $pattern.Matches((Get-Content $_.FullName -Raw))){
        $_.FullName
        $match.Value
    }
} | Set-Content .\output.txt

Pop-Location

vegarjb · August 22, 2021, 9:46pm

krzydoug,

Thank you very much. Sorry for not formatting the first post. I just joined and it was my first post. I was asked to format it in my reply. Sorry for the confusion. It is always easy to understand something if it is yours but sometime difficult to verbalize. (sp?)

Let me try harder:

Step 1 - Search for all files in a folder and it subfolders that contains the word “locals”.

Step 2 - In every file that the word “locals” is found, start at the word “locals” and search for the an open brace.

Step 3 - Starting right after the opening brace , continue searching to the end of line and every line after until a closing brace is found.

Step 4 - Output the filename, textsearched for (in this case “locals”) and the text found between the braces.

Example input:

locals 
{
  location_down = { 
      {country = $(current)
         {state = $(state)
             {city = $(city)
            }
        }
    }
}

Example output:

Filename: main.txt
Word to search for : locals

{
  location_down = { 
      {country = $(current)
         {state = $(state)
             {city = $(city)
            }
        }
    }
}

Note: Sometimes the word “locals” and the opening brace may appear on the same line.

I have not had time to go through your code yet, but I will and respond.

Thanks,

Ray

krzydoug · August 22, 2021, 11:37pm

Hi Ray,

Yeah definitely try the code out, it should be very close to what you’re after. Also, don’t want to nitpick but again you say “output the text found between the braces” and then in your example output show the braces in addition.

krzydoug · August 22, 2021, 11:47pm

You may also try these.

Set-Content -Path .\output.txt -Value $(
    foreach($logfile in Get-ChildItem -Filter *.log) {
        $output = $false
        $braces = 0
    
        switch -Regex (Get-Content -Path $logfile.FullName){
            'locals' {
                "Filename: $($logfile.FullName)"
                "Word to search for : locals"
                ""
                $output = $true
            }

            '{' {
                if($output){
                    $braces++
                    $_
                }
            }

            '}' {
                if($output){
                    $braces--
                    $_
                }
            
                if($braces -eq 0){
                    $output = $false
                    ""
                }
            }

            default {if($output){$_}}
        }
    }
) -PassThru

Or

[regex]$pattern = '(?s)locals(.+?{.+?(?<! )})'

Get-ChildItem -Filter *.log | ForEach-Object {
    foreach($match in $pattern.Matches((Get-Content $_.FullName -Raw))){
        "Filename: $($_.FullName)"
        ""
        "Word to search for : locals"
        $match.Value -replace 'locals'
        ""
    }
} | Set-Content .\output.txt -Passthru

Olaf · August 23, 2021, 12:14am

Here is some great stuff to read about the regex technique relevant for your task:

dicey · August 23, 2021, 8:37am

@vegarjb - this is a bit rough, but it’s gotten late after a long day of restoring a guest bathroom. If it meets your original post, it might provide a framework to continue development.

Process

I created three source files and set each with their own locals array. This was to test the regex against different patterns and false positives. I recommend establishing standards for your procedure and streamlining the process where you may.

Notes

This is demo code. It is a Proof of Concept and does not represent stable production-quality scripting.
The one-liner locals array has a SearchStart of 0. This needs to be handled to prevent misreporting.

Demo Code

using namespace System.Collections;

Class TextBody
{
    hidden [string] $FileName;
    hidden [int] $GroupID;
    hidden [string] $LineContent;
    hidden [int] $LineID;
}

Class TextHeader : TextBody
{
    [string] $Body;
    [string] $FileName;
    [int] $SearchStart;
    [int] $SearchEnd;
    [string] $SearchTerm;
}

[string]$searchTerm = 'locals';
$files = (Get-ChildItem "A:\bin\day\logs").Where({$_.Extension -eq '.log'});
$locals = [ArrayList]::new();
[int]$count = 1;

foreach($file in $files)
{
    if((Get-Content $file.FullName -Raw) -match "\b$($searchTerm)\b(?=\s*[{])")
    {
        $x = (Get-Content $file.FullName);
        if($x.IndexOf( $x.Where({$_ -match "\b$($searchTerm)\b\s*[{]"})) -eq 1)
        {
            [int]$idx = $x.IndexOf( $x.Where({$_ -match "\b$($searchTerm)\b"}))
        }
        else
        {
            [int]$idx = $x.IndexOf( $x.Where({$_ -match "\b$($searchTerm)\b\s*[{]"}))
        }
   
        for($i = $idx ; $i -le $x.IndexOf($x.Where({$_ -match "}"})[-1]); $i++)
        {       
            $y = [TextBody]::new();
            $y.FileName = $file.FullName;
            $y.GroupID = $count;
            $y.LineContent = $x[$i];
            $y.LineID = $i;
            [void]$locals.Add($y);
        }
    }
    $count++;
}

$localsOut = [ArrayList]::new();
for ($j = 1; $j -le ($locals|select -Unique GroupID).Count; $j++)
{
    $u = $locals.Where({$_.GroupID -eq $j})
    $z = [TextHeader]::new();
    $z.Body = $u.LineContent | Out-String;
    $z.FileName = $u.FileName|select -First 1;
    $z.SearchStart = $u.LineID[2];
    $z.SearchEnd = $u.LineID[-1];
    $z.SearchTerm = $searchTerm;
    [void]$localsOut.Add($z);
}
$localsOut | Select-Object FileName,SearchTerm,SearchStart,SearchEnd,Body | Out-File A:\bin\output.txt -Force;

Results

`$localsOut`

Output.txt

The file output can be cleaned with better formatting. The red is lint running in my VS Code.

Topic		Replies	Views
Select-String by pattern then if found actions PowerShell Help	23	957	May 16, 2024
Help searching through text file PowerShell Help	8	326	May 16, 2024
Read text file and find match PowerShell Help	16	912	May 16, 2024
Search for string in a text file recursively PowerShell Help	5	293	May 16, 2024
Search in file PowerShell Help	10	200	May 16, 2024

Search for text then return text inbetween braces

Process

Notes

Demo Code

Results

`$localsOut`

Output.txt

Uptime.log

Downtime.log

Downtime_002.Log

Search for text then return text inbetween braces

Process

Notes

Demo Code

Results

$localsOut

Output.txt

Uptime.log

Downtime.log

Downtime_002.Log

Related topics

`$localsOut`