by Dwayne Dibbley at 2012-10-06 03:20:36
Is it possible to have a rolling buffer eg 5 values, and every time a new value is added the oldest is dropped out?by jonhtyler at 2012-10-06 04:56:07
The idea is to read a log file with a compass heading every line, and smooth out the data by reading for example 5 values into the buffer then reading the average from the buffer?
Thanks
There is nothing native in Powershell or in .NET that would do this natively. I believe you would have to roll your own class to handle the logic of removing the oldest entry when a new one gets queued. If you search for "Circular Buffer in .NET" you can find some examples that might get you on your way. Since Powershell is based on .NET, you can fairly easily create the class and have Powershell implement it.by MattG at 2012-10-06 07:27:35
I think that a queue is what you’re looking for. The following code should get you started. I hope this helps.by Dwayne Dibbley at 2012-10-07 02:26:53[Int] $Bearings = Get-Content .\bearings.txt
$Queue = New-Object System.Collections.Queue(5)
$BufferLength = 5
$RunningTotal = 0
$Count = 0
foreach ($Bearing in $Bearings)
{
$RunningTotal += $Bearing
if ($Queue.Count -eq $BufferLength)
{
$Queue.Dequeue() | Out-Null
$Queue.Enqueue($Bearing)
}
else
{
$Queue.Enqueue($Bearing)
}
$Count++
if ($Count % $BufferLength -eq 0)
{
Write-Host "Average: $($RunningTotal/$BufferLength)"
$RunningTotal = 0
}
}
My bearings.txt consisted of the following values:
0
20
18
322
187
67
98
351
4
93
35
211
Thanks,by MattG at 2012-10-07 07:34:17
If i was reading a standard NMEA log like:$GPRMC,090353.021,A,5137.4042,N,00246.6906,W,029.3,143.7,300912,A78
$GPGGA,090353.071,5137.4038,N,00246.6901,W,1,07,1.3,30.5,M,51.4,M,000076
$GPRMC,090353.071,A,5137.4038,N,00246.6901,W,029.6,143.5,300912,A70
$GPGGA,090353.121,5137.4034,N,00246.6896,W,1,07,1.3,30.6,M,51.4,M,000072
$GPRMC,090353.121,A,5137.4034,N,00246.6896,W,029.8,143.3,300912,A7F
$GPGGA,090353.171,5137.4030,N,00246.6892,W,1,07,1.3,30.6,M,51.4,M,000077
$GPRMC,090353.171,A,5137.4030,N,00246.6892,W,030.1,143.2,300912,A7A
$GPGGA,090353.221,5137.4027,N,00246.6887,W,1,07,1.3,30.6,M,51.4,M,000073
$GPRMC,090353.221,A,5137.4027,N,00246.6887,W,030.4,143.0,300912,A79
$GPGGA,090353.271,5137.4023,N,00246.6882,W,1,07,1.3,30.6,M,51.4,M,000077
$GPRMC,090353.271,A,5137.4023,N,00246.6882,W,030.7,142.9,300912,A76
$GPGGA,090353.321,5137.4019,N,00246.6877,W,1,07,1.3,30.7,M,51.4,M,000071
$GPRMC,090353.321,A,5137.4019,N,00246.6877,W,031.0,142.8,300912,A76
$GPGGA,090353.371,5137.4015,N,00246.6872,W,1,07,1.3,30.7,M,51.4,M,00007D
$GPRMC,090353.371,A,5137.4015,N,00246.6872,W,031.3,142.7,300912,A76
$GPGGA,090353.421,5137.4011,N,00246.6868,W,1,07,1.3,30.7,M,51.4,M,000070
$GPRMC,090353.421,A,5137.4011,N,00246.6868,W,031.5,142.7,300912,A7D
$GPGGA,090353.471,5137.4007,N,00246.6863,W,1,07,1.3,30.8,M,51.4,M,000076
$GPRMC,090353.471,A,5137.4007,N,00246.6863,W,031.8,142.6,300912,A78
$GPGGA,090353.521,5137.4002,N,00246.6858,W,1,07,1.3,30.8,M,51.4,M,00007F
$GPRMC,090353.521,A,5137.4002,N,00246.6858,W,032.1,142.6,300912,A7B
$GPGGA,090353.571,5137.3998,N,00246.6853,W,1,07,1.3,30.8,M,51.4,M,00007C
$GPRMC,090353.571,A,5137.3998,N,00246.6853,W,032.4,142.6,300912,A7D
$GPGGA,090353.621,5137.3994,N,00246.6848,W,1,07,1.3,30.9,M,51.4,M,00007D
$GPRMC,090353.621,A,5137.3994,N,00246.6848,W,032.6,142.6,300912,A7F
$GPGGA,090353.671,5137.3990,N,00246.6843,W,1,07,1.3,30.9,M,51.4,M,000077
$GPRMC,090353.671,A,5137.3990,N,00246.6843,W,032.9,142.6,300912,A7A
$GPGGA,090353.721,5137.3986,N,00246.6838,W,1,07,1.3,30.9,M,51.4,M,000078
$GPRMC,090353.721,A,5137.3986,N,00246.6838,W,033.2,142.7,300912,A7E
$GPGGA,090353.771,5137.3981,N,00246.6833,W,1,07,1.3,31.0,M,51.4,M,000079
$GPRMC,090353.771,A,5137.3981,N,00246.6833,W,033.4,142.7,300912,A71
$GPGGA,090353.821,5137.3977,N,00246.6828,W,1,07,1.3,31.0,M,51.4,M,000070
$GPRMC,090353.821,A,5137.3977,N,00246.6828,W,033.7,142.7,300912,A7B
$GPGGA,090353.871,5137.3973,N,00246.6823,W,1,07,1.3,31.0,M,51.4,M,00007A
$GPRMC,090353.871,A,5137.3973,N,00246.6823,W,034.0,142.7,300912,A71
$GPGGA,090353.921,5137.3968,N,00246.6818,W,1,07,1.3,31.1,M,51.4,M,00007D
and using your stack code to determine the course taken ( if last bearing was 10 and current bearing 12 then that would be a 2 degree turn to the right? ) does the following look correct?$file = new-object System.IO.StreamWriter("C:\test.txt")
$lasttimestamp = [datetime]::ParseExact("000000.000","HHmmss.fff",$null)
$Queue = New-Object System.Collections.Queue(5)
$BufferLength = 5
$RunningTotal = 0
$Count = 0
$SourceFile = "C:\LOG.txt"
$data = get-content $SourceFile
foreach ( $line in $data )
{
if ($line -match ".GPRMC.")
{
$splitline = $line.split(",")
$timestamp = [datetime]],"HHmmss.fff",$null)
$course = [decimal]$splitline[8]
$RunningTotal += $course
if ($Queue.Count -eq $BufferLength)
{
$Queue.Dequeue() | Out-Null
$Queue.Enqueue($course)
}
else
{
$Queue.Enqueue($course)
}
$Count++
if ($Count % $BufferLength -eq 0)
{
$heading = $($RunningTotal/$BufferLength) - $lastbearing
$file.WriteLine("$timestamp heading: $heading")
$lastbearing = $($RunningTotal/$BufferLength)
$RunningTotal = 0
}
}
}
$file.Close()
Without looking into your code too much, I do have some suggestions that you might find useful. Since the NMEA log is simply a CSV file, you could use ConvertFrom-Csv to perform some really interesting analytics on your data. For example, observe the following:by Dwayne Dibbley at 2012-10-07 09:48:37$Data = Get-Content .\log.txt
$GPRMC = $Data -match '^$GPRMC.+'
$Headers = 'MinSentence','FixTime','Active','LatVal','LatDir','LongVal','LongDir','GndSpeed','TrackAngle','Date','MagVarVal','MagVarDir','Chksum'
$Table = ConvertFrom-Csv -InputObject $GPRMC -Header $Headers
$Table | Format-Table -Property * -AutoSize
This will output the following:MinSentence FixTime Active LatVal LatDir LongVal LongDir GndSpeed TrackAngle Date MagVarVal MagVarDir Chksum
----------- ------- ------ ------ ------ ------- ------- -------- ---------- ---- --------- --------- ------
$GPRMC 090353.021 A 5137.4042 N 00246.6906 W 029.3 143.7 300912 A78
$GPRMC 090353.071 A 5137.4038 N 00246.6901 W 029.6 143.5 300912 A70
$GPRMC 090353.121 A 5137.4034 N 00246.6896 W 029.8 143.3 300912 A7F
$GPRMC 090353.171 A 5137.4030 N 00246.6892 W 030.1 143.2 300912 A7A
$GPRMC 090353.221 A 5137.4027 N 00246.6887 W 030.4 143.0 300912 A79
$GPRMC 090353.271 A 5137.4023 N 00246.6882 W 030.7 142.9 300912 A76
$GPRMC 090353.321 A 5137.4019 N 00246.6877 W 031.0 142.8 300912 A76
$GPRMC 090353.371 A 5137.4015 N 00246.6872 W 031.3 142.7 300912 A76
$GPRMC 090353.421 A 5137.4011 N 00246.6868 W 031.5 142.7 300912 A7D
$GPRMC 090353.471 A 5137.4007 N 00246.6863 W 031.8 142.6 300912 A78
$GPRMC 090353.521 A 5137.4002 N 00246.6858 W 032.1 142.6 300912 A7B
$GPRMC 090353.571 A 5137.3998 N 00246.6853 W 032.4 142.6 300912 A7D
$GPRMC 090353.621 A 5137.3994 N 00246.6848 W 032.6 142.6 300912 A7F
$GPRMC 090353.671 A 5137.3990 N 00246.6843 W 032.9 142.6 300912 A7A
$GPRMC 090353.721 A 5137.3986 N 00246.6838 W 033.2 142.7 300912 A7E
$GPRMC 090353.771 A 5137.3981 N 00246.6833 W 033.4 142.7 300912 A71
$GPRMC 090353.821 A 5137.3977 N 00246.6828 W 033.7 142.7 300912 A7B
$GPRMC 090353.871 A 5137.3973 N 00246.6823 W 034.0 142.7 300912 A*71
Also, I was probably premature to suggest using queues. You could simply use a five-element array and use the modulo operator to rotate through the array as can be seen here:$BufferSize = 5
$LatArray = New-Object Single($BufferSize)
$i = 0
foreach ($Waypoint in $Table)
{
$LatArray[$i % $BufferSize] = $Waypoint.LatVal
$i++
}
Without performing a full code review of your work, I think you have a good starting point to do some really interesting NMEA parsing. Have fun!
Thanks again, would i be correct with the following using the measure object on the array to read the average?by MattG at 2012-10-07 10:22:12foreach ($Waypoint in $Table)
{
$LatArray[$i % $BufferSize] = $Waypoint.TrackAngle
$Output = $LatArray | measure-object -average
$Output.Average
$i++
}
Absolutely! That would be a great use of Measure-Object.by Dwayne Dibbley at 2012-10-08 02:56:33
is there a way to tweak the array to account for the shift from example:358 degrees to 0 degrees? as this triggers the average to drop by 70 when i should only drop by 1 or 2?by MattG at 2012-10-08 07:11:10
example showing the array (CourseAverage) and real course (course) and the Diff (HeadingDiff) between last CourseAverage and Current CourseAverageUTCTime:155837.791 CourseAverage:352 Course:356 Speed:31 HeadingDiff:2
UTCTime:155837.891 CourseAverage:354 Course:358 Speed:32 HeadingDiff:2
UTCTime:155837.991 CourseAverage:284 Course:0 Speed:32 HeadingDiff:-70
UTCTime:155838.091 CourseAverage:214 Course:2 Speed:32 HeadingDiff:-70
UTCTime:155838.191 CourseAverage:144 Course:4 Speed:33 HeadingDiff:-70
UTCTime:155838.291 CourseAverage:74 Course:7 Speed:33 HeadingDiff:-70
UTCTime:155838.391 CourseAverage:4 Course:9 Speed:34 HeadingDiff:-70
UTCTime:155838.491 CourseAverage:7 Course:11 Speed:34 HeadingDiff:3
UTCTime:155838.591 CourseAverage:8 Course:12 Speed:35 HeadingDiff:1
UTCTime:155838.691 CourseAverage:10 Course:12 Speed:35 HeadingDiff:2
UTCTime:155838.791 CourseAverage:11 Course:12 Speed:35 HeadingDiff:1
UTCTime:155838.891 CourseAverage:12 Course:12 Speed:36 HeadingDiff:1
UTCTime:155838.991 CourseAverage:12 Course:11 Speed:36 HeadingDiff:0
UTCTime:155839.091 CourseAverage:11 Course:10 Speed:36 HeadingDiff:-1
UTCTime:155839.191 CourseAverage:11 Course:9 Speed:36 HeadingDiff:0
UTCTime:155839.291 CourseAverage:10 Course:7 Speed:36 HeadingDiff:-1
UTCTime:155839.391 CourseAverage:9 Course:6 Speed:36 HeadingDiff:-1
UTCTime:155839.491 CourseAverage:7 Course:5 Speed:35 HeadingDiff:-2
UTCTime:155839.591 CourseAverage:6 Course:4 Speed:35 HeadingDiff:-1
UTCTime:155839.691 CourseAverage:5 Course:3 Speed:35 HeadingDiff:-1
UTCTime:155839.791 CourseAverage:4 Course:2 Speed:34 HeadingDiff:-1
UTCTime:155839.891 CourseAverage:3 Course:1 Speed:34 HeadingDiff:-1
UTCTime:155839.991 CourseAverage:2 Course:0 Speed:34 HeadingDiff:-1
UTCTime:155840.091 CourseAverage:73 Course:360 Speed:33 HeadingDiff:71
UTCTime:155840.191 CourseAverage:145 Course:359 Speed:32 HeadingDiff:72
UTCTime:155840.291 CourseAverage:216 Course:359 Speed:32 HeadingDiff:71
UTCTime:155840.391 CourseAverage:287 Course:358 Speed:31 HeadingDiff:71
UTCTime:155840.491 CourseAverage:359 Course:358 Speed:30 HeadingDiff:72
UTCTime:155840.591 CourseAverage:359 Course:358 Speed:29 HeadingDiff:0
UTCTime:155840.691 CourseAverage:358 Course:358 Speed:29 HeadingDiff:-1
UTCTime:155840.791 CourseAverage:358 Course:359 Speed:28 HeadingDiff:0
UTCTime:155840.891 CourseAverage:359 Course:360 Speed:27 HeadingDiff:1
UTCTime:155840.991 CourseAverage:287 Course:0 Speed:26 HeadingDiff:-72
UTCTime:155841.091 CourseAverage:216 Course:2 Speed:26 HeadingDiff:-71
UTCTime:155841.191 CourseAverage:145 Course:3 Speed:25 HeadingDiff:-71
UTCTime:155841.291 CourseAverage:74 Course:5 Speed:24 HeadingDiff:-71
UTCTime:155841.391 CourseAverage:3 Course:7 Speed:24 HeadingDiff:-71
UTCTime:155841.491 CourseAverage:5 Course:9 Speed:23 HeadingDiff:2
UTCTime:155841.591 CourseAverage:7 Course:12 Speed:23 HeadingDiff:2
UTCTime:155841.691 CourseAverage:9 Course:15 Speed:22 HeadingDiff:2
UTCTime:155841.791 CourseAverage:12 Course:18 Speed:22 HeadingDiff:3
UTCTime:155841.891 CourseAverage:15 Course:22 Speed:21 HeadingDiff:3
Thanks again
What you’re asking is no longer a PowerShell question but a general computer science question. I’ll take a stab at it though. ;Dby Dwayne Dibbley at 2012-10-08 07:47:52
You should ask yourself, how does your algorithm differentiate between say a heading difference of 2 versus a heading difference of -358? What you would need to do to make that determination is calculate the instantaneous rate of change in the course. If the rate of change from the previous samples is positive, then it is most likely that the heading diff is 2. Conversely, if the rate of change is negative, then the heading diff should be -358. This makes sense because while it is unlikely, you could have made a really quick, nearly 360’ turn versus a slight 2’ heading difference. Therefore you must infer the heading diff based upon the previous heading diffs.
Make sense?
ok to bring it back to powershell land if found the following excel formula that works but unsure how to powershellize it?by Dwayne Dibbley at 2012-10-08 08:28:11A1:A10 = Raw Data
B1: =IF(A1>180,A1-360,A1) <drag down to B10
C2: =IF(ABS(A1-A2)>180,1,"") <drag down to C10
D10: =IF(SUM(C1:C10)>0,IF(AVERAGE(B1:B10)<0,360+AVERAGE(B1:B10),AVERAGE(B1:B10)),AVERAGE(A1:A10))
excel result with some test data:
A B C
3 3
2 2
1 1
0 0 1
359 -1
358 -2
359 -1 1
0 0
1 1
2 2 0.5 < average bearing from A1:A10
Thanks again
if have the following so far but cant work out how to do the IF(ABS(A1-A2)>180,1,"") part as this requires a future number? A2. so i have currently just stuck in IF(ABS(A2-A1)>180,1,"") to show the part, maybe i could run a step behind somehow?by Dwayne Dibbley at 2012-10-09 06:58:30foreach ($Waypoint in $Table)
{
Switch($Waypoint.Course)
{
{($_ -gt 180) } { $LatArray[$i % $BufferSize] = $Waypoint.Course - 360}
default { $LatArray[$i % $BufferSize] = $Waypoint.Course }
}
if ([Math]::ABS($latArray[$i % $BufferSize] - $latArray[$i % $BufferSize - 1]) -gt 180)
{
$flag[$i % $BufferSize] = 1
}
$Output = $LatArray | measure-object -average
"Raw:$($Waypoint.Course) CourseAverage]$Output.Average) flag:$($flag[$i % $BufferSize])"
$i++
}
i think i have a working version now for the course that seems to work, might not be the cleanest way but im only a noob :$Data = Get-Content .\LOG10.txt
$GPRMC = $Data -match '^$GPRMC.+'
$Headers = 'ID','UTCTime','Status','Lat','NSInd','Long','EWInd','Speed','Course','UTCDate','MagVar','MagVarInd','Chksum'
$Table = ConvertFrom-Csv -InputObject $GPRMC -Header $Headers
$BufferSize = 5
$CourseArrayA = New-Object Single($BufferSize)
$CourseArrayB = New-Object Single($BufferSize)
$CourseArrayC = New-Object Single($BufferSize)
$i = 0
$flag = New-Object Single($BufferSize)
foreach ($Waypoint in $Table)
{
$CourseArrayA[$i % $BufferSize] = $Waypoint.Course
Switch($Waypoint.Course)
{
{($_ -gt 180) } { $CourseArrayB[$i % $BufferSize] = $Waypoint.Course - 360}
default { $CourseArrayB[$i % $BufferSize] = $Waypoint.Course }
}
if ($i % $BufferSize -ne 0)
{
if ([Math]::ABS($CourseArrayA[$i % $BufferSize] - $CourseArrayA[$i % $BufferSize - 1]) -gt 180)
{
$CourseArrayC[$i % $BufferSize] = 1
}
else
{
$CourseArrayC[$i % $BufferSize] = 0
}
}
else
{
if ([Math]] - $CourseArrayA[4]) -gt 180)
{
$CourseArrayC[$i % $BufferSize] = 1
}
else
{
$CourseArrayC[$i % $BufferSize] = 0
}
}
$first = $CourseArrayC | measure-object -sum
if ($first.sum -gt 0)
{
$temp = $CourseArrayB | measure-object -average
if ( $temp.average -lt 0 )
{
$result = 360 + $temp.average
}
else
{
$result = $temp.average
}
}
else
{
$temp = $CourseArrayA | measure-object -average
$result = $temp.average
}
$result
$i++
}