OK I need some help figuring out how to parse duplicates from a very large csv file with million rows and 26 columns
Columns are Name, P1…P25
I need to find any duplicate cell regardless of row or column.
then export the Name and duplicate cell for each.
For your use case, this is what these are used for.
# get function / cmdlet details (Get-Command -Name Select-String).Parameters Get-help -Name Select-String -Full Get-help -Name Select-String -Online Get-help -Name Select-String -Examples (Get-Command -Name Import-Csv).Parameters Get-help -Name Import-Csv -Full Get-help -Name Import-Csv -Online Get-help -Name Import-Csv -Examples
What have you tried?
Where is your code?
What errors are you having, please explain?
What have you searched for?
There are many examples and articles on how to select data rows from files. A simple search using …
‘powershell select row from csv’
… will give you a solid list of them.
Example(s):
PowerTip: Get Row from CSV File Based On Value
Filtering CSV file based on row value
https://stackoverflow.com/questions/31393777/filtering-csv-file-based-on-row-value
I have searched rows and columns before and often just do compare-object
but in this case I need each column of each row to be a separate object.
So if any object in row one matches any object in any other row add both to an array or export
then row two and so on, but dealing with close to a million rows and now 57 columns.