My question and requirement are listed below:
- I have many files in txt and csv formats(files A, B, C, D)
- These files will load SQL tables (A, B, C, D)
- These files have different metadata and unique keys
- These files will be loaded to target tables as full load the first time
–Subsequent files(A,B,C,D) will be loaded on incremental basis:
----If the primary key column already exist, update non-key columns
----if the primary key column does not exist, insert as new rows
What I read on this link seems to touch on it barely:
https://powershell.org/forums/topic/csv-to-sql-database-insertupdate/
What I have done in the past(I removed the code so it does not cause confusion–Where I passed all my entries as parameters)
Create an array list from a csv file which has the entries for the files name(A, B,C,D)
Loop for each file name and process:
My challenge is, assuming the first load is a full load, subsequent load will either update or insert.
- How would I get all the columns as parameters
- Pick the the key column(s)
- Compare file entries to existing equivalent table
- If primary key exist, update none-key column but if primary key does not exist, insert into target table as new rows
Thank you