Powershell – Merging files to then load into PostgreSQL

Merging a set of files into one large file

Start PowerShell then use CD to move to the Directory with the files to merge

Get-Content .\*.txt | Out-File c:\temp\merged.txt

This will find all .txt (text) files and merge into a single document in c:\temp folder.

However I found the output was in Unicode format… and to then load this into PostgreSQL
So I  then loaded and saved the file using textpad in UTF-8 format
then create the table in PG….

create table houseprice03_msoa_ew
(la_code text,
la_name text,
msoa_code text,
msoa_name text,
price_all_dwellings_2percentile integer,
price_all_dwellings_lower_quartile integer,
price_all_dwellings__median integer,
price_all_dwellings__upper_quartile integer,
price_all_dwellings_98precentilentile integer,
price_all_dwellings__mean integer
)

and then copy the data into that table…

COPY houseprice03_msoa_ew FROM 'C:\Data\GIS\MartynDorey\CensusData\2003 Houseprices\houseprice2003_G080309_2288_GeoPolicy_MSOA.csv' DELIMITER ',' CSV HEADER;