Merge multiple csv files to a single csv file
31 Dec 2016Recently, I tried to merge multiple csv files (with the same header) into a single csv file without header.
For example, I have two csv files, hi.csv
and hi1.csv
:
$ cat hi.csv
uuid,email
"i-am-uuid","i-am-email"
$ cat hi1.csv
uuid,email
"i-am-uuid-in-hi1.csv","i-am-email-in-hi1.csv"
And I want to merge them to be:
"i-am-uuid","i-am-email"
"i-am-uuid-in-hi1.csv","i-am-email-in-hi1.csv"
You can achieve this by:
$ find . -name "hi*.csv" | xargs -n 1 tail -n +2
"i-am-uuid","i-am-email"
"i-am-uuid-in-hi1.csv","i-am-email-in-hi1.csv"
And then you can pip the result into a new file:
$ find . -name "hi*.csv" | xargs -n 1 tail -n +2 > new_file.csv
$ cat new_file.csv
"i-am-uuid","i-am-email"
"i-am-uuid-in-hi1.csv","i-am-email-in-hi1.csv"
For more information about xargs
, please see here
What if I want the new file to have a header?
Well, after you have the new_file.csv
, you can insert the header into the beginning of the new_file.csv
:
$ echo -e "$(head -n 1 hi1.csv)\n$(cat new_file.csv)" > new_file.csv
$ cat new_file.csv
uuid,email
"i-am-uuid","i-am-email"
"i-am-uuid-in-hi1.csv","i-am-email-in-hi1.csv"
You can also combine these two steps into one single command:
$ echo -e "$(head -n 1 hi1.csv)\n$(find . -name 'hi*.csv' | xargs -n 1 tail -n +2 )" > new_file.csv
$ cat new_file.csv
uuid,email
"i-am-uuid","i-am-email"
"i-am-uuid-in-hi1.csv","i-am-email-in-hi1.csv"
I hope it is useful for you.