Combining two files in different folders in Linux - linux

Question

I have two set of folders that have files with the same filenames and structure. The folder structure is something like this:
\outputfolder\
|---\folder1\
| |---file1.txt
| |---file2.txt
|
|---\folder2\
|---file1.txt
|---file2.txt
So what I need to do is to combine (append) all the files with the same name in these folders (file1.txt with file1.txt etc.) into another file inside the outputfolder. After getting these combined files I also need to create a tar.gz file from all of these combined files.
How can I accomplish this in a Linux based command line environment? The folder name (folder1 and folder2 etc) is variable so this needs to be given but the files need not and it should automatically combine all the files with the same name.
Also, these files have headers for column names, so I would need to remove that as well while appending.

Answer 1

Here's some code to get you started
topdir=outputfolder
dir1=folder1
dir2=folder2
for f in $topdir/$dir1/*.txt
do
outf=$topdir/`basename $f .txt`-concat.txt
cp $f $outf
sed -e '1 d' $topdir/$dir2/`basename $f` >> $outf
done
tar czf foo.tar.gz $topdir/*-concat.txt
Edit: added the part removing the header of the 2nd file.

Answer 2

find . -name 'file1.txt' | xargs cat >file1_concat.txt

Answer 3

This will work even if there are some files only in folder1 and some files only in folder2:
concat_files() {
for dir in "$#"; do
for file in "$dir"/*; do
this=$(basename "$file")
{ [[ -f "$this" ]] && sed 1d "$file" || cat "$file"; } >> "$this"
done
done
tar zcvf allfiles.tar.gz *
}
concat_files folder1 folder2
It will work if you have more than 2 folders for your concatenation job.
I assume you want to keep the header in the resulting file.

Answer 4

Have you tried the cat command (concatenation)?
cat file1 file2 >> outputfile
Might want to chuck this in a small bash script to go through the directory. This should start you off.
Best of luck.
Leo