Sed command for merging multiple filesClean up concatenated XML filesImprove sed command to replace first instance of character and all following characters?Remove and replace between two specific strings using sed commandI want to compare and match two files and print them into one fileSed for gzip filesHow can I merge the lines of two files by having common headers?Calculate DateTime difference between data of two filesAppend string after each line except header and footer lineusing sed command to get after a /Using sed for files in multiple directories
Did Shadowfax go to Valinor?
Why doesn't Newton's third law mean a person bounces back to where they started when they hit the ground?
Smoothness of finite-dimensional functional calculus
How to calculate partition Start End Sector?
Mage Armor with Defense fighting style (for Adventurers League bladeslinger)
Roll the carpet
Important Resources for Dark Age Civilizations?
Schoenfled Residua test shows proportionality hazard assumptions holds but Kaplan-Meier plots intersect
Modeling an IP Address
How is it possible to have an ability score that is less than 3?
How can I make a cone from a cube and view the cube with different angles?
What do the dots in this tr command do: tr .............A-Z A-ZA-Z <<< "JVPQBOV" (with 13 dots)
How does strength of boric acid solution increase in presence of salicylic acid?
A newer friend of my brother's gave him a load of baseball cards that are supposedly extremely valuable. Is this a scam?
Fencing style for blades that can attack from a distance
Are the number of citations and number of published articles the most important criteria for a tenure promotion?
Is every diagonalizable matrix is an exponential
Today is the Center
Is it possible to do 50 km distance without any previous training?
Languages that we cannot (dis)prove to be Context-Free
Can an x86 CPU running in real mode be considered to be basically an 8086 CPU?
Is it tax fraud for an individual to declare non-taxable revenue as taxable income? (US tax laws)
In Japanese, what’s the difference between “Tonari ni” (となりに) and “Tsugi” (つぎ)? When would you use one over the other?
"You are your self first supporter", a more proper way to say it
Sed command for merging multiple files
Clean up concatenated XML filesImprove sed command to replace first instance of character and all following characters?Remove and replace between two specific strings using sed commandI want to compare and match two files and print them into one fileSed for gzip filesHow can I merge the lines of two files by having common headers?Calculate DateTime difference between data of two filesAppend string after each line except header and footer lineusing sed command to get after a /Using sed for files in multiple directories
.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty margin-bottom:0;
I want to merge multiple files to single file, and also remove the header and footer for all files except the first file header and last file footer.
For example I have 10 files:
File1
Name
A
B
C
Next date
File2
Name
D
E
F
Next date
...
File10
Name
X
Y
Next date
I need output like:
Name
A
B
C
D
E
F
...
X
Y
Next date
How to do in sed command?
sed
edited Mar 27 at 14:07
αғsнιη
17.1k103069
asked Mar 27 at 10:38
Rajathilaka VeeraputhiranRajathilaka Veeraputhiran
83
add a comment |
I want to merge multiple files to single file, and also remove the header and footer for all files except the first file header and last file footer.
For example I have 10 files:
File1
Name
A
B
C
Next date
File2
Name
D
E
F
Next date
...
File10
Name
X
Y
Next date
I need output like:
Name
A
B
C
D
E
F
...
X
Y
Next date
How to do in sed command?
sed
edited Mar 27 at 14:07
αғsнιη
17.1k103069
asked Mar 27 at 10:38
Rajathilaka VeeraputhiranRajathilaka Veeraputhiran
83
Also would be nice to show that how you tried to do it and what didn't work.
– rush
Mar 27 at 10:47
3
Are you limited tosedonly?
– Sjoerd
Mar 27 at 11:31
Header & Footer in all files are always same and static or have any specific pattern?
– αғsнιη
Mar 27 at 15:02
add a comment |
I want to merge multiple files to single file, and also remove the header and footer for all files except the first file header and last file footer.
For example I have 10 files:
File1
Name
A
B
C
Next date
File2
Name
D
E
F
Next date
...
File10
Name
X
Y
Next date
I need output like:
Name
A
B
C
D
E
F
...
X
Y
Next date
How to do in sed command?
sed
edited Mar 27 at 14:07
αғsнιη
17.1k103069
asked Mar 27 at 10:38
Rajathilaka VeeraputhiranRajathilaka Veeraputhiran
83
I want to merge multiple files to single file, and also remove the header and footer for all files except the first file header and last file footer.
For example I have 10 files:
File1
Name
A
B
C
Next date
File2
Name
D
E
F
Next date
...
File10
Name
X
Y
Next date
I need output like:
Name
A
B
C
D
E
F
...
X
Y
Next date
How to do in sed command?
sed
sed
edited Mar 27 at 14:07
αғsнιη
17.1k103069
asked Mar 27 at 10:38
Rajathilaka VeeraputhiranRajathilaka Veeraputhiran
83
edited Mar 27 at 14:07
αғsнιη
17.1k103069
asked Mar 27 at 10:38
Rajathilaka VeeraputhiranRajathilaka Veeraputhiran
83
edited Mar 27 at 14:07
αғsнιη
17.1k103069
edited Mar 27 at 14:07
αғsнιη
17.1k103069
edited Mar 27 at 14:07
αғsнιη
17.1k103069
17.1k103069
asked Mar 27 at 10:38
Rajathilaka VeeraputhiranRajathilaka Veeraputhiran
83
asked Mar 27 at 10:38
Rajathilaka VeeraputhiranRajathilaka Veeraputhiran
83
asked Mar 27 at 10:38
Rajathilaka VeeraputhiranRajathilaka Veeraputhiran
83
83
Also would be nice to show that how you tried to do it and what didn't work.
– rush
Mar 27 at 10:47
3
Are you limited tosedonly?
– Sjoerd
Mar 27 at 11:31
Header & Footer in all files are always same and static or have any specific pattern?
– αғsнιη
Mar 27 at 15:02
add a comment |
Also would be nice to show that how you tried to do it and what didn't work.
– rush
Mar 27 at 10:47
3
Are you limited tosedonly?
– Sjoerd
Mar 27 at 11:31
Header & Footer in all files are always same and static or have any specific pattern?
– αғsнιη
Mar 27 at 15:02
Also would be nice to show that how you tried to do it and what didn't work.
– rush
Mar 27 at 10:47
Also would be nice to show that how you tried to do it and what didn't work.
– rush
Mar 27 at 10:47
3
3
Are you limited to
sed only?– Sjoerd
Mar 27 at 11:31
Are you limited to
sed only?– Sjoerd
Mar 27 at 11:31
Header & Footer in all files are always same and static or have any specific pattern?
– αғsнιη
Mar 27 at 15:02
Header & Footer in all files are always same and static or have any specific pattern?
– αғsнιη
Mar 27 at 15:02
add a comment |
3 Answers
3
active
oldest
votes
Using head, tail and sed:
set -- File*
head -n 1 "$1"
for file do
sed '1d;$d' "$file"
done
tail -n 1 "$file"
>outfile
This assumes that all the relevant files' names are matched by File* (or whatever pattern you use to match them with in the set command).
The set command sets the positional parameters, i.e. $1, $2 etc., to the filenames that we'd like to work with. These are also available in the list $@.
The head command starts by extracting the header from the first file.
The following sed command in the loop (which iterates over all files that were matched by File*) then outputs the contents of each file, except for the first and last line of each file. The sed expression 1d deletes the first line of input, and $d deletes the last line.
The final tail outputs the footer from the last file.
All output is redirected to a file called outfile.
Testing with four files, each with a single line header and footer:
$ cat File*
FIRST HEADER
1
2
3
FOOTER
HEADER
1
2
3
FOOTER
HEADER
1
2
3
FOOTER
HEADER
1
2
3
LAST FOOTER
Running the commands from above. And then,
$ cat outfile
FIRST HEADER
1
2
3
1
2
3
1
2
3
1
2
3
LAST FOOTER
answered Mar 27 at 13:06
Kusalananda♦Kusalananda
140k17261435
add a comment |
Kindly try with below steps and it worked fine
step1:
for i in file1 file2.........filen; do sed -i -e '1d' -e '$d' $i;cat $i >>final_file; done
step2
sed -i '1i Name' final_file
step3
sed '$s/.*/&nNext date/g' final_file
answered Mar 27 at 15:10
Praveen Kumar BSPraveen Kumar BS
1,7391311
1
Note that this modifies the original files, and that the question specifically asked that the header and footer should be taken from the first and last file (you are giving an explicit header and footer text).
– Kusalananda♦
Mar 27 at 15:39
add a comment |
You can try with this awk :
cat script_awk
NR == 1
# keep first header in a
a = $0
next
FNR == 1
# remove last line of File_n and header of File_n+1
a=""
next
if ( a )
# print previous line
print a
# keep last line in a
a=$0
END
# print the last line (footer of last File)
print a
and call it this way :
awk -f script_awk File1 File2 File10
You can call it this way :
awk -f script_awk File*
but not sure you get the footer of File10
answered Mar 27 at 17:22
ctac_ctac_
1,4301211
add a comment |
Your Answer
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "106"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f508945%2fsed-command-for-merging-multiple-files%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
3 Answers
3
active
oldest
votes
3 Answers
3
active
oldest
votes
active
oldest
votes
active
oldest
votes
Using head, tail and sed:
set -- File*
head -n 1 "$1"
for file do
sed '1d;$d' "$file"
done
tail -n 1 "$file"
>outfile
This assumes that all the relevant files' names are matched by File* (or whatever pattern you use to match them with in the set command).
The set command sets the positional parameters, i.e. $1, $2 etc., to the filenames that we'd like to work with. These are also available in the list $@.
The head command starts by extracting the header from the first file.
The following sed command in the loop (which iterates over all files that were matched by File*) then outputs the contents of each file, except for the first and last line of each file. The sed expression 1d deletes the first line of input, and $d deletes the last line.
The final tail outputs the footer from the last file.
All output is redirected to a file called outfile.
Testing with four files, each with a single line header and footer:
$ cat File*
FIRST HEADER
1
2
3
FOOTER
HEADER
1
2
3
FOOTER
HEADER
1
2
3
FOOTER
HEADER
1
2
3
LAST FOOTER
Running the commands from above. And then,
$ cat outfile
FIRST HEADER
1
2
3
1
2
3
1
2
3
1
2
3
LAST FOOTER
answered Mar 27 at 13:06
Kusalananda♦Kusalananda
140k17261435
add a comment |
Using head, tail and sed:
set -- File*
head -n 1 "$1"
for file do
sed '1d;$d' "$file"
done
tail -n 1 "$file"
>outfile
This assumes that all the relevant files' names are matched by File* (or whatever pattern you use to match them with in the set command).
The set command sets the positional parameters, i.e. $1, $2 etc., to the filenames that we'd like to work with. These are also available in the list $@.
The head command starts by extracting the header from the first file.
The following sed command in the loop (which iterates over all files that were matched by File*) then outputs the contents of each file, except for the first and last line of each file. The sed expression 1d deletes the first line of input, and $d deletes the last line.
The final tail outputs the footer from the last file.
All output is redirected to a file called outfile.
Testing with four files, each with a single line header and footer:
$ cat File*
FIRST HEADER
1
2
3
FOOTER
HEADER
1
2
3
FOOTER
HEADER
1
2
3
FOOTER
HEADER
1
2
3
LAST FOOTER
Running the commands from above. And then,
$ cat outfile
FIRST HEADER
1
2
3
1
2
3
1
2
3
1
2
3
LAST FOOTER
answered Mar 27 at 13:06
Kusalananda♦Kusalananda
140k17261435
add a comment |
Using head, tail and sed:
set -- File*
head -n 1 "$1"
for file do
sed '1d;$d' "$file"
done
tail -n 1 "$file"
>outfile
This assumes that all the relevant files' names are matched by File* (or whatever pattern you use to match them with in the set command).
The set command sets the positional parameters, i.e. $1, $2 etc., to the filenames that we'd like to work with. These are also available in the list $@.
The head command starts by extracting the header from the first file.
The following sed command in the loop (which iterates over all files that were matched by File*) then outputs the contents of each file, except for the first and last line of each file. The sed expression 1d deletes the first line of input, and $d deletes the last line.
The final tail outputs the footer from the last file.
All output is redirected to a file called outfile.
Testing with four files, each with a single line header and footer:
$ cat File*
FIRST HEADER
1
2
3
FOOTER
HEADER
1
2
3
FOOTER
HEADER
1
2
3
FOOTER
HEADER
1
2
3
LAST FOOTER
Running the commands from above. And then,
$ cat outfile
FIRST HEADER
1
2
3
1
2
3
1
2
3
1
2
3
LAST FOOTER
answered Mar 27 at 13:06
Kusalananda♦Kusalananda
140k17261435
Using head, tail and sed:
set -- File*
head -n 1 "$1"
for file do
sed '1d;$d' "$file"
done
tail -n 1 "$file"
>outfile
This assumes that all the relevant files' names are matched by File* (or whatever pattern you use to match them with in the set command).
The set command sets the positional parameters, i.e. $1, $2 etc., to the filenames that we'd like to work with. These are also available in the list $@.
The head command starts by extracting the header from the first file.
The following sed command in the loop (which iterates over all files that were matched by File*) then outputs the contents of each file, except for the first and last line of each file. The sed expression 1d deletes the first line of input, and $d deletes the last line.
The final tail outputs the footer from the last file.
All output is redirected to a file called outfile.
Testing with four files, each with a single line header and footer:
$ cat File*
FIRST HEADER
1
2
3
FOOTER
HEADER
1
2
3
FOOTER
HEADER
1
2
3
FOOTER
HEADER
1
2
3
LAST FOOTER
Running the commands from above. And then,
$ cat outfile
FIRST HEADER
1
2
3
1
2
3
1
2
3
1
2
3
LAST FOOTER
answered Mar 27 at 13:06
Kusalananda♦Kusalananda
140k17261435
edited Mar 27 at 15:02
answered Mar 27 at 13:06
Kusalananda♦Kusalananda
140k17261435
answered Mar 27 at 13:06
Kusalananda♦Kusalananda
140k17261435
answered Mar 27 at 13:06
Kusalananda♦Kusalananda
140k17261435
140k17261435
add a comment |
add a comment |
Kindly try with below steps and it worked fine
step1:
for i in file1 file2.........filen; do sed -i -e '1d' -e '$d' $i;cat $i >>final_file; done
step2
sed -i '1i Name' final_file
step3
sed '$s/.*/&nNext date/g' final_file
answered Mar 27 at 15:10
Praveen Kumar BSPraveen Kumar BS
1,7391311
1
Note that this modifies the original files, and that the question specifically asked that the header and footer should be taken from the first and last file (you are giving an explicit header and footer text).
– Kusalananda♦
Mar 27 at 15:39
add a comment |
Kindly try with below steps and it worked fine
step1:
for i in file1 file2.........filen; do sed -i -e '1d' -e '$d' $i;cat $i >>final_file; done
step2
sed -i '1i Name' final_file
step3
sed '$s/.*/&nNext date/g' final_file
answered Mar 27 at 15:10
Praveen Kumar BSPraveen Kumar BS
1,7391311
1
Note that this modifies the original files, and that the question specifically asked that the header and footer should be taken from the first and last file (you are giving an explicit header and footer text).
– Kusalananda♦
Mar 27 at 15:39
add a comment |
Kindly try with below steps and it worked fine
step1:
for i in file1 file2.........filen; do sed -i -e '1d' -e '$d' $i;cat $i >>final_file; done
step2
sed -i '1i Name' final_file
step3
sed '$s/.*/&nNext date/g' final_file
answered Mar 27 at 15:10
Praveen Kumar BSPraveen Kumar BS
1,7391311
Kindly try with below steps and it worked fine
step1:
for i in file1 file2.........filen; do sed -i -e '1d' -e '$d' $i;cat $i >>final_file; done
step2
sed -i '1i Name' final_file
step3
sed '$s/.*/&nNext date/g' final_file
answered Mar 27 at 15:10
Praveen Kumar BSPraveen Kumar BS
1,7391311
answered Mar 27 at 15:10
Praveen Kumar BSPraveen Kumar BS
1,7391311
answered Mar 27 at 15:10
Praveen Kumar BSPraveen Kumar BS
1,7391311
answered Mar 27 at 15:10
Praveen Kumar BSPraveen Kumar BS
1,7391311
1,7391311
1
Note that this modifies the original files, and that the question specifically asked that the header and footer should be taken from the first and last file (you are giving an explicit header and footer text).
– Kusalananda♦
Mar 27 at 15:39
add a comment |
1
Note that this modifies the original files, and that the question specifically asked that the header and footer should be taken from the first and last file (you are giving an explicit header and footer text).
– Kusalananda♦
Mar 27 at 15:39
1
1
Note that this modifies the original files, and that the question specifically asked that the header and footer should be taken from the first and last file (you are giving an explicit header and footer text).
– Kusalananda♦
Mar 27 at 15:39
Note that this modifies the original files, and that the question specifically asked that the header and footer should be taken from the first and last file (you are giving an explicit header and footer text).
– Kusalananda♦
Mar 27 at 15:39
add a comment |
You can try with this awk :
cat script_awk
NR == 1
# keep first header in a
a = $0
next
FNR == 1
# remove last line of File_n and header of File_n+1
a=""
next
if ( a )
# print previous line
print a
# keep last line in a
a=$0
END
# print the last line (footer of last File)
print a
and call it this way :
awk -f script_awk File1 File2 File10
You can call it this way :
awk -f script_awk File*
but not sure you get the footer of File10
answered Mar 27 at 17:22
ctac_ctac_
1,4301211
add a comment |
You can try with this awk :
cat script_awk
NR == 1
# keep first header in a
a = $0
next
FNR == 1
# remove last line of File_n and header of File_n+1
a=""
next
if ( a )
# print previous line
print a
# keep last line in a
a=$0
END
# print the last line (footer of last File)
print a
and call it this way :
awk -f script_awk File1 File2 File10
You can call it this way :
awk -f script_awk File*
but not sure you get the footer of File10
answered Mar 27 at 17:22
ctac_ctac_
1,4301211
add a comment |
You can try with this awk :
cat script_awk
NR == 1
# keep first header in a
a = $0
next
FNR == 1
# remove last line of File_n and header of File_n+1
a=""
next
if ( a )
# print previous line
print a
# keep last line in a
a=$0
END
# print the last line (footer of last File)
print a
and call it this way :
awk -f script_awk File1 File2 File10
You can call it this way :
awk -f script_awk File*
but not sure you get the footer of File10
answered Mar 27 at 17:22
ctac_ctac_
1,4301211
You can try with this awk :
cat script_awk
NR == 1
# keep first header in a
a = $0
next
FNR == 1
# remove last line of File_n and header of File_n+1
a=""
next
if ( a )
# print previous line
print a
# keep last line in a
a=$0
END
# print the last line (footer of last File)
print a
and call it this way :
awk -f script_awk File1 File2 File10
You can call it this way :
awk -f script_awk File*
but not sure you get the footer of File10
answered Mar 27 at 17:22
ctac_ctac_
1,4301211
answered Mar 27 at 17:22
ctac_ctac_
1,4301211
answered Mar 27 at 17:22
ctac_ctac_
1,4301211
answered Mar 27 at 17:22
ctac_ctac_
1,4301211
1,4301211
add a comment |
add a comment |
Thanks for contributing an answer to Unix & Linux Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f508945%2fsed-command-for-merging-multiple-files%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
-sed
Also would be nice to show that how you tried to do it and what didn't work.
– rush
Mar 27 at 10:47
3
Are you limited to
sedonly?– Sjoerd
Mar 27 at 11:31
Header & Footer in all files are always same and static or have any specific pattern?
– αғsнιη
Mar 27 at 15:02