Sed command for merging multiple filesClean up concatenated XML filesImprove sed command to replace first instance of character and all following characters?Remove and replace between two specific strings using sed commandI want to compare and match two files and print them into one fileSed for gzip filesHow can I merge the lines of two files by having common headers?Calculate DateTime difference between data of two filesAppend string after each line except header and footer lineusing sed command to get after a /Using sed for files in multiple directories
How is the claim "I am in New York only if I am in America" the same as "If I am in New York, then I am in America?
When blogging recipes, how can I support both readers who want the narrative/journey and ones who want the printer-friendly recipe?
How can the DM most effectively choose 1 out of an odd number of players to be targeted by an attack or effect?
What is the logic behind how bash tests for true/false?
I’m planning on buying a laser printer but concerned about the life cycle of toner in the machine
What would the Romans have called "sorcery"?
Shell script can be run only with sh command
Can I interfere when another PC is about to be attacked?
Copenhagen passport control - US citizen
Accidentally leaked the solution to an assignment, what to do now? (I'm the prof)
How did the USSR manage to innovate in an environment characterized by government censorship and high bureaucracy?
What makes Graph invariants so useful/important?
How do I create uniquely male characters?
Email Account under attack (really) - anything I can do?
Why don't electron-positron collisions release infinite energy?
Schwarzchild Radius of the Universe
New order #4: World
How does one intimidate enemies without having the capacity for violence?
Are tax years 2016 & 2017 back taxes deductible for tax year 2018?
DOS, create pipe for stdin/stdout of command.com(or 4dos.com) in C or Batch?
What typically incentivizes a professor to change jobs to a lower ranking university?
What would happen to a modern skyscraper if it rains micro blackholes?
What are these boxed doors outside store fronts in New York?
Is it possible to make sharp wind that can cut stuff from afar?
Sed command for merging multiple files
Clean up concatenated XML filesImprove sed command to replace first instance of character and all following characters?Remove and replace between two specific strings using sed commandI want to compare and match two files and print them into one fileSed for gzip filesHow can I merge the lines of two files by having common headers?Calculate DateTime difference between data of two filesAppend string after each line except header and footer lineusing sed command to get after a /Using sed for files in multiple directories
.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty margin-bottom:0;
I want to merge multiple files to single file, and also remove the header and footer for all files except the first file header and last file footer.
For example I have 10 files:
File1
Name
A
B
C
Next date
File2
Name
D
E
F
Next date
...
File10
Name
X
Y
Next date
I need output like:
Name
A
B
C
D
E
F
...
X
Y
Next date
How to do in sed command?
sed
edited Mar 27 at 14:07
αғsнιη
17.1k103069
asked Mar 27 at 10:38
Rajathilaka VeeraputhiranRajathilaka Veeraputhiran
83
add a comment |
I want to merge multiple files to single file, and also remove the header and footer for all files except the first file header and last file footer.
For example I have 10 files:
File1
Name
A
B
C
Next date
File2
Name
D
E
F
Next date
...
File10
Name
X
Y
Next date
I need output like:
Name
A
B
C
D
E
F
...
X
Y
Next date
How to do in sed command?
sed
edited Mar 27 at 14:07
αғsнιη
17.1k103069
asked Mar 27 at 10:38
Rajathilaka VeeraputhiranRajathilaka Veeraputhiran
83
Also would be nice to show that how you tried to do it and what didn't work.
– rush
Mar 27 at 10:47
3
Are you limited tosedonly?
– Sjoerd
Mar 27 at 11:31
Header & Footer in all files are always same and static or have any specific pattern?
– αғsнιη
Mar 27 at 15:02
add a comment |
I want to merge multiple files to single file, and also remove the header and footer for all files except the first file header and last file footer.
For example I have 10 files:
File1
Name
A
B
C
Next date
File2
Name
D
E
F
Next date
...
File10
Name
X
Y
Next date
I need output like:
Name
A
B
C
D
E
F
...
X
Y
Next date
How to do in sed command?
sed
edited Mar 27 at 14:07
αғsнιη
17.1k103069
asked Mar 27 at 10:38
Rajathilaka VeeraputhiranRajathilaka Veeraputhiran
83
I want to merge multiple files to single file, and also remove the header and footer for all files except the first file header and last file footer.
For example I have 10 files:
File1
Name
A
B
C
Next date
File2
Name
D
E
F
Next date
...
File10
Name
X
Y
Next date
I need output like:
Name
A
B
C
D
E
F
...
X
Y
Next date
How to do in sed command?
sed
sed
edited Mar 27 at 14:07
αғsнιη
17.1k103069
asked Mar 27 at 10:38
Rajathilaka VeeraputhiranRajathilaka Veeraputhiran
83
edited Mar 27 at 14:07
αғsнιη
17.1k103069
asked Mar 27 at 10:38
Rajathilaka VeeraputhiranRajathilaka Veeraputhiran
83
edited Mar 27 at 14:07
αғsнιη
17.1k103069
edited Mar 27 at 14:07
αғsнιη
17.1k103069
edited Mar 27 at 14:07
αғsнιη
17.1k103069
17.1k103069
asked Mar 27 at 10:38
Rajathilaka VeeraputhiranRajathilaka Veeraputhiran
83
asked Mar 27 at 10:38
Rajathilaka VeeraputhiranRajathilaka Veeraputhiran
83
asked Mar 27 at 10:38
Rajathilaka VeeraputhiranRajathilaka Veeraputhiran
83
83
Also would be nice to show that how you tried to do it and what didn't work.
– rush
Mar 27 at 10:47
3
Are you limited tosedonly?
– Sjoerd
Mar 27 at 11:31
Header & Footer in all files are always same and static or have any specific pattern?
– αғsнιη
Mar 27 at 15:02
add a comment |
Also would be nice to show that how you tried to do it and what didn't work.
– rush
Mar 27 at 10:47
3
Are you limited tosedonly?
– Sjoerd
Mar 27 at 11:31
Header & Footer in all files are always same and static or have any specific pattern?
– αғsнιη
Mar 27 at 15:02
Also would be nice to show that how you tried to do it and what didn't work.
– rush
Mar 27 at 10:47
Also would be nice to show that how you tried to do it and what didn't work.
– rush
Mar 27 at 10:47
3
3
Are you limited to
sed only?– Sjoerd
Mar 27 at 11:31
Are you limited to
sed only?– Sjoerd
Mar 27 at 11:31
Header & Footer in all files are always same and static or have any specific pattern?
– αғsнιη
Mar 27 at 15:02
Header & Footer in all files are always same and static or have any specific pattern?
– αғsнιη
Mar 27 at 15:02
add a comment |
3 Answers
3
active
oldest
votes
Using head, tail and sed:
set -- File*
head -n 1 "$1"
for file do
sed '1d;$d' "$file"
done
tail -n 1 "$file"
>outfile
This assumes that all the relevant files' names are matched by File* (or whatever pattern you use to match them with in the set command).
The set command sets the positional parameters, i.e. $1, $2 etc., to the filenames that we'd like to work with. These are also available in the list $@.
The head command starts by extracting the header from the first file.
The following sed command in the loop (which iterates over all files that were matched by File*) then outputs the contents of each file, except for the first and last line of each file. The sed expression 1d deletes the first line of input, and $d deletes the last line.
The final tail outputs the footer from the last file.
All output is redirected to a file called outfile.
Testing with four files, each with a single line header and footer:
$ cat File*
FIRST HEADER
1
2
3
FOOTER
HEADER
1
2
3
FOOTER
HEADER
1
2
3
FOOTER
HEADER
1
2
3
LAST FOOTER
Running the commands from above. And then,
$ cat outfile
FIRST HEADER
1
2
3
1
2
3
1
2
3
1
2
3
LAST FOOTER
answered Mar 27 at 13:06
Kusalananda♦Kusalananda
140k17261435
add a comment |
Kindly try with below steps and it worked fine
step1:
for i in file1 file2.........filen; do sed -i -e '1d' -e '$d' $i;cat $i >>final_file; done
step2
sed -i '1i Name' final_file
step3
sed '$s/.*/&nNext date/g' final_file
answered Mar 27 at 15:10
Praveen Kumar BSPraveen Kumar BS
1,7391311
1
Note that this modifies the original files, and that the question specifically asked that the header and footer should be taken from the first and last file (you are giving an explicit header and footer text).
– Kusalananda♦
Mar 27 at 15:39
add a comment |
You can try with this awk :
cat script_awk
NR == 1
# keep first header in a
a = $0
next
FNR == 1
# remove last line of File_n and header of File_n+1
a=""
next
if ( a )
# print previous line
print a
# keep last line in a
a=$0
END
# print the last line (footer of last File)
print a
and call it this way :
awk -f script_awk File1 File2 File10
You can call it this way :
awk -f script_awk File*
but not sure you get the footer of File10
answered Mar 27 at 17:22
ctac_ctac_
1,4321211
add a comment |
Your Answer
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "106"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f508945%2fsed-command-for-merging-multiple-files%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
3 Answers
3
active
oldest
votes
3 Answers
3
active
oldest
votes
active
oldest
votes
active
oldest
votes
Using head, tail and sed:
set -- File*
head -n 1 "$1"
for file do
sed '1d;$d' "$file"
done
tail -n 1 "$file"
>outfile
This assumes that all the relevant files' names are matched by File* (or whatever pattern you use to match them with in the set command).
The set command sets the positional parameters, i.e. $1, $2 etc., to the filenames that we'd like to work with. These are also available in the list $@.
The head command starts by extracting the header from the first file.
The following sed command in the loop (which iterates over all files that were matched by File*) then outputs the contents of each file, except for the first and last line of each file. The sed expression 1d deletes the first line of input, and $d deletes the last line.
The final tail outputs the footer from the last file.
All output is redirected to a file called outfile.
Testing with four files, each with a single line header and footer:
$ cat File*
FIRST HEADER
1
2
3
FOOTER
HEADER
1
2
3
FOOTER
HEADER
1
2
3
FOOTER
HEADER
1
2
3
LAST FOOTER
Running the commands from above. And then,
$ cat outfile
FIRST HEADER
1
2
3
1
2
3
1
2
3
1
2
3
LAST FOOTER
answered Mar 27 at 13:06
Kusalananda♦Kusalananda
140k17261435
add a comment |
Using head, tail and sed:
set -- File*
head -n 1 "$1"
for file do
sed '1d;$d' "$file"
done
tail -n 1 "$file"
>outfile
This assumes that all the relevant files' names are matched by File* (or whatever pattern you use to match them with in the set command).
The set command sets the positional parameters, i.e. $1, $2 etc., to the filenames that we'd like to work with. These are also available in the list $@.
The head command starts by extracting the header from the first file.
The following sed command in the loop (which iterates over all files that were matched by File*) then outputs the contents of each file, except for the first and last line of each file. The sed expression 1d deletes the first line of input, and $d deletes the last line.
The final tail outputs the footer from the last file.
All output is redirected to a file called outfile.
Testing with four files, each with a single line header and footer:
$ cat File*
FIRST HEADER
1
2
3
FOOTER
HEADER
1
2
3
FOOTER
HEADER
1
2
3
FOOTER
HEADER
1
2
3
LAST FOOTER
Running the commands from above. And then,
$ cat outfile
FIRST HEADER
1
2
3
1
2
3
1
2
3
1
2
3
LAST FOOTER
answered Mar 27 at 13:06
Kusalananda♦Kusalananda
140k17261435
add a comment |
Using head, tail and sed:
set -- File*
head -n 1 "$1"
for file do
sed '1d;$d' "$file"
done
tail -n 1 "$file"
>outfile
This assumes that all the relevant files' names are matched by File* (or whatever pattern you use to match them with in the set command).
The set command sets the positional parameters, i.e. $1, $2 etc., to the filenames that we'd like to work with. These are also available in the list $@.
The head command starts by extracting the header from the first file.
The following sed command in the loop (which iterates over all files that were matched by File*) then outputs the contents of each file, except for the first and last line of each file. The sed expression 1d deletes the first line of input, and $d deletes the last line.
The final tail outputs the footer from the last file.
All output is redirected to a file called outfile.
Testing with four files, each with a single line header and footer:
$ cat File*
FIRST HEADER
1
2
3
FOOTER
HEADER
1
2
3
FOOTER
HEADER
1
2
3
FOOTER
HEADER
1
2
3
LAST FOOTER
Running the commands from above. And then,
$ cat outfile
FIRST HEADER
1
2
3
1
2
3
1
2
3
1
2
3
LAST FOOTER
answered Mar 27 at 13:06
Kusalananda♦Kusalananda
140k17261435
Using head, tail and sed:
set -- File*
head -n 1 "$1"
for file do
sed '1d;$d' "$file"
done
tail -n 1 "$file"
>outfile
This assumes that all the relevant files' names are matched by File* (or whatever pattern you use to match them with in the set command).
The set command sets the positional parameters, i.e. $1, $2 etc., to the filenames that we'd like to work with. These are also available in the list $@.
The head command starts by extracting the header from the first file.
The following sed command in the loop (which iterates over all files that were matched by File*) then outputs the contents of each file, except for the first and last line of each file. The sed expression 1d deletes the first line of input, and $d deletes the last line.
The final tail outputs the footer from the last file.
All output is redirected to a file called outfile.
Testing with four files, each with a single line header and footer:
$ cat File*
FIRST HEADER
1
2
3
FOOTER
HEADER
1
2
3
FOOTER
HEADER
1
2
3
FOOTER
HEADER
1
2
3
LAST FOOTER
Running the commands from above. And then,
$ cat outfile
FIRST HEADER
1
2
3
1
2
3
1
2
3
1
2
3
LAST FOOTER
answered Mar 27 at 13:06
Kusalananda♦Kusalananda
140k17261435
edited Mar 27 at 15:02
answered Mar 27 at 13:06
Kusalananda♦Kusalananda
140k17261435
answered Mar 27 at 13:06
Kusalananda♦Kusalananda
140k17261435
answered Mar 27 at 13:06
Kusalananda♦Kusalananda
140k17261435
140k17261435
add a comment |
add a comment |
Kindly try with below steps and it worked fine
step1:
for i in file1 file2.........filen; do sed -i -e '1d' -e '$d' $i;cat $i >>final_file; done
step2
sed -i '1i Name' final_file
step3
sed '$s/.*/&nNext date/g' final_file
answered Mar 27 at 15:10
Praveen Kumar BSPraveen Kumar BS
1,7391311
1
Note that this modifies the original files, and that the question specifically asked that the header and footer should be taken from the first and last file (you are giving an explicit header and footer text).
– Kusalananda♦
Mar 27 at 15:39
add a comment |
Kindly try with below steps and it worked fine
step1:
for i in file1 file2.........filen; do sed -i -e '1d' -e '$d' $i;cat $i >>final_file; done
step2
sed -i '1i Name' final_file
step3
sed '$s/.*/&nNext date/g' final_file
answered Mar 27 at 15:10
Praveen Kumar BSPraveen Kumar BS
1,7391311
1
Note that this modifies the original files, and that the question specifically asked that the header and footer should be taken from the first and last file (you are giving an explicit header and footer text).
– Kusalananda♦
Mar 27 at 15:39
add a comment |
Kindly try with below steps and it worked fine
step1:
for i in file1 file2.........filen; do sed -i -e '1d' -e '$d' $i;cat $i >>final_file; done
step2
sed -i '1i Name' final_file
step3
sed '$s/.*/&nNext date/g' final_file
answered Mar 27 at 15:10
Praveen Kumar BSPraveen Kumar BS
1,7391311
Kindly try with below steps and it worked fine
step1:
for i in file1 file2.........filen; do sed -i -e '1d' -e '$d' $i;cat $i >>final_file; done
step2
sed -i '1i Name' final_file
step3
sed '$s/.*/&nNext date/g' final_file
answered Mar 27 at 15:10
Praveen Kumar BSPraveen Kumar BS
1,7391311
answered Mar 27 at 15:10
Praveen Kumar BSPraveen Kumar BS
1,7391311
answered Mar 27 at 15:10
Praveen Kumar BSPraveen Kumar BS
1,7391311
answered Mar 27 at 15:10
Praveen Kumar BSPraveen Kumar BS
1,7391311
1,7391311
1
Note that this modifies the original files, and that the question specifically asked that the header and footer should be taken from the first and last file (you are giving an explicit header and footer text).
– Kusalananda♦
Mar 27 at 15:39
add a comment |
1
Note that this modifies the original files, and that the question specifically asked that the header and footer should be taken from the first and last file (you are giving an explicit header and footer text).
– Kusalananda♦
Mar 27 at 15:39
1
1
Note that this modifies the original files, and that the question specifically asked that the header and footer should be taken from the first and last file (you are giving an explicit header and footer text).
– Kusalananda♦
Mar 27 at 15:39
Note that this modifies the original files, and that the question specifically asked that the header and footer should be taken from the first and last file (you are giving an explicit header and footer text).
– Kusalananda♦
Mar 27 at 15:39
add a comment |
You can try with this awk :
cat script_awk
NR == 1
# keep first header in a
a = $0
next
FNR == 1
# remove last line of File_n and header of File_n+1
a=""
next
if ( a )
# print previous line
print a
# keep last line in a
a=$0
END
# print the last line (footer of last File)
print a
and call it this way :
awk -f script_awk File1 File2 File10
You can call it this way :
awk -f script_awk File*
but not sure you get the footer of File10
answered Mar 27 at 17:22
ctac_ctac_
1,4321211
add a comment |
You can try with this awk :
cat script_awk
NR == 1
# keep first header in a
a = $0
next
FNR == 1
# remove last line of File_n and header of File_n+1
a=""
next
if ( a )
# print previous line
print a
# keep last line in a
a=$0
END
# print the last line (footer of last File)
print a
and call it this way :
awk -f script_awk File1 File2 File10
You can call it this way :
awk -f script_awk File*
but not sure you get the footer of File10
answered Mar 27 at 17:22
ctac_ctac_
1,4321211
add a comment |
You can try with this awk :
cat script_awk
NR == 1
# keep first header in a
a = $0
next
FNR == 1
# remove last line of File_n and header of File_n+1
a=""
next
if ( a )
# print previous line
print a
# keep last line in a
a=$0
END
# print the last line (footer of last File)
print a
and call it this way :
awk -f script_awk File1 File2 File10
You can call it this way :
awk -f script_awk File*
but not sure you get the footer of File10
answered Mar 27 at 17:22
ctac_ctac_
1,4321211
You can try with this awk :
cat script_awk
NR == 1
# keep first header in a
a = $0
next
FNR == 1
# remove last line of File_n and header of File_n+1
a=""
next
if ( a )
# print previous line
print a
# keep last line in a
a=$0
END
# print the last line (footer of last File)
print a
and call it this way :
awk -f script_awk File1 File2 File10
You can call it this way :
awk -f script_awk File*
but not sure you get the footer of File10
answered Mar 27 at 17:22
ctac_ctac_
1,4321211
answered Mar 27 at 17:22
ctac_ctac_
1,4321211
answered Mar 27 at 17:22
ctac_ctac_
1,4321211
answered Mar 27 at 17:22
ctac_ctac_
1,4321211
1,4321211
add a comment |
add a comment |
Thanks for contributing an answer to Unix & Linux Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f508945%2fsed-command-for-merging-multiple-files%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
-sed
Also would be nice to show that how you tried to do it and what didn't work.
– rush
Mar 27 at 10:47
3
Are you limited to
sedonly?– Sjoerd
Mar 27 at 11:31
Header & Footer in all files are always same and static or have any specific pattern?
– αғsнιη
Mar 27 at 15:02