Using parameter substitution on a Bash arrayIndexing and modifying Bash parameter array $@Auto-expansion problem with array elements containing an '*' (asterisk)How to do a control loopaccess elements in string array - stray @ symbolDouble quotes in bashUsage of ! in parameter expansionProperly escaping output from command substitution to be used as arguments for another commandHow to add values to an array which contains a variable in the array name in bash?Why is bash history substitution still enabled by default?Unexpected outcome of a=“$@”
What does the word "Atten" mean?
Opposite of a diet
Is there a korbon needed for conversion?
Is there a good way to store credentials outside of a password manager?
Crossing the line between justified force and brutality
Roman Numeral Treatment of Suspensions
Lay out the Carpet
Is the destination of a commercial flight important for the pilot?
Where does the Z80 processor start executing from?
How to Reset Passwords on Multiple Websites Easily?
How to pronounce the slash sign
What does "I’d sit this one out, Cap," imply or mean in the context?
Pre-amplifier input protection
Anatomically Correct Strange Women In Ponds Distributing Swords
How can a function with a hole (removable discontinuity) equal a function with no hole?
How can we prove that any integral in the set of non-elementary integrals cannot be expressed in the form of elementary functions?
Term for the "extreme-extension" version of a straw man fallacy?
Is exact Kanji stroke length important?
What is the intuitive meaning of having a linear relationship between the logs of two variables?
How easy is it to start Magic from scratch?
How does it work when somebody invests in my business?
What is the difference between "behavior" and "behaviour"?
How to be diplomatic in refusing to write code that breaches the privacy of our users
Why Were Madagascar and New Zealand Discovered So Late?
Using parameter substitution on a Bash array
Indexing and modifying Bash parameter array $@Auto-expansion problem with array elements containing an '*' (asterisk)How to do a control loopaccess elements in string array - stray @ symbolDouble quotes in bashUsage of ! in parameter expansionProperly escaping output from command substitution to be used as arguments for another commandHow to add values to an array which contains a variable in the array name in bash?Why is bash history substitution still enabled by default?Unexpected outcome of a=“$@”
I have file.txt that I need to read into a Bash array. Then I need to remove spaces, double quotes and all but the first comma in every entry. Here's how far I've gotten:
$ cat file.txt
10,this
2 0 , i s
30,"all"
40,I
50,n,e,e,d,2
60",s e,e"
$ cat script.sh
#!/bin/bash
readarray -t ARRAY<$1
ARRAY=( "$ARRAY[@]// /" )
ARRAY=( "$ARRAY[@]//"/" )
for ELEMENT in "$ARRAY[@]";do
echo "|ELEMENT|$ELEMENT|"
done
$ ./script.sh file.txt
|ELEMENT|10,this|
|ELEMENT|20,is|
|ELEMENT|30,all|
|ELEMENT|40,I|
|ELEMENT|50,n,e,e,d,2|
|ELEMENT|60,se,e|
Which works great except for the comma situation. I'm aware that there are multiple ways to skin this cat, but due to the larger script this is a part of, I'd really like to use parameter substitution to get to here:
|ELEMENT|10,this|
|ELEMENT|20,is|
|ELEMENT|30,all|
|ELEMENT|40,I|
|ELEMENT|50,need2|
|ELEMENT|60,see|
Is this possible via parameter substitution?
bash shell-script array variable-substitution parameter
add a comment |
I have file.txt that I need to read into a Bash array. Then I need to remove spaces, double quotes and all but the first comma in every entry. Here's how far I've gotten:
$ cat file.txt
10,this
2 0 , i s
30,"all"
40,I
50,n,e,e,d,2
60",s e,e"
$ cat script.sh
#!/bin/bash
readarray -t ARRAY<$1
ARRAY=( "$ARRAY[@]// /" )
ARRAY=( "$ARRAY[@]//"/" )
for ELEMENT in "$ARRAY[@]";do
echo "|ELEMENT|$ELEMENT|"
done
$ ./script.sh file.txt
|ELEMENT|10,this|
|ELEMENT|20,is|
|ELEMENT|30,all|
|ELEMENT|40,I|
|ELEMENT|50,n,e,e,d,2|
|ELEMENT|60,se,e|
Which works great except for the comma situation. I'm aware that there are multiple ways to skin this cat, but due to the larger script this is a part of, I'd really like to use parameter substitution to get to here:
|ELEMENT|10,this|
|ELEMENT|20,is|
|ELEMENT|30,all|
|ELEMENT|40,I|
|ELEMENT|50,need2|
|ELEMENT|60,see|
Is this possible via parameter substitution?
bash shell-script array variable-substitution parameter
3
Is there any reason you need to keep the text in an array, and why you can't let e.g.awk
orsed
do the processing of the data?
– Kusalananda♦
yesterday
@Jeff -- Looping over the array will be a nightmare to implement in the larger script I'm working on.
– Jon Red
yesterday
3
@JonRed I don't know what you are doing, so it's entirely possible that you may not have a choice in the matter, but generally, when you find yourself doing such complex string acrobatics in the shell, that's a very good indication that you should be using an actual programming language. The shell is not designed as a programming language, and while it can be used as one, it really isn't a good idea for more complex things. I strongly urge you to consider switching to perl or python or any other scripting language.
– terdon♦
yesterday
@terdon It's funny, I just got done saying almost the exact same thing to my colleague before I read this post. I basically said this is the final version of this script and that any further requirements will necessitate re-writing in Perl. So yeah, I definitely agree
– Jon Red
yesterday
add a comment |
I have file.txt that I need to read into a Bash array. Then I need to remove spaces, double quotes and all but the first comma in every entry. Here's how far I've gotten:
$ cat file.txt
10,this
2 0 , i s
30,"all"
40,I
50,n,e,e,d,2
60",s e,e"
$ cat script.sh
#!/bin/bash
readarray -t ARRAY<$1
ARRAY=( "$ARRAY[@]// /" )
ARRAY=( "$ARRAY[@]//"/" )
for ELEMENT in "$ARRAY[@]";do
echo "|ELEMENT|$ELEMENT|"
done
$ ./script.sh file.txt
|ELEMENT|10,this|
|ELEMENT|20,is|
|ELEMENT|30,all|
|ELEMENT|40,I|
|ELEMENT|50,n,e,e,d,2|
|ELEMENT|60,se,e|
Which works great except for the comma situation. I'm aware that there are multiple ways to skin this cat, but due to the larger script this is a part of, I'd really like to use parameter substitution to get to here:
|ELEMENT|10,this|
|ELEMENT|20,is|
|ELEMENT|30,all|
|ELEMENT|40,I|
|ELEMENT|50,need2|
|ELEMENT|60,see|
Is this possible via parameter substitution?
bash shell-script array variable-substitution parameter
I have file.txt that I need to read into a Bash array. Then I need to remove spaces, double quotes and all but the first comma in every entry. Here's how far I've gotten:
$ cat file.txt
10,this
2 0 , i s
30,"all"
40,I
50,n,e,e,d,2
60",s e,e"
$ cat script.sh
#!/bin/bash
readarray -t ARRAY<$1
ARRAY=( "$ARRAY[@]// /" )
ARRAY=( "$ARRAY[@]//"/" )
for ELEMENT in "$ARRAY[@]";do
echo "|ELEMENT|$ELEMENT|"
done
$ ./script.sh file.txt
|ELEMENT|10,this|
|ELEMENT|20,is|
|ELEMENT|30,all|
|ELEMENT|40,I|
|ELEMENT|50,n,e,e,d,2|
|ELEMENT|60,se,e|
Which works great except for the comma situation. I'm aware that there are multiple ways to skin this cat, but due to the larger script this is a part of, I'd really like to use parameter substitution to get to here:
|ELEMENT|10,this|
|ELEMENT|20,is|
|ELEMENT|30,all|
|ELEMENT|40,I|
|ELEMENT|50,need2|
|ELEMENT|60,see|
Is this possible via parameter substitution?
bash shell-script array variable-substitution parameter
bash shell-script array variable-substitution parameter
asked yesterday
Jon RedJon Red
698
698
3
Is there any reason you need to keep the text in an array, and why you can't let e.g.awk
orsed
do the processing of the data?
– Kusalananda♦
yesterday
@Jeff -- Looping over the array will be a nightmare to implement in the larger script I'm working on.
– Jon Red
yesterday
3
@JonRed I don't know what you are doing, so it's entirely possible that you may not have a choice in the matter, but generally, when you find yourself doing such complex string acrobatics in the shell, that's a very good indication that you should be using an actual programming language. The shell is not designed as a programming language, and while it can be used as one, it really isn't a good idea for more complex things. I strongly urge you to consider switching to perl or python or any other scripting language.
– terdon♦
yesterday
@terdon It's funny, I just got done saying almost the exact same thing to my colleague before I read this post. I basically said this is the final version of this script and that any further requirements will necessitate re-writing in Perl. So yeah, I definitely agree
– Jon Red
yesterday
add a comment |
3
Is there any reason you need to keep the text in an array, and why you can't let e.g.awk
orsed
do the processing of the data?
– Kusalananda♦
yesterday
@Jeff -- Looping over the array will be a nightmare to implement in the larger script I'm working on.
– Jon Red
yesterday
3
@JonRed I don't know what you are doing, so it's entirely possible that you may not have a choice in the matter, but generally, when you find yourself doing such complex string acrobatics in the shell, that's a very good indication that you should be using an actual programming language. The shell is not designed as a programming language, and while it can be used as one, it really isn't a good idea for more complex things. I strongly urge you to consider switching to perl or python or any other scripting language.
– terdon♦
yesterday
@terdon It's funny, I just got done saying almost the exact same thing to my colleague before I read this post. I basically said this is the final version of this script and that any further requirements will necessitate re-writing in Perl. So yeah, I definitely agree
– Jon Red
yesterday
3
3
Is there any reason you need to keep the text in an array, and why you can't let e.g.
awk
or sed
do the processing of the data?– Kusalananda♦
yesterday
Is there any reason you need to keep the text in an array, and why you can't let e.g.
awk
or sed
do the processing of the data?– Kusalananda♦
yesterday
@Jeff -- Looping over the array will be a nightmare to implement in the larger script I'm working on.
– Jon Red
yesterday
@Jeff -- Looping over the array will be a nightmare to implement in the larger script I'm working on.
– Jon Red
yesterday
3
3
@JonRed I don't know what you are doing, so it's entirely possible that you may not have a choice in the matter, but generally, when you find yourself doing such complex string acrobatics in the shell, that's a very good indication that you should be using an actual programming language. The shell is not designed as a programming language, and while it can be used as one, it really isn't a good idea for more complex things. I strongly urge you to consider switching to perl or python or any other scripting language.
– terdon♦
yesterday
@JonRed I don't know what you are doing, so it's entirely possible that you may not have a choice in the matter, but generally, when you find yourself doing such complex string acrobatics in the shell, that's a very good indication that you should be using an actual programming language. The shell is not designed as a programming language, and while it can be used as one, it really isn't a good idea for more complex things. I strongly urge you to consider switching to perl or python or any other scripting language.
– terdon♦
yesterday
@terdon It's funny, I just got done saying almost the exact same thing to my colleague before I read this post. I basically said this is the final version of this script and that any further requirements will necessitate re-writing in Perl. So yeah, I definitely agree
– Jon Red
yesterday
@terdon It's funny, I just got done saying almost the exact same thing to my colleague before I read this post. I basically said this is the final version of this script and that any further requirements will necessitate re-writing in Perl. So yeah, I definitely agree
– Jon Red
yesterday
add a comment |
5 Answers
5
active
oldest
votes
I would remove what you need to remove using sed
before loading into the array (also note the lower case variable names, in general it is best to avoid capitalized variables in shell scripts):
#!/bin/bash
readarray -t array< <(sed 's/"//g; s/ *//g; s/,/"/; s/,//g; s/"/,/' "$1")
for element in "$array[@]";do
echo "|ELEMENT|$element|"
done
This produces the following output on your example file:
$ foo.sh file
|ELEMENT|10,this|
|ELEMENT|20,is|
|ELEMENT|30,all|
|ELEMENT|40,I|
|ELEMENT|50,need2|
|ELEMENT|60,see|
If you really must use parameter substitution, try something like this:
#!/bin/bash
readarray -t array< "$1"
array=( "$array[@]// /" )
array=( "$array[@]//"/" )
array=( "$array[@]/,/"" )
array=( "$array[@]//,/" )
array=( "$array[@]/"/," )
for element in "$array[@]"; do
echo "|ELEMENT|$element|"
done
1
@JonRed I added a version with parameter substitution but it's complex, cumbersome and ugly. Doing this sort of thing in the shell is very rarely a good idea.
– terdon♦
yesterday
1
Note that if you've removed both spaces and double quotes, these characters becoma available to use instead of yourRANDOMTEXTTHATWILLNEVERBEINTHEFILE
.
– Kusalananda♦
yesterday
1
@Kusalananda yeah, I just read your answer. Should have thought of that! Thanks :)
– terdon♦
yesterday
Directly answers the question, illustrates why my preferred solution isn't ideal, and provides the most viable alternative. You win, best answer.
– Jon Red
yesterday
add a comment |
As far as I can see, there's no need to read it into a bash
array to create that output:
$ sed 's/[ "]//g; s/,/ /; s/,//g; s/ /,/; s/.*/|ELEMENT|&|/' <file
|ELEMENT|10,this|
|ELEMENT|20,is|
|ELEMENT|30,all|
|ELEMENT|40,I|
|ELEMENT|50,need2|
|ELEMENT|60,see|
The sed
expression deletes spaces and double quotes, replaces the first comma with a space (there are no other spaces in the string at this point), deletes all other commas, restores the first comma, and the prepends and appends the extra data.
Alternatively, with GNU sed
:
sed 's/[ "]//g; s/,//2g; s/.*/|ELEMENT|&|/' <file
(standard sed
does not support the combination of 2
and g
as flags to the s
command).
1
with GNU sed, you can use's/,//2g
to remove commas, starting with the 2nd
– glenn jackman
yesterday
2
And, the last 2 s/// commands can bes/.*/|ELEMENT|&|/
but that may be more effort for sed.
– glenn jackman
yesterday
1
@glennjackman Possibly, but it looks rather neat.
– Kusalananda♦
yesterday
Yeah, this is part of a larger script. The array is necessary, not just for the output. Hence my interest in parameter substitution. I could loop over the array with this but that will be a nightmare to implement. Terndon provided a loop-free solution using sed that I'll likely fall back on if parameter substitution is a no-go.
– Jon Red
yesterday
If I wasn't tied to using an array, however, this would be best solution.
– Jon Red
yesterday
|
show 4 more comments
ELEMENT='50,n,e,e,d,2'
IFS=, read -r first rest <<<"$ELEMENT"
printf "%s,%sn" "$first" "$rest//,/"
50,need2
Get out of the habit of using ALLCAPS variable names. You'll eventually collide with a crucial "system" variable like PATH and break your code.
Not parameter substitution. BUT, I was unaware that ALLCAPS variable names was a bad habit in Bash. You make a good point, one that a cursory googling definitely confirms. Thank you for improving my style! :)
– Jon Red
yesterday
1
I've answer questions where the person wrotePATH=something; ls $PATH
and then wondered about thels: command not found
error.
– glenn jackman
yesterday
1
There are nearly a hundred built-in variables that are named in all caps (click through this man page link) to see...
– Jeff Schaller♦
yesterday
add a comment |
[This is essentially a more fully developed version of glenn jackmann's answer]
Building an associative array from the stripped key and value, using the first comma as separator:
declare -A arr
while IFS=, read -r k v; do arr["$k//[ "]"]="$v//[ ,"]"; done < file.txt
for k in "$!arr[@]"; do
printf '|ELEMENT|%s,%s|n' "$k" "$arr[$k]"
done
|ELEMENT|20,is|
|ELEMENT|10,this|
|ELEMENT|50,need2|
|ELEMENT|40,I|
|ELEMENT|60,see|
|ELEMENT|30,all|
add a comment |
You could loop over the array and use an intermediate variable:
for((i=0; i < "$#ARRAY[@]"; i++))
do
rest="$ARRAY[i]#*,"
ARRAY[i]="$ARRAY[i]%%,*","$rest//,/"
done
This assigns to rest
the portion after the first comma; we then concatenate three pieces back into the original variable:
- the portion before the first comma
- a comma
- the replacement in
rest
of every comma with nothing
This was my first thought and is simple enough for the example but this is part of larger script where the array is massive and there's already loops and it would be a whole thing. This would definitely work but would be very cumbersome to implement in the larger project I'm working on.
– Jon Red
yesterday
1
Fair enough; I just tried to answer within the limitations (parameter expansion only).
– Jeff Schaller♦
yesterday
add a comment |
Your Answer
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "106"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f508777%2fusing-parameter-substitution-on-a-bash-array%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
5 Answers
5
active
oldest
votes
5 Answers
5
active
oldest
votes
active
oldest
votes
active
oldest
votes
I would remove what you need to remove using sed
before loading into the array (also note the lower case variable names, in general it is best to avoid capitalized variables in shell scripts):
#!/bin/bash
readarray -t array< <(sed 's/"//g; s/ *//g; s/,/"/; s/,//g; s/"/,/' "$1")
for element in "$array[@]";do
echo "|ELEMENT|$element|"
done
This produces the following output on your example file:
$ foo.sh file
|ELEMENT|10,this|
|ELEMENT|20,is|
|ELEMENT|30,all|
|ELEMENT|40,I|
|ELEMENT|50,need2|
|ELEMENT|60,see|
If you really must use parameter substitution, try something like this:
#!/bin/bash
readarray -t array< "$1"
array=( "$array[@]// /" )
array=( "$array[@]//"/" )
array=( "$array[@]/,/"" )
array=( "$array[@]//,/" )
array=( "$array[@]/"/," )
for element in "$array[@]"; do
echo "|ELEMENT|$element|"
done
1
@JonRed I added a version with parameter substitution but it's complex, cumbersome and ugly. Doing this sort of thing in the shell is very rarely a good idea.
– terdon♦
yesterday
1
Note that if you've removed both spaces and double quotes, these characters becoma available to use instead of yourRANDOMTEXTTHATWILLNEVERBEINTHEFILE
.
– Kusalananda♦
yesterday
1
@Kusalananda yeah, I just read your answer. Should have thought of that! Thanks :)
– terdon♦
yesterday
Directly answers the question, illustrates why my preferred solution isn't ideal, and provides the most viable alternative. You win, best answer.
– Jon Red
yesterday
add a comment |
I would remove what you need to remove using sed
before loading into the array (also note the lower case variable names, in general it is best to avoid capitalized variables in shell scripts):
#!/bin/bash
readarray -t array< <(sed 's/"//g; s/ *//g; s/,/"/; s/,//g; s/"/,/' "$1")
for element in "$array[@]";do
echo "|ELEMENT|$element|"
done
This produces the following output on your example file:
$ foo.sh file
|ELEMENT|10,this|
|ELEMENT|20,is|
|ELEMENT|30,all|
|ELEMENT|40,I|
|ELEMENT|50,need2|
|ELEMENT|60,see|
If you really must use parameter substitution, try something like this:
#!/bin/bash
readarray -t array< "$1"
array=( "$array[@]// /" )
array=( "$array[@]//"/" )
array=( "$array[@]/,/"" )
array=( "$array[@]//,/" )
array=( "$array[@]/"/," )
for element in "$array[@]"; do
echo "|ELEMENT|$element|"
done
1
@JonRed I added a version with parameter substitution but it's complex, cumbersome and ugly. Doing this sort of thing in the shell is very rarely a good idea.
– terdon♦
yesterday
1
Note that if you've removed both spaces and double quotes, these characters becoma available to use instead of yourRANDOMTEXTTHATWILLNEVERBEINTHEFILE
.
– Kusalananda♦
yesterday
1
@Kusalananda yeah, I just read your answer. Should have thought of that! Thanks :)
– terdon♦
yesterday
Directly answers the question, illustrates why my preferred solution isn't ideal, and provides the most viable alternative. You win, best answer.
– Jon Red
yesterday
add a comment |
I would remove what you need to remove using sed
before loading into the array (also note the lower case variable names, in general it is best to avoid capitalized variables in shell scripts):
#!/bin/bash
readarray -t array< <(sed 's/"//g; s/ *//g; s/,/"/; s/,//g; s/"/,/' "$1")
for element in "$array[@]";do
echo "|ELEMENT|$element|"
done
This produces the following output on your example file:
$ foo.sh file
|ELEMENT|10,this|
|ELEMENT|20,is|
|ELEMENT|30,all|
|ELEMENT|40,I|
|ELEMENT|50,need2|
|ELEMENT|60,see|
If you really must use parameter substitution, try something like this:
#!/bin/bash
readarray -t array< "$1"
array=( "$array[@]// /" )
array=( "$array[@]//"/" )
array=( "$array[@]/,/"" )
array=( "$array[@]//,/" )
array=( "$array[@]/"/," )
for element in "$array[@]"; do
echo "|ELEMENT|$element|"
done
I would remove what you need to remove using sed
before loading into the array (also note the lower case variable names, in general it is best to avoid capitalized variables in shell scripts):
#!/bin/bash
readarray -t array< <(sed 's/"//g; s/ *//g; s/,/"/; s/,//g; s/"/,/' "$1")
for element in "$array[@]";do
echo "|ELEMENT|$element|"
done
This produces the following output on your example file:
$ foo.sh file
|ELEMENT|10,this|
|ELEMENT|20,is|
|ELEMENT|30,all|
|ELEMENT|40,I|
|ELEMENT|50,need2|
|ELEMENT|60,see|
If you really must use parameter substitution, try something like this:
#!/bin/bash
readarray -t array< "$1"
array=( "$array[@]// /" )
array=( "$array[@]//"/" )
array=( "$array[@]/,/"" )
array=( "$array[@]//,/" )
array=( "$array[@]/"/," )
for element in "$array[@]"; do
echo "|ELEMENT|$element|"
done
edited yesterday
answered yesterday
terdon♦terdon
133k32265444
133k32265444
1
@JonRed I added a version with parameter substitution but it's complex, cumbersome and ugly. Doing this sort of thing in the shell is very rarely a good idea.
– terdon♦
yesterday
1
Note that if you've removed both spaces and double quotes, these characters becoma available to use instead of yourRANDOMTEXTTHATWILLNEVERBEINTHEFILE
.
– Kusalananda♦
yesterday
1
@Kusalananda yeah, I just read your answer. Should have thought of that! Thanks :)
– terdon♦
yesterday
Directly answers the question, illustrates why my preferred solution isn't ideal, and provides the most viable alternative. You win, best answer.
– Jon Red
yesterday
add a comment |
1
@JonRed I added a version with parameter substitution but it's complex, cumbersome and ugly. Doing this sort of thing in the shell is very rarely a good idea.
– terdon♦
yesterday
1
Note that if you've removed both spaces and double quotes, these characters becoma available to use instead of yourRANDOMTEXTTHATWILLNEVERBEINTHEFILE
.
– Kusalananda♦
yesterday
1
@Kusalananda yeah, I just read your answer. Should have thought of that! Thanks :)
– terdon♦
yesterday
Directly answers the question, illustrates why my preferred solution isn't ideal, and provides the most viable alternative. You win, best answer.
– Jon Red
yesterday
1
1
@JonRed I added a version with parameter substitution but it's complex, cumbersome and ugly. Doing this sort of thing in the shell is very rarely a good idea.
– terdon♦
yesterday
@JonRed I added a version with parameter substitution but it's complex, cumbersome and ugly. Doing this sort of thing in the shell is very rarely a good idea.
– terdon♦
yesterday
1
1
Note that if you've removed both spaces and double quotes, these characters becoma available to use instead of your
RANDOMTEXTTHATWILLNEVERBEINTHEFILE
.– Kusalananda♦
yesterday
Note that if you've removed both spaces and double quotes, these characters becoma available to use instead of your
RANDOMTEXTTHATWILLNEVERBEINTHEFILE
.– Kusalananda♦
yesterday
1
1
@Kusalananda yeah, I just read your answer. Should have thought of that! Thanks :)
– terdon♦
yesterday
@Kusalananda yeah, I just read your answer. Should have thought of that! Thanks :)
– terdon♦
yesterday
Directly answers the question, illustrates why my preferred solution isn't ideal, and provides the most viable alternative. You win, best answer.
– Jon Red
yesterday
Directly answers the question, illustrates why my preferred solution isn't ideal, and provides the most viable alternative. You win, best answer.
– Jon Red
yesterday
add a comment |
As far as I can see, there's no need to read it into a bash
array to create that output:
$ sed 's/[ "]//g; s/,/ /; s/,//g; s/ /,/; s/.*/|ELEMENT|&|/' <file
|ELEMENT|10,this|
|ELEMENT|20,is|
|ELEMENT|30,all|
|ELEMENT|40,I|
|ELEMENT|50,need2|
|ELEMENT|60,see|
The sed
expression deletes spaces and double quotes, replaces the first comma with a space (there are no other spaces in the string at this point), deletes all other commas, restores the first comma, and the prepends and appends the extra data.
Alternatively, with GNU sed
:
sed 's/[ "]//g; s/,//2g; s/.*/|ELEMENT|&|/' <file
(standard sed
does not support the combination of 2
and g
as flags to the s
command).
1
with GNU sed, you can use's/,//2g
to remove commas, starting with the 2nd
– glenn jackman
yesterday
2
And, the last 2 s/// commands can bes/.*/|ELEMENT|&|/
but that may be more effort for sed.
– glenn jackman
yesterday
1
@glennjackman Possibly, but it looks rather neat.
– Kusalananda♦
yesterday
Yeah, this is part of a larger script. The array is necessary, not just for the output. Hence my interest in parameter substitution. I could loop over the array with this but that will be a nightmare to implement. Terndon provided a loop-free solution using sed that I'll likely fall back on if parameter substitution is a no-go.
– Jon Red
yesterday
If I wasn't tied to using an array, however, this would be best solution.
– Jon Red
yesterday
|
show 4 more comments
As far as I can see, there's no need to read it into a bash
array to create that output:
$ sed 's/[ "]//g; s/,/ /; s/,//g; s/ /,/; s/.*/|ELEMENT|&|/' <file
|ELEMENT|10,this|
|ELEMENT|20,is|
|ELEMENT|30,all|
|ELEMENT|40,I|
|ELEMENT|50,need2|
|ELEMENT|60,see|
The sed
expression deletes spaces and double quotes, replaces the first comma with a space (there are no other spaces in the string at this point), deletes all other commas, restores the first comma, and the prepends and appends the extra data.
Alternatively, with GNU sed
:
sed 's/[ "]//g; s/,//2g; s/.*/|ELEMENT|&|/' <file
(standard sed
does not support the combination of 2
and g
as flags to the s
command).
1
with GNU sed, you can use's/,//2g
to remove commas, starting with the 2nd
– glenn jackman
yesterday
2
And, the last 2 s/// commands can bes/.*/|ELEMENT|&|/
but that may be more effort for sed.
– glenn jackman
yesterday
1
@glennjackman Possibly, but it looks rather neat.
– Kusalananda♦
yesterday
Yeah, this is part of a larger script. The array is necessary, not just for the output. Hence my interest in parameter substitution. I could loop over the array with this but that will be a nightmare to implement. Terndon provided a loop-free solution using sed that I'll likely fall back on if parameter substitution is a no-go.
– Jon Red
yesterday
If I wasn't tied to using an array, however, this would be best solution.
– Jon Red
yesterday
|
show 4 more comments
As far as I can see, there's no need to read it into a bash
array to create that output:
$ sed 's/[ "]//g; s/,/ /; s/,//g; s/ /,/; s/.*/|ELEMENT|&|/' <file
|ELEMENT|10,this|
|ELEMENT|20,is|
|ELEMENT|30,all|
|ELEMENT|40,I|
|ELEMENT|50,need2|
|ELEMENT|60,see|
The sed
expression deletes spaces and double quotes, replaces the first comma with a space (there are no other spaces in the string at this point), deletes all other commas, restores the first comma, and the prepends and appends the extra data.
Alternatively, with GNU sed
:
sed 's/[ "]//g; s/,//2g; s/.*/|ELEMENT|&|/' <file
(standard sed
does not support the combination of 2
and g
as flags to the s
command).
As far as I can see, there's no need to read it into a bash
array to create that output:
$ sed 's/[ "]//g; s/,/ /; s/,//g; s/ /,/; s/.*/|ELEMENT|&|/' <file
|ELEMENT|10,this|
|ELEMENT|20,is|
|ELEMENT|30,all|
|ELEMENT|40,I|
|ELEMENT|50,need2|
|ELEMENT|60,see|
The sed
expression deletes spaces and double quotes, replaces the first comma with a space (there are no other spaces in the string at this point), deletes all other commas, restores the first comma, and the prepends and appends the extra data.
Alternatively, with GNU sed
:
sed 's/[ "]//g; s/,//2g; s/.*/|ELEMENT|&|/' <file
(standard sed
does not support the combination of 2
and g
as flags to the s
command).
edited yesterday
answered yesterday
Kusalananda♦Kusalananda
138k17258426
138k17258426
1
with GNU sed, you can use's/,//2g
to remove commas, starting with the 2nd
– glenn jackman
yesterday
2
And, the last 2 s/// commands can bes/.*/|ELEMENT|&|/
but that may be more effort for sed.
– glenn jackman
yesterday
1
@glennjackman Possibly, but it looks rather neat.
– Kusalananda♦
yesterday
Yeah, this is part of a larger script. The array is necessary, not just for the output. Hence my interest in parameter substitution. I could loop over the array with this but that will be a nightmare to implement. Terndon provided a loop-free solution using sed that I'll likely fall back on if parameter substitution is a no-go.
– Jon Red
yesterday
If I wasn't tied to using an array, however, this would be best solution.
– Jon Red
yesterday
|
show 4 more comments
1
with GNU sed, you can use's/,//2g
to remove commas, starting with the 2nd
– glenn jackman
yesterday
2
And, the last 2 s/// commands can bes/.*/|ELEMENT|&|/
but that may be more effort for sed.
– glenn jackman
yesterday
1
@glennjackman Possibly, but it looks rather neat.
– Kusalananda♦
yesterday
Yeah, this is part of a larger script. The array is necessary, not just for the output. Hence my interest in parameter substitution. I could loop over the array with this but that will be a nightmare to implement. Terndon provided a loop-free solution using sed that I'll likely fall back on if parameter substitution is a no-go.
– Jon Red
yesterday
If I wasn't tied to using an array, however, this would be best solution.
– Jon Red
yesterday
1
1
with GNU sed, you can use
's/,//2g
to remove commas, starting with the 2nd– glenn jackman
yesterday
with GNU sed, you can use
's/,//2g
to remove commas, starting with the 2nd– glenn jackman
yesterday
2
2
And, the last 2 s/// commands can be
s/.*/|ELEMENT|&|/
but that may be more effort for sed.– glenn jackman
yesterday
And, the last 2 s/// commands can be
s/.*/|ELEMENT|&|/
but that may be more effort for sed.– glenn jackman
yesterday
1
1
@glennjackman Possibly, but it looks rather neat.
– Kusalananda♦
yesterday
@glennjackman Possibly, but it looks rather neat.
– Kusalananda♦
yesterday
Yeah, this is part of a larger script. The array is necessary, not just for the output. Hence my interest in parameter substitution. I could loop over the array with this but that will be a nightmare to implement. Terndon provided a loop-free solution using sed that I'll likely fall back on if parameter substitution is a no-go.
– Jon Red
yesterday
Yeah, this is part of a larger script. The array is necessary, not just for the output. Hence my interest in parameter substitution. I could loop over the array with this but that will be a nightmare to implement. Terndon provided a loop-free solution using sed that I'll likely fall back on if parameter substitution is a no-go.
– Jon Red
yesterday
If I wasn't tied to using an array, however, this would be best solution.
– Jon Red
yesterday
If I wasn't tied to using an array, however, this would be best solution.
– Jon Red
yesterday
|
show 4 more comments
ELEMENT='50,n,e,e,d,2'
IFS=, read -r first rest <<<"$ELEMENT"
printf "%s,%sn" "$first" "$rest//,/"
50,need2
Get out of the habit of using ALLCAPS variable names. You'll eventually collide with a crucial "system" variable like PATH and break your code.
Not parameter substitution. BUT, I was unaware that ALLCAPS variable names was a bad habit in Bash. You make a good point, one that a cursory googling definitely confirms. Thank you for improving my style! :)
– Jon Red
yesterday
1
I've answer questions where the person wrotePATH=something; ls $PATH
and then wondered about thels: command not found
error.
– glenn jackman
yesterday
1
There are nearly a hundred built-in variables that are named in all caps (click through this man page link) to see...
– Jeff Schaller♦
yesterday
add a comment |
ELEMENT='50,n,e,e,d,2'
IFS=, read -r first rest <<<"$ELEMENT"
printf "%s,%sn" "$first" "$rest//,/"
50,need2
Get out of the habit of using ALLCAPS variable names. You'll eventually collide with a crucial "system" variable like PATH and break your code.
Not parameter substitution. BUT, I was unaware that ALLCAPS variable names was a bad habit in Bash. You make a good point, one that a cursory googling definitely confirms. Thank you for improving my style! :)
– Jon Red
yesterday
1
I've answer questions where the person wrotePATH=something; ls $PATH
and then wondered about thels: command not found
error.
– glenn jackman
yesterday
1
There are nearly a hundred built-in variables that are named in all caps (click through this man page link) to see...
– Jeff Schaller♦
yesterday
add a comment |
ELEMENT='50,n,e,e,d,2'
IFS=, read -r first rest <<<"$ELEMENT"
printf "%s,%sn" "$first" "$rest//,/"
50,need2
Get out of the habit of using ALLCAPS variable names. You'll eventually collide with a crucial "system" variable like PATH and break your code.
ELEMENT='50,n,e,e,d,2'
IFS=, read -r first rest <<<"$ELEMENT"
printf "%s,%sn" "$first" "$rest//,/"
50,need2
Get out of the habit of using ALLCAPS variable names. You'll eventually collide with a crucial "system" variable like PATH and break your code.
answered yesterday
glenn jackmanglenn jackman
52.7k573114
52.7k573114
Not parameter substitution. BUT, I was unaware that ALLCAPS variable names was a bad habit in Bash. You make a good point, one that a cursory googling definitely confirms. Thank you for improving my style! :)
– Jon Red
yesterday
1
I've answer questions where the person wrotePATH=something; ls $PATH
and then wondered about thels: command not found
error.
– glenn jackman
yesterday
1
There are nearly a hundred built-in variables that are named in all caps (click through this man page link) to see...
– Jeff Schaller♦
yesterday
add a comment |
Not parameter substitution. BUT, I was unaware that ALLCAPS variable names was a bad habit in Bash. You make a good point, one that a cursory googling definitely confirms. Thank you for improving my style! :)
– Jon Red
yesterday
1
I've answer questions where the person wrotePATH=something; ls $PATH
and then wondered about thels: command not found
error.
– glenn jackman
yesterday
1
There are nearly a hundred built-in variables that are named in all caps (click through this man page link) to see...
– Jeff Schaller♦
yesterday
Not parameter substitution. BUT, I was unaware that ALLCAPS variable names was a bad habit in Bash. You make a good point, one that a cursory googling definitely confirms. Thank you for improving my style! :)
– Jon Red
yesterday
Not parameter substitution. BUT, I was unaware that ALLCAPS variable names was a bad habit in Bash. You make a good point, one that a cursory googling definitely confirms. Thank you for improving my style! :)
– Jon Red
yesterday
1
1
I've answer questions where the person wrote
PATH=something; ls $PATH
and then wondered about the ls: command not found
error.– glenn jackman
yesterday
I've answer questions where the person wrote
PATH=something; ls $PATH
and then wondered about the ls: command not found
error.– glenn jackman
yesterday
1
1
There are nearly a hundred built-in variables that are named in all caps (click through this man page link) to see...
– Jeff Schaller♦
yesterday
There are nearly a hundred built-in variables that are named in all caps (click through this man page link) to see...
– Jeff Schaller♦
yesterday
add a comment |
[This is essentially a more fully developed version of glenn jackmann's answer]
Building an associative array from the stripped key and value, using the first comma as separator:
declare -A arr
while IFS=, read -r k v; do arr["$k//[ "]"]="$v//[ ,"]"; done < file.txt
for k in "$!arr[@]"; do
printf '|ELEMENT|%s,%s|n' "$k" "$arr[$k]"
done
|ELEMENT|20,is|
|ELEMENT|10,this|
|ELEMENT|50,need2|
|ELEMENT|40,I|
|ELEMENT|60,see|
|ELEMENT|30,all|
add a comment |
[This is essentially a more fully developed version of glenn jackmann's answer]
Building an associative array from the stripped key and value, using the first comma as separator:
declare -A arr
while IFS=, read -r k v; do arr["$k//[ "]"]="$v//[ ,"]"; done < file.txt
for k in "$!arr[@]"; do
printf '|ELEMENT|%s,%s|n' "$k" "$arr[$k]"
done
|ELEMENT|20,is|
|ELEMENT|10,this|
|ELEMENT|50,need2|
|ELEMENT|40,I|
|ELEMENT|60,see|
|ELEMENT|30,all|
add a comment |
[This is essentially a more fully developed version of glenn jackmann's answer]
Building an associative array from the stripped key and value, using the first comma as separator:
declare -A arr
while IFS=, read -r k v; do arr["$k//[ "]"]="$v//[ ,"]"; done < file.txt
for k in "$!arr[@]"; do
printf '|ELEMENT|%s,%s|n' "$k" "$arr[$k]"
done
|ELEMENT|20,is|
|ELEMENT|10,this|
|ELEMENT|50,need2|
|ELEMENT|40,I|
|ELEMENT|60,see|
|ELEMENT|30,all|
[This is essentially a more fully developed version of glenn jackmann's answer]
Building an associative array from the stripped key and value, using the first comma as separator:
declare -A arr
while IFS=, read -r k v; do arr["$k//[ "]"]="$v//[ ,"]"; done < file.txt
for k in "$!arr[@]"; do
printf '|ELEMENT|%s,%s|n' "$k" "$arr[$k]"
done
|ELEMENT|20,is|
|ELEMENT|10,this|
|ELEMENT|50,need2|
|ELEMENT|40,I|
|ELEMENT|60,see|
|ELEMENT|30,all|
answered yesterday
steeldriversteeldriver
37.6k45389
37.6k45389
add a comment |
add a comment |
You could loop over the array and use an intermediate variable:
for((i=0; i < "$#ARRAY[@]"; i++))
do
rest="$ARRAY[i]#*,"
ARRAY[i]="$ARRAY[i]%%,*","$rest//,/"
done
This assigns to rest
the portion after the first comma; we then concatenate three pieces back into the original variable:
- the portion before the first comma
- a comma
- the replacement in
rest
of every comma with nothing
This was my first thought and is simple enough for the example but this is part of larger script where the array is massive and there's already loops and it would be a whole thing. This would definitely work but would be very cumbersome to implement in the larger project I'm working on.
– Jon Red
yesterday
1
Fair enough; I just tried to answer within the limitations (parameter expansion only).
– Jeff Schaller♦
yesterday
add a comment |
You could loop over the array and use an intermediate variable:
for((i=0; i < "$#ARRAY[@]"; i++))
do
rest="$ARRAY[i]#*,"
ARRAY[i]="$ARRAY[i]%%,*","$rest//,/"
done
This assigns to rest
the portion after the first comma; we then concatenate three pieces back into the original variable:
- the portion before the first comma
- a comma
- the replacement in
rest
of every comma with nothing
This was my first thought and is simple enough for the example but this is part of larger script where the array is massive and there's already loops and it would be a whole thing. This would definitely work but would be very cumbersome to implement in the larger project I'm working on.
– Jon Red
yesterday
1
Fair enough; I just tried to answer within the limitations (parameter expansion only).
– Jeff Schaller♦
yesterday
add a comment |
You could loop over the array and use an intermediate variable:
for((i=0; i < "$#ARRAY[@]"; i++))
do
rest="$ARRAY[i]#*,"
ARRAY[i]="$ARRAY[i]%%,*","$rest//,/"
done
This assigns to rest
the portion after the first comma; we then concatenate three pieces back into the original variable:
- the portion before the first comma
- a comma
- the replacement in
rest
of every comma with nothing
You could loop over the array and use an intermediate variable:
for((i=0; i < "$#ARRAY[@]"; i++))
do
rest="$ARRAY[i]#*,"
ARRAY[i]="$ARRAY[i]%%,*","$rest//,/"
done
This assigns to rest
the portion after the first comma; we then concatenate three pieces back into the original variable:
- the portion before the first comma
- a comma
- the replacement in
rest
of every comma with nothing
answered yesterday
Jeff Schaller♦Jeff Schaller
44k1161142
44k1161142
This was my first thought and is simple enough for the example but this is part of larger script where the array is massive and there's already loops and it would be a whole thing. This would definitely work but would be very cumbersome to implement in the larger project I'm working on.
– Jon Red
yesterday
1
Fair enough; I just tried to answer within the limitations (parameter expansion only).
– Jeff Schaller♦
yesterday
add a comment |
This was my first thought and is simple enough for the example but this is part of larger script where the array is massive and there's already loops and it would be a whole thing. This would definitely work but would be very cumbersome to implement in the larger project I'm working on.
– Jon Red
yesterday
1
Fair enough; I just tried to answer within the limitations (parameter expansion only).
– Jeff Schaller♦
yesterday
This was my first thought and is simple enough for the example but this is part of larger script where the array is massive and there's already loops and it would be a whole thing. This would definitely work but would be very cumbersome to implement in the larger project I'm working on.
– Jon Red
yesterday
This was my first thought and is simple enough for the example but this is part of larger script where the array is massive and there's already loops and it would be a whole thing. This would definitely work but would be very cumbersome to implement in the larger project I'm working on.
– Jon Red
yesterday
1
1
Fair enough; I just tried to answer within the limitations (parameter expansion only).
– Jeff Schaller♦
yesterday
Fair enough; I just tried to answer within the limitations (parameter expansion only).
– Jeff Schaller♦
yesterday
add a comment |
Thanks for contributing an answer to Unix & Linux Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f508777%2fusing-parameter-substitution-on-a-bash-array%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
-array, bash, parameter, shell-script, variable-substitution
3
Is there any reason you need to keep the text in an array, and why you can't let e.g.
awk
orsed
do the processing of the data?– Kusalananda♦
yesterday
@Jeff -- Looping over the array will be a nightmare to implement in the larger script I'm working on.
– Jon Red
yesterday
3
@JonRed I don't know what you are doing, so it's entirely possible that you may not have a choice in the matter, but generally, when you find yourself doing such complex string acrobatics in the shell, that's a very good indication that you should be using an actual programming language. The shell is not designed as a programming language, and while it can be used as one, it really isn't a good idea for more complex things. I strongly urge you to consider switching to perl or python or any other scripting language.
– terdon♦
yesterday
@terdon It's funny, I just got done saying almost the exact same thing to my colleague before I read this post. I basically said this is the final version of this script and that any further requirements will necessitate re-writing in Perl. So yeah, I definitely agree
– Jon Red
yesterday