Bash displaying gibberish after specific incoming request2019 Community Moderator ElectionWhat type of terminal to support if implementing a terminal emulator?Replace log file without affecting “Redirection of stdin and stdout” using nohupSyslog to console, drawbacks?How to examine POST request over HTTPS?loop curl get request bashDisplaying and updating a counter in bashAdding Text After Specific Line of File in Bash Scriptget value after specific wordReplace string after last dot in bashTrace apache server activity for a specific http requestadd a header to incoming http request in a serverDoes Bash imitate Locale-Specific Translation from C?bash curl post request set timeout
Extending the spectral theorem for bounded self adjoint operators to bounded normal operators
Filling the middle of a torus in Tikz
Why has "pence" been used in this sentence, not "pences"?
Offered money to buy a house, seller is asking for more to cover gap between their listing and mortgage owed
Why did the EU agree to delay the Brexit deadline?
Indicating multiple different modes of speech (fantasy language or telepathy)
How much character growth crosses the line into breaking the character
Can I sign legal documents with a smiley face?
Will adding a BY-SA image to a blog post make the entire post BY-SA?
why `nmap 192.168.1.97` returns less services than `nmap 127.0.0.1`?
Should I stop contributing to retirement accounts?
Bob has never been a M before
Journal losing indexing services
Are lightweight LN wallets vulnerable to transaction withholding?
Longest common substring in linear time
Is it possible to have a strip of cold climate in the middle of a planet?
How does the reference system of the Majjhima Nikaya work?
Global amount of publications over time
Has Darkwing Duck ever met Scrooge McDuck?
Does a 'pending' US visa application constitute a denial?
Drawing ramified coverings with tikz
Can I use my Chinese passport to enter China after I acquired another citizenship?
How can Trident be so inexpensive? Will it orbit Triton or just do a (slow) flyby?
Can we have a perfect cadence in a minor key?
Bash displaying gibberish after specific incoming request
2019 Community Moderator ElectionWhat type of terminal to support if implementing a terminal emulator?Replace log file without affecting “Redirection of stdin and stdout” using nohupSyslog to console, drawbacks?How to examine POST request over HTTPS?loop curl get request bashDisplaying and updating a counter in bashAdding Text After Specific Line of File in Bash Scriptget value after specific wordReplace string after last dot in bashTrace apache server activity for a specific http requestadd a header to incoming http request in a serverDoes Bash imitate Locale-Specific Translation from C?bash curl post request set timeout
I set up a new, basic Linux server (CentOS in this case) just for testing purposes. In there, without any firewalls and all that I'm running a Python web application. Basically I type python run.py and have the application on foreground, while the application itself is run on some port like 8080, so in web browser I use I just type my_public_ip_addr:8080 and use it just fine. All this happens over SSH from my laptop.
Now, I left my laptop open for a while, and when I came back, the shell was displaying something like this:
83.20.238.86 - - [13/Mar/2019 08:54:43] "GET / HTTP/1.1" 200 -
87.122.83.97 - - [13/Mar/2019 11:55:30] "GET / HTTP/1.1" 200 -
176.32.33.145 - - [13/Mar/2019 12:08:36] "GET / HTTP/1.1" 200 -
176.32.33.145 - - [13/Mar/2019 12:08:36] code 400, message Bad request syntax ('x16x03x01x00xfcx01x00x00xf8x03x03(xd3FMxf5x0eLox17xa3|x1f8xca~#x07xc1x1f&&x14x19x11x10:x824xd23nAx00x00x8cxc00xc0,xc02xc0.xc0/xc0+xc01xc0-x00xa5x00xa3x00xa1x00x9fx00xa4x00xa2x00xa0x00x9exc0(xc0$xc0x14xc0')
176.32.33.145 - - [13/Mar/2019 12:08:36] "��(M�L⎺�≠8#�&&:�4┼A��▮�←�2�↓�/�→�1�↑���������(�$��" 4▮▮ ↑
176↓32↓33↓145 ↑ ↑ [13/M▒⎼/2▮19 14:55:55] "GET / HTTP/1↓1" 2▮▮ ↑
176↓32↓33↓145 ↑ ↑ [13/M▒⎼/2▮19 14:55:55] c⎺de 4▮▮← └e⎽⎽▒±e B▒d ⎼e─┤e⎽├ ⎽≤┼├▒│ ('│16│▮3│▮1│▮▮│°c│▮1│▮▮│▮▮│°8│▮3│▮3│92│8e│°7│9e│1▒│▒2│1e│°8│°bb^│1b│d1│▒1│1e│d2│d1^│1e/└│96_(│beU│▮4│8d≥│d7⎻│°e│▮▮│▮▮│8c│c▮▮│c▮←│c▮2│c▮↓│c▮/│c▮→│c▮1│c▮↑│▮▮│▒5│▮▮│▒3│▮▮│▒1│▮▮│9°│▮▮│▒4│▮▮│▒2│▮▮│▒▮│▮▮│9e│c▮(│c▮$│c▮│14│c▮')
176↓32↓33↓145 ↑ ↑ [13/M▒⎼/2▮19 14:55:55] "���������b^[⎺⎼▒┼±e@ce┼├⎺⎽↑⎺⎼▒┼±e ⎺⎼▒┼±e_±c]$ ^C
[⎺⎼▒┼±e@ce┼├⎺⎽↑⎺⎼▒┼±e ⎺⎼▒┼±e_±c]$ ┌⎺±⎺┤├
C⎺┼┼ec├☃⎺┼ ├⎺ 1▮4↓248↓36↓8 c┌⎺⎽ed↓
▒d▒└@±⎽:·$
▒d▒└@±⎽:·$
▒d▒└@±⎽:·$
▒d▒└@±⎽:·$ ec▒⎺ '▒e┌┌⎺ ⎽├▒c┐ ⎺┴e⎼°┌⎺┬'
▒e┌┌⎺ ⎽├▒c┐ ⎺┴e⎼°┌⎺┬
▒d▒└@±⎽:·$
You can see 3 last "normal" GET requests to /, but then it begins. I know it can be fixed (link1 or link2) and these were some scanning bots, but my question is:
How does it work, that incoming request broke my terminal?
bash string http
asked Mar 13 at 16:41
adamcziadamczi
1315
|
show 1 more comment
I set up a new, basic Linux server (CentOS in this case) just for testing purposes. In there, without any firewalls and all that I'm running a Python web application. Basically I type python run.py and have the application on foreground, while the application itself is run on some port like 8080, so in web browser I use I just type my_public_ip_addr:8080 and use it just fine. All this happens over SSH from my laptop.
Now, I left my laptop open for a while, and when I came back, the shell was displaying something like this:
83.20.238.86 - - [13/Mar/2019 08:54:43] "GET / HTTP/1.1" 200 -
87.122.83.97 - - [13/Mar/2019 11:55:30] "GET / HTTP/1.1" 200 -
176.32.33.145 - - [13/Mar/2019 12:08:36] "GET / HTTP/1.1" 200 -
176.32.33.145 - - [13/Mar/2019 12:08:36] code 400, message Bad request syntax ('x16x03x01x00xfcx01x00x00xf8x03x03(xd3FMxf5x0eLox17xa3|x1f8xca~#x07xc1x1f&&x14x19x11x10:x824xd23nAx00x00x8cxc00xc0,xc02xc0.xc0/xc0+xc01xc0-x00xa5x00xa3x00xa1x00x9fx00xa4x00xa2x00xa0x00x9exc0(xc0$xc0x14xc0')
176.32.33.145 - - [13/Mar/2019 12:08:36] "��(M�L⎺�≠8#�&&:�4┼A��▮�←�2�↓�/�→�1�↑���������(�$��" 4▮▮ ↑
176↓32↓33↓145 ↑ ↑ [13/M▒⎼/2▮19 14:55:55] "GET / HTTP/1↓1" 2▮▮ ↑
176↓32↓33↓145 ↑ ↑ [13/M▒⎼/2▮19 14:55:55] c⎺de 4▮▮← └e⎽⎽▒±e B▒d ⎼e─┤e⎽├ ⎽≤┼├▒│ ('│16│▮3│▮1│▮▮│°c│▮1│▮▮│▮▮│°8│▮3│▮3│92│8e│°7│9e│1▒│▒2│1e│°8│°bb^│1b│d1│▒1│1e│d2│d1^│1e/└│96_(│beU│▮4│8d≥│d7⎻│°e│▮▮│▮▮│8c│c▮▮│c▮←│c▮2│c▮↓│c▮/│c▮→│c▮1│c▮↑│▮▮│▒5│▮▮│▒3│▮▮│▒1│▮▮│9°│▮▮│▒4│▮▮│▒2│▮▮│▒▮│▮▮│9e│c▮(│c▮$│c▮│14│c▮')
176↓32↓33↓145 ↑ ↑ [13/M▒⎼/2▮19 14:55:55] "���������b^[⎺⎼▒┼±e@ce┼├⎺⎽↑⎺⎼▒┼±e ⎺⎼▒┼±e_±c]$ ^C
[⎺⎼▒┼±e@ce┼├⎺⎽↑⎺⎼▒┼±e ⎺⎼▒┼±e_±c]$ ┌⎺±⎺┤├
C⎺┼┼ec├☃⎺┼ ├⎺ 1▮4↓248↓36↓8 c┌⎺⎽ed↓
▒d▒└@±⎽:·$
▒d▒└@±⎽:·$
▒d▒└@±⎽:·$
▒d▒└@±⎽:·$ ec▒⎺ '▒e┌┌⎺ ⎽├▒c┐ ⎺┴e⎼°┌⎺┬'
▒e┌┌⎺ ⎽├▒c┐ ⎺┴e⎼°┌⎺┬
▒d▒└@±⎽:·$
You can see 3 last "normal" GET requests to /, but then it begins. I know it can be fixed (link1 or link2) and these were some scanning bots, but my question is:
How does it work, that incoming request broke my terminal?
bash string http
asked Mar 13 at 16:41
adamcziadamczi
1315
1
Can you runcat -veton that log file and show us the lines from 12:08:36 and 14:55:55 ? There might be a Ctrl-N in there. en.wikipedia.org/wiki/Shift_Out_and_Shift_In_characters
– Mark Plotnick
Mar 13 at 17:09
1
It's possible that someone can guess the byte sequence that was printed, but it might help diagnose the precise byte sequence if you post a hex dump of the bytes that were sent to the terminal. eg:python run.py | tee run.logfollowed byhexdump -C run.log
– Philip Couling
Mar 13 at 17:10
Hi @MarkPlotnick, thanks for your comment. The thing is that I don't have those strings in any logfile, it just copy-pasted content straight from my terminal output
– adamczi
Mar 13 at 18:50
@PhilipCouling thanks for your suggestion, but as I wrote above, it has already happened, so I can only run it again and wait for the next try.
– adamczi
Mar 13 at 18:52
1
while I think JdeBP has a very good point in their answer, I'm voting to close this is as non-reproducible, because like the previous comments say, answering the question about the "bad character" would require seeing the actual data and you said you don't have that.
– ilkkachu
Mar 13 at 19:24
|
show 1 more comment
I set up a new, basic Linux server (CentOS in this case) just for testing purposes. In there, without any firewalls and all that I'm running a Python web application. Basically I type python run.py and have the application on foreground, while the application itself is run on some port like 8080, so in web browser I use I just type my_public_ip_addr:8080 and use it just fine. All this happens over SSH from my laptop.
Now, I left my laptop open for a while, and when I came back, the shell was displaying something like this:
83.20.238.86 - - [13/Mar/2019 08:54:43] "GET / HTTP/1.1" 200 -
87.122.83.97 - - [13/Mar/2019 11:55:30] "GET / HTTP/1.1" 200 -
176.32.33.145 - - [13/Mar/2019 12:08:36] "GET / HTTP/1.1" 200 -
176.32.33.145 - - [13/Mar/2019 12:08:36] code 400, message Bad request syntax ('x16x03x01x00xfcx01x00x00xf8x03x03(xd3FMxf5x0eLox17xa3|x1f8xca~#x07xc1x1f&&x14x19x11x10:x824xd23nAx00x00x8cxc00xc0,xc02xc0.xc0/xc0+xc01xc0-x00xa5x00xa3x00xa1x00x9fx00xa4x00xa2x00xa0x00x9exc0(xc0$xc0x14xc0')
176.32.33.145 - - [13/Mar/2019 12:08:36] "��(M�L⎺�≠8#�&&:�4┼A��▮�←�2�↓�/�→�1�↑���������(�$��" 4▮▮ ↑
176↓32↓33↓145 ↑ ↑ [13/M▒⎼/2▮19 14:55:55] "GET / HTTP/1↓1" 2▮▮ ↑
176↓32↓33↓145 ↑ ↑ [13/M▒⎼/2▮19 14:55:55] c⎺de 4▮▮← └e⎽⎽▒±e B▒d ⎼e─┤e⎽├ ⎽≤┼├▒│ ('│16│▮3│▮1│▮▮│°c│▮1│▮▮│▮▮│°8│▮3│▮3│92│8e│°7│9e│1▒│▒2│1e│°8│°bb^│1b│d1│▒1│1e│d2│d1^│1e/└│96_(│beU│▮4│8d≥│d7⎻│°e│▮▮│▮▮│8c│c▮▮│c▮←│c▮2│c▮↓│c▮/│c▮→│c▮1│c▮↑│▮▮│▒5│▮▮│▒3│▮▮│▒1│▮▮│9°│▮▮│▒4│▮▮│▒2│▮▮│▒▮│▮▮│9e│c▮(│c▮$│c▮│14│c▮')
176↓32↓33↓145 ↑ ↑ [13/M▒⎼/2▮19 14:55:55] "���������b^[⎺⎼▒┼±e@ce┼├⎺⎽↑⎺⎼▒┼±e ⎺⎼▒┼±e_±c]$ ^C
[⎺⎼▒┼±e@ce┼├⎺⎽↑⎺⎼▒┼±e ⎺⎼▒┼±e_±c]$ ┌⎺±⎺┤├
C⎺┼┼ec├☃⎺┼ ├⎺ 1▮4↓248↓36↓8 c┌⎺⎽ed↓
▒d▒└@±⎽:·$
▒d▒└@±⎽:·$
▒d▒└@±⎽:·$
▒d▒└@±⎽:·$ ec▒⎺ '▒e┌┌⎺ ⎽├▒c┐ ⎺┴e⎼°┌⎺┬'
▒e┌┌⎺ ⎽├▒c┐ ⎺┴e⎼°┌⎺┬
▒d▒└@±⎽:·$
You can see 3 last "normal" GET requests to /, but then it begins. I know it can be fixed (link1 or link2) and these were some scanning bots, but my question is:
How does it work, that incoming request broke my terminal?
bash string http
asked Mar 13 at 16:41
adamcziadamczi
1315
I set up a new, basic Linux server (CentOS in this case) just for testing purposes. In there, without any firewalls and all that I'm running a Python web application. Basically I type python run.py and have the application on foreground, while the application itself is run on some port like 8080, so in web browser I use I just type my_public_ip_addr:8080 and use it just fine. All this happens over SSH from my laptop.
Now, I left my laptop open for a while, and when I came back, the shell was displaying something like this:
83.20.238.86 - - [13/Mar/2019 08:54:43] "GET / HTTP/1.1" 200 -
87.122.83.97 - - [13/Mar/2019 11:55:30] "GET / HTTP/1.1" 200 -
176.32.33.145 - - [13/Mar/2019 12:08:36] "GET / HTTP/1.1" 200 -
176.32.33.145 - - [13/Mar/2019 12:08:36] code 400, message Bad request syntax ('x16x03x01x00xfcx01x00x00xf8x03x03(xd3FMxf5x0eLox17xa3|x1f8xca~#x07xc1x1f&&x14x19x11x10:x824xd23nAx00x00x8cxc00xc0,xc02xc0.xc0/xc0+xc01xc0-x00xa5x00xa3x00xa1x00x9fx00xa4x00xa2x00xa0x00x9exc0(xc0$xc0x14xc0')
176.32.33.145 - - [13/Mar/2019 12:08:36] "��(M�L⎺�≠8#�&&:�4┼A��▮�←�2�↓�/�→�1�↑���������(�$��" 4▮▮ ↑
176↓32↓33↓145 ↑ ↑ [13/M▒⎼/2▮19 14:55:55] "GET / HTTP/1↓1" 2▮▮ ↑
176↓32↓33↓145 ↑ ↑ [13/M▒⎼/2▮19 14:55:55] c⎺de 4▮▮← └e⎽⎽▒±e B▒d ⎼e─┤e⎽├ ⎽≤┼├▒│ ('│16│▮3│▮1│▮▮│°c│▮1│▮▮│▮▮│°8│▮3│▮3│92│8e│°7│9e│1▒│▒2│1e│°8│°bb^│1b│d1│▒1│1e│d2│d1^│1e/└│96_(│beU│▮4│8d≥│d7⎻│°e│▮▮│▮▮│8c│c▮▮│c▮←│c▮2│c▮↓│c▮/│c▮→│c▮1│c▮↑│▮▮│▒5│▮▮│▒3│▮▮│▒1│▮▮│9°│▮▮│▒4│▮▮│▒2│▮▮│▒▮│▮▮│9e│c▮(│c▮$│c▮│14│c▮')
176↓32↓33↓145 ↑ ↑ [13/M▒⎼/2▮19 14:55:55] "���������b^[⎺⎼▒┼±e@ce┼├⎺⎽↑⎺⎼▒┼±e ⎺⎼▒┼±e_±c]$ ^C
[⎺⎼▒┼±e@ce┼├⎺⎽↑⎺⎼▒┼±e ⎺⎼▒┼±e_±c]$ ┌⎺±⎺┤├
C⎺┼┼ec├☃⎺┼ ├⎺ 1▮4↓248↓36↓8 c┌⎺⎽ed↓
▒d▒└@±⎽:·$
▒d▒└@±⎽:·$
▒d▒└@±⎽:·$
▒d▒└@±⎽:·$ ec▒⎺ '▒e┌┌⎺ ⎽├▒c┐ ⎺┴e⎼°┌⎺┬'
▒e┌┌⎺ ⎽├▒c┐ ⎺┴e⎼°┌⎺┬
▒d▒└@±⎽:·$
You can see 3 last "normal" GET requests to /, but then it begins. I know it can be fixed (link1 or link2) and these were some scanning bots, but my question is:
How does it work, that incoming request broke my terminal?
bash string http
bash string http
asked Mar 13 at 16:41
adamcziadamczi
1315
asked Mar 13 at 16:41
adamcziadamczi
1315
edited Mar 14 at 10:55
adamczi
asked Mar 13 at 16:41
adamcziadamczi
1315
asked Mar 13 at 16:41
adamcziadamczi
1315
asked Mar 13 at 16:41
adamcziadamczi
1315
1315
1
Can you runcat -veton that log file and show us the lines from 12:08:36 and 14:55:55 ? There might be a Ctrl-N in there. en.wikipedia.org/wiki/Shift_Out_and_Shift_In_characters
– Mark Plotnick
Mar 13 at 17:09
1
It's possible that someone can guess the byte sequence that was printed, but it might help diagnose the precise byte sequence if you post a hex dump of the bytes that were sent to the terminal. eg:python run.py | tee run.logfollowed byhexdump -C run.log
– Philip Couling
Mar 13 at 17:10
Hi @MarkPlotnick, thanks for your comment. The thing is that I don't have those strings in any logfile, it just copy-pasted content straight from my terminal output
– adamczi
Mar 13 at 18:50
@PhilipCouling thanks for your suggestion, but as I wrote above, it has already happened, so I can only run it again and wait for the next try.
– adamczi
Mar 13 at 18:52
1
while I think JdeBP has a very good point in their answer, I'm voting to close this is as non-reproducible, because like the previous comments say, answering the question about the "bad character" would require seeing the actual data and you said you don't have that.
– ilkkachu
Mar 13 at 19:24
|
show 1 more comment
1
Can you runcat -veton that log file and show us the lines from 12:08:36 and 14:55:55 ? There might be a Ctrl-N in there. en.wikipedia.org/wiki/Shift_Out_and_Shift_In_characters
– Mark Plotnick
Mar 13 at 17:09
1
It's possible that someone can guess the byte sequence that was printed, but it might help diagnose the precise byte sequence if you post a hex dump of the bytes that were sent to the terminal. eg:python run.py | tee run.logfollowed byhexdump -C run.log
– Philip Couling
Mar 13 at 17:10
Hi @MarkPlotnick, thanks for your comment. The thing is that I don't have those strings in any logfile, it just copy-pasted content straight from my terminal output
– adamczi
Mar 13 at 18:50
@PhilipCouling thanks for your suggestion, but as I wrote above, it has already happened, so I can only run it again and wait for the next try.
– adamczi
Mar 13 at 18:52
1
while I think JdeBP has a very good point in their answer, I'm voting to close this is as non-reproducible, because like the previous comments say, answering the question about the "bad character" would require seeing the actual data and you said you don't have that.
– ilkkachu
Mar 13 at 19:24
1
1
Can you run
cat -vet on that log file and show us the lines from 12:08:36 and 14:55:55 ? There might be a Ctrl-N in there. en.wikipedia.org/wiki/Shift_Out_and_Shift_In_characters– Mark Plotnick
Mar 13 at 17:09
Can you run
cat -vet on that log file and show us the lines from 12:08:36 and 14:55:55 ? There might be a Ctrl-N in there. en.wikipedia.org/wiki/Shift_Out_and_Shift_In_characters– Mark Plotnick
Mar 13 at 17:09
1
1
It's possible that someone can guess the byte sequence that was printed, but it might help diagnose the precise byte sequence if you post a hex dump of the bytes that were sent to the terminal. eg:
python run.py | tee run.log followed by hexdump -C run.log– Philip Couling
Mar 13 at 17:10
It's possible that someone can guess the byte sequence that was printed, but it might help diagnose the precise byte sequence if you post a hex dump of the bytes that were sent to the terminal. eg:
python run.py | tee run.log followed by hexdump -C run.log– Philip Couling
Mar 13 at 17:10
Hi @MarkPlotnick, thanks for your comment. The thing is that I don't have those strings in any logfile, it just copy-pasted content straight from my terminal output
– adamczi
Mar 13 at 18:50
Hi @MarkPlotnick, thanks for your comment. The thing is that I don't have those strings in any logfile, it just copy-pasted content straight from my terminal output
– adamczi
Mar 13 at 18:50
@PhilipCouling thanks for your suggestion, but as I wrote above, it has already happened, so I can only run it again and wait for the next try.
– adamczi
Mar 13 at 18:52
@PhilipCouling thanks for your suggestion, but as I wrote above, it has already happened, so I can only run it again and wait for the next try.
– adamczi
Mar 13 at 18:52
1
1
while I think JdeBP has a very good point in their answer, I'm voting to close this is as non-reproducible, because like the previous comments say, answering the question about the "bad character" would require seeing the actual data and you said you don't have that.
– ilkkachu
Mar 13 at 19:24
while I think JdeBP has a very good point in their answer, I'm voting to close this is as non-reproducible, because like the previous comments say, answering the question about the "bad character" would require seeing the actual data and you said you don't have that.
– ilkkachu
Mar 13 at 19:24
|
show 1 more comment
1 Answer
1
active
oldest
votes
Let this be a lesson in security. Your program dumps network-supplied input directly to its log, as-is. You dumped the log output directly to a user terminal. You gave attackers out on Internet at large the ability to control output on your terminal.
Pipe your logs through cyclog and multilog or similar, as I explained at https://unix.stackexchange.com/a/505854/5132, so that they go to a set of strictly size-capped, automatically rotated, log files rather than to a terminal. Then read those log files using tools that will sanitize control characters.
The "bad characters" here are well known, and are standardized by ECMA-35 (a.k.a. ISO/IEC 2022) in conjunction with a large registry of character sets. Your terminal emulator implements two switchable portions of the 8-bit character set, known as "GL" and "GR". Various standard control characters and escape sequences switch these two amongst four designated character sets, known as "G0", "G1", "G2", and "G3". These four are in turn mapped to actual character sets by further escape sequences.
The set of byte sequences that can mess up your output is rather large. There are more than just ␎ and ␏, as question comments would lead you to believe. There are four possible shifts of two shiftable areas, and locking and single shifts. The C1 control characters for shifting have two representations. Then there are just under two hundred possible mapped character sets for each of the four shifts, each with their own escape sequence.
It's quite a complex system, and if you are at this point thinking "Surely it's better to just use Unicode?" you will not be the first. The inventors of mosh made it a selling point that their terminal emulator does not implement any of this character set switching. Neither does my console-terminal-emulator. Our terminal emulators simply will not get into these difficulties. Markus Kuhn has been encouraging dropping ISO 2022 character set switching since 1999.
Further reading
- Markus Kuhn (1999). "What are the issues related to UTF-8 terminal emulators?". UTF-8 and Unicode FAQ for Unix/Linux.
Character Code Structure and
Extension Techniques. ECMA-35. 6th edition. 1994. ECMA International.- Keith Winstein, Anders Kaseorg, et al. (2012). "ISO 2022 locking escapes". mosh technical info.
INTERNATIONAL REGISTER OF CODED CHARACTER SETS TO BE USED WITH ESCAPE SEQUENCES. itscj.ipsj.or.jp.- https://unix.stackexchange.com/a/289871/5132
- Jonathan de Boyne Pollard (2019).
console-terminal-emulator. nosh Guide. Softwares.
answered Mar 13 at 18:21
JdeBPJdeBP
37.5k478180
add a comment |
Your Answer
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "106"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f506123%2fbash-displaying-gibberish-after-specific-incoming-request%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
Let this be a lesson in security. Your program dumps network-supplied input directly to its log, as-is. You dumped the log output directly to a user terminal. You gave attackers out on Internet at large the ability to control output on your terminal.
Pipe your logs through cyclog and multilog or similar, as I explained at https://unix.stackexchange.com/a/505854/5132, so that they go to a set of strictly size-capped, automatically rotated, log files rather than to a terminal. Then read those log files using tools that will sanitize control characters.
The "bad characters" here are well known, and are standardized by ECMA-35 (a.k.a. ISO/IEC 2022) in conjunction with a large registry of character sets. Your terminal emulator implements two switchable portions of the 8-bit character set, known as "GL" and "GR". Various standard control characters and escape sequences switch these two amongst four designated character sets, known as "G0", "G1", "G2", and "G3". These four are in turn mapped to actual character sets by further escape sequences.
The set of byte sequences that can mess up your output is rather large. There are more than just ␎ and ␏, as question comments would lead you to believe. There are four possible shifts of two shiftable areas, and locking and single shifts. The C1 control characters for shifting have two representations. Then there are just under two hundred possible mapped character sets for each of the four shifts, each with their own escape sequence.
It's quite a complex system, and if you are at this point thinking "Surely it's better to just use Unicode?" you will not be the first. The inventors of mosh made it a selling point that their terminal emulator does not implement any of this character set switching. Neither does my console-terminal-emulator. Our terminal emulators simply will not get into these difficulties. Markus Kuhn has been encouraging dropping ISO 2022 character set switching since 1999.
Further reading
- Markus Kuhn (1999). "What are the issues related to UTF-8 terminal emulators?". UTF-8 and Unicode FAQ for Unix/Linux.
Character Code Structure and
Extension Techniques. ECMA-35. 6th edition. 1994. ECMA International.- Keith Winstein, Anders Kaseorg, et al. (2012). "ISO 2022 locking escapes". mosh technical info.
INTERNATIONAL REGISTER OF CODED CHARACTER SETS TO BE USED WITH ESCAPE SEQUENCES. itscj.ipsj.or.jp.- https://unix.stackexchange.com/a/289871/5132
- Jonathan de Boyne Pollard (2019).
console-terminal-emulator. nosh Guide. Softwares.
answered Mar 13 at 18:21
JdeBPJdeBP
37.5k478180
add a comment |
Let this be a lesson in security. Your program dumps network-supplied input directly to its log, as-is. You dumped the log output directly to a user terminal. You gave attackers out on Internet at large the ability to control output on your terminal.
Pipe your logs through cyclog and multilog or similar, as I explained at https://unix.stackexchange.com/a/505854/5132, so that they go to a set of strictly size-capped, automatically rotated, log files rather than to a terminal. Then read those log files using tools that will sanitize control characters.
The "bad characters" here are well known, and are standardized by ECMA-35 (a.k.a. ISO/IEC 2022) in conjunction with a large registry of character sets. Your terminal emulator implements two switchable portions of the 8-bit character set, known as "GL" and "GR". Various standard control characters and escape sequences switch these two amongst four designated character sets, known as "G0", "G1", "G2", and "G3". These four are in turn mapped to actual character sets by further escape sequences.
The set of byte sequences that can mess up your output is rather large. There are more than just ␎ and ␏, as question comments would lead you to believe. There are four possible shifts of two shiftable areas, and locking and single shifts. The C1 control characters for shifting have two representations. Then there are just under two hundred possible mapped character sets for each of the four shifts, each with their own escape sequence.
It's quite a complex system, and if you are at this point thinking "Surely it's better to just use Unicode?" you will not be the first. The inventors of mosh made it a selling point that their terminal emulator does not implement any of this character set switching. Neither does my console-terminal-emulator. Our terminal emulators simply will not get into these difficulties. Markus Kuhn has been encouraging dropping ISO 2022 character set switching since 1999.
Further reading
- Markus Kuhn (1999). "What are the issues related to UTF-8 terminal emulators?". UTF-8 and Unicode FAQ for Unix/Linux.
Character Code Structure and
Extension Techniques. ECMA-35. 6th edition. 1994. ECMA International.- Keith Winstein, Anders Kaseorg, et al. (2012). "ISO 2022 locking escapes". mosh technical info.
INTERNATIONAL REGISTER OF CODED CHARACTER SETS TO BE USED WITH ESCAPE SEQUENCES. itscj.ipsj.or.jp.- https://unix.stackexchange.com/a/289871/5132
- Jonathan de Boyne Pollard (2019).
console-terminal-emulator. nosh Guide. Softwares.
answered Mar 13 at 18:21
JdeBPJdeBP
37.5k478180
add a comment |
Let this be a lesson in security. Your program dumps network-supplied input directly to its log, as-is. You dumped the log output directly to a user terminal. You gave attackers out on Internet at large the ability to control output on your terminal.
Pipe your logs through cyclog and multilog or similar, as I explained at https://unix.stackexchange.com/a/505854/5132, so that they go to a set of strictly size-capped, automatically rotated, log files rather than to a terminal. Then read those log files using tools that will sanitize control characters.
The "bad characters" here are well known, and are standardized by ECMA-35 (a.k.a. ISO/IEC 2022) in conjunction with a large registry of character sets. Your terminal emulator implements two switchable portions of the 8-bit character set, known as "GL" and "GR". Various standard control characters and escape sequences switch these two amongst four designated character sets, known as "G0", "G1", "G2", and "G3". These four are in turn mapped to actual character sets by further escape sequences.
The set of byte sequences that can mess up your output is rather large. There are more than just ␎ and ␏, as question comments would lead you to believe. There are four possible shifts of two shiftable areas, and locking and single shifts. The C1 control characters for shifting have two representations. Then there are just under two hundred possible mapped character sets for each of the four shifts, each with their own escape sequence.
It's quite a complex system, and if you are at this point thinking "Surely it's better to just use Unicode?" you will not be the first. The inventors of mosh made it a selling point that their terminal emulator does not implement any of this character set switching. Neither does my console-terminal-emulator. Our terminal emulators simply will not get into these difficulties. Markus Kuhn has been encouraging dropping ISO 2022 character set switching since 1999.
Further reading
- Markus Kuhn (1999). "What are the issues related to UTF-8 terminal emulators?". UTF-8 and Unicode FAQ for Unix/Linux.
Character Code Structure and
Extension Techniques. ECMA-35. 6th edition. 1994. ECMA International.- Keith Winstein, Anders Kaseorg, et al. (2012). "ISO 2022 locking escapes". mosh technical info.
INTERNATIONAL REGISTER OF CODED CHARACTER SETS TO BE USED WITH ESCAPE SEQUENCES. itscj.ipsj.or.jp.- https://unix.stackexchange.com/a/289871/5132
- Jonathan de Boyne Pollard (2019).
console-terminal-emulator. nosh Guide. Softwares.
answered Mar 13 at 18:21
JdeBPJdeBP
37.5k478180
Let this be a lesson in security. Your program dumps network-supplied input directly to its log, as-is. You dumped the log output directly to a user terminal. You gave attackers out on Internet at large the ability to control output on your terminal.
Pipe your logs through cyclog and multilog or similar, as I explained at https://unix.stackexchange.com/a/505854/5132, so that they go to a set of strictly size-capped, automatically rotated, log files rather than to a terminal. Then read those log files using tools that will sanitize control characters.
The "bad characters" here are well known, and are standardized by ECMA-35 (a.k.a. ISO/IEC 2022) in conjunction with a large registry of character sets. Your terminal emulator implements two switchable portions of the 8-bit character set, known as "GL" and "GR". Various standard control characters and escape sequences switch these two amongst four designated character sets, known as "G0", "G1", "G2", and "G3". These four are in turn mapped to actual character sets by further escape sequences.
The set of byte sequences that can mess up your output is rather large. There are more than just ␎ and ␏, as question comments would lead you to believe. There are four possible shifts of two shiftable areas, and locking and single shifts. The C1 control characters for shifting have two representations. Then there are just under two hundred possible mapped character sets for each of the four shifts, each with their own escape sequence.
It's quite a complex system, and if you are at this point thinking "Surely it's better to just use Unicode?" you will not be the first. The inventors of mosh made it a selling point that their terminal emulator does not implement any of this character set switching. Neither does my console-terminal-emulator. Our terminal emulators simply will not get into these difficulties. Markus Kuhn has been encouraging dropping ISO 2022 character set switching since 1999.
Further reading
- Markus Kuhn (1999). "What are the issues related to UTF-8 terminal emulators?". UTF-8 and Unicode FAQ for Unix/Linux.
Character Code Structure and
Extension Techniques. ECMA-35. 6th edition. 1994. ECMA International.- Keith Winstein, Anders Kaseorg, et al. (2012). "ISO 2022 locking escapes". mosh technical info.
INTERNATIONAL REGISTER OF CODED CHARACTER SETS TO BE USED WITH ESCAPE SEQUENCES. itscj.ipsj.or.jp.- https://unix.stackexchange.com/a/289871/5132
- Jonathan de Boyne Pollard (2019).
console-terminal-emulator. nosh Guide. Softwares.
answered Mar 13 at 18:21
JdeBPJdeBP
37.5k478180
answered Mar 13 at 18:21
JdeBPJdeBP
37.5k478180
answered Mar 13 at 18:21
JdeBPJdeBP
37.5k478180
answered Mar 13 at 18:21
JdeBPJdeBP
37.5k478180
37.5k478180
add a comment |
add a comment |
Thanks for contributing an answer to Unix & Linux Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f506123%2fbash-displaying-gibberish-after-specific-incoming-request%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
-bash, http, string
1
Can you run
cat -veton that log file and show us the lines from 12:08:36 and 14:55:55 ? There might be a Ctrl-N in there. en.wikipedia.org/wiki/Shift_Out_and_Shift_In_characters– Mark Plotnick
Mar 13 at 17:09
1
It's possible that someone can guess the byte sequence that was printed, but it might help diagnose the precise byte sequence if you post a hex dump of the bytes that were sent to the terminal. eg:
python run.py | tee run.logfollowed byhexdump -C run.log– Philip Couling
Mar 13 at 17:10
Hi @MarkPlotnick, thanks for your comment. The thing is that I don't have those strings in any logfile, it just copy-pasted content straight from my terminal output
– adamczi
Mar 13 at 18:50
@PhilipCouling thanks for your suggestion, but as I wrote above, it has already happened, so I can only run it again and wait for the next try.
– adamczi
Mar 13 at 18:52
1
while I think JdeBP has a very good point in their answer, I'm voting to close this is as non-reproducible, because like the previous comments say, answering the question about the "bad character" would require seeing the actual data and you said you don't have that.
– ilkkachu
Mar 13 at 19:24