Ttyjfyk

2001: A Space Odyssey's use of the song "Daisy Bell" (Bicycle Built for Two); life imitates art or vice-versa?

Is CEO the profession with the most psychopaths?

If my PI received research grants from a company to be able to pay my postdoc salary, did I have a potential conflict interest too?

Why wasn't DOSKEY integrated with COMMAND.COM?

Can you use the Shield Master feat to shove someone before you make an attack by using a Readied action?

Is there a kind of relay only consumes power when switching?

Is safe to use va_start macro with this as parameter?

Withdrew £2800, but only £2000 shows as withdrawn on online banking; what are my obligations?

Circuit to "zoom in" on mV fluctuations of a DC signal?

8 Prisoners wearing hats

How to react to hostile behavior from a senior developer?

Can melee weapons be used to deliver Contact Poisons?

Around usage results

For a new assistant professor in CS, how to build/manage a publication pipeline

If a VARCHAR(MAX) column is included in an index, is the entire value always stored in the index page(s)?

Do jazz musicians improvise on the parent scale in addition to the chord-scales?

When was Kai Tak permanently closed to cargo service?

How could we fake a moon landing now?

In what way is everyone not a utilitarian

Did MS DOS itself ever use blinking text?

How come Sam didn't become Lord of Horn Hill?

How can I use the Python library networkx from Mathematica?

Compare a given version number in the form major.minor.build.patch and see if one is less than the other

Can a party unilaterally change candidates in preparation for a General election?

find a pair of words that appear the most of the times together

.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty margin-bottom:0;

1

1

I have 10 text files, in each file i have a chapter from a book, i want to find the pair of words that appear the most of the time together in a line i.e:

chapter1:

hello world good boy green sun

good green boy sun world hello

chapter2:

chapter3:

.....etc

Output wanted for chapter1:

hello world (alphabet order)

linux text-processing awk sed grep

share|improve this question

edited 10 hours ago

mosvy

10.2k11237

asked 12 hours ago

John BJohn B

62

New contributor

John B is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.

• 
  
  
  

  
  3
  

  
  
  
  
  
  

  
  
  The output should have "boy green" also, right?
  
  – Guru
  11 hours ago
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  yes "boy green" also, didn't see
  
  – John B
  11 hours ago
  

  
  
  
  


• 
  
  
  

  
  1
  

  
  
  
  
  
  

  
  
  what about if you had a line like Hello world Hello? it should come in output? and how ? Hello world or world Hello?
  
  – αғsнιη
  11 hours ago
  
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  hello world , and this will count for 2 to the pair "hello world" always alphabetical
  
  – John B
  11 hours ago
  

  
  
  
  


• 
  
  
  

  
  1
  

  
  
  
  
  
  

  
  
  how about HellonworldnHello? n is actual new-line character. please edit your question to answer for comments asking for clarifications
  
  – αғsнιη
  10 hours ago
  
  

  
  
  
  

 | 
show 1 more comment

1

1

I have 10 text files, in each file i have a chapter from a book, i want to find the pair of words that appear the most of the time together in a line i.e:

chapter1:

hello world good boy green sun

good green boy sun world hello

chapter2:

chapter3:

.....etc

Output wanted for chapter1:

hello world (alphabet order)

linux text-processing awk sed grep

share|improve this question

edited 10 hours ago

mosvy

10.2k11237

asked 12 hours ago

John BJohn B

62

New contributor

John B is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.

• 
  
  
  

  
  3
  

  
  
  
  
  
  

  
  
  The output should have "boy green" also, right?
  
  – Guru
  11 hours ago
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  yes "boy green" also, didn't see
  
  – John B
  11 hours ago
  

  
  
  
  


• 
  
  
  

  
  1
  

  
  
  
  
  
  

  
  
  what about if you had a line like Hello world Hello? it should come in output? and how ? Hello world or world Hello?
  
  – αғsнιη
  11 hours ago
  
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  hello world , and this will count for 2 to the pair "hello world" always alphabetical
  
  – John B
  11 hours ago
  

  
  
  
  


• 
  
  
  

  
  1
  

  
  
  
  
  
  

  
  
  how about HellonworldnHello? n is actual new-line character. please edit your question to answer for comments asking for clarifications
  
  – αғsнιη
  10 hours ago
  
  

  
  
  
  

 | 
show 1 more comment

I have 10 text files, in each file i have a chapter from a book, i want to find the pair of words that appear the most of the time together in a line i.e:

chapter1:

hello world good boy green sun

good green boy sun world hello

chapter2:

chapter3:

.....etc

Output wanted for chapter1:

hello world (alphabet order)

linux text-processing awk sed grep

share|improve this question

edited 10 hours ago

mosvy

10.2k11237

asked 12 hours ago

John BJohn B

62

New contributor

John B is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.

hello world good boy green sun

good green boy sun world hello

hello world (alphabet order)

share|improve this question

edited 10 hours ago

mosvy

10.2k11237

asked 12 hours ago

John BJohn B

62

New contributor

John B is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.

share|improve this question

edited 10 hours ago

mosvy

10.2k11237

asked 12 hours ago

John BJohn B

62

New contributor

John B is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.

edited 10 hours ago

mosvy

10.2k11237

edited 10 hours ago

mosvy

10.2k11237

asked 12 hours ago

John BJohn B

62

New contributor

John B is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.

asked 12 hours ago

John BJohn B

62

2 Answers
2

active

oldest

votes

0

Try this,

1. Use awk to print each pair of words.


2. Use perl to sort the words in a pair (via).


3. Use sort and uniq -c to count occurrences each pair.

---

awk 'for (i=1;i<NF;i++) print tolower($i)" "tolower($(i+1)) ' file 
| perl -ane '$,=" "; print sort @F; print "n";' 
| sort | uniq -c | sort -b -k1nr -k2

Output:

 2 boy green
 2 hello world
 1 boy good
 1 boy sun
 1 good green
 1 good world
 1 green sun
 1 sun world

share|improve this answer

edited 10 hours ago

answered 11 hours ago

RoVoRoVo

3,960317

• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  can not use perl or pipeline..
  
  – John B
  10 hours ago
  

  
  
  
  


• 
  
  
  

  
  2
  

  
  
  
  
  
  

  
  
  Why can't you, @John? Those are standard utilities on most Linux systems.
  
  – Jeff Schaller♦
  10 hours ago
  

  
  
  
  


• 
  
  
  

  
  1
  

  
  
  
  
  
  

  
  
  Yes, this kind of information should be in your question.
  
  – RoVo
  10 hours ago
  

  
  
  
  

add a comment | 

0

awk '
 
 $0 = tolower($0)
 for (i = 1; i < NF; i++) 
 pair = $i"" < $(i+1) ? $i" "$(i+1) : $(i+1)" "$i
 c = ++count[pair]
 if (c > max) max = c
 
 
 END 
 for (pair in count)
 if (count[pair] == max)
 print pair
 '

share|improve this answer

answered 10 hours ago

Stéphane ChazelasStéphane Chazelas

315k57597955

add a comment | 

Your Answer

StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "106"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);

else
createEditor();

);

function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);



);

John B is a new contributor. Be nice, and check out our Code of Conduct.

draft saved

draft discarded

Sign up or log in

StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);

Sign up using Google

Sign up using Facebook

Sign up using Email and Password

Post as a guest

Name

Email

Required, but never shown

StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f513022%2ffind-a-pair-of-words-that-appear-the-most-of-the-times-together%23new-answer', 'question_page');

);

Post as a guest

Name

Email

Required, but never shown

0

Try this,

1. Use awk to print each pair of words.


2. Use perl to sort the words in a pair (via).


3. Use sort and uniq -c to count occurrences each pair.

---

awk 'for (i=1;i<NF;i++) print tolower($i)" "tolower($(i+1)) ' file 
| perl -ane '$,=" "; print sort @F; print "n";' 
| sort | uniq -c | sort -b -k1nr -k2

Output:

 2 boy green
 2 hello world
 1 boy good
 1 boy sun
 1 good green
 1 good world
 1 green sun
 1 sun world

share|improve this answer

edited 10 hours ago

answered 11 hours ago

RoVoRoVo

3,960317

• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  can not use perl or pipeline..
  
  – John B
  10 hours ago
  

  
  
  
  


• 
  
  
  

  
  2
  

  
  
  
  
  
  

  
  
  Why can't you, @John? Those are standard utilities on most Linux systems.
  
  – Jeff Schaller♦
  10 hours ago
  

  
  
  
  


• 
  
  
  

  
  1
  

  
  
  
  
  
  

  
  
  Yes, this kind of information should be in your question.
  
  – RoVo
  10 hours ago
  

  
  
  
  

add a comment | 

0

Try this,

1. Use awk to print each pair of words.


2. Use perl to sort the words in a pair (via).


3. Use sort and uniq -c to count occurrences each pair.

---

awk 'for (i=1;i<NF;i++) print tolower($i)" "tolower($(i+1)) ' file 
| perl -ane '$,=" "; print sort @F; print "n";' 
| sort | uniq -c | sort -b -k1nr -k2

Output:

 2 boy green
 2 hello world
 1 boy good
 1 boy sun
 1 good green
 1 good world
 1 green sun
 1 sun world

share|improve this answer

edited 10 hours ago

answered 11 hours ago

RoVoRoVo

3,960317

• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  can not use perl or pipeline..
  
  – John B
  10 hours ago
  

  
  
  
  


• 
  
  
  

  
  2
  

  
  
  
  
  
  

  
  
  Why can't you, @John? Those are standard utilities on most Linux systems.
  
  – Jeff Schaller♦
  10 hours ago
  

  
  
  
  


• 
  
  
  

  
  1
  

  
  
  
  
  
  

  
  
  Yes, this kind of information should be in your question.
  
  – RoVo
  10 hours ago
  

  
  
  
  

add a comment | 

Try this,

1. Use awk to print each pair of words.


2. Use perl to sort the words in a pair (via).


3. Use sort and uniq -c to count occurrences each pair.

---

awk 'for (i=1;i<NF;i++) print tolower($i)" "tolower($(i+1)) ' file 
| perl -ane '$,=" "; print sort @F; print "n";' 
| sort | uniq -c | sort -b -k1nr -k2

Output:

 2 boy green
 2 hello world
 1 boy good
 1 boy sun
 1 good green
 1 good world
 1 green sun
 1 sun world

share|improve this answer

edited 10 hours ago

answered 11 hours ago

RoVoRoVo

3,960317

awk 'for (i=1;i<NF;i++) print tolower($i)" "tolower($(i+1)) ' file 
| perl -ane '$,=" "; print sort @F; print "n";' 
| sort | uniq -c | sort -b -k1nr -k2

 2 boy green
 2 hello world
 1 boy good
 1 boy sun
 1 good green
 1 good world
 1 green sun
 1 sun world

share|improve this answer

edited 10 hours ago

answered 11 hours ago

RoVoRoVo

3,960317

answered 11 hours ago

RoVoRoVo

3,960317

answered 11 hours ago

RoVoRoVo

3,960317

0

awk '
 
 $0 = tolower($0)
 for (i = 1; i < NF; i++) 
 pair = $i"" < $(i+1) ? $i" "$(i+1) : $(i+1)" "$i
 c = ++count[pair]
 if (c > max) max = c
 
 
 END 
 for (pair in count)
 if (count[pair] == max)
 print pair
 '

share|improve this answer

answered 10 hours ago

Stéphane ChazelasStéphane Chazelas

315k57597955

add a comment | 

0

awk '
 
 $0 = tolower($0)
 for (i = 1; i < NF; i++) 
 pair = $i"" < $(i+1) ? $i" "$(i+1) : $(i+1)" "$i
 c = ++count[pair]
 if (c > max) max = c
 
 
 END 
 for (pair in count)
 if (count[pair] == max)
 print pair
 '

share|improve this answer

answered 10 hours ago

Stéphane ChazelasStéphane Chazelas

315k57597955

add a comment | 

awk '
 
 $0 = tolower($0)
 for (i = 1; i < NF; i++) 
 pair = $i"" < $(i+1) ? $i" "$(i+1) : $(i+1)" "$i
 c = ++count[pair]
 if (c > max) max = c
 
 
 END 
 for (pair in count)
 if (count[pair] == max)
 print pair
 '

share|improve this answer

answered 10 hours ago

Stéphane ChazelasStéphane Chazelas

315k57597955

awk '
 
 $0 = tolower($0)
 for (i = 1; i < NF; i++) 
 pair = $i"" < $(i+1) ? $i" "$(i+1) : $(i+1)" "$i
 c = ++count[pair]
 if (c > max) max = c
 
 
 END 
 for (pair in count)
 if (count[pair] == max)
 print pair
 '

share|improve this answer

answered 10 hours ago

Stéphane ChazelasStéphane Chazelas

315k57597955

answered 10 hours ago

Stéphane ChazelasStéphane Chazelas

315k57597955

answered 10 hours ago

Stéphane ChazelasStéphane Chazelas

315k57597955