Ttyjfyk

What do you call the main part of a joke?

Do I really need to have a message in a novel to appeal to readers?

Around usage results

Why aren't air breathing engines used as small first stages

Why do we bend a book to keep it straight?

Is it ethical to give a final exam after the professor has quit before teaching the remaining chapters of the course?

How to react to hostile behavior from a senior developer?

Denied boarding although I have proper visa and documentation. To whom should I make a complaint?

Can melee weapons be used to deliver Contact Poisons?

How to answer "Have you ever been terminated?"

Is there a holomorphic function on open unit disc with this property?

Why do the resolve message appear first?

また usage in a dictionary

Do I really need recursive chmod to restrict access to a folder?

Trademark violation for app?

Using et al. for a last / senior author rather than for a first author

What is the escape velocity of a neutron particle (not neutron star)

When a candle burns, why does the top of wick glow if bottom of flame is hottest?

Is it a good idea to use CNN to classify 1D signal?

Why are both D and D# fitting into my E minor key?

Can an alien society believe that their star system is the universe?

Circuit to "zoom in" on mV fluctuations of a DC signal?

Irreducible of finite Krull dimension implies quasi-compact?

Is safe to use va_start macro with this as parameter?

find a pair of words that appear the most of the times together

.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty margin-bottom:0;

1

1

I have 10 text files, in each file i have a chapter from a book, i want to find the pair of words that appear the most of the time together in a line i.e:

chapter1:

hello world good boy green sun

good green boy sun world hello

chapter2:

chapter3:

.....etc

Output wanted for chapter1:

hello world (alphabet order)

linux text-processing awk sed grep

share|improve this question

edited 7 hours ago

mosvy

10.1k11237

asked 9 hours ago

John BJohn B

62

New contributor

John B is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.

• 
  
  
  

  
  3
  

  
  
  
  
  
  

  
  
  The output should have "boy green" also, right?
  
  – Guru
  9 hours ago
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  yes "boy green" also, didn't see
  
  – John B
  9 hours ago
  

  
  
  
  


• 
  
  
  

  
  1
  

  
  
  
  
  
  

  
  
  what about if you had a line like Hello world Hello? it should come in output? and how ? Hello world or world Hello?
  
  – αғsнιη
  9 hours ago
  
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  hello world , and this will count for 2 to the pair "hello world" always alphabetical
  
  – John B
  9 hours ago
  

  
  
  
  


• 
  
  
  

  
  1
  

  
  
  
  
  
  

  
  
  how about HellonworldnHello? n is actual new-line character. please edit your question to answer for comments asking for clarifications
  
  – αғsнιη
  8 hours ago
  
  

  
  
  
  

 | 
show 1 more comment

1

1

I have 10 text files, in each file i have a chapter from a book, i want to find the pair of words that appear the most of the time together in a line i.e:

chapter1:

hello world good boy green sun

good green boy sun world hello

chapter2:

chapter3:

.....etc

Output wanted for chapter1:

hello world (alphabet order)

linux text-processing awk sed grep

share|improve this question

edited 7 hours ago

mosvy

10.1k11237

asked 9 hours ago

John BJohn B

62

New contributor

John B is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.

• 
  
  
  

  
  3
  

  
  
  
  
  
  

  
  
  The output should have "boy green" also, right?
  
  – Guru
  9 hours ago
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  yes "boy green" also, didn't see
  
  – John B
  9 hours ago
  

  
  
  
  


• 
  
  
  

  
  1
  

  
  
  
  
  
  

  
  
  what about if you had a line like Hello world Hello? it should come in output? and how ? Hello world or world Hello?
  
  – αғsнιη
  9 hours ago
  
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  hello world , and this will count for 2 to the pair "hello world" always alphabetical
  
  – John B
  9 hours ago
  

  
  
  
  


• 
  
  
  

  
  1
  

  
  
  
  
  
  

  
  
  how about HellonworldnHello? n is actual new-line character. please edit your question to answer for comments asking for clarifications
  
  – αғsнιη
  8 hours ago
  
  

  
  
  
  

 | 
show 1 more comment

I have 10 text files, in each file i have a chapter from a book, i want to find the pair of words that appear the most of the time together in a line i.e:

chapter1:

hello world good boy green sun

good green boy sun world hello

chapter2:

chapter3:

.....etc

Output wanted for chapter1:

hello world (alphabet order)

linux text-processing awk sed grep

share|improve this question

edited 7 hours ago

mosvy

10.1k11237

asked 9 hours ago

John BJohn B

62

New contributor

John B is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.

hello world good boy green sun

good green boy sun world hello

hello world (alphabet order)

share|improve this question

edited 7 hours ago

mosvy

10.1k11237

asked 9 hours ago

John BJohn B

62

New contributor

John B is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.

share|improve this question

edited 7 hours ago

mosvy

10.1k11237

asked 9 hours ago

John BJohn B

62

New contributor

John B is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.

edited 7 hours ago

mosvy

10.1k11237

edited 7 hours ago

mosvy

10.1k11237

asked 9 hours ago

John BJohn B

62

New contributor

John B is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.

asked 9 hours ago

John BJohn B

62

2 Answers
2

active

oldest

votes

0

Try this,

1. Use awk to print each pair of words.


2. Use perl to sort the words in a pair (via).


3. Use sort and uniq -c to count occurrences each pair.

---

awk 'for (i=1;i<NF;i++) print tolower($i)" "tolower($(i+1)) ' file 
| perl -ane '$,=" "; print sort @F; print "n";' 
| sort | uniq -c | sort -b -k1nr -k2

Output:

 2 boy green
 2 hello world
 1 boy good
 1 boy sun
 1 good green
 1 good world
 1 green sun
 1 sun world

share|improve this answer

edited 8 hours ago

answered 8 hours ago

RoVoRoVo

3,960317

• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  can not use perl or pipeline..
  
  – John B
  8 hours ago
  

  
  
  
  


• 
  
  
  

  
  2
  

  
  
  
  
  
  

  
  
  Why can't you, @John? Those are standard utilities on most Linux systems.
  
  – Jeff Schaller♦
  8 hours ago
  

  
  
  
  


• 
  
  
  

  
  1
  

  
  
  
  
  
  

  
  
  Yes, this kind of information should be in your question.
  
  – RoVo
  8 hours ago
  

  
  
  
  

add a comment | 

0

awk '
 
 $0 = tolower($0)
 for (i = 1; i < NF; i++) 
 pair = $i"" < $(i+1) ? $i" "$(i+1) : $(i+1)" "$i
 c = ++count[pair]
 if (c > max) max = c
 
 
 END 
 for (pair in count)
 if (count[pair] == max)
 print pair
 '

share|improve this answer

answered 7 hours ago

Stéphane ChazelasStéphane Chazelas

315k57597955

add a comment | 

Your Answer

StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "106"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);

else
createEditor();

);

function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);



);

John B is a new contributor. Be nice, and check out our Code of Conduct.

draft saved

draft discarded

Sign up or log in

StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);

Sign up using Google

Sign up using Facebook

Sign up using Email and Password

Post as a guest

Name

Email

Required, but never shown

StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f513022%2ffind-a-pair-of-words-that-appear-the-most-of-the-times-together%23new-answer', 'question_page');

);

Post as a guest

Name

Email

Required, but never shown

0

Try this,

1. Use awk to print each pair of words.


2. Use perl to sort the words in a pair (via).


3. Use sort and uniq -c to count occurrences each pair.

---

awk 'for (i=1;i<NF;i++) print tolower($i)" "tolower($(i+1)) ' file 
| perl -ane '$,=" "; print sort @F; print "n";' 
| sort | uniq -c | sort -b -k1nr -k2

Output:

 2 boy green
 2 hello world
 1 boy good
 1 boy sun
 1 good green
 1 good world
 1 green sun
 1 sun world

share|improve this answer

edited 8 hours ago

answered 8 hours ago

RoVoRoVo

3,960317

• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  can not use perl or pipeline..
  
  – John B
  8 hours ago
  

  
  
  
  


• 
  
  
  

  
  2
  

  
  
  
  
  
  

  
  
  Why can't you, @John? Those are standard utilities on most Linux systems.
  
  – Jeff Schaller♦
  8 hours ago
  

  
  
  
  


• 
  
  
  

  
  1
  

  
  
  
  
  
  

  
  
  Yes, this kind of information should be in your question.
  
  – RoVo
  8 hours ago
  

  
  
  
  

add a comment | 

0

Try this,

1. Use awk to print each pair of words.


2. Use perl to sort the words in a pair (via).


3. Use sort and uniq -c to count occurrences each pair.

---

awk 'for (i=1;i<NF;i++) print tolower($i)" "tolower($(i+1)) ' file 
| perl -ane '$,=" "; print sort @F; print "n";' 
| sort | uniq -c | sort -b -k1nr -k2

Output:

 2 boy green
 2 hello world
 1 boy good
 1 boy sun
 1 good green
 1 good world
 1 green sun
 1 sun world

share|improve this answer

edited 8 hours ago

answered 8 hours ago

RoVoRoVo

3,960317

• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  can not use perl or pipeline..
  
  – John B
  8 hours ago
  

  
  
  
  


• 
  
  
  

  
  2
  

  
  
  
  
  
  

  
  
  Why can't you, @John? Those are standard utilities on most Linux systems.
  
  – Jeff Schaller♦
  8 hours ago
  

  
  
  
  


• 
  
  
  

  
  1
  

  
  
  
  
  
  

  
  
  Yes, this kind of information should be in your question.
  
  – RoVo
  8 hours ago
  

  
  
  
  

add a comment | 

Try this,

1. Use awk to print each pair of words.


2. Use perl to sort the words in a pair (via).


3. Use sort and uniq -c to count occurrences each pair.

---

awk 'for (i=1;i<NF;i++) print tolower($i)" "tolower($(i+1)) ' file 
| perl -ane '$,=" "; print sort @F; print "n";' 
| sort | uniq -c | sort -b -k1nr -k2

Output:

 2 boy green
 2 hello world
 1 boy good
 1 boy sun
 1 good green
 1 good world
 1 green sun
 1 sun world

share|improve this answer

edited 8 hours ago

answered 8 hours ago

RoVoRoVo

3,960317

awk 'for (i=1;i<NF;i++) print tolower($i)" "tolower($(i+1)) ' file 
| perl -ane '$,=" "; print sort @F; print "n";' 
| sort | uniq -c | sort -b -k1nr -k2

 2 boy green
 2 hello world
 1 boy good
 1 boy sun
 1 good green
 1 good world
 1 green sun
 1 sun world

share|improve this answer

edited 8 hours ago

answered 8 hours ago

RoVoRoVo

3,960317

answered 8 hours ago

RoVoRoVo

3,960317

answered 8 hours ago

RoVoRoVo

3,960317

0

awk '
 
 $0 = tolower($0)
 for (i = 1; i < NF; i++) 
 pair = $i"" < $(i+1) ? $i" "$(i+1) : $(i+1)" "$i
 c = ++count[pair]
 if (c > max) max = c
 
 
 END 
 for (pair in count)
 if (count[pair] == max)
 print pair
 '

share|improve this answer

answered 7 hours ago

Stéphane ChazelasStéphane Chazelas

315k57597955

add a comment | 

0

awk '
 
 $0 = tolower($0)
 for (i = 1; i < NF; i++) 
 pair = $i"" < $(i+1) ? $i" "$(i+1) : $(i+1)" "$i
 c = ++count[pair]
 if (c > max) max = c
 
 
 END 
 for (pair in count)
 if (count[pair] == max)
 print pair
 '

share|improve this answer

answered 7 hours ago

Stéphane ChazelasStéphane Chazelas

315k57597955

add a comment | 

awk '
 
 $0 = tolower($0)
 for (i = 1; i < NF; i++) 
 pair = $i"" < $(i+1) ? $i" "$(i+1) : $(i+1)" "$i
 c = ++count[pair]
 if (c > max) max = c
 
 
 END 
 for (pair in count)
 if (count[pair] == max)
 print pair
 '

share|improve this answer

answered 7 hours ago

Stéphane ChazelasStéphane Chazelas

315k57597955

awk '
 
 $0 = tolower($0)
 for (i = 1; i < NF; i++) 
 pair = $i"" < $(i+1) ? $i" "$(i+1) : $(i+1)" "$i
 c = ++count[pair]
 if (c > max) max = c
 
 
 END 
 for (pair in count)
 if (count[pair] == max)
 print pair
 '

share|improve this answer

answered 7 hours ago

Stéphane ChazelasStéphane Chazelas

315k57597955

answered 7 hours ago

Stéphane ChazelasStéphane Chazelas

315k57597955

answered 7 hours ago

Stéphane ChazelasStéphane Chazelas

315k57597955