.

Combining Wordlists

<<

Seen

User avatar

Full Member
Full Member

Posts: 137

Joined: Mon Aug 30, 2010 1:05 am

Post Wed Feb 01, 2012 2:21 pm

Combining Wordlists

What's the best way to combine wordlists?  Just cat them all, and then run sort/uniq?  I've noticed some wordlists have duplicate entries, is this because some password crackers (like Hydra) can miss words?  If I run uniq this would remove all the duplicates, would it be better to just add the lines that aren't in one file to another when combining them?  Sorry for the newbie questions, I've never made a custom wordlist before!
Sec+, eCPPT
<<

dynamik

Recruiters
Recruiters

Posts: 1119

Joined: Sun Nov 09, 2008 11:00 am

Location: Mile High City

Post Wed Feb 01, 2012 2:50 pm

Re: Combining Wordlists

I'm not sure about the duplicates; Hydra, John, etc. aren't going to randomly skip lines.

cat and uniq will do the job of getting everything into one file without duplicates. However, if the word lists are sorted by popularity/frequency, sorting those words into a different order may hurt your efficiency. In that case, you may want to write a script that pulls one line from one file, then one line from the other, (or two from one file, then one from the other, etc., depending on distribution), and then merge things together that way. It totally depends on the situation.
The day you stop learning is the day you start becoming obsolete.
<<

Seen

User avatar

Full Member
Full Member

Posts: 137

Joined: Mon Aug 30, 2010 1:05 am

Post Wed Feb 01, 2012 6:58 pm

Re: Combining Wordlists

Thanks dynamik, I'll try different orderings of my list and see if it makes any difference.
Sec+, eCPPT
<<

dynamik

Recruiters
Recruiters

Posts: 1119

Joined: Sun Nov 09, 2008 11:00 am

Location: Mile High City

Post Wed Feb 01, 2012 7:50 pm

Re: Combining Wordlists

It may, but that would probably just be circumstantial without any in-depth statistical analysis being performed. The point I was making was that you don't want to take a word list where you know passwords are indeed organized by popularity and then run it through sort.
The day you stop learning is the day you start becoming obsolete.
<<

millwalll

Post Thu Feb 02, 2012 4:34 am

Re: Combining Wordlists

I do cat * > wordlist.txt | sort -u

Return to Tools

Who is online

Users browsing this forum: No registered users and 0 guests

.
Powered by phpBB® Forum Software © phpBB Group.
Designed by ST Software