EH-Net

What's the best way to combine wordlists?  Just cat them all, and then run sort/uniq?  I've noticed some wordlists have duplicate entries, is this because some password crackers (like Hydra) can miss words?  If I run uniq this would remove all the duplicates, would it be better to just add the lines that aren't in one file to another when combining them?  Sorry for the newbie questions, I've never made a custom wordlist before!

I'm not sure about the duplicates; Hydra, John, etc. aren't going to randomly skip lines.

cat and uniq will do the job of getting everything into one file without duplicates. However, if the word lists are sorted by popularity/frequency, sorting those words into a different order may hurt your efficiency. In that case, you may want to write a script that pulls one line from one file, then one line from the other, (or two from one file, then one from the other, etc., depending on distribution), and then merge things together that way. It totally depends on the situation.

Thanks dynamik, I'll try different orderings of my list and see if it makes any difference.

It may, but that would probably just be circumstantial without any in-depth statistical analysis being performed. The point I was making was that you don't want to take a word list where you know passwords are indeed organized by popularity and then run it through sort.

I do cat * > wordlist.txt | sort -u