I'm not sure about the duplicates; Hydra, John, etc. aren't going to randomly skip lines.
cat and uniq will do the job of getting everything into one file without duplicates. However, if the word lists are sorted by popularity/frequency, sorting those words into a different order may hurt your efficiency. In that case, you may want to write a script that pulls one line from one file, then one line from the other, (or two from one file, then one from the other, etc., depending on distribution), and then merge things together that way. It totally depends on the situation.
The day you stop learning is the day you start becoming obsolete.