The Art of Software Security Assessment by Dowd, McDonald and Schuh
-Wonderful overall assessment of the modern state of security (this book is HUGE)
Network Security Assessment by Chris McNabb
-This O'Reilly book is one of the best hands on guides I've found.
Linux Hacker Tools by Ivan Sklyarov
-This book explains how to build tools yourself, and in the process explores a lot of the underpinnings of many such tools.
Hacking, the Art of Exploitation by Erickson
-This is a great book that goes through a lot of hands on exercises valuable to penn testers.
Security in Computing by Pfleeger and Pfleeger
-The obligatory textbook to cover everything not covered above

I have to disagree with some of the other recommendations. I find the Hacking Exposed series has jumped the shark and tries to be too much for too many people. You get a real scattershot with that book in the latest edition. I found Art of Deception to be interesting, but it's all about social engineering. I'm not sure that would be in my top 5 for penn testers (I think finding technical security holes is more valuable to penn test clients, but that's just my opinion).
I do agree that a programming book or twenty are useful. At the very least you should memorize the O'Reilly
Practical C Programming by Loudon. If you don't know how to program in a language or use a technology you have to rely on tools to find vulnerabilities.
Building Secure Software by McGraw and Viega is an invaluable resource.
http://www.MadIrish.net