|
Title: New web application crawler Post by: H1t M0nk3y on May 25, 2010, 06:24:14 PM Hey,
I am about 60% done creating a new web application crawler. My goal is not to replace Nikto or any other tool for that matter, but I am more thinking about an application that we can use in the reconnaissance/information gathering phase. So far, all my prototypes have been successful. Some of tasks it performs are already done by existing tools, and I know that. My goal is to put some of these existing functionalities plus many new ones in one single tool. It will hae a GUI and a command line interface. Could be multi-threaded later if people like it. Finally, I will give it for free! So here are the tasks it can/will do against a web site: - Create a wordlist - Find all emails, telephone numbers, fax numbers, etc - Find names and guest possible usernames based on email address - Find broken links - Create a site structure - Display robot.txt file details - Find typical html files - Identify all forms - Search for login screens My questions are: 1) What else would you find useful from a tool like this? 2) Output format from the command line version? I want to launch a usefull tool. Not a script or two... Thanks for you comments! Title: Re: New web application crawler Post by: former33t on May 25, 2010, 10:10:38 PM Well, if you are finding forms, I'd like it to enumerate information about those forms. I like to know going in whether the form is POST or GET (a single GET in a mess of POST forms may indicate someone moved development code to production). I also like to know all the fields, names, types, and values (especially hidden fields). I'd like it to crawl any page with a form a couple of times some user defined time interval apart to see if any pre-defined values for hidden fields change (these are great fuzzing targets). That's about all I can think of at this late hour.
Title: Re: New web application crawler Post by: H1t M0nk3y on May 26, 2010, 06:46:50 AM Thanks former33t for your comment.
It makes sense and I had not thought about this before. I have an idea on what I could do to report this kind of data. Anyone else? Title: Re: New web application crawler Post by: former33t on May 26, 2010, 07:40:36 PM Let me know when it is ready for testing.
Title: Re: New web application crawler Post by: H1t M0nk3y on May 27, 2010, 06:35:51 AM I will post on this forum when it will be in Beta. Probably August or September.
Thanks former33t.
Powered by SMF 1.1.18 |
SMF © 2013, Simple Machines
Joomla Bridge by JoomlaHacks.com |