I'm on placement and I will be soon attempting on writing a web crawler which will be used to test the companies site vulnerabilities I will be starting with SQL injection but will eventually have OWASP Top 10 embedded. I will either be creating this in visual c# or in MVC. It will be a long project so i'd just like some advice on where I should start.
I know that I will need to use things like regular expressions and parse data searches through pattern matching. I will be looking at http://arachnode.net for some ideas on how the process works on the crawler, but if anyone has done something similar, has any links or articles that you could link me to would be great.