The Ethical Hacker Network

millwalll

Guest

« on: November 09, 2011, 11:44:18 AM »

Hi all I am trying to do something in Perl and looking for a bit of advice or a point in the right direction. I have not done hardly any Perl before so be gentle Tongue

I have a file that has list of numbers in these number relate to a website so page for example
http://testsite.com?name= NUMBERPART

I need to loop through the file and for each one go to the webpage and do a grep on it for another key number.

is there a way to do this without download each page ? If I was doing this by command line i would do wget website wait for it to download then do a cat websitename | grep keyword is there way to do it as one command ?

any help or advice would be nice thanks


	Logged

mambru

Jr. Member

Offline

Posts: 98

Re: Perl help need

« Reply #1 on: November 09, 2011, 12:37:08 PM »

By "without download each page" you mean not writing to a file on disk the page and instead keep the contents in a variable? Yes you can, the same way you can with wget (you can dump the page to STDOUT). Check the LWP library, maybe there are many more, it's been a while since I used Perl.


	Logged

millwalll

Guest

Re: Perl help need

« Reply #2 on: November 10, 2011, 03:38:49 AM »

Thanks will have a look at that today and yes I don't want write to a file. as some file contain over 500 number don't want generate 500 temp files or anything like that.


	Logged

millwalll

Guest

Re: Perl help need

« Reply #3 on: November 11, 2011, 09:21:36 AM »

ok need some more help with perl I have a long list of URL like 15 and I want to search the list for a certain link once I have that link I want store it in a variable. Sometime the URL list contains more than one URL I need but they both have MS:MS then a number. Is there a way to grab the URL using regular I thought I could do this or something simular.

m/^/MS:MS/*$\d

The syntax is not correct but just wanted to know if that would work so I grab at the start the MS:MS and at then end look for a digit.

Any help or a better way would be good thanks Tongue


	Logged

mambru

Jr. Member

Offline

Posts: 98

Re: Perl help need

« Reply #4 on: November 11, 2011, 10:00:31 AM »

Certainly you can. Perl is a very powerful language for text processing. If you have your list of URLs in a file, you can load all the contents into an array in a single operation an then traverse the array looking for an array that matches your RE, or you can can read your file line by line and compare it to the RE. You can even extract the specific part you are interested on (the number) from the string when you are comparing against your RE. Be careful how you use the ^ and $.


	Logged

millwalll

Guest

Re: Perl help need

« Reply #5 on: November 11, 2011, 10:52:15 AM »

ok cool I not sure the best way to do it this what I have.

Code:

my $test = HTML::TokeParser->new(\$Page);
$test->get_tag("ul");
my $test2 = $test->get_trimmed_text("/ul");

This is in a loop so for each iteration I get around 15 url I need a way to get the first line and get all the number from it that start with MS:MS then store what it finds in variable so by the end of the loop i should have a list of MS:MS number.

I have tried so many things and think I am getting the syntax wrong as nothing is working. So if you could give me an example that would be cool thanks


	Logged

mambru

Jr. Member

Offline

Posts: 98

Re: Perl help need

« Reply #6 on: November 11, 2011, 11:44:25 AM »

I'm not getting exactly what's the problem you're facing. Do you already have the list with URLs but you don't find one that contains the pattern you want?


	Logged

millwalll

Guest

Re: Perl help need

« Reply #7 on: November 12, 2011, 06:27:45 AM »

I have a massive while loop that takes a url and grabs the contents of that page I then do:

Code:

$test->get_tag("ul");
my $test2 = $test->get_trimmed_text("/ul");

so for each page it grabs everything in the UL tags what contains all the URL. For example $test will contain:

Code:

http://test1 http://test2 http://test3 http://test4 http://test?name=MS:MS000-0000

I need to search the list of URL for a certain word that starts with MS:MS and end in number for example MS:MS00-000 I need to grab that number and store it in another variable. So the time the loops has done I should have a list of lots MS:MS numbers.

Hope that makes it a bit clearer Tongue

Thanks


	Logged

mambru

Jr. Member

Offline

Posts: 98

Re: Perl help need

« Reply #8 on: November 14, 2011, 10:33:44 AM »

If you already have the list of URLs, then it's only matter of comparing your var against a pattern (your regular expression). In Perl the =~ operator is the one you'll use. You can use a capturing group (parentheses operators) to get your number into a var. I hope you're following me


	Logged

millwalll

Guest

Re: Perl help need

« Reply #9 on: November 14, 2011, 11:15:36 AM »

Cool will have a go and see how i get on thanks Tongue


	Logged

Pages: [1] Go Up

« previous next »