Use this Perl Script
#!/usr/bin/perl
# load LWP library:
use LWP::UserAgent;
use HTML::Parse;
# define a URL
my $url = ‘https://www.cnn.com/’;
# create UserAgent object
my $ua = new LWP::UserAgent;
# set a user agent (browser-id)
# $ua->agent(‘Mozilla/5.5 (compatible; MSIE 8; Windows NT 5.1)’);
# timeout:
$ua->timeout(15);
# proceed the request:
my $request = HTTP::Request->new(‘GET’);
$request->url($url);
my $response = $ua->request($request);
#
# responses:
#
# response code (like 200, 404, etc)
my $code = $response->code;
# headers (Server: Apache, Content-Type: text/html, …)
my $headers = $response->headers_as_string;
# HTML body:
my $body = $response->content;
# print the website content:
# print $body;
# do some parsing:
my $parsed_html = HTML::Parse::parse_html($body);
for (@{ $parsed_html->extract_links(qw(a body img)) }) {
# extract all links (a, body, img)
my ($link) = @$_;
print $link . “\n”;
}
Tags:
Perl
Permalink
March 7, 2009 at 12:32 pm
·
If you get 500 Internal Service Error with WordPress, and you are using ModSecurity, here is how to solve the problem:
1, You try to visit your front page, and see this error:
Error Message:
Internal Server Error
The server encountered an internal error or misconfiguration and was unable to complete your request.
Please contact the server administrator, innovita@gmail.com and inform them of the time the error occurred, and anything you might have done that may have caused the error.
More information about this error may be available in the server error log.
2, Check http log file
tail /var/log/httpd/patentdb-error.log
You see
ModSecurity: Output filter: Response body too large (over limit of 524288, total not specified)
3, Now you can fix it:
Open file with vi /etc/httpd/modsecurity.d/modsecurity_crs_10_config.conf, you will see these lines:
SecRequestBodyAccess On
SecResponseBodyAccess On
SecResponseBodyMimeType (null) text/html text/plain text/xml
SecResponseBodyLimit 524288
Change 524288 to larger value, or change option ‘SecResponseBodyAccess’ to Off, save and restart apache
apachectl restart
Tags:
WordPress
Permalink
March 4, 2009 at 1:16 pm
·