Jump to content

Why cURL Fails To fetch Page ?


Recommended Posts


Why is cURL failing to fetch the page ? All this time it worked.
Echoes "Page fetching problem!"

	//Required PHP Files.
include 'config.php';
include 'header.php';
	//1). Set Banned Words.
$banned_words = array("asshole", "nut", "bullshit");
	$url = "http://devshed.com";
// 2). $curl is going to be data type curl resource.
$curl = curl_init();
	// 3). Set cURL options.
curl_setopt($curl, CURLOPT_URL, "$url");
curl_setopt($curl, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($curl, CURLOPT_RETURNTRANSFER, true);
	// 4). Run cURL (execute http request).
$result = curl_exec($curl);
	if (curl_errno($curl))
    echo 'Error:' . curl_error($curl);
	$response = curl_getinfo( $curl );
	//If page is fetched then replace banned words found on page.
if($response['http_code'] == '200' )
    $regex = '/\b';
    $regex .= implode('\b|\b', $banned_words);
    $regex .= '\b/i';
    $substitute = 'BANNED WORD REPLACED';
    $clean_result = preg_replace($regex, $substitute, $result);
    //Present the banned words filtered webpage.
    echo $clean_result;
    //Show error if page fetching failed.
    echo "Page fetching problem!";


Link to comment
Share on other sites

I added this line today ...


echo "$response[http_code]";

Result I get is this: 403.

Googling, I see that the page devshed.com is forbidden. Since I can view the page on y browser and not manage to get cURL to fetch it then I'm guessing devshed.com has put a measurement in place to foil proxies.
Switched the url to another site and that got fetched.
last night, even google did not get fetched and so I suspected maybe I accidently deleted something from my code but could not spot the deletion. Hence, opened this thread to see if anyone else spots it.
Now, I understand, both devshed.com and google were foiling the cURL fetch or proxy fetch.

How well did I do ? ;)

Link to comment
Share on other sites

I now get error that variable $url is undefined on line 72.

Notice: Undefined variable: url in C:\xampp\htdocs\......

If you check line 72, it says between double quotes:

$url = "http://devshed.com"; 

Even if I change the url to a url who's page cURL is able to fetch, I still get the same error.
This does not work either, with single quotes:

$url = 'http://devshed.com'; 

This is very very strange! If the $url variable has not been defined then how is cURL able to fetch the page who's url is on the $url variable value ?
Even though page gets fetched, I still see the error! Weird!

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

  • Create New...