Prevent Pages Being Crawled

[debmc99]

Hello,

 

I just finished the php login tutorial which was great and I learned a lot. My question is how would you go about stopping the php pages with your database connectivity information from being crawled? Do you use a robots.txt file or use a meta tag? Thank you.

[falkencreative]

As far as I know, spiders only see the rendered HTML -- they can't see the actual PHP, so they won't pick up on your database connection info unless you place it somewhere visible to the public.

 

You shouldn't need to use a robot.txt file or a meta tag, though those are used to prevent search engines from spidering certain content.

[debmc99]

Thank you.

[administrator]

Do you use a robots.txt file or use a meta tag?

 

Two things:

 

1. PHP code will not be displayed to the user or search engines because the PHP is processed by the PHP engine and it never gets seen by the outside world. Check out this video on server side vs. client side languages.

 

2. The robots.txt file is used to tell the search engines what pages or directories to crawl and index (or what pages NOT to crawl ...), so yes, that would work too. But as I suggested above, since the PHP code is never seen by the outside world, there is no need.

 

Stefan

[debmc99]

Thanks - that makes sense.