My PHP , Wordpress and Linux Lab

Thanks to a user , i have just tracked down a minor bug which could cause the plugin to breakdown with certain urls .. please download the zip from the download link provided on top right and update both .php files inside your /wp-content/plugins/wp-admin-switcher/ folder.

Just finished my first plugin for wordpress called WP Admin Switcher. You can download this plugin here

After you have installed this plugin you will be able to add information of multiple blogs you manage, the plugin will show you a drop down on top of all pages and let you switch between blogs. When you switch the plugin will attempt to show the same page of the other blogs administration panel. This plugin also works with free blogs from wordpress.com.

The plugin requires CURL to be installed with PHP. Also please make sure to set the permission of the folder “tempfiles” to 777. Another point to keep in mind is that when you enter the URL of the administration panel in the WP Admin Switcher management page , please make sure you enter the correct URL. Some blogs are set to force www , so the page will keep redirecting to http://abc.com/wp-admin/ if you enter the URL like this http://www.abc.com/wp-admin/ . So make sure that enter the correct wp-admin URL.

Have Fun!

2nd Sept , 2007 : As someone pointed out there are several blogs out there that dont use mod_rewrite or permalinks , so thus plugin wont work out of the box for them , they need to create a mod_rewrite rule to point to the main index.php for wp-admin/virtual/ requests. If you dont have mod_rewrite available then this plugin is not going to work for you , maybe the next version will make it independent of mod_rewrite.

24th Aug , 2007 : A bug was found which was leading to a breakdown in the plugins functionality , thanks to a user , this has been weeded out.


5th Aug , 2007 : Some adjustments were made to the script based on user feedback, if you are seeing messages which indicates that your request was blocked , please download the plugin again from the link above and see if it helps.

Today i heard the unexpected news that my modest entry ( and I mean it) at plant source code won the contest. And they allowed me to show this on my site. Thought I would share it with the world. Though some of you might get cracking at me for it being CPU intensive or whatever , but as i said in the entry , it is just a pointer it is not a solution to be used for commercial heavy duty purposes.

Someone put forward a question on the forum regarding how to block leechers and I thought that my answer might make a good post as well.

There is no fool proof way of going about it , cause there are ways in which a script can perfectly pretend to be a valid browser (CURL helps you do exactly that).you will have to put in use several different methods to fight them and reduce the illegal crawlers significantly.Other than the basic idea of banning the IP by getting it by REMOTE_ADDR and HTTP_X_FORWARDED_FOR here are some suggestions.

Method 1:
If there are too many requests from the same IP. Try to locate the ISP/Organization of the IP , like the GeoIP organization and ISP packages give you the database to lookup IP and see the owner. If IP belongs to a valid SE (and provided you want them to crawl) , let them go ahead. You can also do some effort to make a list of valid SE IPs which should override all crawler detections.

Method 2:
On the first request to the site , send a cookie , redirect and check if the cookie can be accessed, if not than redirect to a page asking the user to enable cookies.Generally scripts lack that ability.

Method 3:
Use javascript to set a cookie and then try to access it , if no cookie ask the user to enable javascript. Generic scripts wont be able to process javascript. Unless someone is writing code specifically for your site.

Method 4:
If too many requests show a captcha image which is not so straight forward. If no valid input atleast block that IP from going ahead. Even if alot of users are on that IP , you can show them a captcha again and validate that session_id to browse your site ,even if the IP is the same , a little nuisance but worth it if you have a severe problem.

Method 5:
Always check to see that a user agent header is sent , simple scripts written by newbies (and there are many) forget to send that.

Method 6:
After first entry , each request should contain a HTTP_REFERER logically , so check that too. Newbies forget to send that too.

Method 7:
If the same IP is generating different session_ids , you have a crawler on your hand.

Ok thats all i can think of right now , some of the above might not make sense to some but if all of them are used effectively in combination with each other you got a great crawl blocker system on your hand. I would appreciate if people can suggest other methods as well.

Last couple of days i came across issues with CURL 7.15.3 , PHP 4.4.5 and Apache.

There was this code i had to work on which was written by someone else , the guy was reusing the CURL handle more than one time and i was like …. eh? BUT interesting that code was working a few days ago , somehow during an update to apache etc that code stopped working. And the strangest behaviour was observed. Just imagine my surprise when a function wasnt returning control to the main script. Then i decided to check apache error logs and there it was … segmentation fault , with details

glibc detected *** double free or corruption

Now here is what was happening , a curl handle was generated , some post data was sent and then again that handle was used to send some more post data for a different page. The first request went through , but the second one errored out. Now ideally the handle shouldnt be reused in the first place. But earlier for some reason it was working fine and has been working great for a while.

Now if you experiencing anything similar .. just stop reusing the handle , create a new one for every request and use that. Thats the way it should be done and thats the way you should do it. For instance the CURL wrapper class i have provided here does the same , it always creates a new handle , everything based on that class hasnt ever broken down. So watch out how you use your curl handles!

CURL has become increasingly popular over the past few years specially because of the fact that the web is becoming an active entity rather than a passive one , as it was in the yester years. CURL is a wonderful tool to communicate with other websites. CURL is also widely used in those “auto comment submit” type scripts. It makes the life of PHP developers easy. Alot of developers just consider it as a magic tool where as its nothing more than a way to format the request headers and play around with the response headers. Lets take a brief look at the insides of some of the CURL functions and what it really does. This article will be useful for situations where the curl library is not installed on the server.

$ch = curl_init();
Create the curl handle for use later.

curl_setopt($ch, CURLOPT_URL,”http://www.abc.com/hello.php”);
Tell curl the url you want to process.

curl_setopt ($ch, CURLOPT_HEADER, true);
Tell curl if it needs to return the response header as well along with the data.

curl_setopt ($ch, CURLOPT_FOLLOWLOCATION, true);
Tell curl if it should oblige with the “Location:xyz.php” type response header command.

curl_setopt($ch, CURLOPT_TIMEOUT, 10);
This is the connection timeout limit.

curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
Tell curl if it should return the data or not.

curl_setopt($ch, CURLOPT_USERAGENT, “Mozilla/4.0 (compatible; MSIE 6.0;Windows NT 5.1)”);
Put the user agent type in the request header. This identifies what type of browser is requesting the page

curl_setopt($ch, CURLOPT_REFERER, “http://www.google.com”);
Put the referer url in the request header.Referer carries the page url where the user clicked on a link to request this page.

curl_setopt($ch, CURLOPT_COOKIEJAR, “cookies.txt”);
Provide the file name where curl will store the cookies sent by the web server in response headers.

curl_setopt($ch, CURLOPT_COOKIEFILE, “cookies.txt”);
Provide the file name where cookies are stored , and curl will send the cookies in there in the request headers.

curl_setopt($ch, CURLOPT_POSTFIELDS, array(‘username’=>’abc’ , ‘pass’=>’xyz’));
Provide the post data if any as array , curl will put it in the request headers.

curl_setopt($ch, CURLOPT_PROXY, “11.11.11.11:8080″ );
Tell curl if it should send the request via a proxy , provide the ip and port.

curl_setopt($ch, CURLOPT_PROXYUSERPWD,”user:pass”);
If the above proxy requires authentication , provise the username and password here.

$data = curl_exec($ch);
Execute curl with all the above options and put the data in the $data variable.

curl_close ($ch);
Close the connection

Now lets try to make some code which attempts to do the above with PHP.Please note that i am not trying ot create a full fledge CURL alternative here , i am just showing by example that it can be done easily with PHP , all you need to do is have some experience with headers. This is a not a perfect example script , so feel free to experiment around with it to get desired results.

You can download the below code if my visual presentation skills with wordpress dont strike you as too good - Download the PHP CURL Alternative Example

Every beginner PHP programmer comes across the “headers already sent” error and then they google to find out how to get rid of it. What they dont try to understand is the root level logic behind that error. Surprisingly alot of junior PHP programmers have no idea what headers are.

I will try to explain in brief what headers are actually. I am not an expert on headers but what i put here is what i know from a little bit of experience.

Every HTTP/HTTPS web page request goes to the relevant web server (Apache , IIS etc ) and then the web server responds with the requested page. COOL!!! BUT how do the web server know what page is requested and what form data has been posted etc? HEADERS!

Headers are the means of communication between a browser and a web server. A normal web request from a browser for http://www.abc.com/index.php would look something like this in headers.

GET /index.php HTTP/1.1
Host: www.abc.com
Connection: Close

The first line tells the web server that what transmission method is used GET or POST. Then the page name wanted and the protocol version used.
The second line tells the web server what domain that page should be under, this way apache can look inside its virtual hosts list and see what domain resides in which directory. So if www.abc.com resides in /home/abc/public_html , then the page requested becomes /home/abc/public_html/index.php.

The third line tells the server that after the request is processed the connection will be closed.

So that was a simple request. Now its the servers turn to respond, in our case the page is a .php page , so the php engine will process the requested page and send the response. Before any of the html is sent back to the browser , PHP sends in some information about the server and the data coming going through. Here is a sample response header.

HTTP/1.0 200 OK
Date: Sun, 25 Feb 2007 21:28:38 GMT
Server: Microsoft-IIS/6.0
Content-Type: text/html
Content-Length: 4293
Connection: close

First line gives the HTTP status code of the response. “200 OK” in this case. Which means all good , i found the requested page and i have processed it for you and i am going to send it through. A list of status codes can be found here. Next line is ofcourse the date and time of the server, then comes the name of the web server in use. Next line is content type , which you are probably already familiar with. Content type tells the browser what sort of data is going to come through , so that the browser can show it accordingly. In this case it is text/html , which the browser will show happily. Next comes the content length , ofcourse this is byte size of the data that is going to flow through. Last line as before tells the browser the connection will be closed once the data goes through.

Once the response header is complete , the php script output follows. So important thing to remember header first and then the data from your script.

The PHP header() function basically you can add to the above header response from the server.For instance when you type in your php script..

header(“Location:login.php”);

You are basically adding a line to the above response header..

Location:login.php

Which means that please goto login.php for further processing and the browser obliges and heads to login.php.

When your php script even sends one ” ” (space) basically the data has started to flow to the users browser, as we know before any data flow the header goes out first. Hence that means the header has been sent to the browser, followed by your ” “(space) character. Now once the header has been sent and PHP sees some command like header(“Location:goto.php”) , PHP says … excuse me sir but you have already started to send output from your script , thus i have already sent out the response headers , now how do you expect me to add that line to the reponse header ?

To overcome this problem alot of programmers use ob_start() at the top , basically telling PHP to buffer the output untill the end of the script or flushed explicitly with ob_end_flush or ob_flush. So basically no output is actually sent to the browser and the header command works fine. Now gentlemen that works fine and its a wonderful utility to have at your disposal BUT ….. if you are using ob_start() to get rid of header already sent errors , then you have a flow problem in your script. Fix the flow instead of just using ob_start as a quick band aid , you wont become a good programmer and last in the long run relying on shortcuts!

The ob_start function is best left to greater uses like using it for compression or using it with a callback function to process your output before sending out the script output or various other things.

Hope this article helps someone out there!

Alot of PHP/MySQL programmers out there , specially the ones just starting out make some mistakes while using queries. The ideas i am going to outline aren’t just limited to MySQL , they can be equally applied elsewhere as well. We need to keep in mind the fact that MySQL will use more RAM and CPU than it should if inefficient queries are written.

• For instance one common mistake is to use “SELECT * from table1” , unless you don’t need to use all of the fields i strongly suggest against it. Use something like “SELECT field1,field2 from table1“
• Make sure you know what you are doing when using JOINs. Use “explain” to see how many rows are actually being scanned for the query to execute and bring the result. A lot of times its better to break the query up into several queries. JOINS ARE HEAVY!
• I have seen people use order by clause in queries where they need to get just the count of the records. That’s totally unintelligent. Order by clause uses up RAM as well to sort the data. Avoid it where it can be avoided.
• Saving images in DB is probably not a really good idea after all. Atleast I think so. If you can , doesn’t mean you should.
• Integer primary keys are faster than making username the primary key or something similar.
• Fixed length fields are processed faster than variable length fields.
• Use indexing wisely. Indexing helps in query the database faster BUT insertion and updates are slower.
• Sometimes it better not to create text book database structure aka normalization , sometimes a little redundancy can help minimize number of queries AND/OR load on database, specially when dealing with large databases.
• Keep in mind that in the loop while($array=mysql_fetch_array($queryresult)) , the transaction will keep the query open until the end of the loop. So if there is some heavy processing happening inside the loop on the result data, it is probably a good idea to first run the query loop and store the data you want to process in arrays and then later on process on the array. I find this technique helpful on many occasions.
• Using mysql_pconnect doesnt always help .
• Using mysql_free_result is probably a good idea.
• mysql_insert_id is a useful function to get newest primary key of the record , rather than using “….order by id desc limit 1″
• If you know you will be using some mysql field data alot , like the username of the logged in user , its always a good idea to store it in session , rather than banging on mysqls door again and again for it.
• ENUM field type is very useful , just like fixed length fields. Data processing on ENUM fields is quite fast.
• Using count(*) in queries instead of mysql_num_rows is faster i believe.
• I believe if you are storing signup dates etc , its probably a good idea to have the field as integer instead of datetime and put in mktime() value. This only applies to current and future dates. In cases where past dates are possible datetime should be used. As you never know if that date can be earlier than 1970. Storing mktime gives you faster processing and manipulation of data is easier as well in PHP with the date() function.
• Dont use fulltext unless you really have to.
• And finally DO USE PRIMARY KEYS , dont be affraid to use them , they can help you down the road. For instance “DELETE from table1 WHERE id = 1″ and “DELETE from table1 where firstname=’php’ and lastname=’rox’” , the first query i reckon will run faster.

Those are all i can think of right now. Please remember i am not an expert on MySQL , the above are based on professional experience with PHP and MySQL. Another thing to remember here is , all of these points might seem insignificantly small but when piled up together. They can cause major havoc. Specially when large databases are question.

Hope the tips can really help someone.

I use this 2 functions to delete files and directories recursively. Some of you may find it usefull when

you have to delete a directory which has files and sub directories inside it. Can come in handy i guess

These are a couple of functions i use to get the directories and files from a specific folder. This is a recursive function and brings the sub directories as well. Returns an array.

First param is the folder path to start
Second param tells the function if it needs to bring the files inside the folder
Third param tells if the function needs to bring sub directories as well.

$folders = array();
folderlist(”testserver” , true , true);
print_r($folders);

function folderlist($folder_path = “” , $getfiles = true , $tree=true){
  global $folders;
  $handle = opendir($folder_path.”/.”);
  while (false !== ($file = readdir($handle))) {
   if ($file != “.” && $file != “..”) {
    if(is_dir($folder_path.”/”.$file)){
    if($getfiles){
      $fileshere = filelist($folder_path.”/”.$file);
       $folders[] = array(”foldername” => $folder_path.”/”.$file  ,
                        “files” => $fileshere
                        );
     }else{
     $folders[] = array(”foldername” => $folder_path.”/”.$file);
     }
     if($tree) folderlist($folder_path.”/”.$file);
  }
 } } return $folders;
}

function filelist($folder_path){
  $files = array();
  $handle = opendir($folder_path.”/.”);
  while (false !== ($file = readdir($handle))) {
    if ($file != “.” && $file != “..”) {
     if(!is_dir($folder_path.”/”.$file)){
      $files[] = $folder_path.”/”.$file;
     }
    }
  }
  return $files;
}