My PHP , Wordpress and Linux Lab

Decode IT has announced PHP 5 and MySQL 5 hosting.  After signup customers will be given a choice between version 4 and 5. The servers are dual core servers with high uptime and reliability. Small packages are ideal for Wordpress installations which can be done with clicks via the Fantastico control panel. All accounts contain the all time great CPanel account manager.

Here is the overview….
50 MB Hosting - $2.00 pm
100 MB Hosting - $4.00 pm
200 MB Hosting - $8.00 pm
500 MB Hosting - $20.00 pm
1000 MB Hosting - $35.00 pm

You can find the details of the packages here.

Enjoy!!

Just finished my first plugin for wordpress called WP Admin Switcher. You can download this plugin here

After you have installed this plugin you will be able to add information of multiple blogs you manage, the plugin will show you a drop down on top of all pages and let you switch between blogs. When you switch the plugin will attempt to show the same page of the other blogs administration panel. This plugin also works with free blogs from wordpress.com.

The plugin requires CURL to be installed with PHP. Also please make sure to set the permission of the folder “tempfiles” to 777. Another point to keep in mind is that when you enter the URL of the administration panel in the WP Admin Switcher management page , please make sure you enter the correct URL. Some blogs are set to force www , so the page will keep redirecting to http://abc.com/wp-admin/ if you enter the URL like this http://www.abc.com/wp-admin/ . So make sure that enter the correct wp-admin URL.

Have Fun!

2nd Sept , 2007 : As someone pointed out there are several blogs out there that dont use mod_rewrite or permalinks , so thus plugin wont work out of the box for them , they need to create a mod_rewrite rule to point to the main index.php for wp-admin/virtual/ requests. If you dont have mod_rewrite available then this plugin is not going to work for you , maybe the next version will make it independent of mod_rewrite.

24th Aug , 2007 : A bug was found which was leading to a breakdown in the plugins functionality , thanks to a user , this has been weeded out.


5th Aug , 2007 : Some adjustments were made to the script based on user feedback, if you are seeing messages which indicates that your request was blocked , please download the plugin again from the link above and see if it helps.

Today i heard the unexpected news that my modest entry ( and I mean it) at plant source code won the contest. And they allowed me to show this on my site. Thought I would share it with the world. Though some of you might get cracking at me for it being CPU intensive or whatever , but as i said in the entry , it is just a pointer it is not a solution to be used for commercial heavy duty purposes.

Someone put forward a question on the forum regarding how to block leechers and I thought that my answer might make a good post as well.

There is no fool proof way of going about it , cause there are ways in which a script can perfectly pretend to be a valid browser (CURL helps you do exactly that).you will have to put in use several different methods to fight them and reduce the illegal crawlers significantly.Other than the basic idea of banning the IP by getting it by REMOTE_ADDR and HTTP_X_FORWARDED_FOR here are some suggestions.

Method 1:
If there are too many requests from the same IP. Try to locate the ISP/Organization of the IP , like the GeoIP organization and ISP packages give you the database to lookup IP and see the owner. If IP belongs to a valid SE (and provided you want them to crawl) , let them go ahead. You can also do some effort to make a list of valid SE IPs which should override all crawler detections.

Method 2:
On the first request to the site , send a cookie , redirect and check if the cookie can be accessed, if not than redirect to a page asking the user to enable cookies.Generally scripts lack that ability.

Method 3:
Use javascript to set a cookie and then try to access it , if no cookie ask the user to enable javascript. Generic scripts wont be able to process javascript. Unless someone is writing code specifically for your site.

Method 4:
If too many requests show a captcha image which is not so straight forward. If no valid input atleast block that IP from going ahead. Even if alot of users are on that IP , you can show them a captcha again and validate that session_id to browse your site ,even if the IP is the same , a little nuisance but worth it if you have a severe problem.

Method 5:
Always check to see that a user agent header is sent , simple scripts written by newbies (and there are many) forget to send that.

Method 6:
After first entry , each request should contain a HTTP_REFERER logically , so check that too. Newbies forget to send that too.

Method 7:
If the same IP is generating different session_ids , you have a crawler on your hand.

Ok thats all i can think of right now , some of the above might not make sense to some but if all of them are used effectively in combination with each other you got a great crawl blocker system on your hand. I would appreciate if people can suggest other methods as well.

Over the years I have seen Google emerge out of nothing and rule the planet when it comes to search. But I have also seen lots of junk in my searches and I have seen a thousands of requests for Search Engine Optimization and ads by Search Engine Optimizers claiming to put your site on top of search results for X amount of dollars in Y number of days.

Now probably one of the biggest industries around on the internet is Search Engine Optimization. My understanding is that probably the BEST content on the internet comes from personal small time sites. For instance take look at my blog , no bullshit pure content BUT i will never be able to score high on Search Engines. Why? I DONT KNOW!

Isnt it all about content? Honestly, NO! . Its more about what optimization techniques you follow on your site and what keywords you target. Having said that keyword targeting is earning alot of people alot of money and alot of crap sites are getting loads of hits just because of that and probably alot of great sites with alot more relevant information regarding that keyword come on page 10 , 11 or whatever.

So what does that mean , that means if you want to have real Internet presence and your intentions are all good and you are a guy with principals who doesn’t want to use black hat methods , i have bad news for you. That aint gonna happen! Atleast not the way things are going.

Take for instance the Google Bomb concept , it works and it works well. Page cloaking is another well known technique. Getting these methods to help will cost you $$$$ and some SEO expert is going to get rich. There are also magic ebooks out there which tell you techniques to get on top of search engines at a ’small’ price. OK hold on … I thought the internet was supposed to be an equal oppurtunity domain , where everyone is equal BUT where Google has helped us ALOT it has also created a division in the internet society. Even on the internet the rich guys with pots of money for Search Engine Optimization and Marketing get on the top and the not so rich are always in shadows. Google probably supports the ’secret’ black hat community cause they dont seem to be doing much about it. Lets take the example of Sponsored Ads in Google , have they ever filtered what is actually shown on that right column? I see the same ad on the right side for some searches with something like “Looking for XYZ?” and its the same ad .. with ebay selling me PHP and God knows what.

The point is Google has knowingly or unknowingly put all the good webmasters in a fix. “What should we do to be atleast marginally visible on Google?” . Instead of them concentrating on content , products or services they are more worried about the fact if its even worth the effort. Probably Google should rethink their strategies , if they want to focus on the 1% powerfull people on the internet or the 99% not so powerful people around there.

Probably one of the best things that happened on the internet indexing wise was www.dmoz.org ,
“hand pick the sites that should be included”. Now i am not saying Google should hand pick everything BUT atleast they should be checking what is actually showing on top searched keywords and also i also have another suggestion , there are so many keywords where they have nothing to show in that money column of theirs. What if they started to show some good sites in those spots for free , now probably they can handpick lets say 1000 sites each month and rotate them through in those spots? But anyway the point is , where are we going with Google , by WE i mean the not so powerful lot on the internet, the honest  webmasters! . We are the losers in the game cause we believe that our content , service or product will get us on the top but thats dreamland , hope Google can squeeze us in there somewhere on their money pages.

Last couple of days i came across issues with CURL 7.15.3 , PHP 4.4.5 and Apache.

There was this code i had to work on which was written by someone else , the guy was reusing the CURL handle more than one time and i was like …. eh? BUT interesting that code was working a few days ago , somehow during an update to apache etc that code stopped working. And the strangest behaviour was observed. Just imagine my surprise when a function wasnt returning control to the main script. Then i decided to check apache error logs and there it was … segmentation fault , with details

glibc detected *** double free or corruption

Now here is what was happening , a curl handle was generated , some post data was sent and then again that handle was used to send some more post data for a different page. The first request went through , but the second one errored out. Now ideally the handle shouldnt be reused in the first place. But earlier for some reason it was working fine and has been working great for a while.

Now if you experiencing anything similar .. just stop reusing the handle , create a new one for every request and use that. Thats the way it should be done and thats the way you should do it. For instance the CURL wrapper class i have provided here does the same , it always creates a new handle , everything based on that class hasnt ever broken down. So watch out how you use your curl handles!

CURL has become increasingly popular over the past few years specially because of the fact that the web is becoming an active entity rather than a passive one , as it was in the yester years. CURL is a wonderful tool to communicate with other websites. CURL is also widely used in those “auto comment submit” type scripts. It makes the life of PHP developers easy. Alot of developers just consider it as a magic tool where as its nothing more than a way to format the request headers and play around with the response headers. Lets take a brief look at the insides of some of the CURL functions and what it really does. This article will be useful for situations where the curl library is not installed on the server.

$ch = curl_init();
Create the curl handle for use later.

curl_setopt($ch, CURLOPT_URL,”http://www.abc.com/hello.php”);
Tell curl the url you want to process.

curl_setopt ($ch, CURLOPT_HEADER, true);
Tell curl if it needs to return the response header as well along with the data.

curl_setopt ($ch, CURLOPT_FOLLOWLOCATION, true);
Tell curl if it should oblige with the “Location:xyz.php” type response header command.

curl_setopt($ch, CURLOPT_TIMEOUT, 10);
This is the connection timeout limit.

curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
Tell curl if it should return the data or not.

curl_setopt($ch, CURLOPT_USERAGENT, “Mozilla/4.0 (compatible; MSIE 6.0;Windows NT 5.1)”);
Put the user agent type in the request header. This identifies what type of browser is requesting the page

curl_setopt($ch, CURLOPT_REFERER, “http://www.google.com”);
Put the referer url in the request header.Referer carries the page url where the user clicked on a link to request this page.

curl_setopt($ch, CURLOPT_COOKIEJAR, “cookies.txt”);
Provide the file name where curl will store the cookies sent by the web server in response headers.

curl_setopt($ch, CURLOPT_COOKIEFILE, “cookies.txt”);
Provide the file name where cookies are stored , and curl will send the cookies in there in the request headers.

curl_setopt($ch, CURLOPT_POSTFIELDS, array(’username’=>’abc’ , ‘pass’=>’xyz’));
Provide the post data if any as array , curl will put it in the request headers.

curl_setopt($ch, CURLOPT_PROXY, “11.11.11.11:8080″ );
Tell curl if it should send the request via a proxy , provide the ip and port.

curl_setopt($ch, CURLOPT_PROXYUSERPWD,”user:pass”);
If the above proxy requires authentication , provise the username and password here.

$data = curl_exec($ch);
Execute curl with all the above options and put the data in the $data variable.

curl_close ($ch);
Close the connection

Now lets try to make some code which attempts to do the above with PHP.Please note that i am not trying ot create a full fledge CURL alternative here , i am just showing by example that it can be done easily with PHP , all you need to do is have some experience with headers. This is a not a perfect example script , so feel free to experiment around with it to get desired results.

You can download the below code if my visual presentation skills with wordpress dont strike you as too good - Download the PHP CURL Alternative Example

Every beginner PHP programmer comes across the “headers already sent” error and then they google to find out how to get rid of it. What they dont try to understand is the root level logic behind that error. Surprisingly alot of junior PHP programmers have no idea what headers are.

I will try to explain in brief what headers are actually. I am not an expert on headers but what i put here is what i know from a little bit of experience.

Every HTTP/HTTPS web page request goes to the relevant web server (Apache , IIS etc ) and then the web server responds with the requested page. COOL!!! BUT how do the web server know what page is requested and what form data has been posted etc? HEADERS!

Headers are the means of communication between a browser and a web server. A normal web request from a browser for http://www.abc.com/index.php would look something like this in headers.

GET /index.php HTTP/1.1
Host: www.abc.com
Connection: Close

The first line tells the web server that what transmission method is used GET or POST. Then the page name wanted and the protocol version used.
The second line tells the web server what domain that page should be under, this way apache can look inside its virtual hosts list and see what domain resides in which directory. So if www.abc.com resides in /home/abc/public_html , then the page requested becomes /home/abc/public_html/index.php.

The third line tells the server that after the request is processed the connection will be closed.

So that was a simple request. Now its the servers turn to respond, in our case the page is a .php page , so the php engine will process the requested page and send the response. Before any of the html is sent back to the browser , PHP sends in some information about the server and the data coming going through. Here is a sample response header.

HTTP/1.0 200 OK
Date: Sun, 25 Feb 2007 21:28:38 GMT
Server: Microsoft-IIS/6.0
Content-Type: text/html
Content-Length: 4293
Connection: close

First line gives the HTTP status code of the response. “200 OK” in this case. Which means all good , i found the requested page and i have processed it for you and i am going to send it through. A list of status codes can be found here. Next line is ofcourse the date and time of the server, then comes the name of the web server in use. Next line is content type , which you are probably already familiar with. Content type tells the browser what sort of data is going to come through , so that the browser can show it accordingly. In this case it is text/html , which the browser will show happily. Next comes the content length , ofcourse this is byte size of the data that is going to flow through. Last line as before tells the browser the connection will be closed once the data goes through.

Once the response header is complete , the php script output follows. So important thing to remember header first and then the data from your script.

The PHP header() function basically you can add to the above header response from the server.For instance when you type in your php script..

header(”Location:login.php”);

You are basically adding a line to the above response header..

Location:login.php

Which means that please goto login.php for further processing and the browser obliges and heads to login.php.

When your php script even sends one ” ” (space) basically the data has started to flow to the users browser, as we know before any data flow the header goes out first. Hence that means the header has been sent to the browser, followed by your ” “(space) character. Now once the header has been sent and PHP sees some command like header(”Location:goto.php”) , PHP says … excuse me sir but you have already started to send output from your script , thus i have already sent out the response headers , now how do you expect me to add that line to the reponse header ?

To overcome this problem alot of programmers use ob_start() at the top , basically telling PHP to buffer the output untill the end of the script or flushed explicitly with ob_end_flush or ob_flush. So basically no output is actually sent to the browser and the header command works fine. Now gentlemen that works fine and its a wonderful utility to have at your disposal BUT ….. if you are using ob_start() to get rid of header already sent errors , then you have a flow problem in your script. Fix the flow instead of just using ob_start as a quick band aid , you wont become a good programmer and last in the long run relying on shortcuts!

The ob_start function is best left to greater uses like using it for compression or using it with a callback function to process your output before sending out the script output or various other things.

Hope this article helps someone out there!

My Simple CURL Wrapper Class

Alot of times i have to use CURL with PHP for getting data from remote sites. Its a wonderfull library which makes alot of things easier for developers who want to interact or get data from remote sites and use them inside their own code. This is where my class comes in handy
For simpler uses you can get data easily simply by:
$data = implode(”" , file(”http://www.abc.com/page1.html”));

But what to do when you have to login to another site and get data from inside the secure page?

So here is what i do with this class :

include(’curl.class.php’);
$browser = new extractor(true);

$loginpost[’username’] = “xyz”;
$loginpost[’password’] = “xyz”;
$formaction = “http://www.abc.com/login.php”;
$somesecurepage = “http://www.abc.com/myaccount.php”;

$data = $browser->getdata($formaction , $loginpost , “” , true , false);
$mypage = $browser->getdata($somesecurepage , array() , “” , true , true);

and most of the time i end up logging in to the remote site and getting the page data from secure pages.

Here is the what this class has to offer.

Constructor function has 3 parameters :

First parameter is to allow cookie usage or not , true or false.
Second parameter is to set the time out of connections. Integer value in seconds.
Third parameter is to allocate a filename for the storage of the cookie for this session. This can be usefull when you have to carry the login session of the remote site to multiple scripts in your own code. I generally pass the session id to this parameter , this way if i have to use the remote cookie on next pages as well , i can use it easily. You have to create a folder called “cookies” and set its permission to 0777 for this to work correctly. All cookies will be stored there.

The class has some accessable variables as well. They help you use proxies for your remote requests
var $proxyaddr; //— you can set the proxy ip or host to use
var $proxyport; //– proxy port
var $proxyuser; //– proxy username
var $proxypass; //– proxy password

The getdata function lets you actually request the remote url and get its data. The function has these parameters.

First parameter is the url you want to request.
Second parameter is the post data you want to send to the remote url. For login, search etc.
Third parameter is the custom referrer you want to send.
Fourth parameter is wether to set cookie or not.
Fifth parameter is wether to use the cookie made earlier.
Sixth parameter is to send a custom user agent you want to send for the request.
Seventh parameter is rarely used , sometimes even when you are doing every thing right , you dont get the expected results after sending post data. Setting this parameter to true , sends the post data as string to curl instead of sending it as an array.

Another usefull function in the class is the search function. This helps you get specific regions out of the html. For instance if you want to get all the <p> inside the html , you can do this.

$return_array = $browser->search(”<p” , “</p>” , $data);

The last parameter of the search function lets you remove the start and end strings from the returned results. The result from this function is an array.

This class i developed has made alot of things easier for me over the years . This makes alot of things easier for you when using curl with php , saves you a bundle of time.

Hope this helps someone else as well.
Here is the download link

Since the switch over to mediamax , streamload sux.

I have received several comments and have experienced the bad service myself as well. So i have been trying to find good alternatives to FTP upload to remote storage. Havent come up with many yet . But i will definitely post here if i do.

Some easy alternatives for people with online storage needs are www.box.net , the amazon S3 service , www.xdrive.com and even the G-Drive hack
I guess streamload will be another victim of the capatalist idealogy. People who generally take over good services/comapnies for some odd reason end up screwing them up. Another example i went through myself was www.mesopia.com now www.netbunch.com. The guys at webhostplus.com made sure NO ONE EVER goes the netbunch way again. I hope the FBI case on webhostplus people picks up pace soon.