Avinashsing Sunkur

Very rarely do I find the time to check the raw access logs for my websites. This is mainly because I’ve got Google Analytics installed and since they provide pretty much all of the information I require, the need to check my server logs is not really necessary.

However, Google Analytics does not show you which search engine bots has been crawling your website because the bots do not execute javascript which analytics uses to track visitors to the site. I had to download my server logs and check whether a particular webpage has been crawled. I did a search for “Googlebot” and got an audit trail of which pages have been crawled for the past month or so but I also noticed a completely different IP for Googlebot. I know the standard IP range starts with 66 (eg 66.249.65.49) but this was one 114.152.179.5. So I did a reverse DNS check on that IP and guess what, the IP which claimed to be googlebot is not googlebot at all. It’s for someone from japan who’s done a program to leech content off my website.

The server logs revealed that it has requested a significant amount of pages and I thought that google was really loving my website and decided to do a deep crawl but in reality, somebody was stealing content off my site. You can guess my frustration! I have now blocked the IP address through htaccess but I will be checking my raw access logs more often in an attempt to ban future leechers. If you’re a leecher and reading this, beware because I’m on to you now!

Onto my next project, I wanted to start a website but was looking to target an international audience. The idea behind this is that the more people I can get to the website, the better it is for me and there’s no reason why I can’t do this because English is spoken in many countries like UK and USA.

The population of the UK is around 61,000,000 and that of USA is 300,000,000. This means that United States of America has 5 times more people than here in England. It makes sense to target the USA as well and since the website is going to be mostly informative, it shouldn’t really cause a problem because I wouldn’t have to target services locally.

However people in the UK tend to click on .uk website extensions more than any other websites. Research carried out by Nominet showed that 77% or British are more likely to favour .uk domains rather than .com which means that if I register a .com website and manage to get a first page google ranking, I am more likely not to get any clicks from people searching in the UK. In America, people will favour either .com domains or .us, .net or even .org and a .uk domain is not really appealing to them. Even if you benefit from a country level domain for SEO purposes, you will not do well in terms of clickthroughs because people are biased towards domain name extensions. I part of that group as well and when I do searches in google.co.uk and find .com websites listed, I usually skip them until I reach a .co.uk or .org.uk listing. This normally happens when I’m looking for something specific to the UK.

So I was faced with the difficult decision of choosing the best domain extension for my website now. I then ran a search for the number of searches according to country (UK and USA) and I found out that the keywords received 40,500 searches locally for the month of Feb for UK compared to 1600 for USA. Now this is a significant difference and the conclusion becomes obvious. I’d rather focus on the UK market than the USA because of the traffic volume and this means that going forward with a .co.uk domain is the way forward.

Research is an important part of SEO and targetting your local market is made easier through a country specific TLD. It is pointless to have a good ranking website but an extremely low clickthrough rate because people think that your website is not relevant to them because of incorrect domain name extension.

After much enjoyment of being at number #4 in the SERPs for a popular phrase, I today found out that my webpage has disappeared once again. It was only a week ago that google reinstated my rankings (actually improved my rankings because I was ranked #5 or #6 before) and it was after a really long wait of 4 weeks. Now everything seem back to how it was again. I believe there are changes which are coming our way and that’s why the UK Serps are not stable for the time being. It is really upsetting not being able to find the webpage out of 60 result pages containing 10 results each and it seems the same thing has happened again – the ranking for the webpage has just vanished once more.

I’m not going to be doing anything and hope the rankings are going to come back by themselves again. I shall wait and if things do not pick up after 4-6 weeks, then I’ll start making the changes which I believe could be the cause.

In an earlier post about disappearing from SERPs completely, I thought that too many internal links in the sitewide navigation of my site were causing the penalty for that particular page to be removed from the SERPs without any trace of it. I was hasty to remove the link to that webpage in an attempt to regain my previous rankings but then it occured  to me that the other pages which were in the sitewide navigation would have shown similar penalties if that was the case. 5 days after removing the link, I re-included the link with the same anchor text as before because I was convinced it was not the case.

I then thought that it must have been a temporary glitch in the system which would have dropped the webpage for its keywords from the rankings. The webpage was ranking among 50 million pages without any external links (or maybe just 1 link from an article directory) and that shows that it had the domain and page level trust from google and that alone was enough to rank the webpage on the first page of search engine results. I decided to wait for a couple of weeks and see if the rankings would come back.

After 3 weeks, I was beginning to be really nervous as the rankings were not coming back. I wondered whether there was an Over Optimisation Penalty (OOP) on that page or whether something else was causing the problem. I ran the webpage though a spam checker tool and there was nothing much going on there, nothing spammy at all. I decided to validate the webpage through W3C standards and there were some errors but nothing major to cause disappearance from the SERPs. So the next thing was to look at the over optimisation filter.

I found that the keywords were underlined, bolded, italised as well. Now this looks too spammy, right? Well I concluded that I must have tipped over the OOP filter and that’s why the webpage was not ranking. Mind you that the webpage in question was still indexed and had a PageRank of 1 still. So the problem was just with ranking for those keywords it was optimised for. I removed the underline, bold and italics from the content and changed my signature on an active forum for link to that webpage and made a post hoping that it will get crawled quickly and Google will re-evaluate the webpage and re-rank it.

Next morning when I checked the SERPs, I had my ranking back. Wow, I thought! Things have worked out in the end but wait a minute, was the penalty caused by the over optimisation of that webpage? I decided to check my server logs to see if google fetched (crawled) that webpage the previous day after I made the changes but I was shocked to find out that it wasn’t the case. Now this means that none of the changes I made had any effect on getting back my rankings and this was confirmed when I viewed a cached copy of the webpage in Google.

I couldn’t leave the changes in place because on the next crawl google would re-evaluate the changes and adjust the rankings accordingly and since the changes did not bring back my rankings, I decided that I was better off removing them and bringing the webpage back to its original state. This is what I did and now I’ll be checking the SERPs over the next few days to see if the rankings disappear again.

The conclusion is that it took nearly 4 weeks for google to re-instate my rankings without me having to do anything. This means that if you see your rankings disappear for any reason, before you start making changes, wait a couple of weeks (as long as 6 weeks) and you might see your rankings back again. Otherwise you might end up messing everything up and lose your rankings forever!

I was ranking on the 1st page of Google for a long tail keyword (4 words) for months (or years) now and everything was fine until a couple of days ago when the ranking disappeared. My site was on the first page 7th/8th position for that search phrase and to my surprise it got knocked down through the major UK SERP update that happen the 2nd week of February 2010.

I thought that the ranking would be a couple of places further down at least or a couple of pages down in extreme cases but i was shocked to find that my website had disappeared completely from the search result. Actually not my website but the actual webpage which was ranking for that search phrase. I searched the top 500 listings (first 50 SERP pages) and I couldn’t find a trace of it. Google wouldn’t let me go past 50 pages btw.

Now this is a massive blow in my face because I haven’t purchased any links to boost the ranking of this page. As a matter of fact, this webpage doesn’t even have a single backlink but it was ranking amongst 48 million results! I haven’t changed anything on the site and the website if a couple of years old as well, so there’s no reason why the ranking would have disappeared all of a sudden.

The only thing that i can think of is that i keep updating the site daily which means that there are more and more pages everyday. Now this is a good thing but the problem that i have on my site is that i’ve got a sitewide navigation which links to this page in question. I believe i have crossed the tipping point for being seen as trying to manipulate the rankings through excessive internal linking. Think about it, if google can penalise you for footer links (which are sitewide), then obviously there’s a mechanism to detect excessive links especially those ones which the same anchor text directing to the same webpage. I experienced the same phenomenon some time ago and I’m sure I did something about it then and there.

For the sake of it, I have now removed the link to that webpage from the sitewide navigation. I need to change the whole static sitewide navigation thing but i cannot make too many changes at once, otherwise i will lose ranking across the whole site.

WordPress outputs a lot of crap in the head section of the html. So if you want to get rid of them, you shouldn’t just delete the call to wp_head() in your header.php file as this will break plugins which uses that function (eg All In One SEO Pack, Platinum SEO Pack etc). What you should do is place the following lines in your theme’s functions.php file right after the <?php> tag:

remove_action(‘wp_head’, ‘feed_links’, 2);
remove_action(‘wp_head’, ‘feed_links_extra’, 3);
remove_action(‘wp_head’, ‘rsd_link’);
remove_action(‘wp_head’, ‘wlwmanifest_link’);
remove_action(‘wp_head’, ‘index_rel_link’);
remove_action(‘wp_head’, ‘parent_post_rel_link’);
remove_action(‘wp_head’, ‘start_post_rel_link’);
remove_action(‘wp_head’, ‘adjacent_posts_rel_link’);
remove_action(‘wp_head’, ‘check_and_publish_future_post’);
remove_action(‘wp_head’, ‘wp_print_styles’);
remove_action(‘wp_head’, ‘wp_generator’);
remove_action(‘wp_head’, ‘rel_canonical’);

If you want to just remove the comments rss feed, then include just remove_action(‘wp_head’, ‘feed_links’, 2) and note the 2 in the argument list. If you don’t include that, it won’t work.

If you have set your blog as private (I would like to block search engines, but allow normal visitors), then you will see that WordPress will automatically include a “noindex, nofollow” for the robots meta tag. You need to change it to be publicly visible to everyone by going into Settings and Privacy in the wordpress admin panel. Otherwise you’ll see your pages dropping out of the index quickly.

UPDATE

If you want to delete the previous and next post links from wp_header in WordPress 3.0, you will have to remove the following:

remove_action(‘wp_head’, ‘adjacent_posts_rel_link_wp_head’, 10, 0 );

If don’t see the links removed, you need to look at the default-filters.php file (found in wp-includes) and check how the add_action have been written and use the same format (including the parameters) to remove them. You can just copy the whole line and replace add_action with remove_action.

I’ve been playing around with wp_hive plugin for a while a day or two now and I wanted to see how I can import my other blogs into it. The reason is that I want to have a central point where I can manage all the blogs instead of having to update and upgrade at different places all the time. My other blogs that I wanted to import run on wordpress but they were an older version though. I had a look at the number of tables and they were 10 while the new wordpress 2.9.1 had 11. So the first thing that I did was to update wordpress on the other blogs. If your blogs run on some other platform, then you will need to get it into wordpress format before you can import the blogs to use with WP Hive.

Once you’ve installed wordpress on your primary domain eg DomainA.com (the place where you want central management), you just need to copy the wp_hive plugin contents into the plugins folder in wordpress. Next make sure the wp_content is writable otherwise you wouldn’t be able to activate the plugin. Once that is done, you can start adding other domains into WP Hive. You do that by inputting the new domain name eg DomainB.com and then put “/” as the subfolder because you want it to point to the main installation directory which in this case is DomainA.com. Use a unique table prefix for this new domain eg dmb_ as all the tables for the hives will be created in the same database and the table names are important to load the files specific to a particular domain.

The above paragraph is really for adding new domains (blogs) but for existing blogs you will need to export a copy of the database (through PHPMyAdmin) and find and replace table names (wp_) with another prefix. You do that only for table names – it takes about 1 min and all you have to do is make sure that you’re replacing the prefix only for create and insert statements and not actual input values. Once this is done, you go back to the main domain to import this database (for your existing blog) in your main database (the main database you’ll be using for wp hive). You then proceed to add a new site by going to the options in wp hive on your main wordpress installation. You will need at this point to add the domain name as a parked domain. Don’t try to make it an addon domain! Just park the domain on top of your main one. If you have the default theme, everything should work fine now.

However if you have another theme, you will see a blank page at this point because wordpress is not able to load the theme specified in the wp_options table. You just need to copy the theme from your previous blog into your main one and voila, everything is back!

Remember that DNS propagation takes time and sometimes you will get DNS errors or 404s because nameservers changes haven’t completed. Just be patient!

Blogging is a great way to create content and bulk up a website. However if you run a couple of blogs, you’ll soon realise that it becomes a nightmare to upgrade them as soon as a new version becomes available. I’m really talking about wordpress here because installing wordpress is really easy and there are millions of themes available for you to change the look and feel of your website.

So in an attempt to have multiple domains powered through just one software installation, I had a go with WordPress MU (multi user). I didn’t know what I was getting into and I now regret it because I’ve spent a whole day trying to make it work. Let me talk about my requirements first:

1. I want to be able to create different blogs (different topics) easily
2. I want to be able to upgrade them easily as well
3. Something which is free (open source)
4. Ability to map multiple domains on a single software installation
5. Ability to have different look and feel for all those websites
6. Ability to use shared hosting to perform the tasks

WordPress MU (WPMU) can do all these things but if you’re on shared hosting, then it becomes a problem because you will need to be able to do wildcard mapping for all subdomains, that is, mapping “*” to the main website. Most shared hosting companies do not allow this and you also need access to modify the VirtualHost component as well (httpd.conf). Although my hosting company did the subdomain wildcard mapping for me so that each new blog is created as a subdomain which forwards to the main script (index.php) to retrieve the relevant data for that blog, I still had to use a plugin to enable multiple domain mapping onto the installation. I couldn’t get it work and it was just too much work considering the drawbacks of wordpress mu which are mainly incompatibility of certain plugins, being a few versions back from the main wordpress software and the difficulty in setting it up in the first place.

To be honest, WordPress MU is too complicated to set up and you’ll probably get so frustrated before you get it to run that you’ll think it’s just not worth the effort (just like me). So I came across this plugin (wp hive) which you can activated and get multiple domains run from one single installation of the normal wordpress software. You don’t need to add wildcard subdomain mapping or do anything complicated. You just install wordpress as normal and install the wp hive plugin. Once activated, you can start adding new domains to your existing wordpress installation. One thing to remember though is that you need to have wp_content writeable before you activate the plugin, otherwise it’s not going to work.

You need to park the additional domains on the same host for it to work as well. Just normal domain parking works and as soon as dns propagation has completed, you will be able to log in to the new domain as if wordpress was install on that domain. Sweet, eh?

So if you want multi domain, multi user, multi blogs, then i recommend wp hive over wordpress mu because of the ease of installation and use.

Yahoo Answers is good for driving targetted traffic to your website. If people see that you’re an expert in your field, they will usually click on the link to your website for more information (provided you’ve included a link in your report). However, you need to be at least a Level 2 account holder to be able to insert clickable links, otherwise the links will be plain text and people will have to copy and paste the url in their browser and the chances of that happening are very slim.

So i decided to become Level 2 in yahoo answers and i had to accumulate 250 points. You get 100 points by signing up to yahoo answers, 1 point for logging in everyday and 2 point for each answer provided. Asking questions cost you 10 points and receiving violation incurs you points as well. If you’re voted the best answer, you get 10 points for that but i didn’t really focus on that one and concentrated my efforts on answering questions as 75 answers will get me to Level 2. Since you are only allowed to provide 20 answers a day for Level 1 accounts, it took me 3 days to reach Level 2 because I answered my daily limits and a couple of my answers were chosen as best answers.

All the links that i provided when i was Level 1 became active and clickable and so were any subsequent links. I wanted to see how long it was going to take me to answer 20 questions. So i decided to do a little promotion for this guy for his SEO website. All my answers were really great and helpful and it took me 1 and half hours to complete the task. That was not worth my time at all but i got a surprise email the next day informing me that my yahoo answer account was suspended. I quickly appealed against it because i held this yahoo account since the year 2000 and wanted to keep it clean but the response i got was that i violated the rules and it’s been banned now. I emailed them lot once more asking them to check the quality of my answers and not go by the links i provided (well i know i got suspended for including too many links) but they never replied back.

I later found out that you need to keep all your questions and answers on yahoo private because other users will try to get you banned. Like i said, i wasn’t just inserting links but was providing helpful answers but once your yahoo answers is suspended, there is no way to get it back. The second thing i learned was that you need to keep a balance on how many answers you will include links and how many answers you won’t. Put it this way, if you include a link in all your answers, it doesn’t look good and will get you banned sooner or later. Best thing to do is keep a low profile and promote your website by putting the URL in the source section when you answer a question occasionally.

I’m  a bit upset about this specific yahoo answer account being banned but i’ve got someone to open 5 new accounts for me now and they are all promoting my sites and because the answers and questions are private (so nobody can view the activity on these accounts), they are all doing fine. If you haven’t tried uahoo answers as a method of traffic, it’s not late to start now and this could be one of the finest SEO strategies that you can employ.

If you are a one-man team, it’s very difficult to cope with the demands of maintaining, updating and marketing a website. Even big websites can sometimes get really exhausting and the need for outsourcing work arises. Getting somebody else to do your job has many implications depending on what type of work that person will be doing. For content-writing, you can always check the content before you upload it to your site but even if you haven’t done that, you can later on get it removed from your site quite easily. However for things like off-page optimisation, things can get very nasty.

In an effort to market a website of mine, I recently hired someone to do forum posting for me in a particular niche. The outcome was not very disappointing. If I had done it myself, I would have saved more time and money. I had to constantly chase up the guy for the work because I paid him in advance and did not deliver when it was supposed to and kept on dragging the job. When it was finally completed, the posts were done on forums which has nothing to do with my site. This means that any links to my site will look spammy. Here’s the scenario, you’re browsing a technology forum and you find herbal remedies links. Now this looks weird right because they are not even closely related. The second problem I had was that even on related niche forums, you cannot just drop a link so that you can have a backlink because it’s spamming at the end of the day and what I noticed is that people on the forums have started to get frustrated seeing this new guy on the forum dropping links to a site which does not have any connection with what is being discussed. You can only drop a link when it makes sense in the discussion!

Now the main problem with that is that it hurts your brand name. People who see these links will automatically think that your site is a spammy one and you start losing credibility online. Word can spread like fire and in no time you can see that your website is losing traffic and becomes blacklisted. Any major drawback with off-page optimisation is that you do not have control over the backlinks created and hence cannot get them removed. If it was your own site, you can do that easily but when you have thousands of links spread on thousands websites, you cannot contact all the webmaster asking them to remove your links (well you can but it will take an enormous amount of time and you’re not guaranteed that all links will be removed). Google is a clever search engine and when it sees that links to your site are being dropped all over the place, it will soon penalise you.

It’s always better to do things yourself but in some circumstances, you can’t always do it. You should always try to protect your brand and find someone who will do a good job for you and not a crap work just to get your money. You can lose more than you bargained for if you get the wrong SEO guy to promote your site, so be careful.