Using full text search with fluent nhibernate

Nhibernate (or fluent) doesn’t support full text searching because full text search differs from one database to another. Nhibernate allows you to switch your data store (mysql, ms sql server, oracle etc) easily and if it were to support full text searching, then it will tie itself to the sql required to perform full text search for a particular database. So you need to have a workaround to get full text search working with fluent nhibernate. Here’s my solution:


public virtual IList<RelatedArticle> RelatedArticles(ISession session)
 {
 IQuery query = session.CreateQuery("select new RelatedArticle(A.Title, A.Slug, A.Description, C.Slug) from Article as A inner join A.Category as C WHERE  CONTAINS((A.Subject, A.Content), :title)");
 query.SetParameter("title", "\"" + Title + "\"");

 // no more than 5 related articles
 return query.SetMaxResults(5).List<RelatedArticle>();
 }

This works fine but contains does not give me the option to sort out the result by rank, that is, the most relevant result first. Note that RelatedArticle is an unrelated class mapped using fluent nhibernate to Article class and that by using session.CreateQuery, the query needs to be written in HQL (Hibernate Query Language) and not native SQL. To be able to do that now, I have to use CONTAINSTABLE or FREETEXTTABLE (I’m choosing freetexttable because I don’t need the exact phrase but something related to that). However FREETEXTTABLE in Microsoft SQL Server will return a table and you can no longer use HQL but have to resort to native SQL using session.CreateSQLQuery and this is where the code will fail. Since I’m mapped RelatedArticle using ImportType<RelatedArticle> in ArticleMap class, nhibernate complains withe the following error message “No persister for: RelatedArticle”. What this means is that it is having difficulty finding the mapping for the RelatedArticle class. My solution was as follows:


public virtual IList<RelatedArticle> RelatedArticles(ISession session)
 {
 IQuery query = session.CreateSQLQuery("select A.Title, A.Slug, A.Description, C.Slug AS CategorySlug from Article A inner join Category C ON C.Id = A.CategoryId INNER JOIN FREETEXTTABLE (Article, (Subject, Content), :title, 10) AS ktl ON A.Id = ktl.[KEY] WHERE ORDER BY ktl.RANK DESC").SetResultTransformer(NHibernate.Transform.Transformers.AliasToBean(typeof(RelatedArticle)));
 query.SetParameter("title", Subject);

 return query.SetMaxResults(5).List<RelatedArticle>();
 }

I’m not sure why CreateSQLQuery is having problems while HQL could execute the code without complaining though but I ain’t got time to sort it out yet. As long as the above is working, I suppose I can come back to it later.

Mapping an unrelated class in Fluent NHibernate

There are times when you will need to map unrelated classes through fluent nhibernate. By an unrelated class, I mean a class which is not directly mapped to a database table. This is useful when you create DTOs (Data Transfer Object) for a lightweight object to pass around in your application. Consider the following – you have an Employee class with Id, Firstname, Lastname, DoB, Marital Status, JoinedDate, DeptId. Now in your presentation layer, if you just need to display the Id and Lastname of the employee, you might consider writing a subclass of Employee and called it EmployeeDTO which has only Id and Firstname as fields. This is just to simplify things.

Your Employee class will be mapped to your Employee table through EmployeeMap. To map the EmployeeDTO, you will need to use ImportType in fluent nhibernate as follows:


public class EmployeeMap : ClassMap<Employee>
 {
 public EmployeeMap()
 {
 ImportType<EmployeeDTO>();

 Id(x => x.Id);
 // rest of your mapping here
 }
 }

With this in place, your unrelated class is now mapped to your Employee class.

How to fetch data in a class which is not directly mapped to a table in fluent nhibernate?


public virtual IList<EmployeeDTO> FindEmployees(ISession session)
 {
 IQuery query = session.CreateQuery("select new EmployeeDTO(E.Id, E.Firstname) from Employee as E WHERE E.DeptId = :deptid");
 query.SetParameter("deptid", 1);
 return query.SetMaxResults(5).List<EmployeeDTO>();
 }

Notice that the above query is using HQL (Hibernate Query Language) and is already casting the result to EmployeeDTO type which means you’re getting back a strongly typed object and also we’re using parameterised queries to prevent sql injection.

Invalid postback or callback argument in ASP.NET

While converting some php code to asp.net, I ran into the famous “Invalid postback or callback argument” on one of my webpages. This problem usually occurs when dealing with AJAX calls but my one was completely unrelated to that because I was not using any ajax at that point. The simplest solution that many people choose is to turn off Event Validation in their pages but you should never have to do that as the consequences can be far worse. You should therefore take the time to debug your application to understand where the problem is coming from.

For me the problem was caused by a nested form tag. ASP.NET allows only one form (with the runat attribute) per webpage whilst in PHP, you can have as many form tags as your heart wishes. While porting the code, I forgot to remove the nested form tag for a search functionality (actually I deliberately left the code for the form because I was testing something else) and that’s why asp.net was complaining about for its event validation. Once I’ve got that removed so that there was only one form per page, the error disappeared and the page worked as it should.

I might come back to this post to update with problems using AJAX and invalid callbacks when I implement ajax calls on the site I’m working on but for the time being, that should be it.

Cleaning up ASP.NET head tag with Control Adapters

ASP.NET offers a lot of functionality but has always suffered from being SEO unfriendly. It injects a lot of javascript for postback, tables rather than div for server controls like GridView and inline meta tags in the head section of the HTML which makes it difficult to read. The title tag is the one most important part of search engine optimisation and when you look at an asp.net page you will find that the title tag is spread on 3 lines in the html source code as follows:

<head id="Head"><title>
	Test Page
</title><meta name="keywords" content="test keywords" />...
</head>

If you have meta description tag, css, favicon or any other elements in the head section, they will be rendered inline. The title has white spaces around it. So how do you solve this problem? The answer lies in control adapters. You basically override the default rendering of the controls as follows:

<pre>using System.Web.UI;
using System.Web.UI.Adapters;

namespace Gis.CMS.Presentation
{
 public class HtmlTitleAdapter : ControlAdapter
 {
 protected override void Render(HtmlTextWriter writer)
 {
 writer.WriteLine();
 writer.WriteFullBeginTag("title");
 writer.Write(Page.Title);
 writer.WriteEndTag("title");
 writer.WriteLine();
 }
 }
}

And you will need a .browser file in your App_Browsers folder in your .net application as follows:

<pre><browsers>
 <browser refID="Default">
 <controlAdapters>
 <adapter controlType="System.Web.UI.HtmlControls.HtmlTitle"
 adapterType="Gis.CMS.Presentation.HtmlTitleAdapter" />
 </controlAdapters>
 </browser>
</browsers>

The above code is only to fix the title so that it displays properly:

<pre><title>Test Page</title>

I haven't included code to fix meta tags like keywords and description and link for css and favicon though to keep things simple.

Caution
If you use the Page.MetaKeywords property available in .net 4, you will find that your keywords are not displayed when using the control adapter. You will have to do one of the following to fix it:
Insert a blank keywords meta tag in your html and then override it in your code behind or programmatically add the meta keywords elements using HtmlMeta control.

ASP.NET ValidateRequest not working

ValidateRequest in ASP.NET is a really good security measure as it prevents cross-site scripting which basically means that it prevents malicious codes from being injected into your application. A typical example would be if you have a textbox on your webpage to capture the name of a visitor and the user enters something the following : <script>alert(‘hello world’);</script> instead of his name, then this could lead to potential hacks. Imagine if your code was to immediately write back the name of the person after submitting the form – instead of writing back the name of the person, the above script would run and an alert box would pop up to say “hello world”. Although this will not cause any problems, people with bad intention can really mess up your application and hack your website as well as the people visiting your site.

This is where ASP.NET ValidateRequest comes into play. When set to true on a page, it will raise an exception if it finds that unsafe html is being sent through your web form. In previous versions of asp.net, you could just add ValidateRequest=”false” if you wanted to turn off this feature but in ASP.NET version 4, you need to add <httpRuntime requestValidationMode=”2.0″ /> to your web.config file under <system.web>. Just including ValidateRequest=”false” in the page directive  is not enough because asp.net 4 sets ValidateRequest to true for all requests by default and the only way to set it to false on a page by page basis is to tell asp.net to use the validation mode of asp.net by including the above code in the web.config file.

Although there are scenarios where you want to disable ValidateRequest like if you want people to send code samples or include formatting tags like bold, italics in comments, you need to ensure that you are validating the input on the server side before you process them. You can use htmlencode so that any XSS is rendered harmless.

So if ValidateRequest is not working for you, then make sure that you have included the code above in your web.config if you’re using asp.net 4 runtime environment.

Adsense earnings down – Find out why!

Webmasters who have chosen to work with Adsense to generate income from their websites need to understand how volatile the whole Adsense system is. You cannot expect to earn a fixed amount of money with it and that’s in some way a curse and a blessing as well. This means you have the opportunity to get even more cash if you are able to grow your website’s traffic and conversion. However many times you will find that even though you are getting more traffic than before, you are either not getting any more money from the extra traffic or your adsense revenue is decreasing.

The hard work is not paying off and you’re getting frustrated and disappointed as well. This is why it is imperative to understand what may cause fluctuations in adsense income. Below is a list of the reasons why your adsense earnings might have dropped:

You have lost traffic

This means that your rankings as not as good as they were before in the SERPs. You will need to find which keywords have dropped from the search engine results pages and do more SEO work on them. If you have changed the content of your webpages (either by adding/removing text or changing theme/layout of your website), then you may lose your rankings as well. Many times changing the title tag could lead to drop in rankings.

You are being smart priced

Smart pricing is Google’s way to protect advertisers. If the clicks leading to an advertiser’s site are not converting for the latter, then you may be smart priced to compensate for the advertiser’s loss. For example, if mortgage ads are appearing on your site and the visitors clicking to the advertiser’s site are not filling out their mortgage quote form, instead of getting the usual $3 for a click, you may only get $0.50 for the same click. The advertiser defines the conversion that he wants to be actionable on his site and it can be anything from having a sale to signing up to his newsletter.

One way to combat smart pricing is to block the advertiser for at least a week. This works because smart pricing is reset or recalculated/re-applied on a weekly basis. If you do not tackle smart pricing, you may earn considerably less with adsense because one poorly converting ad can lead to your whole adsense account being smartpriced.

US dollars are getting weaker against your local currency

Many people do not realise that adwords accounts are managed in US dollars. So if you’re in the UK, you’re still going to pay in US dollars for the keywords you choose in your adwords campaign. As a publisher, unless you’re in the US, you are going to see your adsense earnings in your local currency, that is, the conversion from US dollars to whatever currency is local to you. This is done on the fly from exchange rates on the day you’re looking at your adsense report.

So if an advertiser was bidding $1.5 for a keyword and the conversion from US dollars to say British Pound was 1:1 (for the sake of simplicity), you will get £1.5 for that click. Now if the exchange rate changes and the British Pound gets stronger (US dollar weakens) and the new exchange rate becomes 1:1.5 which means you get $1.5 for each British pound, you will then receive only £1 for the same click instead of £1.5. Therefore as long as the US dollar remains strong against your local currency, you are fine but the time that it gets weak, you will see a drop in your earnings provided everything else stays the same.

You have seasonal content (Xmas, Easter, Mother’s day etc)

If your website has content which depends on the season, you will see a boost in traffic when that period approaches but when it ends, you will lose the visitors as well as the money. For instance, if you operate a site which deals with Chrismas, weeks leading up to Xmas will be really good for you in terms of keywords searches for Xmas but when that festival ends, you will barely see any traffic stemming from those keywords. Think about it – would you search for Chrismas presents in March? No!

Your website has content to do with trends

If you’re writing news articles, then chances are that when the news becomes outdated, you will lose the traffic. When something is hot or trendy, there are many searches for that particular keyword of interest. If you have content which talks about such things, then you would get a lot of attention from visitors which will nicely convert for you but as soon as the trend is over, you can forget about it all.

Adwords advertisers are stopping their campaigns or reducing their maximum CPC

Depending on what works best for advertisers, they may introduce new campaigns at different times of the year or pulling old ones out. If advertisers find that some months work better for them, they will stick to these months rather than having a campaign running throughout the year. For example, an advertiser will advertise only during March/April for Easter promotions and when that period is over, so will the revenue coming from these promotions. So if you don’t have the advertiser for the keywords for your content, you won’t make a single penny.

The second reason relating to advertisers is quite interesting. Since Adwords works on a bidding system, if there are more advertisers for the same keywords, you will get a higher CPC and if there are only a small number of competitors, then obviously, you will be paid peanuts. However sometimes advertisers may lower the Cost Per Click depending on the time of the day or country of origin for the click. This again has do to with what works best for the advertiser. A UK advertiser may reduce the CPC during night times because he knows there are not going to be any conversions at this time or he may decrease or reject clicks coming from outside the UK because ke knows foreigners will not buy anything from him.

Better conversion rate for the advertiser doesn’t mean more money for the publisher

Adsense tries to serve the ads which would give you the maximum revenue. This does not mean that the adsense ads which appear on your website are the highest paying ads for the keywords matching your content. Contrary to common belief, ads go through the bidding system to elect the highest bidder but at the same time are filtered down to the ones which have a better Click Through Rate (CTR). So you might be serving an ad which gives you 25% CTR but costs $2 a click rather than an ads which has a 10% CTR and costs $5 per click. A high CTR is good because it means the ad is contextually relevant to your content and will give you more money but if you get 100 impressions with a 25% CTR and making $1 per click, you’ll get $25 in adsense revenue compared to $50 for an ad which has a 10% CTR and costing $5 per click for the same number of impressions. So watch out for those supposedly higher CTR ads with lower value or placement targeted ads as they could be eating your income.

Conclusion

There are many reasons why your adsense earnings can drop and the reasons are the more common ones. I haven’t included targeting high paying keywords because I’m focusing here on why your revenue are decreasing rather than how you can make more with Google Adsense. If you’re in a niche with a relatively low CPC, it might be worth trying something else than adsense.