Avinashsing Sunkur

I was playing around with some table creation and deletion and I noticed that after issuing a DROP command for a particular table, I was not able to recreate the table. I’ve installed MySQL on my Windows 7 laptop and by default tables are created with the InnoDB engine. This is how the problem occurs – you create a table (testtable) and immediately after that you issue the following command:

DROP TABLE testtable;

You will get a command executed successfully but when you try to create a table with the same name, you will get the following error:

MySql Error Code 1050 – Table ‘testtable’ already exists

Now if you look at your database schema, you will not find the table name and the same applies if you issue a “SHOW TABLES” command. If you look at the directory where MySQL data are stored, you will not find the .frm file for that specific table. The location is C:\ProgramData\MySQL\MySQL Server 5.1\data (not that it’s not Program Files or Program Files (x86) but ProgramData on Windows 7.

To solve the problem, you will need to restart the MySQL service. Once MySQL is restarted, you can create a table with the same name as the table you dropped earlier. If you drop another table and want to recreate a table with the same name, you will have to stop/start MySQL service again. I think MySQL stores a dictionary of the tables created and only checks whether they are still valid at a regular interval or when the service is restarted.

Let me tell you first how I got into writing my 1st python program. I wanted to have a look at the database schema of OSQA (Open Source Question Answer) and there was no database model available online. So my only other choice was to download the source code and setup the application. Now OSQA runs on Django, python and MySql. I already have a running instance on MySql on Windows 7. So I had to download python and installed it on Windows. I got an MSI installer from the official python website and it installed with no problems. OSQA requires you to run manage.py syncdb -all to create the database it requires. I tried that command in IDLE but it complained about Django. So I went to download Django framework and got it installed as well. Since OSQA source files was in a different directory from where python was installed, I was having a hard time trying to get the command to work. Therefore I tested a line command from the IDLE command prompt (print hello world). This worked which means the python interpreter is OK. I then saved this code in a file called hello.py and saved that file in the OSQA folder. Surely this must work now but it didn’t. You have to add python to the Environment variables on Windows to ensure it works correctly. I also added the Scripts folder to environment variables (C:\Python27\Scripts). Once that is done, hello.py worked fine. So this was my first Hello World python program.

Now my problem was still not solved. The manage.py program would still not run and when I checked the errors, I saw that it was looking for MySql library. MySql for Python was required and a few other libraries as well to make OSQA work. I was only going to add the libraries which are required to get the database schema, so I downloaded MySql for Python and tried to install it on Windows 7. Big problem! The list of problems are as follows:

1. If you don’t have development bits for MySql, it won’t work. So modify your current instance of MySql and check that development bits are selected. After successful install, you will have an include folder in your current MySql installation path with .h files.
2. Second problem is that on Windows 7, the registry for MySql is located at SOFTWARE\Wow6432Node\MySQL AB\MySQL Server 5.1 instead of SOFTWARE\MySQL AB\MySQL Server 5.1 because Windows 7 is 64 bits rather than 32. So make sure the site.cfg file of MySQL-python-1.2.3 has the above registry and change MySQL Server 5.0 to 5.1 if you’re running MySQL 5.1
3. Third problem is that you need to modify your C:/Python27/Lib/distutils/msvc9compiler.py and add ld_args.append(‘/MANIFEST’) after ld_args.append(‘/MANIFESTFILE:’ + temp_manifest). Make sure you edit this file in IDLE (the python scripter) otherwise you’ll get errors because of white space. It will complain about indentation. I tried editing this in Notepad++ and although it looked fine there, when I open the file in IDLE, there was spacing problems.
4. Fourth problem is that MySQL-python-1.2.3 requires Visual Studio 2008 and since I’m running Visual Studio 2010, it’s not getting the right libraries to install MySQL for Python.

I’ve left it as that for the time being because it’s just too much work for something that’s not really necessary!

UPDATE : I managed to install MySQL-Python-1.2.3 through an installer instead (http://www.codegood.com/archives/129)

UPDATE 2 : On top on MySql for Python, I had to install Markdown and Html5lib as well to get the OSQA database created.

Debugging your application is a great way to find the cause of a problem. However you may sometime encounter the dreaded “The breakpoint will not currently be hit. No symbols have been loaded for this document.” error. You can find fixes to this problem by following this link.

If like me, none of the above methods work for you, then there could be something else that’s preventing the debugger from working. So I tried creating a new application (web) using Visual Studio 2010 and set a breakpoint to see if it works. As soon as I start debugging from VS2010, the ASP.NET Web Development server kicks in and tries to open the webpage and then switches back to visual studio on my breakpoint. This means that debugging should work fine. I tried it on Firefox and Internet Explorer but it doesn’t work though. By that time, I’ve come to the conclusion that the debugger is not attaching itself correctly to the web server which is hosting my application. Note that the default web server that comes with visual studio opens the webpage either in Firefox or Internet Explorer but uses something like localhost:45609 to accomplish that.

My web app was hosted on IIS7.5 under Windows 7 and I had to attach the debugger to w3wp.exe worker process responsible for asp.net requests. Once this is done, the breakpoints work like a charm.

I’m a .net guy and love programming in c#. It’s rare for me to look at other programming languages because I tend to stick to aso.net because I can easily get a project off the grounds as I know pretty much the inside outs of the syntax etc. I can code in php as well but I never really liked it as I get more control in .net. From a developer’s point of view, coding in the latest technology is the best way to stay motivated and learn the new stuff. That’s what we want cutting edge technology! However from a business perspective, it’s a different story. For the business, profit is what dictates its strategy. If there are no real benefit to using a certain technology, then most likely it’s not going to be adopted.

For a small company, microsoft products can be quite expensive. For easy programming in .net, you really need to have Visual Studio. Of course you need IIS as your web server and many .net programmers will want Microsoft SQL Server as well. This is all good but the costs of these things can amount to a lot. So you small business owners ditch .net for open source technologies? Well that really depends on the situation. Apache is a good webserver and is free as well as MySql. The standard approach is to use PHP to develop the website and host it on a linux box which is considerably cheaper than having to pay the licence for Windows Server (2008) and getting Microsoft SQL Server 2005/2008. With the Windows Server, you get IIS as your web server to run your website but you still need a database. MS SQL Server is very expensive, it costs pretty much the same as a dedicated server.

When I was looking at a dedicated Windows Server, you have to pay £99 per month for the server, £25 a month for Windows 2008 Standard Edition (the licence fees) and £99 for Microsoft SQL Server 2005/2008 Standard Edition (the licence fees). Now these are significant costs as you end up paying £223 per month for a .net site compared to £99 a month for a site done in PHP running on Apache with a MySql database.

When I thought about the costs involved, I wantet to switch to open source technologies so that I don’t end up paying so much money for my websites. I was looking into Django as it claims to make website designing quicker and better but after my extensive research, I came to the conclusion that it was not really worth it. As I explained in the post on DJango, there are a lot of things to consider before making such a decision. For example, putting aside the differences in programming syntax for Django (python) vs asp.net, I conclude that it’s going to take me more time to deploy a website using Django because of various configuration needed at the server level. I think that an IDE is really important when programming. Take Visual Studio for instance, you have intellisense which makes it quicker for you to code. You can go to a method’s definition to see what’s happening there and have debugging facility as well. You can write unit tests to see whether things are working as expected. You get to compile your website and see errors beforehand. I’m probably biased but I think I’ve got more control with .net than the open source languages.

There are a lot of advantages when programming in asp.net and it is a true object oriented programming language (C#). However I still find that Microsoft SQL Server is very expensive. I wouldn’t change C# as my main coding language but I’ll definitely look into an open source database like MySql or postgresql. The reasoning behind this is very simple; I do not need the special capabilities of MS SQL Server at the moment, I simply need it as a storage engine. I use nHibernate for the database abstraction and that makes it even easier to switch to another data store. At the end of the day, you have to pay to get a good product with all the facilities that you want. Just like you pay for MySql if you want their extended support, you need to pay Microsoft for using their technology!

Django seems to be the new craze these days like Ruby on Rails was some time ago. The adoption of Django seems to be rising each month while Ruby seems to be quiet. So what exactly is Django about?

Django was developed by a group of developers with a budget restriction. So they made use of things that were free – chose Apache as the webserver, mysql as the database and python as the programming language. That’s all good and it gets even better because Django is sort of like Model View Controller (MVC). Why sort of? It’s just because the view is not really a view; you need to have a template (the real view) associated with it. From the looks of it, it’s quite easy to program in django, however when you’re used to certain programming language like C#, it might not be so obvious.

The problem with Django is that you really need to know Apache or Linux. At the moment, it is not widely supported. So if you have built your website in Django and want to publish it, you might run into problems because chances are that your hosting company doesn’t know how to get Django to work and if it’s shared hosting, they probably won’t give you access to modify the server properties. If like me, you’re not really into that apache stuff, then it will be more of a problem because apache needs to be configured a certain way. On top of that, django does not serve media files from what I understand, so your images, videos, audios and style sheets will have to be passed on to apache through another configuration. I personally find it hard to believe that the guys who developed django said it was meant to make web site development faster and cheaper.

When programming in django, you will have to use the command line utility extensively to create projects, compile etc. It’s like going back to the DOS era. There’s no IDE specific to django at the moment although you could get plugins to highlight the syntax. It’s like programming in notepad with no intellisense.

Personally I don’t think Django is the right choice for me. It can be for other people but the disadvantages far outweigh the advantages for me.

Nhibernate (or fluent) doesn’t support full text searching because full text search differs from one database to another. Nhibernate allows you to switch your data store (mysql, ms sql server, oracle etc) easily and if it were to support full text searching, then it will tie itself to the sql required to perform full text search for a particular database. So you need to have a workaround to get full text search working with fluent nhibernate. Here’s my solution:

public virtual IList<RelatedArticle> RelatedArticles(ISession session)
 {
 IQuery query = session.CreateQuery("select new RelatedArticle(A.Title, A.Slug, A.Description, C.Slug) from Article as A inner join A.Category as C WHERE  CONTAINS((A.Subject, A.Content), :title)");
 query.SetParameter("title", "\"" + Title + "\"");

 // no more than 5 related articles
 return query.SetMaxResults(5).List<RelatedArticle>();
 }

This works fine but contains does not give me the option to sort out the result by rank, that is, the most relevant result first. Note that RelatedArticle is an unrelated class mapped using fluent nhibernate to Article class and that by using session.CreateQuery, the query needs to be written in HQL (Hibernate Query Language) and not native SQL. To be able to do that now, I have to use CONTAINSTABLE or FREETEXTTABLE (I’m choosing freetexttable because I don’t need the exact phrase but something related to that). However FREETEXTTABLE in Microsoft SQL Server will return a table and you can no longer use HQL but have to resort to native SQL using session.CreateSQLQuery and this is where the code will fail. Since I’m mapped RelatedArticle using ImportType<RelatedArticle> in ArticleMap class, nhibernate complains withe the following error message “No persister for: RelatedArticle”. What this means is that it is having difficulty finding the mapping for the RelatedArticle class. My solution was as follows:

public virtual IList<RelatedArticle> RelatedArticles(ISession session)
 {
 IQuery query = session.CreateSQLQuery("select A.Title, A.Slug, A.Description, C.Slug AS CategorySlug from Article A inner join Category C ON C.Id = A.CategoryId INNER JOIN FREETEXTTABLE (Article, (Subject, Content), :title, 10) AS ktl ON A.Id = ktl.[KEY] WHERE ORDER BY ktl.RANK DESC").SetResultTransformer(NHibernate.Transform.Transformers.AliasToBean(typeof(RelatedArticle)));
 query.SetParameter("title", Subject);

 return query.SetMaxResults(5).List<RelatedArticle>();
 }

I’m not sure why CreateSQLQuery is having problems while HQL could execute the code without complaining though but I ain’t got time to sort it out yet. As long as the above is working, I suppose I can come back to it later.

There are times when you will need to map unrelated classes through fluent nhibernate. By an unrelated class, I mean a class which is not directly mapped to a database table. This is useful when you create DTOs (Data Transfer Object) for a lightweight object to pass around in your application. Consider the following – you have an Employee class with Id, Firstname, Lastname, DoB, Marital Status, JoinedDate, DeptId. Now in your presentation layer, if you just need to display the Id and Lastname of the employee, you might consider writing a subclass of Employee and called it EmployeeDTO which has only Id and Firstname as fields. This is just to simplify things.

Your Employee class will be mapped to your Employee table through EmployeeMap. To map the EmployeeDTO, you will need to use ImportType in fluent nhibernate as follows:

public class EmployeeMap : ClassMap<Employee>
 {
 public EmployeeMap()
 {
 ImportType<EmployeeDTO>();

 Id(x => x.Id);
 // rest of your mapping here
 }
 }

With this in place, your unrelated class is now mapped to your Employee class.

How to fetch data in a class which is not directly mapped to a table in fluent nhibernate?

public virtual IList<EmployeeDTO> FindEmployees(ISession session)
 {
 IQuery query = session.CreateQuery("select new EmployeeDTO(E.Id, E.Firstname) from Employee as E WHERE E.DeptId = :deptid");
 query.SetParameter("deptid", 1);
 return query.SetMaxResults(5).List<EmployeeDTO>();
 }

Notice that the above query is using HQL (Hibernate Query Language) and is already casting the result to EmployeeDTO type which means you’re getting back a strongly typed object and also we’re using parameterised queries to prevent sql injection.

While converting some php code to asp.net, I ran into the famous “Invalid postback or callback argument” on one of my webpages. This problem usually occurs when dealing with AJAX calls but my one was completely unrelated to that because I was not using any ajax at that point. The simplest solution that many people choose is to turn off Event Validation in their pages but you should never have to do that as the consequences can be far worse. You should therefore take the time to debug your application to understand where the problem is coming from.

For me the problem was caused by a nested form tag. ASP.NET allows only one form (with the runat attribute) per webpage whilst in PHP, you can have as many form tags as your heart wishes. While porting the code, I forgot to remove the nested form tag for a search functionality (actually I deliberately left the code for the form because I was testing something else) and that’s why asp.net was complaining about for its event validation. Once I’ve got that removed so that there was only one form per page, the error disappeared and the page worked as it should.

I might come back to this post to update with problems using AJAX and invalid callbacks when I implement ajax calls on the site I’m working on but for the time being, that should be it.

ASP.NET offers a lot of functionality but has always suffered from being SEO unfriendly. It injects a lot of javascript for postback, tables rather than div for server controls like GridView and inline meta tags in the head section of the HTML which makes it difficult to read. The title tag is the one most important part of search engine optimisation and when you look at an asp.net page you will find that the title tag is spread on 3 lines in the html source code as follows:

<head id="Head"><title>
	Test Page
</title><meta name="keywords" content="test keywords" />...
</head>

If you have meta description tag, css, favicon or any other elements in the head section, they will be rendered inline. The title has white spaces around it. So how do you solve this problem? The answer lies in control adapters. You basically override the default rendering of the controls as follows:
<pre>using System.Web.UI;
using System.Web.UI.Adapters;

namespace Gis.CMS.Presentation
{
 public class HtmlTitleAdapter : ControlAdapter
 {
 protected override void Render(HtmlTextWriter writer)
 {
 writer.WriteLine();
 writer.WriteFullBeginTag("title");
 writer.Write(Page.Title);
 writer.WriteEndTag("title");
 writer.WriteLine();
 }
 }
}

And you will need a .browser file in your App_Browsers folder in your .net application as follows:
<pre><browsers>
 <browser refID="Default">
 <controlAdapters>
 <adapter controlType="System.Web.UI.HtmlControls.HtmlTitle"
 adapterType="Gis.CMS.Presentation.HtmlTitleAdapter" />
 </controlAdapters>
 </browser>
</browsers>

The above code is only to fix the title so that it displays properly:
<pre><title>Test Page</title>

I haven't included code to fix meta tags like keywords and description and link for css and favicon though to keep things simple.
Caution

If you use the Page.MetaKeywords property available in .net 4, you will find that your keywords are not displayed when using the control adapter. You will have to do one of the following to fix it:

Insert a blank keywords meta tag in your html and then override it in your code behind or programmatically add the meta keywords elements using HtmlMeta control.

ValidateRequest in ASP.NET is a really good security measure as it prevents cross-site scripting which basically means that it prevents malicious codes from being injected into your application. A typical example would be if you have a textbox on your webpage to capture the name of a visitor and the user enters something the following : <script>alert(‘hello world’);</script> instead of his name, then this could lead to potential hacks. Imagine if your code was to immediately write back the name of the person after submitting the form – instead of writing back the name of the person, the above script would run and an alert box would pop up to say “hello world”. Although this will not cause any problems, people with bad intention can really mess up your application and hack your website as well as the people visiting your site.

This is where ASP.NET ValidateRequest comes into play. When set to true on a page, it will raise an exception if it finds that unsafe html is being sent through your web form. In previous versions of asp.net, you could just add ValidateRequest=”false” if you wanted to turn off this feature but in ASP.NET version 4, you need to add <httpRuntime requestValidationMode=”2.0″ /> to your web.config file under <system.web>. Just including ValidateRequest=”false” in the page directive  is not enough because asp.net 4 sets ValidateRequest to true for all requests by default and the only way to set it to false on a page by page basis is to tell asp.net to use the validation mode of asp.net by including the above code in the web.config file.

Although there are scenarios where you want to disable ValidateRequest like if you want people to send code samples or include formatting tags like bold, italics in comments, you need to ensure that you are validating the input on the server side before you process them. You can use htmlencode so that any XSS is rendered harmless.

So if ValidateRequest is not working for you, then make sure that you have included the code above in your web.config if you’re using asp.net 4 runtime environment.