This one is for the love of my life. I thank Chris Hadfield for making your eyes shine.

# Category Archives: Uncategorized

Geraldo Rivera knows exactly why Trayvon Martin an unarmed black teenager was shot dead. He wore a hoodie, a hoodie on a black person is scary! we all know that, don’t we ? Only some people should wear hoodies, or hoods of any form.

Geraldo also offered a wonderful textbook notpology to all those offended by his extremely valuable advice.

Thanks Geraldo, we all get your advice. Now we know whose hoodies to be scare of and whose not to be scared of.

## Dinesh Derailed

Dinesh Trivedi just presented a Railway Budget. He also got fired. What happened? Well, he wasn’t ‘populist’ enough that’s what happened. He apparently did not keep the best interests of the ‘aam admi’ (common man) in mind while deciding to hike fares.

Mr Trivedi proposed a series of increases in Rail fares cutting across classes. The increases he proposed were given out as paise per kilometer. The opposition accused him of misrepresenting in paise what would actually be sizeable increases in passenger fare. Well here is the truth behind it, a paise/kilometer increase makes a lot of sense! The increase in rates has not been uniform across classes, for the suburban trains most indispensable for the common man, the increase comes up to Rs 2/100 kms. How many people use suburban trains to travel a 100kms? Only those people would face even a Rs 2 increase in their fares.

One thing that stands out clear is that while many support Mr Trivedi’s budget given its strong emphasis on safety , modernization and a reduction in the operating ratio, all dire requirements for the railways, very few oppose the fare increases as a whole. Not even Trinamool Congress Leader Mamta Bannerjee or her sidekick Derek O’Brien (He should have stuck with his know it all school kids at the Bournvia Quiz Contest) are opposed to fare hikes in the classes above the sleeper class, that is, the AC 2^{nd} and 1^{st} class. They oppose the increases in the lower classes as only that would affect the aam admi. This shows that everybody, even the hardcore communists, recognize the need to increase revenues within the railways. Before going into ways in which revenue can be increased, let’s look at why it needs to be increased.

1) Operating ratio has to be decreased, currently 95 rupees are spent for every 100 rupees earned by the railways. The railways are owned by the government, ruining its financial health would not be in the best interests of the aam admi.

2) SAFETY! – The Indian Railways transports a large sea of humanity. Around 30 million people travel by train every day! Given this figure the number of accidents may seem like a relatively small percentage, however is being a part of small percentage any consolation to those who lose their lives? Small percentages translate to large numbers in India, making safety a major priority. The list of accidents can be seen here on wiki. Safety costs money, Trivedi’s plan includes modernization of tracks, signaling systems and manning all level crossings. The Aam admi values his life.

3) Modernization-Hygiene- More Safety – How many of us who have travelled by our beloved Indian trains (even that really cool looking Shatabdi tween Chennai and Bangalore often preferred over flights) can call them hygienic ? The stench that accompanies the railways is so characteristic that its considered ‘un-Indian’ to complain about it. Some of us have noticed recently with glee that the open-toilet systems are changing; many of the trains now have greener toilets (Lalu Prasad Yadav initiated this venture). Under Trivedi’s plan 2,500 more coaches would have green toilets by 2013. The open toilet system is dreadfully unhygienic, especially for those with homes near railway tracks. It also corrodes the tracks and costs the railways around 350 crores of rupees per year.

4) Capacity! – Despite being such a large network serving countless number of people, there is always more demand for railway tickets. (We all have at some point woken up at 8am and restlessly hit the refresh button on the tatkal bookings site of IRCTC with our fingers crossed hoping to get lucky). Trivedi has set aside Rs 4410 crores to augment capacity.

5) RnD (Design) – The Indian Railways don’t really look like the ones in Japan do they? The railways are a solid system and do their job well, it could do with a revamp in design though. Trivedi plans to put money into a dedicated Railway design wing at the National Institute of Design.

6) Many more reasons can be found in the budget highlights.

To do all of the above Dinesh Trivedi has proposed borrowing from the market Rs 15,000 crores and also a nominal hike in the fares (after ten long years!).

The opposition thinks that fares should be increased only in the upper classes and not in the lower. I’d like to talk about why this is not such a great idea.

The Indian Railways is a price discriminating monopoly. Prices are different based on whether you are a student/senior citizen/physically handicapped/female senior citizen etc. Prices are different across different trains even for the same classes. The Indian railways also forces consumers to reveal their preferences by offering a range of products. All this is done with the differences in price elasticities in demand across the categories kept in mind. That is acknowledging the fact that increasing the prices have an impact on the quantity of ticket sales, and thereby on the revenue, but this impact is different across different categories.

While I do not have any data to substantiate my claims, I shall consider a case where there is a high increase in prices in the upper classes and none in the lower ones. I broadly guess that given the high fares in the AC 1^{st} and 2^{nd} classes and presence of low cost airlines as close substitutes, these classes would have relatively more elastic demand, that is an increase in prices would cut down travel by AC 1^{st} class a lot, Making them a lot less competitive when compared to airlines which have a great advantage of saving on time. (Make my trip allowed me to book tickets from Delhi to Chennai by a low cost airline at Rs 4,700 as long as I booked really early, the corresponding Rajdhani ticket price on a 1^{st} class was Rs 4,500). Some of the current AC travelers may also switch to the already overwhelmed second class sleepers given the large difference in prices, again bringing no new revenue and putting the ‘aam admi’ in further trouble with regard to lack of capacity.

For short distances buses might substitute the sleeper class trains, however as far as I have seen, the buses are more expensive than the trains and do not work very well for longer distances. Therefore the most inelastic segment of demand would be that for long distance travel by sleeper class, which is where the revenue killing has to be made. The price hike here is pretty nominal a Rs 5 per hundred kilometers is not going to put anybody off traveling.

We would all like safer more hygienic travel even if it meant shedding an extra hundred rupees. The hue and cry is over nothing. Kudos to Mr Trivedi for standing upright through this whole mess and refusing to roll back his changes despite being threatened with ouster. Those who oppose his changes should take a look at this.

## Comparing Classifiers – Revisiting Bayes

I have been quite interested in data and its analysis lately. One of the major tasks involved in dealing with data is classifying it. Is a customer credit worthy or not? Would a customer be interested in buying the latest flavor of ice cream? Or better still, which flavor/brand is she likely to choose? While these questions require predicting the future, more specifically they require you to classify people/objects into different bins based on what has been observed historically.

To address this issue many types of classifiers have been developed by mathematicians, statisticians and computer scientists. Most of these make some kind of assumptions about underlying data and are varied in their complexity as well as accuracy. As a rule of thumb, the more complex classifiers make less stringent assumptions about the underlying data and thereby give more accurate results for data which isn’t as well behaved as a statistician would ideally like it to be.

Since this piqued my interest I decided to test out two well known classifiers very varied in their level of complexity on the famous iris data. The classifiers I tried out are the Naïve Bayes Classifier and the Multinomial Logistic Regression Model.

I think I’ll talk a little about the data first, the Iris dataset is pretty famous and is available as one of the pre-loaded data sets in R(Open source Statistical Software).

The dataset is a table of observations collected by the botanist Edgar Anderson. It has measurements of the Petal Width, Petal Height, Sepal Width and Sepal Height of three species of the Iris flower namely Iris Setosa, Iris Virginica and Iris Versicolor.

Sepal.Length | Sepal.Width | Petal.Length | Petal.Width | Species |

6.2 | 2.9 | 4.3 | 1.3 | versicolor |

4.8 | 3.4 | 1.9 | 0.2 | setosa |

6.4 | 3.2 | 4.5 | 1.5 | versicolor |

6.6 | 2.9 | 4.6 | 1.3 | versicolor |

7.9 | 3.8 | 6.4 | 2.0 | virginica |

6.0 | 2.9 | 4.5 | 1.5 | versicolor |

Our objective would be to create this classifier that could learn from this data and classify future observations about iris flowers into one of these three species based on these parameters.

The Naïve Bayes classifer is an extremely simple classifier based on Bayes Theorem. It makes the strict assumption that each of the attributes (Petal Width , Petal Height, Sepal Width) are conditionally independent that is the probability of a flower having a larger petal width wouldn’t depend on the fact that it has a large petal length, once you know which type of flower species it is. If this is true, we would expect that there wouldn’t be any correlation between a set of attributes within a flower species.

A quick look at the scatter plot below (click and expand the gallery) would tell us that this isn’t exactly true, looking at the third box on the last row, there is an evident correlation between Petal width and Petal Length given that the species is Versicolor (Red). There are plenty of observations which look not too correlated as well. I am going to go ahead and use the Naïve Bayes Classifier and see what it does anyway. Naïve Bayes is a classifier whose algorithm breaks down into simple counting and so its very easy to understand and computationally simple to implement. Professore Andrew Moore’s website is an excellent source for understanding these and other algorithms in data analysis.

The Mulinomial Logistic Regression Model uses a more complex estimation process. It is an extension to multiple groups of the logistic regression model which provides estimators for data which can be classified into binary groups. Here we use multinomial logit rather than the basic logit model as the data has 3 groups namely Versicolor, Setosa and Virginica. Mainly a regression analysis is done for two of these classes while one of them is regarded as the base class. Therefore 5(number of parameters = 4, namely Sepal Length, Sepal Width, Petal Length and Petal Width + 1 (intercept)) parameters are estimated for each regression, bringing it to a total of 10 parameters. The estimation involves a Bayesian concept called Maximum a Posteriori estimation which is far more complex than the simple counting of the Naïve Bayes Classifier.

So why go into all this trouble? The answer is that multinomial logistic regression models make far less stricter assumptions about the nature of the underlying data. The assumption made in the case of this model is one called Independence of Irrelevant Alternatives. That is adding another category to the existing three categories of species should not change the relative odds between any two of the species already listed. This condition only applies to the choice variable (Species) and says nothing about the attributes unlike the conditional independence assumption in the Naïve Bayes classifier.

So I used both the classifiers. The iris dataset has 150 rows of data , that is 150 flowers were observed and recorded in terms of the attributes mentioned and their species. In order to test these classifiers, I used only 75% of the data to train them and the other 25% to test the predictions made by them with the true value of categories that are available.

A simple function to calculate their respective error rates/accuracy was written in R.

The result ? Mlogit was more accurate but only marginally so. Over 10 runs, on average the Naïve Bayes classifier gave 95% accurate results and the Mlogit gave 97% accurate results. Small price to pay for getting rid of complex computation ? Maybe not if you have powerful processors and efficient algorithms but every mistake could cost you a lot, Maybe so if you just want to make quick classifications and a little loss in accuracy wont cost you a lot compared to the gains in speed. Points to ponder..?

*You can download the code to do these comparisons here — It would work with any data frame as long as the last column is the choice variable.(Or so I believe)* .Classifiers

Photo Credits

Wiki Commons –

Setosa – Radomil

Versicolor – Daniel Langlois

Virginica – Frank Mayfield