Thursday, April 23, 2009

'Star Ideas' for Injustice and the Idea Star Singer 2008: The Katcheri (Classical Music) Round Performance of Somdas and the Mockery of Evaluation

(For Readers' Comments: Click Here)

The Idea Star Singer programme telecasted by the television channel Asianet is touted as the ideal platform for budding and emerging young singers to polish and develop their musical talents and skills. The programme is claimed as a ‘reality show’, its recorded and edited broadcast notwithstanding! The programme managers and the evaluating (judges) persons often proclaim from the roof top that they are adopting a transparent, unbiased and unprejudiced approach in evaluating the performances of the competitors/participants. Nonetheless, in many a situation the ‘show’ showed that it is not at all transparent, unbiased and unprejudiced. Recently also, they aired one episode to underscore this stark, plain truth.

This note is prompted by the subjective evaluation to the hilt made by the nondescript so-called celebrity expert judge Mr. Sankaran Namboothiri and the uncharacteristic remarks uttered by the evaluator named Mr. Sarath and the anchor lady while awarding marks and making remarks for the performance of competitor/participant Mr. Somadas in his—Classical music (Katcheri) round—telecasted on 9th of April, 2009 (8:30pm-09:30 pm slot). The latter lady, whether expressing her own views or resonating her masters’ (programme manager’s) voice, made a scathing remark against Somadas, wherein she attempted to seem that she is consoling Mr. Somadas but patently disgraced and humiliated him in an ‘artful and skilful’ manner. And pontificated in a subtle mode from the roof top of Idea Star Singer 2008 that; viewers—you should not vote through SMSs to Somadas as he rather made a poor performance and not at all a good singer and viewers—you are supposed to cast SMS votes to the singer whom ‘we designate’ as the good or best singer! Dear Asianet, your anchor lady was openly treading through a terrain which was not meant for an anchor lady. Or she naively plodded through the dotted lines under the diktat of top officials of the Asianet like the programme manager etc?

Now dissect the evaluation process.

Evaluation of Mr. Sankaran Namboothiri

After the Kutcheri performance of Somadas, he remarked that the performance was a disappointment and awarded only 3 (three) marks out of 25 (twenty five). Slicing the merit worthiness of his lengthy comments is inconsequential, as he awarded only 3 marks out of 25. Everyone knows that there are two important parameters in Kutcheri or in the rendition of Carnatic music and they are the composition (krithi or keerthana) and the improvisation (including raga, alapana, niraval,kalpana swaras, thanam,pallavi etc.). So the evaluation criteria used (Did he aware of any such objective evaluation criteria?!) for awarding marks must also take into account the presentation of these two parameters. Somdas might have failed (miserably?) in the presentation of its finer details, but the impregnable fact need to be conceded, that he made at least a very basic rendition of Kutcheri. Hence the interrogative is how much marks should be awarded for such a basic level of performance? Three marks amounts only twelve percent of the maximum marks. Mr. Sankaran Namboothiri never could argue that the Kutcheri performance of Somdas was not Kutcheri at all and it was something else. So long as it holds, Somdas deserved at least 35 (thirty five) percent of the maximum marks for his performance or at least 9 marks.


Over all, considering other aspects of his performance he deserved 11-12 marks at least, from the most sceptical and meticulous evaluator. But Sankaran Namboothiri gave only the pittance. Some may argue that the authority of the evaluator in awarding marks should not be questioned but it is only a wrong and naïve notion. Awarding marks is not charity. Objective evaluation criteria should be there and it must be made public at least to the participants and the evaluator indeed has the responsibility to justify his marking scheme with respect to that. Sorry to say that Mr. Sankaran Namboothiri miserably failed in all these respects.

The evaluation on the basis of awarding marks ranging from 0 to 25 implies that the marking is on a ‘continuous scale’. Such a scale requires a high level of expertise and finesse from the part of the evaluator. The evaluation scale implies that the evaluator has to distinguish, when arranged in an ascending or descending order, at least 25 different types of performance gradually inching towards pinnacle of perfection or abyss of rendition. Note that what is needed is not the parading of 25 different types of performance but what is required is, performances, arranged in an ascending or descending order gradually inching towards pinnacle of perfection or abyss of rendition.

Simply put, this distinction of 25 different types of performances is very impossible. It may be possible for a maestro or for a genius but it is beyond the reach of majority. It is worth noting the fact that only when we are able to distinguish 25 different performances, it would be objective and meaningful to give marks like 3, 15, 16, 17 and so on. In the presence of difficulty in making such a fine distinction one cannot claim that this performance deserves only 3 marks or 4 marks and so on. If one evaluator is adamant that this performance deserves only 3 marks, then he has to explain and show which performance deserve 2, 4, 5 marks and so on. Otherwise the awarding of marks cannot be considered as fair, objective and error free.

In the context of the above said difficulty, evaluators generally follow a common sense practice of adding ‘some more’ (1-3 marks) marks to counteract the above said type of error (due to difficulty in making fine distinctions) which arises from the subjectivity inherent in the evaluation based on marks. This practice has been followed with the sole objective of protecting the participant/competitor from any adverse judgement that arises from the flaws of the evaluation process pointed out above. It is to be noted that these additional marks may sometimes exaggerate the evaluation but it ensure that nobody is hurt or thrown into a disadvantageous situation for some other’s fault like the one explained above (in the instant case the fault is of Mr. Sankaran Namboothiri). To sum up, by awarding just three marks for Mr. Somadas, Mr. Sankaran Namboothiri conveyed that he is unwilling to compensate for the error that would creep into his own judgement and thereby made an erroneous evaluation, his apparent scrupulous posture throughout the commentary notwithstanding!

Imperative for Moderation

The theoretical justification of moderation implemented in the process of evaluation need to be punctuated. The element of subjectivity in the evaluation process and the consequent variation in marks awarded by different evaluators requisite a procedure often referred as ‘moderation’ to contend with the variability in marks awarded. Put simply, if there is considerable variation in marks awarded by different evaluators than what is normally expected, that variation is due to some extraneous reason and hence must be dealt with appropriately.

The statistical tool ‘standard deviation’ is used to quantify the variation in marks. Put simply, standard deviation (SD) is a measure of (average) variation in marks given by different evaluators. The SD is computed for all the evaluations and that historical SD is used to compare the SD of each evaluation. If there is any significant difference in SD for a particular evaluation with respect to the historical SD, it would be surmised that there was an extraneous influence in the evaluation process. That extraneous influence must be neutralised through the procedure of ‘moderation’ where in additional marks are given to bring the SD almost equal to the historical SD. The table given below furnishes the numerical example.






It can be seen that the SD ranges between 0.5—1.41 for the evaluations. But the SD for Somdas is four times higher than this and is 6.18. This requisite a correction and if moderation is implemented to bring back the SD from 6.18 to at least 1.49 for lending fairness, addition of at least 9.5 marks is required which would take the total marks to around 59. To wind up, addition of at least 10 marks are required to give Somdas justice and maintain objectivity and fairness in the evaluation process.

Comments of Mr. Sarath and the anchor lady

The prime responsibility of any evaluator is to perform the evaluation helpful to ‘perfect the performance’ in future, by instilling confidence into the competitor/participant. By scrupulously adhering to the evaluation criteria, the evaluator must dissect the performance to describe where it faltered and excelled and the ways through which it could be improved. Eventually, even a pathetic performer would amass confidence and become a star performer as a result of the evaluation. In the instant case, the evaluator Mr. Sarath with his uncharacteristic remarks, virtually discharged the confidence of the low profile yet unassuming and affable Somdas by making him tear publicly. Unacceptable behaviour from anybody, however great he may be, let alone one Mr. Sarath!

The anchor lady, after the performance of Somdas, as pointed out above, disgraced and humiliated Somdas in an ‘artful and skilful’ manner. She told that, it was only because of the audience (ie. with their SMSs) that he scraped through eighteen rounds. What is its implication? It has only one implication that Somdas came to this penultimate round not because of his musical talent and skills but just because of the countless SMSs he got. If this is not humiliation then tell Asianet, what else is this? It must be accentuated that this indiscreet anchor lady with this remarks even downplayed and contradicted the open stance of Asianet. Asianet through out took the stance that SMSs sent by the audience are counted as a proxy of evaluations made by the viewing public. When the anchor lady attempt to downplay, the much valued evaluations of the viewing public as per the official stance of Asianet, she is humiliating her own employer also! She also demeaned musical wizards like Johnson master etc who gave Somdas 23 marks or so with her remark that Somdas scraped through eighteen rounds only because of the SMS support of audience.

Or what that anchor lady was talking—the evaluations of the viewing public through SMSs, solicited under the auspices of Asianet, are just money spinning mechanisms or machinations? Dear Asianet, you owe an explanation.

Before winding up, one heartening aspect need to be highlighted and appreciated. It is the evaluation and the comments made by the evaluator Mr. Venuopal. His remarks were balanced and encouraging yet pointed out the flaws in the performance of Somdas. The single word remark of Ms. Usha Uthup, (“disappointment”) should also be put under the scanner (for judging improvement?!) but well could be ignored as she said only one word!

Hence, to restore credibility and to lend justice and fairness to Somdas, Asianet must make appropriate corrections. The top management of Asianet may not be in the thick of details of the programme. Hence, the top management should take appropriate steps to resolve the issue for the future of the show.

Wednesday, April 8, 2009

Readers' Comments on "Star Ideas" for Injustice and the Idea Star Singer 2008: The Katcheri Round Performance of Somdas and the Mockery of Evaluation

Sankholi said... (on April 27, 2009 5:46 AM at blog titled "Asianet Idea Star Singer: Evaluation Intrigues")

Awesome Review. It is happy to see people like u thinking on such aspects of society. Please make a review on the revenue side of the show if possible .

========================================================


Anonymous said... (April 29, 2009 11:50 PM at blog titled "Asianet Idea Star Singer: Evaluation Intrigues")

Sir, Are you a lecturer? By going through the review, it is felt that you are not at all qualified to be an evaluator. You actually belong to the school of thought run by the education minister 100% marks for all! Your designation, given here suggests that you are a lecturer in the PG dept. of a Govt. College. But there are no separate posts of lecturers for PG courses in Govt. Colleges. If you are an evaluator for PG courses, it is spring time for students. Actually, the celebrity guest gave 03 marks, even though the participant made a mockery of karnatic music!

Dear Friend,

Please note that I did not argue for giving full marks for any body, let alone Mr. Somdas. Nobody can argue that what Somdas performed was not Karnatic Music. It needs to be conceded that he indeed presented at least a very basic performance of Karnatic music. It might not have gone to the heights of the finer details of Karnatic music, true but it was indeed a performance in Karnatic Music. Lastly, I only mentioned that I am a lecturer at a post graduate department.

I am not afraid of critical comments and I respect your freedom to take any stand according to your views.

Santhosh T Varghese


======================================================

Vijay said... (on April 30, 2009 7:51 AM at blog titled "Asianet Idea Star Singer: Evaluation Intrigues")


Well said sir.
From the two seasons I have noticed that,

1) they will bring 1 person from a poor background and they will highlight his poverty, his poor parents, his moderate background etc to increase the TRP ratings, This will continue this till the penultimate round and then they will suddenly realise that he doesn’t know how to sing & wonder how he got thru this far without knowing anything about singing and they will start giving gyan to the voting public as to how you should use their responsibility properly. (I really pity these guys - Sanni & now Somu, I am sure there will be some guy in the next episode as well in the same genre- Asianet really needs them to mint money & for the TRP)

2) There will be one person with disability - Everything same as above except this person will be shown the door a little early. ( the idea is to give lot of opportunities to didi to shed tears - meaning TRP )

3) The best person always never win ( last time it was the S/E ( poor chap he resigned his job for this drama, this time I thought Rahul was better even though my personal favourite was Prashobh )

4) Judges will always compensate with marks to avoid losing the good singers ( whom they think, doesn’t mean that they are always right in their choice of best singers). You should also see how the anchor asks the celebrity guest first for the marks & the actual judges compensating for any so-called errors in the judgement of the celebrity judge. Most cases you will see one guy being awarded 81 marks & other guy being given 75, even though the person who got 75 must have sung better than the first guy. I guess your SD might explain this :)

5) Lack of transparency - Sometimes they say there is a difference of 50 marks & you will need lakhs & lakhs of SMS to compensate.
I mean why don’t they announce 1 Judge vote = x SMS, and why don’t they announce the # of SMS got by somebody at the end of the elimination

6) Lastly - lack of propriety and lack of all its synonyms - politeness, decorum, modesty, good manners, respectability, decency on the part of (barring a teeny weeny number) almost all the judges & especially the anchor


======================================================

Anonymous said... (on April 27, 2009 4:41 PM at blog titled "Facts Unmasked")

I agree 100% with your point of view. There is no transparency in selecting the winner. Renjini (the anchor lady )is trying to promote her favourite contestant. you have noticed she was requesting votes for one of the contestant who done a poor performance in one episode. Whatever be their commend Somdas has got talent .so that will help him to reach heights. Anchor lady should do her job only i.e anchoring not giving commends over the performance of the contestants.

Regards one regular viewer one.

======================================================

Anonymous said... (on May 7, 2009 4:29 PM at blog titled "Facts Unmasked")

Dear Santhosh sir,

You have said absolutely right what was in every genuine viewers mind since the cssical round of Mr.Somadas was aired. It was just clear and fabricated humiliation and indecency shown to a contestant. If Soma Das were in a foreign country, he could sue them for insulting him.(I don't know if this right is practised in India.)Also the anchor lady should not be allowed in a reputed show like this if Asianet gives any value to morality for the least am very ashamed to tell that there was some of her nude pictures posted in Mallu terminal and it shows she is not more than a sex worker. And the grand finale showed that the audience of Thiruvananthapuram gave little support or respect to her. Asianet should open their eyes and be more sensible. And once again I'd like to ask the innocent people of Kerala to wake up and think before sending SMs to your favourites. Better save that money for some in needy.