The Idea Star Singer programme telecasted by the television channel Asianet is touted as the ideal platform for budding and emerging young singers to polish and develop their musical talents and skills. The programme is claimed as a ‘reality show’, its recorded and edited broadcast notwithstanding! The programme managers and the evaluating (judges) persons often proclaim from the roof top that they are adopting a transparent, unbiased and unprejudiced approach in evaluating the performances of the competitors/participants. Nonetheless, in many a situation the ‘show’ showed that it is not at all transparent, unbiased and unprejudiced. Recently also, they aired one episode to underscore this stark, plain truth.
This note is prompted by the subjective evaluation to the hilt made by the nondescript so-called celebrity expert judge Mr. Sankaran Namboothiri and the uncharacteristic remarks uttered by the evaluator named Mr. Sarath and the anchor lady while awarding marks and making remarks for the performance of competitor/participant Mr. Somadas in his—Classical music (Katcheri) round—telecasted on 9th of April, 2009 (8:30pm-09:30 pm slot). The latter lady, whether expressing her own views or resonating her masters’ (programme manager’s) voice, made a scathing remark against Somadas, wherein she attempted to seem that she is consoling Mr. Somadas but patently disgraced and humiliated him in an ‘artful and skilful’ manner. And pontificated in a subtle mode from the roof top of Idea Star Singer 2008 that; viewers—you should not vote through SMSs to Somadas as he rather made a poor performance and not at all a good singer and viewers—you are supposed to cast SMS votes to the singer whom ‘we designate’ as the good or best singer! Dear Asianet, your anchor lady was openly treading through a terrain which was not meant for an anchor lady. Or she naively plodded through the dotted lines under the diktat of top officials of the Asianet like the programme manager etc?
Now dissect the evaluation process.
Evaluation of Mr. Sankaran Namboothiri
After the Kutcheri performance of Somadas, he remarked that the performance was a disappointment and awarded only 3 (three) marks out of 25 (twenty five). Slicing the merit worthiness of his lengthy comments is inconsequential, as he awarded only 3 marks out of 25. Everyone knows that there are two important parameters in Kutcheri or in the rendition of Carnatic music and they are the composition (krithi or keerthana) and the improvisation (including raga, alapana, niraval,kalpana swaras, thanam,pallavi etc.). So the evaluation criteria used (Did he aware of any such objective evaluation criteria?!) for awarding marks must also take into account the presentation of these two parameters. Somdas might have failed (miserably?) in the presentation of its finer details, but the impregnable fact need to be conceded, that he made at least a very basic rendition of Kutcheri. Hence the interrogative is how much marks should be awarded for such a basic level of performance? Three marks amounts only twelve percent of the maximum marks. Mr. Sankaran Namboothiri never could argue that the Kutcheri performance of Somdas was not Kutcheri at all and it was something else. So long as it holds, Somdas deserved at least 35 (thirty five) percent of the maximum marks for his performance or at least 9 marks.
Over all, considering other aspects of his performance he deserved 11-12 marks at least, from the most sceptical and meticulous evaluator. But Sankaran Namboothiri gave only the pittance. Some may argue that the authority of the evaluator in awarding marks should not be questioned but it is only a wrong and naïve notion. Awarding marks is not charity. Objective evaluation criteria should be there and it must be made public at least to the participants and the evaluator indeed has the responsibility to justify his marking scheme with respect to that. Sorry to say that Mr. Sankaran Namboothiri miserably failed in all these respects.
The evaluation on the basis of awarding marks ranging from 0 to 25 implies that the marking is on a ‘continuous scale’. Such a scale requires a high level of expertise and finesse from the part of the evaluator. The evaluation scale implies that the evaluator has to distinguish, when arranged in an ascending or descending order, at least 25 different types of performance gradually inching towards pinnacle of perfection or abyss of rendition. Note that what is needed is not the parading of 25 different types of performance but what is required is, performances, arranged in an ascending or descending order gradually inching towards pinnacle of perfection or abyss of rendition.
Simply put, this distinction of 25 different types of performances is very impossible. It may be possible for a maestro or for a genius but it is beyond the reach of majority. It is worth noting the fact that only when we are able to distinguish 25 different performances, it would be objective and meaningful to give marks like 3, 15, 16, 17 and so on. In the presence of difficulty in making such a fine distinction one cannot claim that this performance deserves only 3 marks or 4 marks and so on. If one evaluator is adamant that this performance deserves only 3 marks, then he has to explain and show which performance deserve 2, 4, 5 marks and so on. Otherwise the awarding of marks cannot be considered as fair, objective and error free.
In the context of the above said difficulty, evaluators generally follow a common sense practice of adding ‘some more’ (1-3 marks) marks to counteract the above said type of error (due to difficulty in making fine distinctions) which arises from the subjectivity inherent in the evaluation based on marks. This practice has been followed with the sole objective of protecting the participant/competitor from any adverse judgement that arises from the flaws of the evaluation process pointed out above. It is to be noted that these additional marks may sometimes exaggerate the evaluation but it ensure that nobody is hurt or thrown into a disadvantageous situation for some other’s fault like the one explained above (in the instant case the fault is of Mr. Sankaran Namboothiri). To sum up, by awarding just three marks for Mr. Somadas, Mr. Sankaran Namboothiri conveyed that he is unwilling to compensate for the error that would creep into his own judgement and thereby made an erroneous evaluation, his apparent scrupulous posture throughout the commentary notwithstanding!
Imperative for Moderation
The theoretical justification of moderation implemented in the process of evaluation need to be punctuated. The element of subjectivity in the evaluation process and the consequent variation in marks awarded by different evaluators requisite a procedure often referred as ‘moderation’ to contend with the variability in marks awarded. Put simply, if there is considerable variation in marks awarded by different evaluators than what is normally expected, that variation is due to some extraneous reason and hence must be dealt with appropriately.
The statistical tool ‘standard deviation’ is used to quantify the variation in marks. Put simply, standard deviation (SD) is a measure of (average) variation in marks given by different evaluators. The SD is computed for all the evaluations and that historical SD is used to compare the SD of each evaluation. If there is any significant difference in SD for a particular evaluation with respect to the historical SD, it would be surmised that there was an extraneous influence in the evaluation process. That extraneous influence must be neutralised through the procedure of ‘moderation’ where in additional marks are given to bring the SD almost equal to the historical SD. The table given below furnishes the numerical example.
It can be seen that the SD ranges between 0.5—1.41 for the evaluations. But the SD for Somdas is four times higher than this and is 6.18. This requisite a correction and if moderation is implemented to bring back the SD from 6.18 to at least 1.49 for lending fairness, addition of at least 9.5 marks is required which would take the total marks to around 59. To wind up, addition of at least 10 marks are required to give Somdas justice and maintain objectivity and fairness in the evaluation process.
Comments of Mr. Sarath and the anchor lady
The prime responsibility of any evaluator is to perform the evaluation helpful to ‘perfect the performance’ in future, by instilling confidence into the competitor/participant. By scrupulously adhering to the evaluation criteria, the evaluator must dissect the performance to describe where it faltered and excelled and the ways through which it could be improved. Eventually, even a pathetic performer would amass confidence and become a star performer as a result of the evaluation. In the instant case, the evaluator Mr. Sarath with his uncharacteristic remarks, virtually discharged the confidence of the low profile yet unassuming and affable Somdas by making him tear publicly. Unacceptable behaviour from anybody, however great he may be, let alone one Mr. Sarath!
The anchor lady, after the performance of Somdas, as pointed out above, disgraced and humiliated Somdas in an ‘artful and skilful’ manner. She told that, it was only because of the audience (ie. with their SMSs) that he scraped through eighteen rounds. What is its implication? It has only one implication that Somdas came to this penultimate round not because of his musical talent and skills but just because of the countless SMSs he got. If this is not humiliation then tell Asianet, what else is this? It must be accentuated that this indiscreet anchor lady with this remarks even downplayed and contradicted the open stance of Asianet. Asianet through out took the stance that SMSs sent by the audience are counted as a proxy of evaluations made by the viewing public. When the anchor lady attempt to downplay, the much valued evaluations of the viewing public as per the official stance of Asianet, she is humiliating her own employer also! She also demeaned musical wizards like Johnson master etc who gave Somdas 23 marks or so with her remark that Somdas scraped through eighteen rounds only because of the SMS support of audience.
Or what that anchor lady was talking—the evaluations of the viewing public through SMSs, solicited under the auspices of Asianet, are just money spinning mechanisms or machinations? Dear Asianet, you owe an explanation.
Before winding up, one heartening aspect need to be highlighted and appreciated. It is the evaluation and the comments made by the evaluator Mr. Venuopal. His remarks were balanced and encouraging yet pointed out the flaws in the performance of Somdas. The single word remark of Ms. Usha Uthup, (“disappointment”) should also be put under the scanner (for judging improvement?!) but well could be ignored as she said only one word!
Hence, to restore credibility and to lend justice and fairness to Somdas, Asianet must make appropriate corrections. The top management of Asianet may not be in the thick of details of the programme. Hence, the top management should take appropriate steps to resolve the issue for the future of the show.
