What is the purpose of television evaluation?

On occasion, I have heard statements among television executives, colleagues of experience and respect, according to which the secret of the success of soap operas or television programs is still to be discovered: Things like "nobody has the last word" or "whoever knows him should be working in the offices of the best television companies in the world are said. world." I wonder sincerely why after more than fifty years, the television industry, experts and producers have not hit the nail on the head and have no indication of the "key to success". Could it be that we have not been able to investigate and systematize the successes to analyze them and intelligently manage the good results, replicating them in future projects?

I don't think an industry like television has been able to get stronger during all this time and become big, relying on the success of its products to chance, intuition and "good rumor". With the aim of providing a constructive point of view in this search and convinced that it is possible to find and manage the keys to success, I want to look at the theories of educational evaluation, successfully applied in the Children Television Workshop.

Evaluation is not a mere theoretical disquisition; it is conceived as an effective method to facilitate decision-making and the execution of actions. The starting point of an effective educational process takes into account the situation and conditions of the student, prior to learning (diagnostic evaluation). In commercial television, this translates into the study of the "target". In addition, the evaluation serves to monitor the educational process (in our case, production) and verify the coherence between the steps taken with the proposed objectives (formative evaluation) Finally, the evaluation verifies the achievement of the proposed objectives and compares them with the results obtained (summative evaluation) Tyler and Stufflebeam, among other researchers, they develop in their works these theories conducive to such valuable activity.

The investments required for the production and programming of a commercial television company force investors to minimize risks. The need to achieve the levels of attunement that allow the sale of programs, leads television producers to adopt previous evaluation criteria and objective measurement tools different from the rating (because they are a posteriori) and beyond the common sense of the programmer, for the production, purchase and broadcast of programs. These are powerful reasons to assume, methodically and responsibly, evaluation as a fundamental task in the production and programming process.

In the production of television programs, the evaluation has to take a specific program, from the moment of its gestation, and serve as sustenance and support in all the development. During the preparation stage prior to the recording, elements of the diagnostic evaluation can be applied and contrast, for example, the thinking and criteria of the company against the contents of the programs offered. Studies of the broadcast schedules of the programs can be carried out by looking at the age, sex and social class of potential viewers; their tastes are ascertained, the previously successful programs in the audience slots where the new programs would be broadcast; the expectations of the public towards the programs at a certain time are investigated; the costs of the different alternatives are evaluated. Likewise, the quality (how difficult!) of the scripts, the way of narrating, the ability to handle the drama, etc. are studied. Not a few companies also evaluate the products of competitors to face them successfully.

The role of evaluation at this stage is diagnostic; that is, the evaluators dictate the state of the company in the face of the challenge of programming profitably at a certain time. This work requires method, clarity of data and must produce a production strategy of the programs. The depth and quality of the work of this first stage can depend on the success or failure of a company. This same task of diagnostic analysis must be fulfilled before acquiring and programming / transmitting national or foreign productions, elaborated by others.

The challenge for educational television producers is similar. The daily bombardment of programs of all kinds and content makes the production of "educational" programs difficult and demanding and, although intentionally instructional programs have captive audiences or are broadcast at "educational" times, the mind of viewers is accustomed to seeing attractive television proposals that compete strongly with such productions.

Educational television seeks to motivate the change of attitudes of viewers, explain and clarify difficult points in specific areas of knowledge, etc. Educational programs, if they want to achieve changes in attitudes, must attract an audience, must compete with the formats that the television audience finds in commercial television. And that's not just the result of a good amount of money available for production; it is also the product of clarity of goals in educational institutions and program makers. If educational television programs bore instead of attracting, they fail in the fundamental purpose that gave rise to them.

With the formative evaluation, the production work properly begins. The previous stage concluded with the formulation and setting of objectives, with the selection of a specific project that now becomes recordings, filming, editions, musicalizations, etc. For this, it was necessary to review the inventories, reread scripts, analyze pilot programs, check studies and, finally, make the decisions. That is the fundamental function of evaluation: to provide data to make decisions.

In the production or development of a drama series or a telenovela for example, the evaluators monitor the writing of the scripts to see, especially, the fulfillment of the minimum standards (they have to exist!) of the television narration, the development of the conflict, the definition of characters; then, during the recordings, the coherence between the libretto and its interpretation, the respect for the drama, the direction of actors and the staging. It is also necessary to monitor the technical elements such as: lighting, music, scenography, setting, costumes, as well as the final edition. Television should be art, but it provokes measurable reactions that guide us about its success or failure.

This work can be annoying for directors and producers, and annoying for evaluators. To the former, it may seem as if they distrust their work. For evaluators, it can be a source of conflict, for example, to suggest that the intent of a sentence, of a scene, has been changed and that the dramatic conflict has lost strength. Many producers leave in the hands of the director the delicate task of staging and on screen the product that can mean the success or bankruptcy of the company. Others think differently and believe that objective and serene teamwork can bring many satisfactions to all.

Because of the investments required by the programs, not a few companies prefer to produce "pilot" programs. Sometimes in the United States these "pilots" are two-hour telefilms. Depending on the level of harmony and acceptance of these prototypes, the largest investments are made or not, trying to minimize the risks. But, whatever the format of the program: contest, musical, interview, documentary, newscast, magazine, drama, the formative evaluation provides information on how the product fulfills the proposed objectives.

It is possible to objectively measure with potential audiences the levels of acceptance of "pilot" programs, the credibility generated by their arguments, the attention, understanding and attitudes of the audience. There are sophisticated computer systems by means of which computers collect almost second by second data from the audience: the time of attention, the level of taste or dislike, the degree of acceptance of the program against another, etc. These analyses can be done with programs that are already being broadcast and the results will facilitate decision-making about necessary changes.

Thus, at this stage of program production, evaluation tools help to "shape" the product and keep it alive. They help to discover causes of problems so that the remedies are adopted. They verify the coherence between the objectives and the reality at hand.

In educational television it is necessary to ensure the understanding of the content by the audience. It is necessary to monitor the coherence between the libretto and the realization, between the objectives that the audience must achieve and the television proposal. The format chosen must be appropriate, attractive and interesting, even if it does not seem "formally instructive". Needless to say, for results to be reliable, there must be complete independence between evaluators and filmmakers. No one can be, at the same time, judge and party. Of course, it is potential or actual viewers who best judge and receive the product.

Commercial television has developed sophisticated measurement systems to verify the number of people or households watching the programs. Today it is possible to determine quite accurately the number of children, women, heads of household or housewives who watch a series, a soap opera or a sporting event. Audience composition is determinable and verifiable. By this means sales are argued. These measurements allow to reinforce promotional aspects, change sales strategies or, if necessary, change the program.

However, companies specialized in these measurement systems are not able to determine to what extent the contents of the programs are assimilated and brought to daily life. The discussion about the effects of television on the family in many places is nothing more than a good intention that has no verifiable and objective basis that clarifies whether or not there is any cause-effect relationship between the contents that are transmitted and the behavior of this or that segment of the population.

The impact of certain messages on the behaviour and attitudinal changes of population groups are the fundamental object of evaluative studies at the end of a series of programmes. If it is possible to credibly determine the pre-existing conditions (attitudes, knowledge, inclinations, etc.) of a group before its exposure to certain programs and compared with those of the same group after the broadcast, it would be possible to define behavioral changes attributable to television programs and make proposals for modifications to their contents. I think that experiments of this nature, although they are difficult to control due to the innumerable variables that intervene in them, are the contributions that commercial television needs and can only be born of unprejudiced spirits and free of commitments. On the other hand, determining the causes of possible errors in the production of the programs will allow them not to be repeated in the future. It will also make it easier to guarantee successes, capitalize on successes, identify the keys to attunement and acceptance in different audience groups.

I believe that the principles of educational evaluation can be used valuablely in the process of programming and production of commercial television and the methodologies used by the leaders of "educational" television in the realization of their programs can be applied in commercial television. The marketing techniques applied in commercial television can guide the educational work in television and make it more efficient, more attractive to the viewer. Although commercial television producers do not have openly instructional intentions, the work of issuing messages, actions, dramas, produces social effects, generates criteria, fashions, buying habits; and all of that is necessarily "educational."

Hopefully these reflections will encourage many to take on the task of evaluating as a necessary, valuable and conscious routine, which can guarantee the effectiveness of the messages, the acceptance of the audience, the achievement of educational purposes, the survival of the company, the construction of a better human being.