Evaluating Philosophy for Children
Johan Niklasson, Ragnar Ohlsson and Monika Ringborg

(Before entering "Philosophical Discussions in Four Different Classes. Summary of the philosophy assistant's report" you should at least read the story The doll.)

In recent years the validity of different methods of testing the effects of implementing the IAPC-program, as well as evaluating the results has been discussed. Christina Slade ("Creative and Critical Thinking", Analytic Teaching, Vol. 13 No. 1 Nov. 1992) and Marina Santi ("Philosophizing and Learning to Think: Some Proposals for a Qualitative Evaluation" Thinking, Vol. 10, No. 3, 1993) both advocate qualitative methods which analyze the structure and content of classroom discussions. Slade proposes that in this way we will be able to evaluate the progress in both critical and creative thinking (between which there is no necessary incompatibility according to Slade). Santi's opinion is that analysis of transcripts of children's discussions will make it possible to find qualitative indicators of the cognitive progress of the group.

We agree with them on the main aims of evaluation methods for philosophy with children. Analyzing transcriptions of discussions is one method we used in our research project on philosophy with children. For some years, we have had an on-going research project with Philosophy for Children within some Swedish compulsory schools. The project is financed by the National Board of Education in Sweden. In a few classes ranging from grade 1 to 3 (the children being 7-10 years old) we have had continuous philosophy lessons for three years. In one class we used mainly Elfie and Kio and Gus. We had two lessons a week and in the main followed the method worked out by Professor Matthew Lipman of the Institute for the Advancement of Philosophy for Children at Montciair State University in New Jersey. In another class, with the same student composition, we used other books, mainly Le petit prince by Saint-Exupery as well as Swedish books for children. This class got fewer lessons than the other class. Our intention was to test for long-range effects of Philosophy for Children with slightly different methods and materials. Two-and-a-half years after the last lesson in grade 3 we returned to the classes, now in the 6th grade, to see whether we could find any significant differences between the children who had been trained in philosophy and children who had not.

Two assistants – one a philosophy student and the other a student of pedagogics – who had no previous contact with the classes, were instructed to give two lessons in philosophy in four different classes. Two of the classes had had no philosophy at all, the other two belonged to the above mentioned classes.

Three of the classes, including the experimental class, belong to the same school in a suburb of Stockholm. This suburb has average sociological levels and cultural patterns. It is neither a high nor low status milieu. But the fourth class, where we tested the material and which was included in the written report, is different: these children live in a relatively high status area of the city of Stockholm. Their parents are mostly middle class, many of them have occupations within the media branch or do some artistic work. Some "immigrant" children attend this class, but these immigrant families are rather special: the parents usually are researchers within academic disciplines. Consequently, these children are rather above the average when it comes to verbal performance, general education, self-reliance, and so forth. This class was chosen as one of the control groups, because we wanted to compensate for the possibility that our experimental class by luck should be very intelligent and prone to argue. If it was found that the philosophy class argued better and more self-reliantly than even this rather special class, our results would have a higher degree of validity. However, in this experiment, the assistants knew that this was not an experimental class, which certainly diminishes the value of the comparison. On the other hand, we have also performed some written tests with the same groups of children, where we could cover up the source of the papers. And all tests tend in the same direction: "our" children are rather special.

The pedagogic assistant was instructed to observe the "climate" of the classes, to make notations of how involved the pupils were in the discussion, how many joined in the discussion, whether they talked to each other or just to the teacher and so forth. The philosophy assistant edited four short texts, two about distributive justice and two about time travels. He also acted as the teacher during the lessons. The two assistants tried out the texts and the lessons in one class. Then they gave the same lessons to three other classes. In each class, the philosophy assistant gave two lessons of about 40-50 minutes each. One lesson was focused upon principles of distributive justice. The other discussed problems connected with time and the possibility of time-travel. Both lessons had the same structure: the children were presented with one text, the "teacher" asked about the children's opinions upon the text. Next the discussion commenced. When the discussion appeared to be exhausted and the same arguments seemed to return, a new text with a slightly different angle on the same problem was presented and discussed. The discussion about distributive justice thus had two starting points: the first was a short text adapted from the Manual to Matthew Lipman’s Lisa about the fair distribution of candies. The second text was adapted from a Swedish philosopher Torbjörn Tannsjö (it was taken from an unpublished manuscript: Classical Hedonistic Utilitarianism. A Defense). For the lesson on time two text were used, the first was adapted from Gareth Matthew's Dialogues with Children (pp. 76-79), the second from some IAPC manual (we have been unable to trace the text when writing this article). All texts are included in the Appendix.

None of the assistants were trained in the pedagogical methods developed by the Institute for The Advancement of Philosophy for Children. They had to find their own methods. It seemed to us that this would be the "fairest" to the classes that had not been trained in philosophy. If we were to find differences between the classes this should not be dependent upon the children being acquainted 'with the method. This fact explains why so many children are silent in all the classes: the philosophy teacher made no conscious effort to put everyone into the discussion; we wanted to see if there could be detected differences between the classes, which could be seen by anyone.

The assistants were asked to note their spontaneous reflections upon the classes directly after the lessons. When the lessons had been transcribed from the audio tapes, each assistant made a detailed analysis of the lessons, with a different focus: the pedagogy assistant was to focus on patterns of communications, student interest and involvement and so forth, and answer a number of specific questions related to these aspects; the philosophy assistant was to focus on philosophically interesting ideas, the quality of argumentation, and so forth. The two questionnaires were formulated in advance; some of our questions were taken from an evaluation scheme constructed at the British Center for Philosophy for Children.

Thus we got eight detailed reports on the discussions in these four classes. We here present the summaries of these reports. Note that these reports and the summaries were written before the assistants knew which of the classes had had philosophy continuously for three years. The experiment-class is the one called "The Tuesday-class" and the class from the high-status area of the city of Stockholm is the "Wednesday-class", which the reader can keep in mind when reading the report.

Philosophical Discussions in Four Different Classes. Summary of the philosophy assistant's report

"There are some similarities (as well as differences) between the four classes I have had the opportunity to discuss philosophical topics with. The similarities and differences primarily concern their thoughts on philosophical topics and the way they present their ideas of tackle the arguments of their friends. I will make a short summary of what I found characteristic for each one of the four classes.

The Wednesday Class (High-status)

"The Wednesday class was the most talkative. They seemed to find the philosophical problems discussed very interesting. On several occasions, many students gave long exegeses on what might be the most just solution, how we should relate to time or on other problems that arose. Nevertheless, they rarely offered arguments for their positions. Fairly often, I had to ask them to bring forward arguments. There are some examples of students using thoughts of others as bricks in their own argumentation, but it did not happen very often.

"In the four lessons during which we discussed justice with the different classes, only three students in this class and one in the Thursday class consistently argued for one and the same theory of justice throughout both texts on the subject. In this class, two girls and a boy consistently claimed that an equal distribution is the most just one in all situations. This could be seen either as a sign of dogmatism or as a sign of deep conviction based on thoroughly thought-out argumentation. Which of the two applies to this group of students is hard to tell and I have chosen not to comment on that question. Most other students seemed to be of the opinion that the choice of justice system should depend on the specific situation.

"The Wednesday class did not find the characters in the stones plausible, and they consistently commented on details in the text. They criticized the text on several occasions, but their criticism rarely had anything to do with he central problems expressed in the text. Philosophically formulated examples often highlight the issue in an almost overly clear manner. This can in some respects make the examples less plausible but not less interesting. From a scientific point of view, it is very important to be able to imagine the possibility of something, plausible or lot. The students in the Wednesday class did not show any skill at this.

The Monday class (The comparative class)

"The Monday class seemed both uninterested in the philosophical problems that were raised and bothered by my questions. I often had to act as a producer of questions, rather than as a guide through the texts. On the other hand, the class always commented on the problems I presented. None of the students used arguments of other students to any larger extent. Furthermore, the students often changed their views. I did not find any one who consistently argued for one and the same idea. In the justice lesson, I believed that this could be interpreted as if the students meant that what should be considered just always depend on the situation. However, this cannot be linked to any specific statement. The most popular theory of justice was the acquisition theory of Locke, which is unique for the class."
 
 

The Tuesday class (The experiment class)

"Although the students were rather passive in the beginning of both lessons, this was the group where the discussions worked best. At one time during the discussion of time travels, I remained quiet for ten minutes and allowed the students to pursue the investigation themselves. It worked excellently. They did not deviate from the central subject nor did the discussion get out of hand.

"The class immediately seemed to perceive where my questions were aiming and the students spontaneously came up with ideas that I had to present myself in the other classes. The participating students not only listened to each other's argument, but also elaborated on them and used them in their own arguments.

"During the discussion on whether one travels in time when passing the dateline, the following was said:

                            Ester: "No, you are always at the same place."

"It is interesting to note that Fredrik seems to be of the same opinion as Kristian, but still wants him to give reasons for his view.

"When my reasoning or their fellow students' theories were criticized, one could perceive a degree of modesty and mutual understanding. The students explained why they found the argument or theory indefensible (often with an example). I found that the students had a critical approach to both what their classmates and I had to say. This was the only class that criticized me. The students also interpreted the texts in a fruitful way and envisaged different consequences of the theories that were discussed.

"The students rarely stuck to a single theory for more than a couple of minutes. All sorts of ideas were argued or and rejected. However, most students could be said to accept the idea hat justice is dependent on the situation.

The Thursday class

"A problem arose when I was about to start my analysis of the Thursday lass. The analysis was complicated because very few students were present at the first lesson, and one single student dominated the second lesson. However, there is no doubt that in the Thursday class showed little or no interest in the problems of time and justice. I often requested arguments underpinning the students' comments, but they mainly chose to be silent.

"A common feature was inconsistent argumentation of the class. Exceptionally, one boy, throughout the second text on justice, spoke in favor of distribution by merit. The boy remained quiet during the discussion of the first text on justice. It is uncertain if he believed that distribution by merit should be used as a general principle or only in specific cases. Among the other active students, the most popular solution was to leave the doll – the object to be distributed – to avoid the problems of distribution."

Conclusion

"The fact that all classes defended philosophically interesting views and hat the students used arguments and theories known from the history of philosophy ought not to be interpret as if all students are great philosophers. I tries to give the discussions a philosophical bias and the texts that were used illustrated both well-known and interesting standpoints.

"Nevertheless, there is one class, or rather some students in one class, who showed several distinguishing qualities. I am referring to the Tuesday class, where the students had a critical attitude to what was said during the discussion, had no problem with abstract thinking and also seemed to be able to see the consequences that implementations of a theory might bring. Another sign of philosophical qualities could be that no one presented a firm idea of justice. Philosophy is known for not producing truths, tanks to its high demands on what could be considered as true. Despite my conclusion that none of the Tuesday class students had a firm idea of justice, I do not think it should be seen as a sign of philosophical quality. I support this assertion by the fact that together only four students consequently argued for one and the same theory of justice. Thus, not having a firm idea of justice does not make the Tuesday class unique.

"The only standpoint I have found common to all classes is that Sara ought not to have the doll. Sara represents a Buchanan contract theory. I believe that the students would have another attitude towards Sara if her character were to be slightly changed. Most students regarded Sara as evil. They felt that evil people do not deserve advantages. Most students wanted Elin to have the doll. Elin was designed to represent Rawls equality theory, but the students came up with other arguments to support Elin as well. One was Bentham's utilitarian argument and the other was an argument based on virtue. The virtue argument was also used as a justification for why Sara ought not to have the doll. If the characters were changed but their arguments were kept intact I do not believe that all students would have argued in favor of the same characters in the text. For example, one might try to turn Elin into a vicious little creature and Sara into a kind, reasoning person.

"I have been unable to identify a common pervading characteristic relating to the discussion on time, except that the students' intuitions as regards time problems seems far weaker than their intuition as far as problems of justice are concerned.

Communicative Interaction in the Classroom in Four Different Classes. Summary of the pedagogics assistant's report.

"The purpose of my observation was to investigate in a contrasting way the differences and similarities between the pattern of communication of the four different classes. The guidelines through the observation were the following eight questions:

1. How does the pattern of communication (PC) develop during the lesson? (By "pattern of communication, is here meant the verbal interaction between the pupils and the teacher and between the pupils themselves).

2. How does the PG change from lesson I to lesson II?

3. Is the PG different depending on what text is discussed? I was also interested to evaluate some further dimensions of the discussions:

4. Flexibility – i.e., the children's ability to see a problem in different perspectives.

5. Argumentative accuracy – i.e., how often the children gave reasons for their standpoints, and how relevant and well formulated these were.

6. Tolerance of uncertainty – i.e., could the children deal with the fact that reasons can be given for several different solutions to the same philosophical problem?

7. Openness – i.e., were the children prepared to see the pros and cons of new possibilities which came up in the discussion?

8. Independence – i.e., how independent were the children in respect to the opinions of the teacher, other authorities, and their classmates?

"Since I had no acquaintance with the classes and no background information about them, I could just observe what happened during the lessons and analyze the transcripts from the tape. I could not speculate about what factors made the children say what they said, nor could I formulate any hypotheses about the reasons behind the children's behavior. Thus, in a way, my observation was rather pure and ascetic. Since I had two opportunities to observe each class, I do believe that some of the patterns which reoccurred were not accidental but somehow were characteristic of the different classes.

"The differences between the four classes, as far as the PC's are concerned, are significant, although the discussion of text I in all classes has a similar PC-at Teacher->Students Interaction, i.e., the teacher asks and the students respond with "yes," or mmm... or try to guess what answer the teacher is aiming at. But after the first half-hour or so of the first lessons, the differences between the classes became significant.

"In some classes, the students were responding collectively and in other classes they hardly ever responded collectively. In some classes there was no interaction at all between the classmates. This was most obvious in the Wednesday class and the Thursday class. In the Monday class the students responded collectively, but one main reason for this can be that the teacher asked the class as a whole – "Do you all think ..." – more frequently than he did in the other classes.

"The different PC's in the classes during Lesson II are most easily described graphically, as follows in Figures 1-4.

{Figures 1, 2, 3 & 4}

"The differences between the Monday, Wednesday and Thursday classes on the one hand and the Tuesday class on the other hand are significant. The figures show that there is a much more intense communication between the pupils in the Tuesday class than between the pupils in the other classes. The figures also show that a larger part of the Tuesday class takes part in the discussion. There are also more pupils who make many contributions to the discussions compared with the other classes. (One boy is noted for 124(!) utterances.) A further analysis of the arguments given shows furthermore, that the students in this class were discussing the subject matter in a more conscious way that the others. They were able to grasp the abstract principles discussed. The students in the Wednesday class were especially noted for using material from their life domain, rather than discussing in the symbolic domain. The students were more subjectively connected to the subject than the students of the Tuesday class.

"The following differences during the discussions about text III and IV are of interest. The PC of the Monday class was changed from lesson I to lesson II. Only one student was active during lesson II, compared to four-five students during lesson I. "Active" is here defined as making more than 10 contributions to the discussion. During lesson II in the Monday class, the interaction was almost exclusively between one student and the teacher. The students did not interact at all with their classmates.

"The PC of the Tuesday class was almost the same during the two lessons, but the PC of the second lesson was even more intense and dynamic, almost dialectic of character. The interaction of the active students was however too marked by the students' personal roles to be really constructive. In that way the interaction was static.

"The PC of the Wednesday class was the same during the two lessons, i.e., Teacher <->Students interaction.

"The PC of the Thursday class changed dramatically, because of one dominant student, who was absent during lesson I. The interaction during lesson II was primarily between that student and the teacher and had the character of private conversation. The other students tried to break the dominant interaction without succeeding.

"In all four classes the students were not as active in the beginning of the lessons as they were at the end of them. The students of the Tuesday class became especially more active as the time passed and they did not seem to care when it was time to finish the lesson. They continued to discuss the subject long after the lesson officially ended. The teacher eventually had to stop the discussion.

"The flexibility and the ability of seeing the subject matter in different perspectives varied between the classes as well as in the same class, depending upon what was discussed. Text II was the most discussed text, where more children took part in the discussion than was the case with any of the other texts. This might depend on the vicinity between the children's own experiences and those of the kids in the text, but it might also be a result of the text presenting five distinct standpoints to choose between, which might have facilitated the discussion.

"The great exception was the Tuesday class. The students of this class also showed flexibility when discussing texts about time travels which were far from their own experience. Some individuals in the Monday class showed the same flexibility and ability as well. The students of the Wednesday and the Tuesday classes showed the highest argumentative accuracy. However, the difference between the two classes is due to what basis the students were discussing the subject, from the life domain or in the symbolic domain: the Monday class mostly chose their examples from their own experience, while the pupils of the Tuesday class could keep more in touch with the abstract subject matter. The other two classes were heavily dependent upon the teacher and seldom gave reasons for their standpoints.

"Most students in all four classes seemed to be satisfied with pure facts or anecdotes. During the discussion of Text II, the students showed some tolerance, due to the several solutions they came up with.

"There are variances in and between the four classes when it comes to openness, but the most important difference was between the Tuesday class and the Wednesday class. The students of the Wednesday class showed no openness at all, while the students of the Tuesday class listened to the arguments of their classmates, changed their positions in view of new arguments, and built their own arguments on those of their comrades. They seemed more open to different solutions of the problems.

"The students of the Tuesday class showed the highest independence. They discussed the subject among themselves and rather independently. The teacher was most of the time a partner in the conversation, rather than an authoritative person. The students in the Wednesday class were also independent versus the teacher, but their independence seems to follow a more conventional pattern formed by the society.

Conclusion

What is most striking about these reports is the difference in communication patterns. This is neatly illustrated by the drawings of the pedagogic assistant. The philosophy pupils talk to each other they use the arguments of peers, they question their arguments. They even question the arguments of the "teacher". (This might be a problem for these children later in school.)

Furthermore, these children seem more involved in the discussion. They focus upon the central problems and carry on the discussion themselves. Though they often sighed during the three years of philosophy and questioned the value of philosophy, more children in this class seem to have developed more of a feeling for philosophy than the other classes.

The quality of their argumentation seems also higher, but here the results are uncertain, since the eloquent children in the Wednesday class impressed one of the assistants. A closer look at the arguments used shows that the relevance and ingenuity of the experiment class is higher, although they might not be able to formulate what they think as easily as the Wednesday class. But when compared with children from their own milieu, our children show greater skill in formulating their thoughts.

The result seems to confirm the idea that systemized training in philosophy according to the IAPC method results in significant differences in the way the children discuss philosophical problems. The differences are not so striking when it comes to the ability to propose philosophical ideas: many children seem to be able to discuss philosophical problems and come up with good ideas, which confirms the hypothesis behind Philosophy for Children, that children in a way are natural philosophers. The most significant differences are to be found in the way the children talk together, how they use the ideas of their comrades, and how they are able to question the ideas of the "teacher". The philosophy-trained children seem more autonomous and self-reliant. At the same time they form a community of inquiry in a way the other children do not.

Of course, no certain conclusions can be drawn from such a limited material. But the experiment can be performed several times (as long as we have classes who have been trained in philosophy and money for the evaluation). We have instead tried to evaluate the results of Philosophy for Children by using several different methods in the same classes. Since some patterns are constant and these patterns convincingly can be explained by the training in philosophy, we have reasons to presume that these patterns are effects of the philosophy lessons.

These findings might not astonish people who have long been involved with Philosophy for Children. Nonetheless, we were surprised that these effects were so clearly shown, and that they remained after two-and-a-half years without a single lesson in philosophy. As other evaluations, which have been carried out within the project, seem to show these qualities seem to spread to other subjects, like for example math. (See Filosofi med barn, pp. 173-200.)