The Trouble With Rubrics

Kohn, A. (2006). The trouble with rubrics. English Journal, 95(4), 12-15.

Summary/Analysis

In this article, Kohn analyses rubrics in regard to the assessment of writing and finds them problematic and insufficient in terms of assessing student work. By comparing them to letter grades or scores he highlights the way that rubrics are used to measure student achievement and/or provide extrinsic motivation- rather than provide feedback that helps to improve student work and engagement.

Kohn rejects the idea that rubrics should be adopted because they make assessment quicker and easier for teachers. The fact that it is easier to standardize writing conventions mean that rubrics may often be over-reliant on this one aspect of writing, rather than others. Other consequences of the use of rubrics are, according to Kohn, vacuous writing, lack of confidence and unwillingness to take risks.

Quotes

“research shows three reliable effects when students are graded:  They tend to think less deeply, avoid taking risks, and lose interest in the learning itself.[2] The ultimate goal of authentic assessment must be the elimination of grades. But rubrics actually help to legitimategrades by offering a new way to derive them.  They do nothing to address the terrible reality of students who have been led to focus on getting A’s rather than on making sense of ideas.”

"Consistent and uniform standards are admirable, and maybe even workable, when we’re talking about, say, the manufacture of DVD players.  The process of trying to gauge children’s understanding of ideas is a very different matter, however. It necessarily entails the exercise of human judgment, which is an imprecise, subjective affair.   Rubrics are, above all, a tool to promote standardization, to turn teachers into grading machines or at least allow them to pretend that what they’re doing is exact and objective."

"But I worry more about the success of rubrics than their failure.  Just as it’s possible to raise standardized test scores as long as you’re willing to gut the curriculum and turn the school into a test-preparation factory, so it’s possible to get a bunch of people to agree on what rating to give an assignment as long as they’re willing to accept and apply someone else’s narrow criteria for what merits that rating." 

"A B+ at the top of a paper tells a student very little about its quality, whereas a rubric provides more detailed information based on multiple criteria.  Therefore, a rubric is a superior assessment.

The fatal flaw in this logic is revealed by a line of research in educational psychology showing that students whose attention is relentlessly focused on how well they’re doing often become less engaged with what they're doing."

"Studies have shown that too much attention to the quality of one’s performance is associated with more superficial thinking, less interest in whatever one is doing, less perseverance in the face of failure, and a tendency to attribute the outcome to innate ability and other factors thought to be beyond one’s control.[7]  To that extent, more detailed and frequent evaluations of a student’s accomplishments may be downright counterproductive."

"What’s our reason for trying to evaluate the quality of students’ efforts?  It matters whether the objective is to (1) rank kids against one another, (2) provide an extrinsic inducement for them to try harder, or (3) offer feedback that will help them become more adept at, and excited about, what they’re doing."

"Neither we nor our assessment strategies can be simultaneously devoted to helping all students improve and to sorting them into winners and losers.  That’s why we have to do more than reconsider rubrics."

Evaluation

This article is a damning critique of those rubrics that are as Kohn describes. While these rubrics may make a rating clearer than just a letter or number grade, they show children where they are on a scale that is problematic in its composition and may well have the same detrimental effect that simple grades have. 

However, Kohn's forceful critique of rubrics seems to be limited to a narrow definition that does not fit with the different ways that I have seen rubrics used. Rubrics are not always about sorting students but can be useful in terms of making it plain to students what a teacher is looking for.

Reflections

Kohn's article appeals to me because ranking kids seems to be the opposite of what a teacher should do. Assessment of children into grades or numbers or categories is time consuming and what are you left with? At best you have a list from which you can formulate next steps. At worst, you have learners who spend more time comparing themselves to others, bruised egos, fixed mindsets and disengagement. Surely it is better to assess children based on what they are doing well and what they need to improve on?

However, I do think that it is good for any learner to have a clear idea about what makes a great piece of work. A rubric can help draw this out. It should be a tool for the student and the teacher in formulating next steps - rather than measuring.

Supporting Self-Directed Learners: Five Forms of Feedback

Costa, Arthur L., & Garmston, Robert, J. (2013). Supporting Self-Directed Learners: Five Forms of Feedback. ASCD Express, 8 (18). http://www.ascd.org/ascd-express/vol8/818-costa.aspx.

Summary/Analysis:

This article describes five forms of feedback, ranked in order of effectiveness for developing “student’s capacity for self-directedness:”

1. Reflective questioning.

2. Nonjudgmental data.

3. Inferences, causality, and Interpretations.

4. Personal opinions and preferences.

5. Evaluations and judgments.  

The authors define each form, offer examples, and explain how it may promote or discourage self-directed learning.   The fifth section, “Evaluations and judgments,” is twice as long as any other; here,  with reference to relevant literature, the authors discuss the possible effects of praise and rewards on learning.  The authors conclude that the first two forms of feedback support self-directed learning while the remaining three work against this goal.  Apart from the examples of each type of feedback, the article provides no quantitative or qualitative data; it is a series of reasonable hypotheses around language and cognition, as opposed to a full research study.  In the final section, “Feedback for Learning,” the authors explain the importance of self-directed learning for nurturing creative, resilient, and successful children.

Quotes:

“Posing mediating questions has the highest potential for developing self-directedness, as the intent is to alert the students to the data that will serve to provide self-feedback, process that feedback, construct meaning from it, and set goals to self-modify as needed to achieve desired results.”

“Evaluative feedback makes the smallest contribution to learning and behavior change. When the teacher makes value-laden comments, this sends a signal that she is the final arbitrator of what is good or bad. The teacher may think that making judgments—either positive or negative—is helpful or reinforcing for students, but the opposite is true. Such comments shift the focus from feedback to evaluation.”

“Our wish is for creative students who are eager to learn. Having this disposition means always striving for improvement, always growing, always learning, always modifying. Experiencing problems, situations, tensions, conflicts, and circumstances provides valuable opportunities to gather feedback and to learn.”

U.S. Elementary and Secondary Schools: Equalizing Opportunity or Replicating the Status Quo?

Rouse, C. E., & Barrow, L. (2006). US Elementary and secondary schools: equalizing opportunity or replicating the status quo?. The Future of Children, 16(2), 99-123.

Summary:

This article looks at the correlation between socioeconomic status and educational success of students. It takes a few factors into account, but focuses on the the nature versus nurture dilemma. The article touches on the difficulties in conducting research with human, specifically relating genetics to certain behaviors. The researchers found ways to conduct ethical experiments by observing adopted children placed  families that were not genetically related. It concludes by discussing different ideas that may help equalize opportunities in students with differing socioeconomic statuses.

Quote:

Researchers have used other strategies to estimate the extent to which family income determines children's educational achievement. Again, because they cannot assume that family income is unrelated to other factors (such as inherited ability) that determine both children's socioeconomic status and their educational attainment, they must look for changes in family income that are unrelated to family characteristics such as whether the parents are highly educated or have high genetic “ability.” (Rouse and Barrow)

The Influence of Teacher Background on the Inclusion of Multicultural Education: A Case Study of Two Contrasts

Smith, R. W. (2000). The influence of teacher background on the inclusion of multicultural education: A case study of two contrasts. The Urban Review, 32(2), 155-176.

Summary: The article "The Influence of Teacher Background on the Inclusion of Multicultural Education: A Case Study of Two Contrasts" focuses on effective preparation for teachers in multicultural classrooms. It is critical of what is not happening to effectively prepare teachers in their teaching programs. It splits the essential needs of teacher into 3 important elements ultimately sating that in order to be successful teachers must take into account their background because it plays a large role in the interpretation of classroom situations, and behavior. The first background area is a combination of race, gender and social class membership. The second being prior experience with diversity and the third is support for ideologies of individualism. These factors are what the author finds to be most important through her study of trying to effectively teach in classrooms that are multicultural.

Quote: "In addition to inexperience with culturally diverse people, members of the dominant cultural groups often view the structure of society from a culturally privileged perspective (Boyle-Baise, 1995). Consequently, preservice teachers often support ideologies of individualism opposing structural explanations of, for example, chronic child poverty (Davis, 1995)."

Comment: This quote struck me because it made me think of my experiences and how it plays a role as to who I am as a teacher. I grew up a latino lower middle class student with family on both sides of the spectrum. While I come from parents who were very poor, I never had to deal with the issues they struggled with daily. That being said, I wonder how things might be different for me as an educator if that were the case.

Creative Community Organizing: A Guide For Rabble-Rousers, Activists, and Quiet Lovers of Justice

Kahn, S. (2010). Creative community organizing: A guide for rabble-rousers, activists, and quiet lovers of justice. Berrett-Koehler Store.

Summary: “Creative Community Organizing” is a memoir from former founder of the non-profit Grassroot Leadership, Si Kahn. Aside from being a working musician, Kahn’s experience with community organizing is rooted in the Civil Rights movement, where he first landed in Mississippi during the summer of 1964 to organize black citizens to register to vote. After that historic summer, Kahn’s work covered other historical social movements, working alongside various labor unions in the 1970’s and 80’s to protesting against the existence of for-profit prisons in the last two decades. His book serves as both a memoir and a how to, as Kahn uses his own organizing experience to provide detailed and accessible guidance on how to affect change in your own community. However, his book and ideas are not merely meant for political activist. Kahn states in his introduction that what separates his work and the campaigns he had led was their creativity. Through the book, he is providing innovative structures in which people can collect, meet, and critically think about their community. Those lessons could not only be valuable for any teacher trying to daily mold the hearts and minds of young students, but also for any school looking to actively and positively engage it’s own community. I have not read the entire book, but simply by the introduction and first two chapters, Kahn lives up to that expectation, blending his own rich narrative with practical advice in the organizer’s tool belt found at the end of every chapter.

Quote: “One of the greatest skills an organizer can have is the ability to frame and ask questions in way that make people not only want to answer them, but also think deeply, and in unexpected ways, about what the answers might be… Creativity community organizing can transform us into visionaries, prod us to learn new skills, and encourage us take risks for our and our children’s future.”

Teacher Characteristics and Gains in Student Achievement

Hanushek, Eric. "Teacher characteristics and gains in student achievement: Estimation using micro data." The American Economic Review 61.2 (1971): 280-288.

Summary:

The article “Teacher Characteristics and Gains in Student Achievement” was set out to understand exactly what background experience of teachers has to do with how well or poorly students achieve greatness inside the classroom. By using micro-data, researchers placed variables in the place of students and teachers to conduct studies on classroom behavior, achievement, and advancement. Researchers wanted to see if there was a correlation between what the students were learning and how much education the teachers had.

The studies did not prove to be as true as the researchers thought they’d be. The original hypothesis was that teachers with education reaching higher than a bachelor’s had more qualifications to help students succeed in the classroom than teachers with only bachelor’s degrees. The testing was completed over several years in several diverse communities. Researchers studied Title 1 schools, inner city schools, private schools, and predominantly white schools. All of the research came out the same: student achievement had more to do with the teacher’s character than with the teacher’s educational background or experience.

It was also mentioned that teachers with more years experience tried less hard to have the students succeed. Researchers mentioned that this was not on purpose, however, as time passed, teachers were less likely to learn something new and more likely to stick to what they had been doing in previous years.

Quotes:

“There is a suggestion that one can measure other dimensions of teacher and school quality. These include attitudes of teachers and administrators, verbal facility (and perhaps general ability) of teachers, quality of physical plant, quality of teacher education, background of teachers, and more” (Eric Hanushek, 2002).

“The analysis indicates that teaching experience and graduate education do not contribute to the gains of in student achievement scores” (Eric Hanushek, 2002).

Comments:

In reading this article, I found it fascinating that it spoke so heavily against hiring teachers simply for their educational background. It’s interesting, also, that I work for an organization that focuses less on the educational background of teachers, and more on their ability to connect with students, prepare and produce meaningful projects, and truly understand progressive education. I appreciated that this article looked at all sides of education - from different classrooms, to different teachers, even to different equations (with multiple variables) to try and centralize the hypotheses centered around teacher education vs. teacher background.

I also appreciated how the article specifically states that this should only be taken as a primary (beginning stages) research article. They are admitting that this study only reaches a small educational circle, and that there are plans to expand the study and see other findings. I found it interesting that throughout the article the idea of education can help both the teacher and the students if all are willing to utilize those strengths. It isn’t enough to simply attain a master’s degree if the teacher is not going to use what he or she learned in the program to apply it to his or her students’ learning.

In Parent Choice Suit, U.S. Department of Justice on Wrong Side of History

Fuller, H. and Chavous, K. (2013). In Parent Choice Suit, U.S. Department of Justice on Wrong Side of History. http://www.redefinedonline.org/2013/10/in-school-choice-suit-doj-on-wrong-side-of-history/

This article was of particular interest, after watching and reading discussions of Howard Fuller's with the High Tech schools regarding equity and understanding of race within education. Fuller and Chavous make an interesting argument that quality education trumps the notion of impacting desegregation in schools. I too agree, that should families and students qualify for scholastic scholarships that could potentially advance their education, the DOJ should not interfere with their desire to do so. How is this any different than students who leave other public schools around the US, to go to private or charter schools that best meet their needs- even if they do not go via scholarship?

Quote~
From Judge Robert Carver, the articles quotes, "The immediate and urgent need of the black urban poor is the attainment, in real life terms and in settings of virtually total black-­‐white school separation, at least of some of the benefits and protection of the constitutional guarantee of equal educational opportunity that Brown requires."