Encouraging Critical Listening and Thinking in New Graduate Students

My experience as a teacher of leadership has convinced me that several of the key terms–e.g., leader and leadership–are used in multiple, inconsistent, and superficial ways. For example, in much of the public discourse and a surprising amount of the academic discourse the term leader could be replaced with an expression such as excellent manager or admirable person without any significant shift in meaning (except to reveal more clearly how the speaker defines or fails to define leader). I see this as evidence that the speaker has not given sufficient thought to how he or she might define leader in comparison to a variety of related concepts.

I believe that a person who earns a master’s degree in leadership–among other things–should:

1. Listen to (or read) statements about leadership carefully and critically, accepting that persons can and will legitimately use words in discourse-community-specific or even idiosyncratic ways, and recognizing that all concepts (or all definitions) are not equally excellent.
2. Recognize that one important, conceptual dimension concerns whether leadership is conceived as an attribute of the leader, (“leader-centric”) or an attribute of the follower (“follower-centric”). For purposes of this assignment leader-centric and follower-centric were defined according to chapters 2 and 3 of A very short, fairly interesting, and reasonably cheap book about studying leadership (2nd ed) by B. Jackson and K. Parry.
3. Understand that everyone talking (or writing) about leadership does not mean the same thing (and, in some cases, may not have given much thought to what he or she means).
4. Have a personal–and evolving–personal understanding (or definition) of leader and/or leadership, and be able and willing to articulate and defend one’s view.

I hoped to use the assignment (on which my ITEL project focused) to contribute to these pedagogical objectives, especially the first three.

My assignment (as revised in consultation with my ITEL cohort) consisted of referring students to two on-line, publically available presentations. These two presentations were selected because: (a) the two presentations were almost precisely equal in duration; (b) the two speakers differed in age, sex, and skin color; and (c) each of the presentations could be categorized as expressing either a leader-centric (Bob Davids) or a follower-centric (Natalie Warne) perspective, but each also included elements of the other perspective, making discernment and argument necessary. To locate the two videos the students were told:

• Go to www.trendhunter.com/course/leadership
• Pick one of the two following presentations
• Leadership Without Ego” by Bob Davids (duration 12:50)
• “Youthful Leaders” by Natalie Warne (duration 12:51)

I hoped that the video presentations would provide students with a brief, coherent (though not entirely self-consistent), and engaging stimulus for evaluation. The preparation and delivery (and subsequent editing) of a video presentation encourages expressing oneself directly, clearly, and succinctly–these characteristics would, I expected, result in a good (for my purposes) stimulus object.

Three waves of data provide some evidence of the effectiveness of the assignment: faculty appraisal, peer feedback, and student evaluations.

The first set of data consisted of the papers submitted by students and graded by the instructor (me). Grades on the assignment ranged from A- to B- (mode = B). This grade distribution is comparable to the grade distribution in previous years and not particularly informative because the assignment was significantly different this year.

However, my subjective assessment is that the papers showed a high level of desired impact. I found that the students were engaged and perceptive in seeking to understand how the video speaker defined and understood leadership, thus contributing to objective #1 (“Listen to ... statements about leadership carefully”). I was impressed–but less so–with their efforts to correctly place a speaker’s perspective in either a leader-centric or a follower-centric category, objective #2.

I was also (and unexpectedly) struck by the amount of energy which the students had exerted toward mastering the content of the two most relevant chapters in the textbook. A number of the papers included passages that I would call “trying to clarify my understanding by putting it on paper,” in other words, long summaries of textbook content. I was also favorably impressed by student efforts (and accuracy) in bringing textbook content into the classroom discussions–this easily surpassed similar behavior in past years. Given some success with objective #1 and objective #2, I assume that there was also some movement toward objective #3 and objective #4.

Based on my evaluations of the student papers, I created an “assignment debrief” which I incorporated into the students’ second week-end on campus. In addition to the concepts already discussed, the de-brief materials reflected my conclusion that the students were overly respectful of the video speaker and unwilling to describe the speaker as inconsistent of imprecise, even when the text of the paper clearly indicated (at least to me) that the student was aware of the speaker’s deficiencies. Thus my de-briefing efforts included some explicit admonitions to the students (in their post-graduate role) to challenge the materials to which they were being exposed.

The second body of data consisted of peer feedback papers. This was a feature retained from past years and one that seemed to become more effective this year. The assignment consists of assigning each student to read and comment on the paper written by a different member of the cohort. I do not assign a letter grade to this assignment–but I do note whether or not the paper is completed on time and includes appropriate content (and count student performance, along with attendance, toward a “participation” grade).

This year I assigned each student to read a paper that addressed the same video stimulus that he or she had written about (either the speech by Davids or the speech by Warne). And this year the peer feedback papers included in several cases (I regret that I did not keep count) arguments about the proper interpretation of a speaker and the placement of the speaker in either a leader-centric or a follower-centric category. Thus, the peer feedback papers gave additional evidence of progress toward objective #1 and objective #2, along with what I regard as strong implicit evidence of progress toward objective #3 (sensitivity to differing definitions) and objective #4 (developing and expressing a personal view).

The third body of data consisted of anonymous student feedback on the assignment (82% response rate). Results indicated that students found the assignment helpful in mastering the material (mean = 3.83; sd = 0.60; 1-5 scale), but not as helpful as I would have liked. I was somewhat more pleased to see that the assignment de-brief had been well received (mean = 4.28; sd = 0.65; 1-5 scale).

At least some of the learning from the assignment emerged from the peer feedback process. Some students reported more learning from giving feedback (17%); some, from receiving feedback (11%); and most, learning equally from giving and receiving feedback (72%). A content analysis of responses to an open question (“please briefly describe what you learned”) found that 78% of the students referenced one of more of the four pedagogical objectives. The most frequently referenced objective was #3 (awareness of a diversity of viewpoints), mentioned by 61 percent of the students. Students also made comments related to objective #2 (correctly categorizing perspectives). A few made comments related to objective #4 (clarifying one’s own views).

Finally, as a matter of curiosity, I asked whether the student had watched leadership videos in addition to the one commented on. (The speeches to be analyzed were part of a larger collection and I wondered whether the students would take the time to explore other video presentations. The assignment could be completed by watching only a single video.) More than half of the students reported watching both designated videos. Almost forty percent of the students also reported watching a significant portion of at least one video in addition to the two that had been designated–the additional viewing ranged from one to five video presentations.

My conclusion, based on the available data is that the assignment worked well, certainly better than the previous assignment, but that there is room for improvement.

This assignment will now be a permanent part of the course.

I see several ways in which to improve the assignment. However, I’m not sure that those improvements (e.g., introduce the assignment earlier, look for a more easily digestible introduction to leadership perspectives, tweak the syllabus description of the assignment), have any resonance outside the specific context of my course.

I plan to develop other assignments of this type: Finding and incorporating free on-line resources.

And I plan to continue the practice of requiring students to respond to each other’s work. That aspect of the assignment seems to have been the portion that moved students toward the “higher” objectives (#3 and #4) in my list.