Archive | Rubric RSS feed for this section

Theory Application Rubric

In creating my rubric, I tried to think what I as an instructor would be wanting a student to demonstrate. So, I ended up with three sections: Explanation, Application, and Reflection.

I applied my rubric to Suzanne’s application of genre theory to the Underground Press Syndicate. Overall, Suzanne did a great job in the explanation and application sections. She gave an overview of the theory, justified its application to the UPS, and thoroughly explained the UPS in terms of genres. If I were providing her with feedback with this rubric, I would encourage her to further develop the reflection section.

Theory Application Rubric

Theory Application Rubric

Surprisingly, the rubric worked really well with Suzanne’s application. Because we had used different questions to write the case study specifically in terms of networks, I actually expected the rubric to be much more problematic. The success of it, however, makes me realize how much both the structure of a theoretical application and rubrics themselves are genres. I know a couple of people in the class collaborated on a rubric and shared it with the rest of us–when I looked at theirs (in Maury’s subfolder in shared class Google Drive folder), I was again surprised to see how similar the rubrics were. This is telling of our expectations. Even though they only grouped their criteria into two sections, the components were the same–they just combined what I considered reflection with the application. Suzanne’s analysis so closely fit both mine and the collaborative rubric that its clear we all have the same conception of what was expected in applying a theory.

Additionally, the format of the rubric was similar. A table on the left hand side with room for comments on the right hand side. As teachers and students of writing, we all clearly value a place for discussion more than we value a checklist through which a writer accumulates points.

I know that my discussion is supposed to focus on what I would change about my rubric, but I like my rubric and its now made me realize what I didn’t do in my own case study. Of course, to do everything my rubric requires in 1000 words  might too difficult, so I guess if I were to tailor it to fit this assignment more closely, I’d try to decide which elements are most important for the student to demonstrate. Since we are summarizing and discussing the theories in other places in the course, I think I would place more value on the application and reflection for the sake of space in this assignment.


Theory Application Rubric: A Class Construction

This rubric really is a social construct: class members collaborated (with a great deal of momentum generated by Maury’s contributions) on the beginnings of our rubric. While each of us likely added or removed bits of the collaborative work to personalize the rubric, I’m proud to be part of this socially constructed, class-sourced rubric development process.

We recognized two major areas of focus for the rubric:

  • Articulation and contextualization of the theory
  • Application of theory to specific OoS (explained with clarity)

After reading the hypertext theory readings, I recommended a third area of focus, which I’ve included in my rubric:

  • Mapping of theory to local context (praxis)
seattle awareness map from 1978

Seattle Awareness Map, 1978: Mapping Seattle’s historical landmarks. Creative Commons licensed image from Flickr user Rob Ketcherside.

While we can apply theory to our OoS, I think it’s important to be able to map the theory to localized instantiations of the OoS. If theory can’t be mapped to specific aspects of practice in the field, then it hardly seems useful (in a pragmatic sense) to the field or its scholars and practitioners. Not that every theory needs a Spinuzzi-like operationalized exemplar to be valid — but we need to be able to identify specific ways that teachers in local contexts will be able to apply theoretical constructs and principles to pedagogy, and how scholars will be able to apply theory to specific recommendations for action in the field.

We also discussed how or whether to assign grades or points to each aspect of the rubric. Most of us chose to avoid assigning grades: our goal was to develop a rubric that could be applied to both assessment and praxis, and my sense is that assessment needs to be localized at the assignment level rather than generalized at the development level (see Discussion below for more on this subject). As a result, I did not include point values, nor would I want to do so without first sharing the rubric with the person to whom I applied it.

My (class-sourced) Theory Application Rubric

Articulated and Contextualized (Theoretical Understanding)

  • Theorist(s) who developed the theory
  • Influential predecessors to the theory to theorist
  • Main premise(s) of the theory and key attributes
  • Limitations of the theory
  • Relationship to other theories in the field and importance to the field
  • Existing canonical or well-respected applications of the theory

Applied to Object of Study and Explained (OoS Understanding, Application)

  • OoS contextualized and explained
  • Theory attributes mapped to OoS attributes
  • Portion(s) of the theory used and discarded, and why
  • Contribution to understanding or re-seeing the OoS
  • Practical benefits of applying the theory
  • Limitations (blind spots) of this theory as applied to this OoS
  • Additions to the body of knowledge surrounding OoS and/or the discipline

Mapped to Local Context (Praxis)

  • Local context(s) to which theory can be mapped
  • Specific person(s) responsible for activated mapping
  • Social and political boundaries defined by theory
  • Aspects of theory mapped to specific lived experience
  • Anticipated social action to be achieved by mapping
  • Assessment process of localized mapping defined

Applying the Rubric

Foucauldian Analysis of Live-Action Role-Playing Games as Networks

Maury applied specific aspects of network construction with Foucaultian theory to LARPs. Below are the results of applying my rubric to her case study.

Theoretical Understanding
Characteristic Addressed Comments
Theorist(s) who developed the theory Yes Foucault
Influential predecessors to the theory to theorist No The assignment did not call for the need to contextualize the theorist among others.
Main premise(s) of the theory and key attributes Partially Foucault offers a broad range of theories; those applicable to the OoS were appropriately selected.
Limitations of the theory Partially The limitations of the theory may have been demonstrated by absence in the case study.
Relationship to other theories in the field and importance to the field No This was not a required component of the assignment.
Existing canonical or well-respected applications of the theory N/A The scope of the assignment did not require this level of exploration of the theory.
OoS Understanding & Application
Characteristic Addressed Comments
OoS contextualized and explained Yes Thorough explanation of the OoS and its context made it accessible to a complete noob.
Theory attributes mapped to OoS attributes Yes Of special note were connection to archive, positivity, absence, and monument.
Portion(s) of the theory used and discarded, and why No It’s difficult to nail down Foucault to a single theoretical stance or even set of stances; as a result, this is an appropriate omission.
Contribution to understanding or re-articulating the OoS Yes Among the strongest aspects of the case study. Application revealed relational and contingent character of the game’s discourse.
Practical benefits of applying the theory Yes Among benefits noted are recognizing the change in meaning that occurs as the game is played.
Limitations (blind spots) of this theory as applied to this OoS No Given the broad range of theoretical position Foucault offers, it’s difficult to identify limitations.
Additions to the body of knowledge surrounding OoS and/or the discipline Yes The networked description of the OoS via Foucault focuses attention on specific connections within the game, and it broadens an understanding of Foucault’s archive and monument.
Praxis
Characteristic Addressed Comments
Local context(s) to which theory can be mapped Yes LARP as distinguished from cosplay, historical re-enactment, creative anachronism, and boffer-style LARP.
Specific person(s) responsible for activated mapping Yes Very detailed; notable are Game Masters along with many other actors on the network.
Social and political boundaries defined by theory Yes The field of game play is clearly articulated and connected to the field of discourse.
Aspects of theory mapped to specific lived experience Yes Another strength of the case study, mapping specific lived experiences of LARP to theoretical aspects.
Anticipated social action to be achieved by mapping Yes Closing statement addresses the specific social action expected: multiplicity of discourse emerging from a single LARP.
Assessment process of localized mapping defined Yes In the same closing statement, successful mapping with be demonstrated by multiple discourses from a single LARP.

Discussion

Theoretical Understanding

The rubric we crowd sourced was intended to address broadly the way a theory is constructed in its time-space and context. Since our assignment was to apply a theory that we had all worked on together in class, neither the assignment nor our execution was expected to spend a great deal of time explaining the key components of the theory, its place among theories, or other contexts related to the theory itself. It was assumed that we’d bring to the assignment that understanding without having to articulate it in the blog post.

However, as a hermeneutic, the rubric offers a useful set of tools for assessing and presenting major theoretical aspects to a reader. Of particular importance as we move forward in our case studies will be explaining more of the influential context of the theory — its predecessors, its influences, its turns and negations, its relationship to other theoretical stances. And a conference paper-length application would certainly be expected to use a literature review to place the theoretical stance(s) in appropriate context. As a result, although this case study implicitly precluded most of the contextual background of Foucaultian theory, the rubric itself is likely hermeneutically sound.

OoS Application and Explanation

In the case of Foucault, nailing down a single theoretical stance, or even a set of theoretical positions, is quite difficult. As a result, omitting some of Foucault’s theoretical positions is necessary in anything but a monograph-length study (and even then, I’m not sure). These omissions don’t necessarily mean they don’t apply to the OoS or that there are no mappings between the theory and the OoS. I take these omissions to be practical realities, and would likely consider them so even in a graded assignment (unless major issues were left unaddressed, like statements or discursive formation). That same breadth of theoretical perspective necessitates the OoS itself to define its limits within the frame of theoretical reference. In a more narrowly focused theoretical stance, I’d expect more explicit statements about the OoS boundaries as defined by the theory. In the case of Foucault, I sensed little of LARPs that Foucault would not address. While this was never explicitly stated or even implied in the analysis, the results speak for themselves — there is no shortage of LARP when applying Foucault. As a result, even though the application does not always address every aspect of the rubric, I don’t think the rubric is faulty.

Praxis

I surprised myself in finding the Praxis section of the rubric the most informative and applicable section of the rubric. I found concrete mappings between theory and localized context. I don’t consider this section to repeat the OoS application and explanation section; to me, the object of study is not necessarily localized. In the case of LARPs, for example, the localized mapping went so far as to specify a single LARP (Three Muskateers), while the OoS itself remained a more general discussion of LARPs. However, even this general discussion worked to localize the LARP by differentiating it from other similar activities. My mapping the OoS in lived experience to theory, both LARP and theoretical understanding benefitted. As a reader with no LARP experience, the localized mapping offered a clear theoretical underpinning to the concept and practice of the LARP while clarifying in concrete examples some of the more difficult concepts of Foucaultian theory. Mapping theory to localized experience offered a win/win experience for me as reader, and I believe that same experience applies to extending knowledge and understanding of both fields.

visualization of map of the internet, 2005

Map of the Internet, 2005: Mapping a global theory to a localization. Creative Commons licensed image from Flickr user Cesar Harada.

Localization

This rubric, like all others, requires a flexible, local application bound up in real experience. The fact that the assignment did not fulfill all requirements of the rubric makes neither the assignment nor the rubric unsuccessful. The assignment called for different expectations than the rubric (which, of course, reveals in practice the importance of developing rubrics prior to, rather than in response to, assignments), so the rubric could not be fully applied to the assignment. Furthermore, the rubric addressed a broader conception of theory and OoS than the format and length of the assignment could achieve. I believe it’s important to recognize ways the rubric can’t or won’t measure exactly what it needs to measure in each and every instance. Every assignment — and every response to every assignment — is a localization, and each requires a flexible application of the rubric. This does not make the rubric an inefficient or inaccurate measurement tool; on the contrary, it reveals the value and significance of local context in measurement.

[Top of page: rubric - Creative Commons licensed image by Flickr user Diana]

Assignment: Applying a Theory Rubric

Developing the Rubric:

I started this assignment with the Zoetewey chapter in mind. The two aspects of "usability" and "usefulness" can be productively adapted for theoretical applications. Can the application be "used" by the readers? Can it be "useful" for them?

I outlined how I see these two terms working in the context of theory application below with a clean copy of the rubric following. 

Usability:

Objectives: Is the application of this theory usable - clear and understandable? Could it be reapplied to another object/problem/question? Would another researcher arrive at a similar application?
  • Key terms are defined so reader has clarity.
  • Theory has been interpreted accurately. Application does not misuse theory or remove it from context in such a way that distorts the theoretical argument.
  • Theory is summarized for context.
  • Object/problem/question to which theory has been applied is explained for understanding.
  • Work applies at least three elements from the theory for triangulation.  

Usefulness:
Objectives: Is the application of this theory useful - relevant and significant? Does it have implications for the discipline? Does it reveal meaning or answers that can further research?
·         Application yields insights, observations, and/or answers about the object/problem/question being analyzed.
·         The conclusions drawn from the application are relevant or significant to the field in which the researcher is working.
·         The application suggests new questions or avenues for exploration in further research - possibly by discussing unused elements of theory, theory shortcomings/limitations/, or unresolved aspects of the object/problem/question.

Assessment Area:
Excellent
Acceptable
Under-developed
Not represented
Usability:




  • Key Terms




  • Accuracy




  • Theory Summary




  • Object Explained




  • Multiple Elements Used




Usefulness:




  • Yields insights




  • Relevance to field




  • Further research







Applying the Rubric:

Randomly selecting a Case Study from the list of blogs on our class page, I came to Chvonne's work with Snapchat.

Assessment Area:
Excellent
Acceptable
Under-developed
Not represented
Usability:




  • Key Terms
X - uses in-text definitions following the inclusion of key terms. Keeps the discussion flowing nicely while still giving information needed. 



  • Accuracy
X - demonstrates a very clear understanding of the concepts used



  • Theory Summary

X - perhaps could include more information about rhetorical situation beyond the three key concepts of audience, constraints, and exigence used.


  • Object Explained
X - very thorough and makes good use of links to Snapchat blog postings



  • Multiple Elements Used
X - exigence, audience, and constraints. Each is applied in more than one way as well.



Usefulness:




  • Yields insights
X - makes several observations about how Snapchat fundamentally differs as a social media platform and the nature of ephemeral action versus documented action.



  • Relevance to field
X - implications for multiple fields including networks and social media.



  • Further research
X - poses several intriguing questions that stem from the application that could be explored in additional thinking




So how did it work? Well, it works as a rubric for assessment, but rather subjectively. If I were a student trying to use this rubric for developing my work, I might be confused by a few points. First, there are more items in the "usability" category, which could lead to someone thinking that was weighted more heavily. Since there are no weights or points assigned to any of the categories, that might be difficult. In my mind, the usefulness categories would be most important, so perhaps revising this I would add percentages (maybe 40/60 top and bottom) to give some information for where work should be concentrated. It is still highly subjective though and I would not want to add a certain amount of points to each item because different theories could yield more or less in one area. I would want to use the rubric to gain an overall sense of usefulness and overall usability to determine a grade from that collective impression.

Second, there are no format criteria included on a rubric, so I can imagine that could be an issue. For example, it does not have an item for length or any required formatting like MLA citation or the inclusion of visuals. Surely an assignment would entail these items (or maybe I've been working with freshman too long), but a good rubric I think speaks to all aspects of an assignment. 

Lastly, the subjectivity could be problematic. For example, what is the difference between an "excellent" and "acceptable"? I think this would require some more explanation, but even still it comes down to interpreting qualities of creativity, thoughtfulness, insightfulness... These are not easily measured by rubric boxes, nor are they easily explained to a student. It comes down to an impression or feeling about whether some area of the rubric was present in the application and perhaps how it compared to other examples from the class. Students when presented with a rubric can often focus narrowly on the items and not realize the full potential of their thinking. It can be limiting at the same time it can be helpful for students to gain access to difficult work. I am not sure how I could remove some subjectivity or even if I would want to, but it is part of this rubric as it stands now.

Overall, I like the rubric and think it would be helpful for constructing and assessing the application of theory. 

Theory Application Rubrics: This Is Only A Test

This week, we were asked to create a “theory assessment” rubric as a means of deepening our understanding of how we might apply theory to objects of study (at least, I think that’s the reasoning!). Given my sense that I’m … Continue reading