User:Ryan McGrady/395proposal/Assessment

__NOINDEX__



Important! Read First!
This page is currently designed to serve four functions for five audiences (one hypothetical):
 * 1) as a proposal for COM/ENG 395
 * 2) *the Justification section on the main course page, for example, is intended for this audience
 * 3) as a course design project for CRD 704
 * 4) *the Scholarly Narrative and Assessment Plan sections are intended for this audience
 * 5) as means for other educators and Wikipedians to provide feedback on what I have planned
 * 6) as a personal workspace
 * 7) *before this course is opened to students, a lot will be moved to my personal (private) wiki that I already use for organizing my COM 110 teaching materials
 * students are the fifth (hypothetical) audience for which several of these sections were written, but this page will be redesigned before actual use by students

=Assessment Plan=

This section initially overlapped considerably with the Scholarly Narrative, so I have since removed from the latter the section on assignment descriptions, which added relatively little beyond what I go over here.

Assignment Descriptions
There are four components to students' grades: class participation, online discussion, a group project, and a final paper. I found this to be the best way to blend criticism and practice.

Participation (20%)
In the spirit of collaboration and community, this will not be a course with many unidirectional lectures. I therefore make clear to students that it's imperative they do the readings and come to class prepared to enter into the discussion or activity of the day. A pattern of unpreparedness and/or absences will significantly affect their grade. I plan to take attendance every day and keep notes of who made significant or conspicuously missing contributions. I made this 20% of their final grade, a larger percentage than it counts for in most cases because I really want to emphasize the importance of active discussion and participation. I can talk endlessly about Wikipedia, but that's of limited use both to me as someone who wants to learn from my students and to them who would be robbed of first-hand engagement with the materials.

Online Discussion (20%)
The first item on the schedule for each week is a link to a discussion prompt. When students click the link, they will be taken to a wiki page dedicated to that week's question. They are instructed to edit the page, add a new section with their name at the bottom, and post a response of 300-500 words by the start of class that week. Questions are based on class discussions and try to connect what we've talked about so far to the readings and activities for the coming week.

During the week, they are asked to respond to at least one classmate's posts by again editing the page and writing a comment at the bottom of the classmate's section. I don't specify a length for this as any back-and-forth it might spark would accomplish its purpose.

My goal for these assignments aren't to task them with performing completion of the readings due that same day, but rather to get them to reflect on their thoughts of the previous topics in such a way that leads them into what we will talk about. Further, it's a weekly exercise in sharing ideas on the subject and in using Wikipedia. In my experience, simply spending time editing articles encourages you to explore different styles, techniques, and interactions.

This, too, is worth 20% of the final grade. It was higher when I intended it to revolve around the readings to be done (I would take 5% from participation and add it here), but I thought it would be more valuable to a) get their reflections/opinions on past discussions, and b) encourage linking of ideas across the course, focusing on what they already have experience with rather than on what they might not yet understand.

Group Project (35%)
Using the tools and techniques covered in class, groups of 3-4 will be responsible for finding and choosing an article on a topic related to this course which either does not exist or only exists as a stub. Throughout the course each group is responsible for writing and developing that article toward the goal of attaining Good Article status. Good articles are well-written, a proper length, accurate, verifiable, comprehensive, neutral, stable, properly formatted, organized well, follow style guidelines, and use media when appropriate. An article must be nominated for this designation, whereby outsiders will review it and provide feedback before approving or rejecting.

A step up from Good Articles are Featured Articles, judged according to the Wikipedia:Featured article criteria. As I write this, 1 in 286 articles have the Good Article label, but only 1 in 1,120 have been awarded Featured status (and the Good designation is much newer). Students are not expected to attain Featured Article status, but the reward for doing so (a very big task) is an A on this assignment.

This project will require an understanding of not just suitable writing techniques and technical competency with a wiki, but also familiarity with the processes and conventions of the Wikipedia community.

At 35%, it is the biggest piece of their grade. It may not seem like a lot of work at first glance, but speaking from experience, it certainly is. Anybody can click edit and start typing, but it takes effort to write within the genre, with strangers watching your every move, expecting you to conform to particular styles, to discuss what you're doing, and so on. Featured Articles have been a gold standard for almost as long as Wikipedia has existed, but their requirements are so stringent and often not specifically stated, that Good Articles came along some years later as a way to reward hard work and bring attention to an article not quite ready for FA. I think the criteria themselves are useful, but the real reason I'm holding them to this standard is beacuse of the value of the nomination process by which an article is granted these titles. They are effectively forced to interact with the community, solicit peer-review, make changes, and learn new conventions, and they will often be pointed toward different committees and community pages to get more specific types or sources of feedback. All of these interactions and feedback are then archived forever, available to those who look at what is effectively a "revise and resubmit" and to me.

Final Paper (25%)
The final paper will be based on the group projects, but written individually. Students are to describe the process of writing, developing, editing, and otherwise crafting their article. I ask them to go into the processes their team undertook, including struggles with different aspects of the assignment: writing, organization, formatting, sourcing, style, group dynamics, outside contributors, peer-review processes, and so on. For example, what kinds of skills did they have to learn that may be valuable? Did their opinion of the site change? As Wikipedia is completely transparent, I will be looking for links to challenges and interactions described, so I will encourage them to keep a weekly or biweekly journal of activity to look back on when writing this paper, although I don't intend to collect these.

This paper is central to my assessment research question, and will be discussed more below.

Assessment
My primary research question is "how do students feel about their collaborative writing/learning experience?" I am most interested in what the students have to say. I already know about these topics, and about Wikipedia, but where I don't have a lot of experience is getting students to see it in new ways. Since this is a course I hope to teach again and bring with my as I look for a job in academia, I plan to continually develop it. The audience for the assessment plan and its findings is thus myself. A secondary audience down the road might be other educators at the Wikipedia:School and University Projects initiative, where I do plan to share my thoughts on how the course went once it's over.

The aspect of wikis (and Wikipedia) most often highlighted as a tool for assessment is its transparency. Every edit a student makes, every contribution to a conversation, every addition to a page, every request for feedback, all feedback it receives, all associated dates, the size of each contribution, how their words have been changed since a particular time, and a host of other data is publicly available through History and User Contribution pages. I do plan to use this to make sure everybody is contributing to their group projects, to make sure discussion posts were done on time, and to make sure proper conventions are being followed (as opposed to, say, adding the Good Article header without it being approved first). But transparency doesn't answer my research question.

The most important part of the course in terms of this assessment plan is the final paper. Toward the end of the group project, once everybody has had a chance to interact with the Wikipedia community, write article content, play with wikitext, encounter difficulties with style and content guidelines and policies, etc. they will start work on an individual paper. While based on experiences in the group, each will be done independently. I will encourage them to include course concepts when appropriate, but only when appropriate.

My hope is to get responses that touch upon expectations vs. reality, complexities of the community, greater understanding of how articles are crafted, a different sense of trust for Wikipedia, a more informed critical eye when looking at Wikipedia (or, preferably, information/texts in general), difficulties and benefits associated with collaborative knowledge-building work, opinions on collective intelligence, how important they felt their contributions were, how they felt about their finished product, and whether or not they were proud enough to point their peers toward their work outside of class.

Most importantly, however, I want to know (and will explicitly ask):
 * did group work affect the way you understood the readings or class discussions?
 * did the readings or class discussions affect the way you understood the group work?
 * was this a valuable learning exercise?

This kind of feedback will be invaluable to me as I develop this course. I'm confident in my ability to talk about and encourage discussions about Wikipedia, but I'm less confident (read: lacking experience) in my course design. It's the best that I can do, but before putting it into practice it's my nature to question whether or not it will work. Only one way to find out.