Wikipedia:Snuggle/Study/Affordances vs motivation

So, users of Snuggle don’t interact with newcomers very often through Snuggle. Instead, they mostly perform categorizations. I hypothesize that this is largely due to the affordances of the software. Snuggle’s current design hides the “action menu” and makes the categorization menu prominent. By hiding one feature and making another more prominent, Snuggle encourages one action and discourages another.

An alternative hypothesis is that Snuggle users would rather not interact with newcomers themselves and prefer the less socially-consequential activity of categorizing despite the fact that categorization itself is not, by itself, a productive activity.

I propose a natural experiment designed to test whether users’ behavior can be substantially changed by altering the affordances of the interface to make Snuggle’s “action menu” equally prominent.

Overview
Let the time that the software change is deployed be t and the duration of an experimental phase be d.

Before t-d, we’ll gather a list of active Wikipedians who fill mentorship roles in Wikipedia but have not yet used Snuggle.

In time t-d through t, we’ll draw a random sample of 50% of the list to go through a demo-interview session (described below).

At time t, I’ll deploy the software change to Snuggle that evens out the affordances of “categorization” and the “action menu”

In time t through t+d, I’ll draw the remaining Wikipedians from the list and bring them through a demo-interview session.

Demo-interview session
These sessions will consist of: A 15 minute demo of the software that includes the Wikipedian using the software on their own A semi-structured interview about mentoring activities and Snuggle

Demo
Participants will be given a brief overview of Snuggle’s functionality and asked to give the tool a test ride for ~15 minutes. The participant will be asked to share any thoughts he/she has while using the interface. The interviewer will be present to answer questions and provide instruction upon request.

Issue: What if different users see significantly different data when they do the task?

Interview
The goal of these interview questions is to (1) figure out how the participant thinks of newcomers and the importance of socialization/mentoring in Wikipedia and (2) understand how the participant would like to use Snuggle.

[Ask for a story first -- first/most-recent interaction with a newcomer]

General mentoring/socializing questions

 * How do you interact with newcomers during your wiki activities? Is it mostly positive or negative experience?
 * What got you interested in the activities & struggles of newcomers?
 * How common do you think it is that a newcomer runs into trouble and doesn’t know where to go to get help?
 * Do you have a way of finding such newcomers?
 * What is your goal when interacting with a newcomer? (Why?)
 * (clarify: Support vs. monitor and mitigate)

Snuggle UI

 * What do you think of the (check), (question) and (X) buttons?
 * How do you feel about categorizing newcomers into good/bad-faith categories?
 * What do you think happens when you categorize a newcomer?
 * Did you notice the action menu?
 * When might you want to perform the available actions?
 * Are there any important actions you feel are missing?
 * What do you think that the icons on the right-hand side of the user view mean?
 * Is it helpful to know what types of messages are on a newcomer’s talk page?
 * Do the icons express this information well?

Locations

 * WER
 * EotW
 * Teahouse
 * Adopt-a-user
 * AfC
 * WT:Snuggle
 * Mentoring IdeaLab discussion

Personal Snuggle demo & interview
Hello NAME. I've been working on a tool to support the work of mentors on Wikipedia called Snuggle. Given your work in SOCIALIZATION SPACE, I figure that you're one of few editors that are particularly capable of giving me some insightful feedback. Would you be interested in scheduling some time to sit down for a demonstration of the tool and a brief interview about your newcomer support work in Wikipedia? Thanks, --EpochFail (talk &bull; contribs) 19:32, 30 July 2013 (UTC)