User talk:ErrantX/Essays/RFA Study

< BACK to study page

This is excellent stuff :o) Pesky  ( talk ) 05:38, 30 March 2011 (UTC)

Pesky's Thoughts
Who/when to ask: I would say one for immediately after, one during (particularly if it seems to have got heated / unpleasant), and one after a cool-off period.

Real Life Effects:
 * can you give a parallel to something that has actually affected you in real life?
 * Can you give examples of the effects? (maybe this should be a 'tick all boxes that apply' question, with the option to add 'other'  (insomnia, loss of appetite, depression, anxiety, poorer judgement than usual, irritability, relationship problems, nightmares, etc. etc.)

On-Wiki Effects: We can add more examples in here, too. Check boxes again?

About Your RfA: Further questions:
 * What / who did you feel was most unfair about the process? (checkboxes)
 * False accusations (and did it generate a free-for-all pile-on?
 * Insignificant stuff being blown out of all proportion / taken out of context?
 * Ancient sins being dredged up and treated as current?
 * Your 'personal style' being attacked, rather than your abilities?
 * (other)


 * How do you feel your RfA could have been improved?

Since RfA: More checkboxes ...
 * Have you stopped interacting with people you worked with before?
 * Have other people treated you differently from before?
 * Do you feel your RfA has made people view you differently?
 * Have you felt tempted to resign altogether?
 * Or maybe just edit as an IP to be less visible?
 * (other) - name it/them

Pesky ( talk ) 05:38, 30 March 2011 (UTC)

Comment from Kudpung
This an excellent idea and will  greatly  support the assumptions we're working  on  with  our goals for reform to  be proposed by  User:Kudpung/RfA reform|this RfA reform  task-force. Similar polls have been tried in the past  but  perhaps this time it will  get  response. It needs to  be done quickly  though. --Kudpung กุดผึ้ง (talk) 08:03, 30 March 2011 (UTC)

Thoughts
It's a great idea. I do think it should be given to the candidate very soon after and preferable with a spiel that they are still a valued member of the community, adminship isn't for everyone, you can always try again etc.

I wonder if any of the following questions would be a good idea
 * Do you agree with the communities concerns?
 * Was the feedback constructive?
 * Do you think you will run for adminship again?
 * Do you have a clear idea how to carry on as a member of this community?
 * What do you plan to do now?

The great thing about the survey is that it confirms we still value the editor's opinion, and that should be reinforced with a welcome blurb. WormTT  &middot; &#32;(talk) 08:14, 30 March 2011 (UTC)

I think the poll should be targeted at every RfA candidate over the last 12 months - the feedback would be invaluable to the project  we're working  on at here. Kudpung กุดผึ้ง (talk) 08:19, 30 March 2011 (UTC)
 * Worm; my thinking was that initially such a survey could be used as Kudpung suggests - to inform the task force he has going by surveying a wider array of people. Then in the future, yes, we could have some sort of ongoing process to get feedback. --Errant (chat!) 08:23, 30 March 2011 (UTC)
 * Sounds sensible to me - I'll adjust my perceptions accordingly ;)  WormTT   &middot; &#32;(talk) 08:27, 30 March 2011 (UTC)

Trial questionnaire run
I think  the poll  should be carried out anonymously on  a check-box and answer field form, we have developers for that. A bot could send the form  to  all  admin candidates over the past  12 months, whether successful, failed, or withdrawn, but  not  to  NOTNOW closures. I don't  mind my  answers here being  public on  this trial  run. My RfA was very  recent. The answers are true. I hope this feedback helps.

Proposed survey questions
Real Life Effects
 * Did your RFA cause real life stress? Yes.
 * If Yes, how much: (a little/not much/some/a lot/severe) a lot.
 * If Yes, did it cause more stress than you have previously experienced due to Wikipedia? (Yes/Same/No) Yes.
 * If you are happy to do so please give some details of how real: Having to  keep  cool  in  the face of some of the most  contemptuous commenting  I  have ever been involved with on  Wikipedia. 

On-Wiki Effects
 * During the RFA process did your on-wiki editing activities change? (Yes/No) Yes
 * If Yes, did they: increase/decrease: stayed the same but obviously  with  a shift  of focus to  the RfA
 * If Yes, did you do more, less or the same amount of your usual activities (i.e. article work, anti-vandalism)? Yes/No:
 * No 'Yes/No'  answer possible to  this question. Continued doing  some less stressful activities such  as NPP.
 * Positive or Negative process? Overall, a negative experience

About the RFA (to answer the following questions you could link to the RFA itself and let us fill in the detail)
 * Was your RFA successful? Yes
 * How many previous RFA's did you have? None
 * What length of time elapsed between your previous two RFA's? N/A
 * Were you nominated or self-nominated? Nominated and co-nominated. Both noms were admins.
 * How many edits did you have at the time? around 28,000
 * In article space? 35% : about 9,000
 * How many good or featured articles did you have at the time? 5 GA

Since RFA
 * Has your editing changed since RFA? Yes
 * In a positive or negative way? Positive
 * If negative explain why:

Real Life Effects:  On-Wiki Effects''': We can add more examples in here, too. Check boxes again?
 * can you give a parallel to something that has actually affected you in real life? Yes. Nasty things about  me put on  social  networking  sites by  people I  have had to  fire. (I have never subscribed to  a social  networking  site. I  think  they  are evil.)
 * Can you give examples of the effects? (maybe this should be a 'tick all boxes that apply' question, with the option to add 'other'  (insomnia, loss of appetite, depression, anxiety, poorer judgement than usual, irritability, relationship problems, nightmares, etc. etc.): '''Insomnia, because I  live in  the opposite time zone (east  Asia). Gritting  my  teeth in  the face of adversary, pacing  round the house, with  my  wife not  understanding  what  was going  on. Overwhelmed with  joy  when editors came to  my  defence and  embracing  my  wife with  her not  understanding  what  was going  on. Generally  displaying  rare emotions at  home that are not to  be expected from  admins -  but  RfA  is a very  personal  thing.

About Your RfA: Further questions:
 * What / who did you feel was most unfair about the process? (checkboxes)
 * False accusations (and did it generate a free-for-all pile-on)?: Yes, and Yes.
 * Insignificant stuff being blown out of all proportion / taken out of context? Yes, and Yes.
 * Ancient sins being dredged up and treated as current?: Yes
 * Your 'personal style' being attacked, rather than your abilities?: Yes
 * (other)


 * How do you feel your RfA could have been improved? All the reasons in my  project  at  User:Kudpung/RfA reform

Since RfA: More checkboxes ...
 * Have you stopped interacting with people you worked with before? No. But I have reduced my work in  some Wikipedia areas that  I  had a great  interest  for.
 * Have other people treated you differently from before? Not that I have noticed
 * Do you feel your RfA has made people view you differently? Not that I have noticed
 * Have you felt tempted to resign altogether? While the RfA was going very badly, I had considered retiring (without a speech) if it  failed,  but  in  hindsight  I  would probably  not  have done.
 * Or maybe just edit as an IP to be less visible? No
 * (other) - name it/them


 * Do you agree with the communities concern: (Question too  vague)
 * Was the feedback constructive? No, not really. Nothing I didn't know already.
 * Do you think you will run for adminship again? N/A
 * Do you have a clear idea how to carry on as a member of this community? Yes
 * What do you plan to do now? I have shifted my  focus slightly  towards more involvement in  policy and process improvements

Additional possible questions:

 * How long had you  been a regular editor to  Wikipedia before your  RfA?: 5 years
 * How many times had you voted on RfA before your own RfA? around 100
 * How many times have you voted on RfA since your own RfA? around 12 (RfA was 4 weeks ago)
 * Did you take part in discussions at WT:RfA before your RfA? Yes
 * Did you read any RfA advice/instruction pages such as Requests for adminship before your RfA? Yes
 * Did you read any RfA essays before your RfA? Yes
 * Did you read any RfA user essays before your RfA? Yes
 * Did you read any users' RfA criteria essays before your RfA? Yes
 * What was the total number of support+oppose+neutral votes on the final  tally? 137
 * Do you think enough people turned up to vote? Yes
 * How many additional questions were posed on your RfA in  the question section (count compound questions as separate questions): 16
 * What do you think would have been a reasonable maximum number? 10
 * Would you have preferred for the additional questions to be posed, and answered by you before the voting starts?
 * Did you find the questions posed by voters reasonable? Most of them
 * Did you find any question(s) totally irresponsible, impertinent, or completely irrelevant to the RfA process? Yes
 * Did you consult your nominator or another editor off-Wiki for help  in  answering  the questions? No
 * Do you think response to additional questions should be optional? Yes
 * Would it have been helpful to  you if the  bureaucrat had  provided a closing  summary/rationale? No particulary, the RfA closed with  a clear majority  for support
 * If your RfA had failed, would you have considered running  again?: probably not
 * If your RfA failed, would you consider running  again?: N/A

Additional questons I missed or that  were added later

 * Did any of the following occur:
 * False accusations or assumption of bad faith: Yes
 * Insignificant (in your opinion) behaviour/action being given undue prominence: Yes
 * Historical (> 6 Months) indiscretions (blocks etc.) being brought up as current: No (I have never been blocked or warned for anything).
 * Personal attacks (i.e. on your character): Yes, quite rudely. I would have placed a civility  warning  on  such  a user's talk  page without  hesitation.

Kudpung กุดผึ้ง (talk) 11:02, 30 March 2011 (UTC)

Polling !voters
This would be more difficult. Of the 207 or so RfA over the last  12 months, this could run  into possible  1,000s. Apart from the 50 or so 'regular' voters, many  are one-time voters. However, We're working on  getting  a statistical voter profile made for the same period. --Kudpung กุดผึ้ง (talk) 11:25, 30 March 2011 (UTC)

What do voters check?
I like the idea of surveying the voters, but would suggest a few more questions:  Ϣere Spiel  Chequers  12:51, 30 March 2011 (UTC)
 * 1) Which of the following do you check before deciding to !Vote?
 * 2) The candidates contributions
 * 3) The candidate's deleted contributions
 * 4) The candidate's block log
 * 5) The candidate's talkpage and its history
 * 6) The candidate's userpages
 * 7) The Nomination and acceptance section in the RFA
 * 8) The question section in the RFA
 * 9) The Support section in the RFA
 * 10) The Oppose section in the RFA
 * 11) The Neutral section in the RFA
 * 12) The Talkpage of the RFA
 * 13) Before you !vote in an RFA, how long do you spend reviewing the candidate and the discussion in that RFA.
 * 14) Less than ten minutes
 * 15) Ten minutes to half an hour
 * 16) Half an hour to an hour
 * 17) More than an hour
 * 18) When you !vote in an RFA do you watchlist it and follow the subsequent debate?
 * 19) Have you ever changed your !vote as a result of other things emerging in the RFA?
 * 20) Have you ever changed your !vote as a result of responses to your voting rationale.
 * Also, how about a more general - "what one thing do you look for most in a candiate?" I like this idea, but I'm not sure it covers everything without letting the voter have a general place to talk.  WormTT   &middot; &#32;(talk) 13:21, 30 March 2011 (UTC)
 * Good stuff people! :) --Errant (chat!) 13:24, 30 March 2011 (UTC)
 * Good stuff WSC, but  how many  would answer it  honestly, even if it were 100% anonymous? An endemic problem at  RfA is the high  number of rare, or one-time !voters. They  run  into  the 1,000s. I've asked SW if he can run up  a stats table to  be able to  extrapolate a voter profile. Kudpung กุดผึ้ง (talk) 13:41, 30 March 2011 (UTC)
 * Speaking of stats - wasn't there a tool which you could use to see how someone had voted at RfA? Is that still about?  WormTT   &middot; &#32;(talk) 13:45, 30 March 2011 (UTC)
 * It's still about but  for some reason  access to  it  through  the X-tools home interface has been disabled. You  can reach  it through  here, you  just  need to  change the user name in  your browser bar once it's loaded, and then reload. Kudpung กุดผึ้ง (talk) 14:12, 30 March 2011 (UTC)
 * Re honesty of response. Actually I don't want an honest response to the amount of scrutiny that most voters give candidates, if I wanted an honest response I would include options for under a minute and 1-5 minutes. I'd rather use the survey to gently chide a number of our voters into thinking that perhaps they should put a bit more effort into their own research before they voted. As for those who've only voted once, I see no harm in excluding them from this study, if we ever survey them it should be with a tailored survey.  Ϣere Spiel  Chequers  14:37, 30 March 2011 (UTC)

Might be an idea to ask them why they only voted once? (In case it was something along the lines of 'so disgusted / put off by whole thing that I never wanted to go back there') Pesky  ( talk ) 02:53, 31 March 2011 (UTC)


 * Possible checkbox answers:
 * Support voters:
 * The candidate gave me a barnstar
 * The candidate is in my class at school
 * The candidate emailed me
 * The candidate sent me a text message
 * The candidate is my big sister's boy friend
 * The candidate is my big brother
 * The candidate gave me $10 to vote
 * The candidate gave me $100 to vote
 * The candidate promised me a new skateboard if I voted
 * I thought it would be cool to vote at RfA
 * Oppose voters:
 * The candidate redirected my article about my elementary school
 * The candidate CSD'd/PRODed/AfD'd my article
 * The candidate reported me to ANI for calling him an 'obtuse jerk'
 * The candidate is in my class at school and s/he's the teacher's pet
 * The candidate is an atheist
 * The candidate has barnstars on their talk page
 * No special reason.
 * I thought it would be cool to vote at RfA
 * Kudpung กุดผึ้ง (talk) 17:50, 1 April 2011 (UTC)

Preview questionnaire
I've created a "preview" questionnaire for candidates based on what we have so far, on Survey Monkey. It's not intended to be ready for the wild or anything (if you fill it in the results go nowhere :D) but it was mostly to test the abilities of the tool and get a feel for the format/style of questions. http://tool.surveypirate.com/SurveyPage.aspx?surveyid=27689

FWIW I'm unimpressed with the overall result of SM.. so will look at other tools too.

I noticed above that someone talked about doing it on-wiki with dev support; given the timeframe I don't think this is possible. But if it is - what options are available? --Errant (chat!) 13:24, 30 March 2011 (UTC)


 * Yes, I mentioned it. Polls have made before and it's quite easy  to  do. There's no  harm  in  asking  dev support if they  can do it  quickly  enough. They  could do  it  in  less than the time you  spent  on SurveyPirate, especially  if they  had that  as a model, and it  would be hosted on  the WMF servers and be quicker and more secure. --Kudpung กุดผึ้ง (talk) 14:06, 30 March 2011 (UTC)
 * Well there is Special:SecurePoll, which gets used in elections. I dug through the code for it (as there is no documentation) and it could be usable *except* it is designed for elections so it's more an approval voting system at the moment than a survey. In addition it only has support for radio check boxes - so no text-box entries etc. However, it could be useful, I will try to ping someone about its use. Certainly for a quick "end of RFA feedback" that we use day-to-day it would be *perfect* --Errant (chat!) 14:37, 30 March 2011 (UTC)
 * There's also Special:SimpleSurvey under special pages. Always been curious as to what that was...  WormTT   &middot; &#32;(talk) 14:41, 30 March 2011 (UTC)
 * Well, would you look at that. I knew I should go digging more often..  WormTT   &middot; &#32;(talk) 14:44, 30 March 2011 (UTC)
 * Oh cool, that would work. Assuming the syntax for creating them is as it is on the SVN then that should be pretty simple --Errant (chat!) 14:51, 30 March 2011 (UTC)

Hey, that questionnaire was fun! Pesky ( talk ) 16:52, 30 March 2011 (UTC) @Worm - that looks like just the biscuit :o) Pesky  ( talk ) 16:53, 30 March 2011 (UTC)

Wording
Query: My first run was a self nom, my second I was nominated. How should I answer this question, and can the question be reworded? "Was your most recent RFA" would be clear, but adding a third answer
 * Was your RFA...
 * 1) A Self-Nomination
 * 2) A Nomination by another editor
 * 3, "I've run as a self nomination and also when nominated by another editor." would be better.


 * At the time of your MOST RECENT RFA, how many ARTICLE space edits did you have?
 * It's never a good idea to ask people a survey question which they won't know the answer to. You risk them guessing or wandering off and looking this up. If you must ask it put in a link to the answer - in this case it will be the talkpage of their most recent RFA, better still you could retrieve that info if they tell you their username. However that raises a secondary issue with this sort of question, Privacy. The exact number of article edits at the time of your last RFA is on its own a unique identifier for most people who have had an RFA, combine that with another variable as a tie breaker and you could uniquely identify anyone completing the survey. Much better to band it or ask "to the nearest 1,000".


 * At the time of your MOST RECENT RFA, how many Good and/or Featured Articles (combined) did you have?
 * I would suggest "had you contributed" or "had you been a significant author of" instead of "did you have".
 *  Ϣere Spiel  Chequers  11:13, 31 March 2011 (UTC)
 * All good, sound sense. Pesky  ( talk ) 12:32, 31 March 2011 (UTC)