Wikipedia:WikiProject United States Public Policy/Assessment

 Project Research Update  edit | watch

Article quality assessment is the primary research goal for the WikiProject: United States Public Policy page. As the basis for evaluation overall article quality improvement, article assessment is essential to the Public Policy Initiative (PPI).

Stuff To Do
Are you interested in research and assessment or just looking to help out? Here's a list of tasks with varying skill and commitment levels, so everyone can participate!


 * Tag articles to be included in WikiProject United States Public Policy (WP:USPP):
 * 1) Click on the edit page of an article relating to United States Public Policy
 * 2) Copy the text between the squiggly brackets.
 * 3) Paste the text at the top of the edit page below other project tags.
 * 4) Preview your changes.
 * 5) Save your tag!
 * Edit articles.


 * A fall participant in the project said, "If you know something about Absentee ballot, and you edit that page, since Wikipedia is always in the top 5 of a Google search, then everyone who ever looks up Absentee ballot will be better off because you improved that article."


 * Assess articles within WP:USPP


 * Follow the steps and watch the tutorial video below.


 * Join the Assessment Team.
 * This team's work is proving that Wikipedians have the highest and most consistent standards of article quality.
 * Contact, Amy is the Research Analyst for the project.

Quantitative Article Quality Assessment Metric
At the start of the project, participants worked together to create a quantitative metric so that we could measure improvement in article quality. This quantitative metric shows consistent results among Wikipedians, results in ratings that align with the 1.0 Assessment ratings, and most importantly captures Wikipedia principles, especially neutrality and article quality.

There are two ways to tag articles for WP:USPP, for the article quality rating metric described on this page, paste this code on an article's talk page: which produces output like this:

You can also use the standard assessment ratings to tag articles using just the class parameter instead of the quantitative metric tag:

How to use the Quantitative Metric
This rubric is based Wikipedia's policies and expectations for high-quality articles. It has detailed breakdowns of scores for different aspects of article quality, but it also can translate into the standard Stub/Start/C/B scale and thus feed into the 1.0 assessment system without too much duplicated effort. The language is for what is expected for high-quality articles is mostly adapted from the featured article criteria.

Comprehensiveness
The article covers all significant aspects of the topic, neglecting no major facts or details and placing the subject in context. Any score from 1 to 10 is possible.
 * The article is comprehensive, going into appropriate detail about all significant aspects of the topic, and using summary style where appropriate. - 10 points
 * The article is mostly comprehensive but falls short in one or more significant aspects of the topic. - 7 points
 * The article is well-developed in some aspects but requires major expansion in others. - 4 points
 * The article goes beyond a preliminary introduction, with at least some detail beyond a brief overview, but is far from comprehensive. - 3 points
 * The article is a stub, consisting of only a paragraph or two of brief introduction to the topic. - 1 point

Sourcing
The article is well-researched. It is verifiable and cites its sources, with inline citations to reliable sources for any material that is likely to be challenged and for all quotations. Any score from 0 to 6 is possible.
 * The article is well-sourced, such that readers can determine which information comes from which source. The most appropriate source are used, including journal articles and scholarly monographs where possible. - 6 points
 * The article is mostly well-sourced, but has some material that is not sourced or does not use the most appropriate sources. - 4 points
 * A significant portion of the article is well-sourced, but the majority of it is not adequately sourced. - 2 points
 * The article contains only a bibliography, or only a small portion of the article is well-sourced. - 1 point
 * The article does not reference any reliable sources. - 0 points

Neutrality
The article has a neutral point of view, accurately representing significant points of view on the topic without advocating or placing inappropriate weight on particular viewpoints.
 * The article follows the NPOV policy. - 3 points
 * The article follows the NPOV policy, with only minor exceptions. - 2 points
 * Minor exceptions include subtle imbalances in the ways different comparably significant viewpoints are described, the exclusion of minor but still significant viewpoints when all major viewpoints are covered, etc. Such an article is neutral on the whole, but may have a few small problem areas.
 * The article mostly follows the NPOV policy for the viewpoints represented, but other major viewpoints are absent - 1 point
 * The article falls significantly short of following the NPOV policy. - 0 points

Readability
The prose is engaging and of a professional standard, and there are no significant grammar problems.
 * The article has excellent style and grammar and is highly readable. - 3 points
 * The article is comprehensible and reasonably clear, but a need for copy editing is apparent. - 2 points
 * The organization, style and/or grammar of the article detract significantly from the reading experience. - 1 point
 * The article is difficult to understand and requires a thorough re-write. - 0 points

Formatting
The article is organized and formatted according to Wikipedia standards and generally adheres to the manual of style.
 * The article is well-formatted and is mostly consistent with itself and with the manual of style. - 2 points
 * The article has modest deficiencies in format and/or deviates significantly from the manual of style. - 1 point
 * The article is poorly formatted such that the formatting detracts significantly from the reading experience. - 0 points

Illustrations
The article is illustrated as well as possible using images (and other media where appropriate) that follow the image use policy and have acceptable copyright status. The images are appropriately captioned and have alt text.
 * The article is well-illustrated, with all or nearly all the appropriate images and captions. - 2 points
 * The article is partially illustrated, but more or better images should be added. - 1 point
 * The article has few or no illustrations, or inappropriate illustrations. - 0 points

Translation of Quantitative Scores to 1.0 Ratings
Numerical scores can be translated into the different classes on the 1.0 assessment scale. For the lower classes, comprehensiveness and sourcing are the main things that differentiate articles of different classes; things like neutrality, style, layout, and illustrations quickly become important as well for the higher tiers of the assessment scale. GA-class and higher require separate reviews, but high numerical scores can indicate whether an article is a likely candidate for one of these ratings. For everything except GA and FA, the ratings are automatically determined by the banner template if detailed scores are present.


 * Stub - An article with a 1 or 2 in comprehensiveness is Stub-class.
 * Start - An article with a 3 or higher in comprehensiveness that does not qualify for a higher rating is Start-class.
 * C - An article must have at least a score of 4 in comprehensiveness and 2 in sourcing to qualify as C-class.
 * B - An article must have at least a score of 7 in comprehensiveness, 4 in sourcing, 2 in readability, and 2 in neutrality to qualify as B-class.
 * GA - An article with at least 8 in comprehensiveness, 5 in sourcing, 3 in neutrality, 2 in readability, 2 in formatting and 1 in illustrations may be a good candidates to be nominated for Good Article status. (B is the highest rating automatically assigned by a numerical assessment.)
 * A - An article with a 10 in comprehensiveness, 6 in sourcing, 2 in readability, 3 in neutrality, 2 in formatting, and 2 in illustrations may be good candidates for an A-class review.
 * FA - An article with full points in every category may be a good Featured Article Candidate; even then, additional work may be necessary to comply fully with the manual of style.

Proof! The Metric Really Works
The first assessment performed by the WP:USPP assessment team tested the quantitative article quality metric. Three main conclusions can be drawn from the metric assessment.
 * 1) The quantitative article quality metric generates the same scores as the 1.0 rating system when used by Wikipedians.
 * 2) The quantitative metric shows that Wikipedians tend to rate article quality with a high degree of consistency.
 * 3) The metric assessment indicates that Wikipedians are tougher critics of article quality than subject matter experts. (The reasons for this are discussed on the talk page and in the full report.)

Assessment Team
The assessment team is crucial to the success of the PPI. Their work uses a quantifiable measurement of Wikipedia article quality. Their work shows that Wikipedia assessment of article quality is tougher and more consistent than non-Wikipedian subject matter expert assessment. Their work helps define success of the project. In addition to the Wikipedia assessors listed below there are a few subject matter experts who assess via a Google Group.

Wikipedia Assessors

 * User:Bejinhan


 * User:CasualObserver'48


 * User:Cordless_Larry


 * User:Fetchcomms


 * User:JayJasper


 * User:Mabeenot


 * User:Mike_Christie


 * User:Pjoef


 * User:RexxS


 * User:Ronk01


 * User:MacMed


 * User:MikeLynch


 * User:MikeBeckett

If you would like to join the Assessment Team, your participation is welcome, contact.

Fall Semester

 * 1) Metric Analysis - this assessment tests how well the quantitative metric works
 * 2) Article Feedback Tool - this assessment compares Wikipedian, expert, and reader article quality scores
 * 3) Student Pre Test - this assessment benchmarks article quality before students work on them for their class assignment
 * 4) Student Post test - this assessment quantifies article improvement through student work on the project

Spring Term
The first assessment request is posted! Things are pretty exciting this term: LOTS of interesting topics and students are making some big improvements to content. The assessment team sees firsthand the impact of the project.


 * 1) Student Pre Test 2.1 Requested 1 April 2011, Completed 15 April 2011- this assessment benchmarks article quality before students work on them for their class assignment
 * 2) Student Post test 2.1 Requested 9 May 2011, Completed 20 May 2011 - this assessment quantifies article improvement through student work on the project
 * 3) Student Pre Test 2.2 Requested 23 May 2011, Completed 30 May 2011 - this assessment benchmarks article quality before students work on them for their class assignment
 * 4) Student Post test 2.2 Requested 10 June  2011, Completed 20 June 2011 - this assessment quantifies article improvement through student work on the project

Project Evaluation
Evaluation of the Public Policy Initiative has several layers. More information about evaluation and research for this project can be found on the Evaluation and Research page of the outreach wiki. For more context on article quality experiments, see "Experiments with article assessment" from The Signpost, 2010-09-13. For an overview of the Public Policy Initiative in general, see "Introducing the Public Policy Initiative" from the 2010-06-28 issue. If you would like to participate in project evaluation at a deeper level, please join the discussion on the outreach wiki or contact.