Wikipedia:Identifying reliable sources (science)


 * See also: Identifying reliable sources, Wikipedia:No original research, and Identifying reliable sources (medicine)

Wikipedia's science articles are not intended to provide formal instruction, but they are nonetheless an important and widely used resource. Scientific information should be based on reliable published sources and should accurately reflect the current state of knowledge. Ideal sources for these articles include comprehensive reviews in independent, reliable published sources, such as reputable scientific journals, statements and reports from reputable expert bodies, widely recognized standard textbooks and handbooks written by experts in a field, expert-curated databases and reference material, or high-quality non-specialist publications. Although news reports are inappropriate as reliable sources for the technical aspects of scientific results or theories, they may be useful when discussing non-technical context or impact of science topics, particularly controversial ones.

The scope of this page includes the natural, social and formal sciences. For articles about medicine, see Reliable sources (medicine-related articles). For queries about the reliability of specific sources for a given purpose, use the reliable sources noticeboard or the talk page of a relevant WikiProject.

Definitions

 * A primary source in science is one where the authors directly participated in the research. They filled the test tubes, analyzed the data, or designed the particle accelerator, or at least supervised those who did. Many, but not all, journal articles are primary sources—particularly original research articles. An appropriate primary source is one that was peer reviewed and published by a reputable publisher.
 * A secondary source is a source presenting and placing in context information originally reported by different authors. These include literature reviews, systematic review articles, topical monographs, specialist textbooks, handbooks, and white papers by major scientific associations. News reports are also secondary sources, but should be used with caution as they are seldom written by persons with disciplinary expertise. An appropriate secondary source is one that is published by a reputable publisher, is written by one or more experts in the field, and is peer reviewed. University presses and other publishing houses known for publishing reliable science books will document their review process. Do not confuse a scientific review (the article/document) with peer review (the activity).
 * A tertiary source usually summarizes a range of secondary sources. Encyclopedias, general textbooks, popular science books, and tables of values are tertiary sources.

Respect secondary sources
In general, scientific information in Wikipedia articles should be based on published, reliable secondary sources, or on widely cited tertiary and primary sources. Sources that are robust in methodology, published in high quality venues, and authored by widely cited researchers are preferred. Especially for surprising or extraordinary results, the description should adhere closely to the interpretation of the data given by the authors or by reliable secondary sources (see No original research).

Primary sources may be used when discussing recent research directions or a particular result. When citing a primary source, be especially mindful of the policy on undue weight, as primary sources are more prone to misuse than secondary or tertiary sources. An individual primary source should never be cited or juxtaposed so as to "debunk" or contradict the conclusions of a reliable secondary source, unless the primary source itself directly makes such a claim (see Wikipedia:Synthesis of published material that advances a position). Primary sources favoring a minority opinion should not be aggregated or presented devoid of context in such a way as to undermine proportionate representation of expert opinion in a field.

If a reliable and comprehensive review article cites a study, result, or idea, the review should usually be cited in preference to the primary source. If a primary source is cited by few or no reliable sources outside the originating lab, the primary source may be removed as not reporting an important result. Wikipedia does not apply any special emphasis to breaking news, but seeks an overall survey of the literature as it has been synthesized by the experts in a field.

Tertiary sources can provide a valuable overview of a topic, but often oversimplify complex material. It is usually better to cite the secondary or primary literature directly.

Although popular-press news articles and press releases may tout the latest experiments, they often exaggerate or speak of "revolutionary" results where the researchers refer to the context of the gradual progress of the field. Including an accessibility link to such a source may aid in reader comprehension, but the language of the actual study should be used; more detailed and less sensational lay sources are preferred.

In all cases, the reliability and relevance of a work is determined by other researchers in the relevant field. Using high-quality sources ensures that our articles reflect the current state of knowledge and proportionately represent the aspects and controversies considered most important by the experts in a field.

Respect primary sources
A primary source, such as a report of a pivotal experiment cited as evidence for a hypothesis, may be a valuable component of an article. A good article may appropriately cite primary, secondary, and tertiary sources. Use of primary sources should always conform to the No original research policy.

However, primary sources describing genetic or genomic research into human ancestry, ancient populations, ethnicity, race, and the like, should not be used to generate content about those subjects, which are controversial. High quality secondary sources as described above should be used instead. Genetic studies of human anatomy or phenotypes like intelligence should be sourced per WP:MEDRS.

Summarize scientific consensus
The prevailing scientific consensus should be presented as the dominant view and articles should be framed accordingly. Scientific consensus can be found in e.g. recent, authoritative review articles, high quality journal articles, or widely used postsecondary textbooks. Significant minority views should be accorded due weight and presented in the context of their acceptance by experts in the field. If mainstream secondary sources in a field do not consider a detail or opinion relevant, it may not be appropriate to cover it at that article; such details and opinions may be desirable at an article on a sub-topic or at a separate article, with linking governed by WP:SPINOUT and WP:ONEWAY.

The fact that a statement is published in a refereed journal does not make it true. Even a well-designed experiment or study can produce flawed results or fall victim to deliberate fraud. (See the Retracted article on neurotoxicity of ecstasy and the Schön affair). There is an informal hierarchy of journals, abetted by the publish or perish culture of academia. Preference should be given to citing articles in top tier journals wherever possible. Similarly, if you find dubious unreferenced or poorly referenced text in an article, your first question should be does including this material add to the full and accurate summary of the topic rather than can I track down a source somewhere that supports this.

The fact that a statement is published in a refereed journal does not make it relevant. Many ideas are proposed and disregarded in the context of scientific discourse. If an idea is cited by a small minority of researchers, but rejected or ignored by the majority of researchers in a field, it should receive limited weight according to its acceptance; ideas held by a tiny minority of researchers need not be reported in our articles, except in articles devoted to these ideas. Very new papers should be used sparingly until enough time has passed to make this assessment - there is no deadline. Additionally, material that is appropriate for a highly focused article on one specific part of a field may not be appropriate for a higher level article about the field as a whole.

Make readers aware of legitimate uncertainty or controversy within the particular field of study. A well-referenced article will point to specific journal articles or specific theories proposed by specific researchers. Wikipedia neither accepts nor rejects any particular position - describe any disputes and their place in the scientific discourse, but do not engage in them. Many values, such as the masses of transuranian elements or the isotopic composition of the solar system, have an associated uncertainty, and even up-to-date highly reliable sources may report slightly different values. Where there is no clear reason to report solely one of several values, discussion on the article's talkpage or the appropriate Wikiproject can help determine which value(s) to use. For values or classes of values affecting many articles, consistency across articles and Wikiproject-level discussion should be preferred.

Political, social, and historical context and impact and public perceptions are important when deciding whether to cover an idea at an article, but should not be considered when assessing scientific consensus.

Assess evidence quality
Editors should be careful to avoid engaging in original research, but the quality of available evidence should be kept in mind when assessing whether a particular idea or viewpoint is well-accepted by the relevant academic community. Such evidence should include reviews of the literature including the work of several different research groups. Individual papers often disagree with each other, but there are several indicators that may be assessed even without specialist knowledge to differentiate high quality papers from low, including:


 * The paper has been appropriately reviewed through formal or informal peer review. Any serious scientific journal is formally peer-reviewed, though white and gray literature may be less transparent in their review methodology.
 * Experimental and mathematical methods are clearly explained and are appropriate to the experiment.
 * Model fitting and statistical analysis are meaningful and appropriate.
 * Uncertainty and the paper's place in the wider scientific discourse are acknowledged.
 * Funding sources and any potential conflicts of interest are disclosed.
 * The authors and the paper itself are widely cited by other researchers in the paper's field. In most scientific fields, the order of the author list usually indicates importance of each researcher's contribution to the article, except that the final author is commonly the senior researcher in charge of the laboratory or research group where the work was done. These conventions may vary by field, journal, and paper.
 * Recognized experts in the field have commented or offered informal opinion.

Cutting edge science is built on the foundation of previous research, and paradigms almost always change only slowly. Preliminary results, whether reported in the popular press, a conference abstract, or a peer-reviewed journal, are a form of anecdote and generally fall below the minimum requirements of reliable science sources. Exceptional or surprising claims should not be presented as authoritative, nor should the description of a broad consensus view be presented as less well-founded until such exceptional claims are replicated or widely cited. Be careful of material in a journal that is not peer-reviewed, especially if reporting material in a different field (see Marty Rimm and the Sokal affair).

Speculative proposals and early-stage research should not be cited in ways that suggest wide acceptance. For example, ideas and results that have been reported only in conference proceedings or on a researcher's website are unlikely to be appropriate for inclusion except when reported as such in the author's biography. A secondary source reporting on preliminary results might be appropriate as part of a well-documented section on research directions in a field. To prevent misunderstandings, the text should clearly identify the level of research cited. If a result does not accurately indicate its place in the scientific discourse, it is unlikely to be reliable.

For example, every year, people propose modifications to general relativity or publish results that call some aspect of the theory into question. Usually these ideas are proposed by serious researchers who pose a question as part of an endeavor to understand the results more deeply: how can these results be understood in terms of the theory they seem to contradict? Such nuances are often missed in popular press reports, but should be included in articles if the proposed modification is cited. Sometimes "revolutionary" ideas are proposed by cranks or are otherwise ignored by researchers; such ideas should be presented only in the context of the broader field and only in articles devoted to the proponent(s) or specific to the idea. Until a significant fraction of the astrophysics community indicates doubt as to the general validity of the theory, the articles treating general relativity should not imply any such doubt.

Use up-to-date evidence
While articles should be kept up to date by citing current literature, care should be taken to avoid recentism, focusing too much on new sources that have not yet been evaluated by the relevant community.

Here are some rules of thumb for keeping an article up-to-date while maintaining the more important goal of reliably reflecting the current state of a field of research. These guidelines are appropriate for actively-researched areas with many primary sources and several reviews, and they may need to be relaxed for mature fields or in areas where little progress is being made and few reviews are being published.


 * Look for reviews published in the last five years or so, preferably in the last two or three years. The range of reviews examined should be wide enough to catch at least one full review cycle, containing newer reviews written and published in the light of older ones and of more-recent primary studies.
 * Within this range, things can be tricky. Although the most-recent reviews include later research results, do not automatically give more weight to the review that happens to have been published most recently. The prominence of the publishing journal, the quality and comprehensiveness of the review, and the respectability of the authors should also be taken into account.
 * Prefer recent reviews to older primary sources on the same topic. If recent reviews do not mention an older primary source or result, the older source is dubious. For example, the articles superconductivity and List of superconductors might mention the hot-off-the-presses latest material or model found to undergo the transition, but such observations should be treated as tentative until confirmed by another research group or affirmed by a broad review of the field. More detail should be devoted to discussion supported by recent reviews.

These are just rules of thumb. There are exceptions:


 * History sections often cite older work, for obvious reasons.
 * An older primary source that is seminal, replicated, and often-cited in reviews is notable in its own right and can be mentioned in the main text in a context established by reviews.
 * Consider scope and focus: articles on broader topics and more mature fields should contain less primary research than articles on narrow, actively researched topics.
 * Editors should be especially leery of citing papers making exceptional claims until the relevant community has evaluated the evidence. If a result is cited only by the research group originating the claim and ignored by the rest of the field, it should probably not be included even if present in a review authored by the group. Blogs by relevant subject matter experts may be useful in talk page evaluation of the relevance of very new results, though they should rarely be cited themselves (see below).
 * Sometimes scientific results have or are taken to have political or social relevance. Wikipedia articles should avoid sensationalism, and should follow the relevant research community in according weight to such results. Reporting on political and social impacts and controversies is often done in separate article sections, and sometimes separate articles. Sourcing for political and social aspects and controversies is beyond the scope of this guideline, but is governed by the reliable sources content guideline.

Use independent sources
Many scientific claims lack independent replication or confirmation of the legitimacy of statements made by proponents. In such cases, reliable sources may be much more difficult to find and unreliable sources can often be more readily available. Especially when writing about ideas not supported by or contradicted by mainstream research, it is vital that third-party, independent sources be used. Sources written and reviewed by the advocates of such marginal ideas can be used to describe notable personal opinions, but extreme care should be taken when using such sources lest the more controversial aspects of their opinions be taken at face value or, worse, asserted as fact. If the only independent sources discussing a subject are of low quality, then it is likely that the subject itself is not notable enough for inclusion. For example, coverage of individual perpetual motion machines should focus on their importance to the creator's biography (if notable) or actual impact (did a large company invest in the inventor? did an eminent scientist comment on the device?) rather than a detailed recapitulation of the supposed principles involved.

Choosing sources
No source is universally reliable. Each source must be carefully weighed in the context of an article to judge whether it is reliable for the statement being made and is the best such source.

Scientific journals
Articles published in respected peer-reviewed scientific journals are preferred for up-to-date reliable information. Scientific literature contains two major types of sources: primary publications that describe novel research for the first time, and review articles that summarize and integrate a topic of research into an overall view. Journals generally publish a mix of primary and secondary sources, though some may concentrate on particular types. The line between primary and secondary sources is not always clear. In general, primary sources include descriptions of an individual experiment or a series of experiments by the same research group; secondary sources include independent review articles summarizing a line of research or rectifying apparently discordant results. It is usually best to use review articles where possible, as these give a more balanced and general perspective of a topic, and can be easier to understand.

Many journals serve their community by also publishing less technical material such as biographies and obituaries. Although almost all such material will count as a reliable source, not all the material is equally useful. Journal articles come in many types, including: original research, reviews, expert summaries, news, editorials, advocacy pieces, speculation, book reviews, correspondence, biographies, and eulogies. Original research papers are primary sources; although they normally contain a review of previous works that functions as a secondary source, these sections are typically less reliable and comprehensive than reviews. A general narrative review of a subject by an expert in the field makes a good secondary source that can be used to cover various aspects of a subject within a Wikipedia article. Such reviews typically contain no original data but can make interpretations and draw conclusions from primary sources that no Wikipedia editor would be allowed to do. A systematic review uses a reproducible methodology to select primary studies meeting explicit criteria in order to answer a specific question. Such reviews should be more reliable, accurate and less prone to bias than a narrative review. However, systematic reviews focus on answering one or a few specific questions, so that complementing with other sources may be necessary to more broadly cover a topic.

Core basic science journals include such publications as Science, Nature, and subject-specific journals published by professional associations. A listing of academic journals can be found in Category:Academic journals and its subcategories.

Books
When using a book as a source, books should be chosen that are up-to-date and published by experts in the field. Postsecondary science textbooks published by academic publishers are often excellent secondary sources, though they may need to be supplemented with more recent research. If a book has students as its declared target audience, it may not be as complete as a monograph or chapter in a book intended for professionals or postgraduates. Major academic publishers and university presses publish specialized book series with good editorial oversight. Volumes in these series summarize the latest research in narrow areas usually in a more extensive format than journal reviews. Specialized encyclopedias published by such established publishers are often of good quality, but may be too terse for detailed articles. Some monographs may overemphasize the importance of the researchers or laboratory groups who authored them, without fully reflecting the views of other experts. If monographs are used as sources, they should therefore be accorded appropriate weight and checked against prevailing viewpoints in the relevant field.

Popular science books can be useful tertiary sources, though information may be oversimplified or lacking in nuance or the full range of opinion in a field may not be adequately represented. Even in such cases, it may be useful to seek them out as an example of the material being presented in a fashion accessible to non-scientists.

Most books and monographs that are self-published or published by vanity presses undergo no independent fact-checking or peer review and consequently are not reliable sources.

White and grey literature
Many organizations research, produce, and publish white papers and grey papers discussing or summarizing various aspects of a field. These papers are typically not peer reviewed in the traditional sense, but may nonetheless provide accurate and accessible information. When assessing the suitability of such a source, consider the reputation of the publishing organization, the reliability and proper use of the sources cited, and how the source is in turn cited or discussed by the relevant academic community.

The various national societies, such as the Royal Society, the American Physical Society, or the Royal Australian Chemical Institute, occasionally produce formal scientific reports, which can be as reliable as the best traditional journal papers. Public guides and service announcements have the advantage of being freely readable, but are generally less authoritative than the underlying literature. Such organizations often contain working groups and subcommittees, which cannot be presumed to speak for the society as a whole.

Government agencies and non-governmental organizations often produce reports that are internally vetted and reviewed. When using such a report as a source, consider the purpose of the organization, its reputation in the desired context, and the reception of the specific report.

Advocacy organizations formed for a specific purpose or to advance a cause may be composed of scientists and mimic the structure and naming conventions of the general purpose societies. Statements and reports from such organizations are not reliable except to cite the organization's opinion or position. If such statements are necessary to the coverage of a topic, they should be attributed and the role of the organization made clear.

Popular press
The popular press is readily accessible and can contain valuable supplemental information of a social, biographical, current-affairs, or historical nature. However, news articles should be used with caution when describing scientific results, studies, or hypotheses. Science news articles may fail to discuss important issues such as the uncertainty range of a conclusion, how a result has been received by experts in the field, the context of related results and theories, and barriers to widespread adoption or realization of an idea.

Articles in newspapers and popular magazines generally lack the context needed to judge experimental results. Be particularly wary of any result reported as surprising or revolutionary, which may be an indication of exaggeration or worse. Popular press articles tend to overemphasize the certainty and importance of any result, for instance presenting a new theory as overturning previous knowledge or a new technology as just around the corner. Newspapers and magazines may also publish articles about scientific results before those results have been published in a peer-reviewed journal or reproduced by other experimenters. Such articles may rely uncritically on a press release, which can be a biased source even when issued by the public relations department of a university or national laboratory. News articles also tend neither to report adequately on the scientific methodology and the experimental error, nor to express risk or uncertainty in meaningful terms.

A news article should therefore not be used as a sole source for a scientific fact or figure, nor should they be considered when describing what aspects of a field the relevant experts consider interesting, surprising, or controversial. Editors are encouraged to seek out the scholarly research behind the news story; good quality science news articles will indicate their sources. One possibility is to cite a higher-quality source along with a more-accessible popular source, for example with the laysummary parameter of Cite journal.

On the other hand, the high-quality popular press can be a good resource for presenting science to a non-technical audience, and often as a source in its own right to supplement (but not supplant) the peer-reviewed literature. For example, while popular science magazines such as Scientific American, Discover, and Popular Science are not peer-reviewed, they sometimes feature articles written by experts that explain scientific subjects in plain English. As the quality of press coverage of science ranges from excellent to irresponsible, use common sense, and see how well the source fits the verifiability policy and the general reliable sources guideline.

Curated databases
Some scientific databases can be used as sources in their own right. Such databases need to have evidence of being A) manually curated/reviewed, i.e. not fully automated; B) by more than one expert, i.e. not a pet project of a single individual; and C) well-established, i.e. cited by others. This is separate from whether inclusion in such a database is sufficient to support notability.

Other sources
Press releases, blogs, newsletters, advocacy and self-help publications, and other sources offer a broad spectrum of scientific information ranging from factual to fraudulent, with a high percentage being of low quality. As much as possible Wikipedia articles should cite the literature directly, and editors should bear in mind that a particular source may introduce a spin not present in the original paper or present a result not supported by the research. Conference abstracts are often incomplete and preliminary, and may be contradicted if and when the data are published; they should be avoided. Patents and patent applications likewise do not receive the critical review necessary for reliability in this context, and should be avoided except when the patent itself is under discussion; the United States Patent and Trademark Office has granted perpetual motion patents as recently as this decade. Personal or group blogs from prominent scientists writing in their field of expertise may be usable when properly attributed. Nature Blogs, ScienceBlogs, and Discover blogs host many such experts, as do more specific portals such as the public outreach and service blogs at the Large Hadron Collider blogs or the more STEM policy oriented blog hosted by the American Physical Society.

Searching for sources
Search engines and academic databases are often used to find sources. When searching for sources, it is wise to skim-read everything available (including abstracts of papers you cannot fully access) to get a feel for expert opinion on the most important aspects of a topic. Each system has quirks, advantages, and disadvantages. It typically takes experience to recognize when a search has not been effective; even if you find useful sources, you may have missed other sources that would have been more useful, or you may find large amounts of less-than-useful material. A good strategy for avoiding sole reliance on search engines is to find a few recent high-quality sources and follow the citations backwards and forwards to see what your search engine may have missed. Limiting a general search using the key words (usually listed under a paper's abstract), or using a semantic search engine may help focus results to the relevant topic. Some resources, such as Google Scholar and Physical Review, also list the papers citing a particular paper; these results may not be comprehensive, especially tending to miss citations that are not well-formed, but the results can be useful both in finding additional sources and as a rough metric of the impact of a particular paper on the field in general. It can also be helpful to perform a plain web search rather than one of scholarly articles only.


 * Other useful search engines include:


 * Web of Science
 * InfoTrac
 * Scopus
 * PubMed
 * Google Scholar
 * Google Books often offers readers a few sentences even when full access is not granted, and can help editors find reliable sources quickly, either by looking at the book's references or by citing the book itself. Check that a particular book is published by a reliable academic publishing house.
 * arXiv is a preprint server; near-final versions of many physics and astronomy papers may be read freely, but these papers have not yet undergone peer review, and any citation should be checked against the final version.
 * Astrophysics Data System covers astronomy and physics papers.
 * University librarians are often aware of specialized resources, and can be exceedingly helpful when approached in a friendly and open fashion.
 * Journals occasionally devote all or most of an issue to a particular topic or sub-field. Such issues can provide a valuable snapshot of the current state and research directions of a field.

Approaching the problem from the other end, many large research organizations and funding agencies publish research highlights. These summaries can be helpful in recognizing the most important result of a piece of research or in ascertaining current research directions, though press releases should generally not be used directly.

Sometimes a paper or series of papers will be summarized by an expert in the field, usually in a research journal with a target audience of other researchers in the field. Such articles provide context for the impact of a result or relative importance of a line of research. If you have access to both the original source(s) and the summary and you find the summary helpful, it is good practice to cite both sources together (see Formatting citations for details).

Accessibility
Try to avoid citing a source having read only its abstract, as the abstract necessarily presents a stripped-down version of the conclusions and omits the background that can be crucial for understanding what the source says. You may need to visit a university library to access the full text, or ask somebody at the WikiProject Resource Exchange or at a relevant WikiProject either to provide you with a copy or to read the source for you and summarize what it says. If neither is possible you may need to settle for using a lower-impact source or even just an abstract, with an eye to updating or replacing the text when a better or more complete source becomes available.

The requirement for a fee or a subscription does not affect the reliability of a source. However, when all else is equal it is preferable to cite a source whose full text is freely readable so that readers can more easily follow the link to the source and editors can double check the content. Journals more likely to be available at a reader's local university library should also be preferred. Although most high-quality journals require a payment or subscription for access, some, such as Proceedings of the National Academy of Sciences and similarly all National Research Council reports are freely-available. Others, such as Physical Review, publish a few freely-readable articles even though most are not free; still others use delayed open access.

There is a growing movement towards allowing the public "open access" to scientific research, particularly since much of the research is publicly-funded. Even for journals where there is no open access, the vast majority of journals allow for self-archiving of either preprints or postprints. Google Scholar can often aid in finding pre-and-postprints. Editors should always cite to the version which they actually read; if the editor can only access the preprint of a published paper, the preprint can be cited (with reliability similar to grey literature) with the citation to be eventually replaced with the final version later by someone who has it available double-checks.

Formatting citations
A citation should document precisely how to access a source. Normally, citations should contain a digital object identifier (DOI) if available. A common practice is to supply a URL to a source if and only if full text is freely readable. Check that the URL given does not depend on a cookie on your machine or IP-based subscription access. Some journals offer free access for only a limited period after publication, so check for linkrot when updating references. WP:CHECKLINKS semi-automates this process. If the Cite journal template is used, all this information can be supplied with the doi, and url parameters, respectively. If you are citing a source along with an expert summary, it is helpful to list them together, with the main source first to indicate that it is more authoritative. For example:


 * Summary:

If a source is available in both HTML and some other form, normally the HTML form should be linked, as it is more likely to work on a wider variety of browsers. If the full text of a source is found in a location other than at the publisher's website, check that the copy does not violate copyright before linking it and be aware that the text may have been altered from the original version.