Skip to Main Content

Systematic Reviews

What is grey / gray literature?

Grey literature is “information produced  by all levels of government, academics, business and industry in electronic and print formats not controlled by commercial publishing”. (GreyNet)

Grey literature may include, but are not limited to the following types of materials: reports (pre-prints, preliminary progress and advanced reports, technical reports, statistical reports, memoranda, state-of-the art reports, market research reports, etc.), theses, conference proceedings, technical specifications and standards, non-commercial translations, bibliographies, technical and commercial documentation, and official documents not published commercially (primarily government reports and documents) (Alberani, 1990).

It is important to search for unpublished or grey literature in systematic review as it helps to reduce publication bias and increase the comprehensiveness of the review to provide a balanced view of the available evidence (Paez, 2017).  

Due to selective publication of clinical research, negative studies are less likely to be published.  A review can be biased when it fails to report crucial information that may be hidden in some grey literature.   This may threaten the validity of systematic reviews or meta-analyses to inform clinical and policy decisions based on overly optimistic estimates of treatment effectiveness. “Underreporting of negative results introduces bias into meta-analysis, which consequently misinforms researchers, doctors and policymakers.” (Mlinarić, A)

Alberani, V., De Castro Pietrangeli, P., & Mazza, A. M. (1990). The use of grey literature in health sciences: a preliminary survey. Bulletin of the Medical Library Association78(4), 358–363.

Paez, A. (2017). Grey literature: An important resource in systematic reviews. J Evid Based Med. https://doi.org/10.1111/jebm.12265

Mlinarić, A., Horvat, M., & Šupak Smolčić, V. (2017). Dealing with the positive publication bias: Why you should really publish your negative results. Biochemia medica, 27(3), 030201-030201. https://doi.org/10.11613/BM.2017.030201

Grey literature resources

One useful checklist for evaluating grey literature is using AACODS (Authority, Accuracy, Coverage, Objectivity, Date and Significance)

Authority - Is the author credible?

Accuracy -  Is it supported by documented and authoritative references? Is there a clearly stated methodology? Is it in line with other work on the same topic

Coverage - Have limitations been imposed and are these stated clearly?

Objectivity - Can bias be detected?

Date - Can't find the date? Rule of the thumb is to avoid such material

Significance - Is it relevant? Would it enrich or have an impact on your research?