Writing a Literature Review

This short guide describes what goes into writing a literature review. A quick glance online will show you there is plenty of such advice, both to read and on video. Many UK universities have published online guidance to help their students write dissertations; other public and private institutions from around the world have posted guides of varying length and detail. There are so many sources of help that we really need a review of published guidance on writing a literature review to help us make sense of it all.

That, in a nutshell, is the challenge you face when writing a literature review: managing the huge amount of possibly relevant information efficiently so that you can produce a useful document within your timescale and with reasonable effort.

This guide is designed to be accessible and quick to read, and includes links to online information that you may also find useful.

  1. The literature review: what and why

2.1 What is a literature review?

A literature review is a search and evaluation of the available literature in your given subject or chosen topic area. It documents the state of the art with respect to the subject or topic you are writing about.

Royal Literary Fund

A literature review is a summary of published evidence in a specific subject area, presented in a way that is integrated, appropriate to the target readership, and organised so that the information is accessible. It is a critical analysis that identifies gaps in knowledge, appraises theories and opinion in the light of evidence, and formulates hypotheses and conclusions.

There are several things a literature review is not:

  • an uncritical summary of published work
  • a description or list of what’s been written
  • an encyclopaedia of everything that’s ever been written about a subject
  • a systematic review or a meta-analysis.

Why do it?

It’s useful to have an overview of an area of practice before embarking on a research project so that it’s clear what has already been attempted and what questions need to be answered.

A literature review will often not reveal anything new. It’s like drawing a line in time that shows how far we’ve come and explains how we’ve got there. By reading conflicting and critical reports:

  • You can learn which methodologies have worked or failed and why, and avoid repeating the mistakes of the past
  • You may identify fellow researchers who could be useful future contacts
  • You will become an expert in your chosen field
  • You will be able to refine the question that your research will attempt to answer.

  1. Planning: research question and scope

The trick to managing the large volume of information that goes into a literature review is to adopt a systematic approach. This will minimise the risk of distractions, wasted effort and inefficiency.

The first stage in organising and planning your literature review is to define the question you aim to answer and the scope of the review.

3.1 What is your question?

What is the aim of your literature review? In other words, what is the question you want to answer?

You should ensure that this precisely and succinctly describes what you want to know. The more specific you can be, the more efficient your literature search and evaluation will be. If you don’t feel up-to-speed on current thinking, carry out a preliminary overview – for example, reading several recent reviews relevant to the topic.

Your question should also be framed with the target audience in mind: different people will be interested in different areas, be that nursing practice, service organisation or clinical management.

Most importantly, don’t move on to the next step until you are clear about your goal.

EXAMPLE

Taking extended half-life factors as an example, there are a number of different research perspectives that a literature review could take:

  • What fears and expectations do parents have about extended half-life factors?

  • What are the side effects of extended half-life factors?

  • Will extended half-life factors improve quality of life compared with older treatment options?

  • Are the benefits of extended half-life factors worth the extra cost?

  • What are the implications for nurse staffing of extended half-life factors?

3.2 The scope of your review

Setting boundaries determines the scale of the task. Here, you should think about time period, languages/countries, demographic, type of study design, and the type of outcomes you want to include.

What time period will be covered?

The later limit is usually a convenient recent time point – for example, the end of the previous year or (if near) the approaching end of the year.

It’s inevitable that, sooner or later, something relevant will be reported after the end of your literature search. As your expertise grows, you will identify key sources you can monitor for new evidence.

The early limit might depend on when a technology was introduced into clinical practice, when professional practice changed, the introduction of legislation or when the last significant literature review was published.

  • It’s unusual to set a start date several decades earlier –there will normally have been a review since then so, if it’s of acceptable quality, there’s no need to duplicate it.
  • Older publications are more difficult to access (fewer are online); and health care and professional practice will have changed significantly, making older evidence less compatible with current thinking.
  • You should be aware of any publications that, though old, were of seminal importance in changing thinking; you can find them easily because they will consistently be cited in reviews.

What languages and countries of origin will you include?

It’s common to confine a review to publications in English. This is clearly a limiting factor but, due to the constraints of time and funding, it’s often unavoidable. It is mitigated by the fact that English is the universal scientific language.

Conversely, it may be feasible to include other languages when reviewing a very narrow subject, especially if practice in more than one country is relevant. For example, what qualifications are required for nurse clinical leaders in different health economies?

What sources will you be looking at?

  • Is it critical to look for every publication, no matter how rare?
  • Or is it sufficient to focus on the mainstream in easily accessible sources?

Who is involved?

What demographic will your review focus on?

  • Are you interested in people with haemophilia or parents/carers, or both?
  • Are you interested in adults or children?
  • Do you want to know about only health professionals, or are social workers and managers relevant too?

Which study designs are acceptable?

  • Will your review focus only randomised controlled trials, or will clinical trials of any design be included?
  • What about the type of study?
    • Interventional and/or observational studies?
    • Prospective and retrospective studies?
    • Quantitative or qualitative –questionnaires or focus groups?

Which outcomes are important?

  • Are you interested in patient-reported outcome measures (PROMs) or clinical events?

  1. Finding your evidence

As you carry out your literature search, note down what you find at each step. You will need to include these figures in your methods section. The list below, which follows the process of how you will find evidence to include in your review, outlines the information/figures you should note at each stage.

  1. What is your search strategy (the MeSH headings and other key words, and how you linked them)?
  2. How many articles and other sources did you identify in total?
  3. How many did you include in a longlist after screening by title?
  4. How many did you include in a shortlist after screening by abstract?
  5. How many did you ultimately included in your review after reading them in full?

4.1 Where to start

The first step is to identify relevant publications, normally using computerised biomedical databases. Some examples are:

However, depending on your research question, other sources are becoming increasingly important, notably social media and traditional print media. Evaluating the views of patients or parents/carers should include a review of online forums, for example.

It’s been estimated that the rate of scientific output in papers, books, datasets and websites is growing at the rate of 8–9% per year, meaning a doubling in volume every nine years.MEDLINE, the premier database of the US National Library of Medicine that makes up about 98% of PubMed content, contains more than 24 million references to journal articles in the biomedical sciences. Elsevier’s biomedical database, EMBASE, has over 32 million records and adds more than 6,000 new records each day. Of course, the number that is relevant to your research is small – as of January 2018, for example, there were ‘only’ 27,000 records in PubMed that were indexed with the term ‘hemophilia’.

A major challenge in conducting a literature review is to identify the publications you need to review from the enormous amount of information available. Biomedical databases can be searched relatively easily, but you need to strike a balance between selecting material that might be relevant and excluding what may not. If you are too focused on your topic, there is a risk of missing something important; if your outlook is too broad, you will be overwhelmed by the volume of literature.

For a truly comprehensive strategy, you will also need to search other sources that may be relevant to your topic (e.g. social media) [link to section below]. This may sometimes include a manual search of journals that are not accessible elsewhere (e.g. professional journals held by the medical library that are not available online). It might help to get advice from others familiar with the topic about how extensive your search needs to be.

One example of a comprehensive literature search, in this case for an evidence-based guideline on pruritus by the British Association of Dermatologists, is available here. This is a good example of how extensive a search strategy needs to be if it is to be exhaustive.

Hopefully, you will compile a longlist of publications that you can trim to a feasible reading list. There are no shortcuts to doing the reading and, with a shortlist of perhaps 30 or 40 publications not unusual, this is time-consuming.

Choosing which databases to use and searching them efficiently requires expertise. You will be able to get advice from your institution’s library about how to access them, and this will be invaluable when formulating an efficient search strategy.

4.2 Understanding PubMed

Of the databases listed above, we’ll only look at PubMed in detail. If you need to know about the other databases, click on the links above to visit the websites. The general principles of searching them are broadly similar, but the way in which articles are indexed will be different. You will need the support of your institution’s librarian or information specialist to manage this effectively.

PubMed is the internet portal through which the world can access MEDLINE free of charge. MEDLINE is the database created by the US National Library of Medicine (NLM) from 5,600 (mostly) academic journals published worldwide; though over 90% are in English, it includes about 40 languages in total. The NLM describes MEDLINE coverage as:

Biomedicine and health, broadly defined to encompass those areas of the life sciences, behavioral sciences, chemical sciences, and bioengineering needed by health professionals and others engaged in basic research and clinical care, public health, health policy development, or related educational activities. MEDLINE also covers life sciences vital to biomedical practitioners, researchers, and educators, including aspects of biology, environmental science, marine biology, plant and animal science as well as biophysics and chemistry.

A subset of PubMed content is available as PubMed Central, which contains full text articles that are free to access. It’s much smaller than PubMed, covering about 40% of the journals, but it does include book reviews.

PubMed offers short video tutorials that will teach you the basics of online searching. General videos are available at https://learn.nlm.nih.gov/rest/training-packets/T0042010P.html and https://www.nlm.nih.gov/bsd/disted/pubmedtutorial/cover.html.For a section aimed specifically at nurses, visit https://www.nlm.nih.gov/bsd/disted/nurses/cover.html.

Before moving on to the next section, it’s worth spending time on these tutorials, as what you will learn here is useful for working with other databases.

You can go to PubMed now and carry out a search with no training whatsoever. Just type a keyword in the box and you will retrieve publications with some relevance to that word. But to use this resource efficiently, it helps to understand how MEDLINE (and therefore PubMed) is constructed.

  • If you would like to find out about how to use the MEDLINE medical dictionary (Medical Subject Headings; MeSH) [LINK 1]
  • For an explanation of the Boolean operators AND, NOT and OR [LINK 2]
  • To look at how to use subheadings [LINK 3]
  • To learn about quick ways of refining your search [LINK 4]

4.3 EMBASE

EMBASE deserves special mention. It is available only via subscription, but NHS employees who are conducting research can (depending on arrangements with their local trusts) get a password to access EMBASE via the Athens gateway or through Ovid. Individuals registered for a university course will also have free access.

EMBASE is like PubMed on steroids. It includes all of MEDLINE plus another 2,000 journals. It offers better coverage of European journals (though MEDLINE has the most frequently cited titles), drugs and pharmacy journals, and conference abstracts. It has its own indexed headings (EMTREE) to which MeSH headings are automatically re-indexed. It allows searches with ‘natural’ language and, like PubMed, offers many ways of limiting and refining your search.

This powerful tool requires expertise to get the best results, and it is therefore a good idea to recruit an information specialist to help with your search. Subscribers to EMBASE have access to online tutorials and some video tutorials are available on the internet. Those from publisher Elsevier are several years old (https://www.youtube.com/user/ElsevierEmbase), but more recent videos have been posted by academic institutions.

A table summarising the similarities and differences of PubMed, Medline and EMBASE is available here. Coverage by Medline and PubMed overlaps by 80–90%, but this may not be reflected in all topics – for example, one research paper found an overlap of only 5% in a search about family medicines. The opposite is also a problem: a search of both MEDLINE and EMBASE will retrieve duplicates of many articles. Deduplication software has been developed for use within and between databases, but there is currently no reliable automated way to exclude all duplicates (though reference management software – see below – can filter out some). The Web of Science, a gateway to several databases, provides searching of several databases simultaneously, but it does not cover EMBASE.

Despite these challenges, EMBASE will probably be an essential part of your literature search.

4.4 A practical example with PubMed

In this section [LINK] we’ll go through a simple search of PubMed to explore some of the pitfalls. If you have a research question, you can use that instead of the example. Please note that the screenshot illustrations in this section are of PubMed; the database will look different and the process of entering search terms is different when you access MEDLINE using Ovid.Also note that this example was written in January 2018. When you access PubMed, you will retrieve different numbers of articles because the database is constantly being updated.

4.5 Other places

You might have noticed, if you have attended meetings where your research topic was discussed, that you found no mention of research that you know about in PubMed. This is a major limitation of PubMed in particular (because it does not cover conferences) but also of other databases. Meetings, especially smaller professional ones, are poorly covered and much of the real-world research we are familiar with goes unreported. Further, many professional journals such as the Nursing Standard and the Pharmaceutical Journal, where this type of research is published, are not covered by MEDLINE. These important sources of information should not be ignored simply because they are not available online.

The nature of your research project will determine whether it’s worthwhile searching smaller sources such as:

  • Reports of small professional meetings (sometimes distributed by pharmaceutical companies)
  • Professional journals not indexed by databases
  • Social media – Facebook groups, Twitter, Mumsnet
  • Newspapers and magazines
  • NHS sites (e.g. NHS England)
  • NICE Evidence Services
  • British Nursing Index
  • Healthcare Management Information Consortium (HMIC)
  • Allied and Complementary Medicine Database (AMED)

Some are available online via non-indexed or indexed databases, but others – the proceedings of small meetings, especially – you will only find by word of mouth, regularly reading professional journals and joining with the research community in its events.

There are apps for searching social media. For example, you can use Google’s free text search or, for greater control, uvrx social search, which allows you to select the sites you search. Limited use ofSocial Searcheris free and covers Twitter, Google+, Facebook, YouTube, Instagram, Tumblr, Reddit, Flickr, Dailymotion and Vimeo. Others, like talkwalker, are aimed at businesses.

  1. Managing your sources

5.1 Abstracts, free text and paid for

The articles you identify with your search strategy will be listed on-screen in a standard format. For PubMed, the default is ‘Summary’:

Use this format to screen out articles that are clearly not relevant to your project. If, on the other hand, you are uncertain, include them for closer review in the next stage.

As we saw in the example of the PubMed search, you can view records in several layouts (there are equivalent options in other databases):

When you have a longlist of articles selected for potential interest on the basis of their titles, you will need to scan them to select those of direct relevance. For a long longlist, this is best done with abstracts that you can read on the screen or download using reference manager software (or simply copy and paste into a word processor).

PubMed gives you a choice of download formats under the ‘Send to’ tab:

Reference management software is freely available online (e.g. Mendeley, Endnote,Refworks). These programmes format citations in a standard way and work with word processor software to make the task of writing more efficient. Your institution’s library will probably use one and will be able to help you master it. If you don’t use reference management software, the download from PubMed will be unformatted – it’s not pretty to look at but still usable:

  1. Patient Prefer Adherence. 2015 Nov 11;9:1623-30. doi: 10.2147/PPA.S92985.

eCollection 2015.

 

Understanding patient preferences and willingness to pay for hemophilia

therapies.

 

Chaugule SS(1), Hay JW(1), Young G(2).

 

Author information:

(1)Department of Clinical Pharmacy, Pharmaceutical Economics and Policy,

University of Southern California, Los Angeles, CA, USA.

(2)Hemostasis and Thrombosis Center, Children’s Hospital Los Angeles, University

of Southern California, Keck School of Medicine, Los Angeles, CA, USA.

 

BACKGROUND: Despite clearly improved clinical outcomes for prophylaxis compared

to on-demand therapy, on average only 56% of patients diagnosed with severe

hemophilia receive prophylactic factor replacement therapy in the US. Prophylaxis

rates generally drop as patients transition from childhood to adulthood, partly

due to patients becoming less adherent when they reach adulthood. Assessment of

patient preferences is important because these are likely to translate into

increased treatment satisfaction and adherence. In this study, we assessed

preferences and willingness to pay (WTP) for on-demand, prophylaxis, and longer

acting prophylaxis therapies in a sample of US hemophilia patients.

METHODS: Adult US hemophilia patients and caregivers (N=79) completed a

discrete-choice survey that presented a series of trade-off questions, each

including a pair of hypothetical treatment profiles. Using a mixed logit model

for analysis, we compared the relative importance of five treatment

characteristics: 1) out-of-pocket treatment costs (paid by patients), 2) factor

dose adjustment, 3) treatment side effects, 4) availability of premixed factor,

and 5) treatment effectiveness and dosing frequency. Based on these attribute

estimates, we calculated patients’ WTP.

RESULTS: Out-of-pocket treatment costs (P<0.001), side effects (P<0.001), and

treatment effectiveness and dosing frequency (P<0.001) were found to be

statistically significant in the model. Patients were willing to pay US $410 (95%

confidence interval: $164-$656) out of pocket per month for thrice-weekly

prophylaxis therapy compared to on-demand therapy and $360 (95% confidence

interval: $145-$575) for a switch from thrice-weekly to once-weekly prophylaxis

therapy.

CONCLUSION: Improvements in treatment effectiveness and dosing frequency,

treatment side effects, and out-of-pocket costs per month were the greatest

determinants of hemophilia treatment choice and WTP. The positive preferences and

WTP for longer acting prophylactic therapies suggest that the uptake is likely to

increase adherence, improving treatment outcomes. These preferences should also

inform the Food and Drug Administration’s assessment of new longer acting

hemophilia therapies.

 

DOI: 10.2147/PPA.S92985

PMCID: PMC4646600

PMID: 26635471

Remember that an abstract is, like the entire article, the copyright of the publisher and cannot be reproduced without permission. It is common practice to download abstracts for personal research use. (The abstract above is from a free text article published by Dove Medical Press, which allows non-commercial reproduction.)

When you at last have a shortlist, you will need to obtain full copies of the papers. A few, as in the example above, are available free online and you simply follow the links. Most articles have to be paid for, either individually by credit card or via the subscription fees of an institutional account holder such as an NHS trust or university library. You then have permission to download a pdf of the article (sometimes for a limited period such as 24 hours) for personal and research purposes.

5.2 Managing volume

Organise the publications you have selected for review so that you can easily find the information and data you need as you write. There are no shortcuts: you must read the information you have retrieved. As you do so, make a note of the most important points of each article, and remember that published research is not always right and never wrong. You can use reference management software to annotate the record for each article as you read it, which will also provide a virtual way to organise your documents systematically. (Such software can also format the references in a variety of styles to match the requirements of different journals.)

Alternatively, you can create your own table. For example, this table [LINK 5] summarises publications about parents’ experiences of transition. It lists the citation (by author, with a numbered reference), the client group, study design, which outcomes were measured and what the main findings were. The process of creating such a table helps you to learn about the topic and is an invaluable tool when writing.

The critical appraisal of methodology is an important aspect of a literature review, so study design is an obvious and useful criterion to start with. It creates several smaller subgroups that are easier to manage than the whole collection of papers.

Categories to consider are:

  • Reviews (including systematic reviews and meta-analyses)
  • Clinical trials
  • Observational studies
  • Qualitative studies (e.g. focus groups)
  • Commentary (e.g. editorials, letters)

Reviews come first because they will provide an overview of published work and a benchmark for your project. Remember that the data in these reviews will include some of the clinical trials and observational studies that you have identified, so be aware of the risk of duplication.

  1. At last you can start writing!

You may like to read the notes on effective writing for nurses who are starting their writing careers that Haemnet has published in its Toolbox.

6.1 Preparation

Decide what the structure of your review will be, give every section a heading, and add to each a shortlist of the items they will include. There’s usually little flexibility about structure: it’s basically the same for all biomedical journals.

This structure, recommended by Imperial College for its students, is a good model (there are many more online):

How you organise the text within the main body of your review depends on the topic and how it can be best presented.

The standard format is methods ˃results ˃discussion. You don’t have to write the introduction before the main body – it might be easier to do it afterwards, when you know what you are going to say. But definitely leave the discussion until you’ve completed the methodology and results sections, because you will then be familiar with your findings.

This approach may not suit a review, especially a qualitative one. Instead, you may prefer to identify key themes and address each in turn.

Regardless of the approach you take, establish the structure of your review at the outset so that you avoid having to rewrite paragraphs as you find yourself adapting to an evolving layout.

When you have your structure, organise all the notes and source material you will need for each part. Formulate a writing plan that sets out a realistic timetable. If you fall behind, revise the plan to take this into account and set more achievable deadlines.

6.2 Support

Only one person can use one keyboard at any one time, but writing need not be a solitary occupation and the results are arguably better when it is collaborative.

Co-authors can share the workload, help to solve problems, exchange ideas, inspire each other and supportively appraise one another’s work.

Whether you are writing solo or as part of a team, consider asking more experienced colleagues for advice if you encounter difficulties. It’s normal to struggle with at least one part of a review but there should always be someone to call on for support – perhaps a member of your clinical team or research group. If you are the sole author, ask a colleague if they’ll act as a sounding board to help you overcome these obstacles.

6.3 The first draft

It is a major feat to finish the first draft –most of the work is now done. The bits that remain are, however, critically important to getting your work from your desk to the reader. It’s useful at this stage to bear some general points in mind:

  • A first draft is a start; it’s not pristine or untouchable. Drafts get changed and they are better for it.
  • Check the draft for errors – repetition, grammar, spelling mistakes, typos, misquoted references.
  • Ask colleagues and co-authors to read the draft, discuss their feedback with them and amend the draft accordingly. This is the stage when big changes are easiest to make. Recognising the need for structural changes can be disheartening, but better to do it now than to have the journal reject your submission because a referee criticises it.
  • Go over the first draft until you are confident you have done all you can, taking into account the support and feedback you’ve received. You will get weary of it, and over-familiarity will mean you will begin to miss errors, so ask a colleague to proofread it.

By now you will have an idea which journal you would like to publish your review. Download the guidelines for contributing authors from the journal website and format your draft to adhere to them exactly. As an example, look at the manuscript submission guide of The Journal of Haemophilia Practice.

Journals vary in their requirements, but they usually specify what kind of submissions will be accepted, the structure of the submitted article, its length, how authors must be listed (and who constitutes an author), how to cite references in the text and at the end (even down to where a full stop should be), and how to present tables and figures. If you don’t follow these instructions your draft will be returned for rewriting.

  1. REVIEW

7.1 Feedback and criticism

Norman Vincent Peale, author of The Power of Positive Thinking(1952), is reputed to have said:

“The trouble with most of us is that we’d rather be ruined by praise than saved by criticism.”

If you’re a book enthusiast, you’ll know that publishers often reject early drafts and that editors work with authors to refine their writing. In the world of movies, it’s not uncommon for the screenwriter to be on set and rewrite scenes on a daily basis. Professional writers routinely allow time for one or two redrafts. In short, revision and rewriting are the norm when writing–and the scientific world is no different.

It’s a truism that if you ask someone what they think, they’ll tell you. What they tell you will hopefully be helpful and supportive, but it may be challenging. You will come across two types of feedback: someone will raise an issue that hadn’t occurred to you or you’ve missed, or they’ll disagree with your interpretation of the evidence or question your conduct of the literature review. (And everyone will correct your grammar.)

There is no question that you should correct omissions and oversights. A literature review is a big task and it’s no surprise that things sometimes go astray or fall under the radar. The draft will always be better as a result.

Conversely, disagreements and contradictions may or may not improve a draft. The difficulty facing a writer is to control the sensitivity they naturally have when their hard work is criticised. It’s easy to say “Don’t take this personally” when you’re not the one who’s invested so much time at the computer, but – provided criticism is presented in a constructive way – try not to take it personally. You should find out what evidence underlies critical feedback. If you can, discuss feedback with the person who gave it. If the comments have any merit you should act on them. If they don’t, and you’re clear why your view is the right one, ignore them. Do not, however much you feel wronged, fire off a blistering email.

There’s plenty of advice online about dealing with criticism, including many examples of simple rules to bear in mind.

7.2 Referees’ comments

Peer review is an essential part of medical publishing. It helps to ensure the accuracy of published research and assesses the findings in the context of current knowledge. There’s an interesting description of peer review here. (Note that this article about peer review has itself been reviewed by a peer reviewer.) The procedure a journal follows is usually available on its website (see Nature journals as an example; this site includes links to advice on dealing with referees’ comments).

Peer review is carried out anonymously by two or three referees who are usually members of a journal’s editorial panel, eminent researchers in the relevant field of practice, or individuals who have published work similar to the project being submitted. Their comments are provided to the journal editors and they may (depending on the journal’s procedures) recommend acceptance, acceptance (usually) with amendments, or rejection. Editors will normally share the referees’ comments with the author, who can then rewrite accordingly.

A submission may be rejected because the referees or editors consider it does not match the readership’s interests; because it’s too technical or not technical enough; it’s not relevant to the core readership or more relevant to another journal; a similar article may have been published recently, or is soon to appear; or perhaps they believe the standard isn’t up to what they require. This is an opportunity to revise the draft in light of the referees’ comments and submit the review to another journal.

  1. SUMMARY

Writing a literature review is a major undertaking best done collaboratively. It calls for long-term commitment and effort, but it is straightforward when managed efficiently. The rewards are correspondingly great. The process of researching and reading will equip you with skills in information management and critical appraisal. Composing and reviewing your work will improve your communication skills in both the written and spoken word. As your knowledge of the subject grows, you will form networks and share ideas with like-minded researchers. Ultimately, the patients and families who use your service will benefit from the way you have changed clinical practice and from the improvement in personal performance that comes with greater knowledge and experience.