Commissioning an evidence review for the Portal

Purpose of an evidence review

The key purpose of an evidence review is to provide a summary of the best available evidence to support decision making. You might commission an evidence review if:

you need an independent and rigorous synthesis of the evidence base
you need to know ‘what works’, or if a particular intervention or program is effective and under what circumstances.

Evidence reviews also identify knowledge gaps where further research might be needed.

Where do I start?

1. Contact the Evidence Portal team

If you want your evidence review to be included in the Evidence Portal, first, contact the Evidence Portal team. We can provide you with guidance to ensure the review meets the requirements for the portal. Your evidence review can contribute to the body of high-quality evidence the human services sector can use to design, commission and deliver evidence-informed services for clients and communities.

Send initial enquiries to the Evidence Portal mailbox EvidencePortal@dcj.nsw.gov.au. We can provide feedback on:

the feasibility of the evidence review
likely costs
how this fits with other planned reviews
how to develop your research question.

2. Review the Evidence Portal Technical Specifications

All evidence reviews included on the portal are conducted using the Evidence Portal Technical Specifications. These describe the method researchers must follow to conduct an evidence review. They provide guidance, explanations and example to ensure the process is applied consistently.

3. Engage a specialist researcher or organisation

Particular technical skills, competencies and experience are required to use the Evidence Portal Technical Specifications. You will need to procure these skills from a specialist researcher or organisation. Further information on where you might find a suitable researcher or organisation is provided below.

4. Plan your evidence review and clearly define your research question

A good evidence review requires careful planning and execution. This starts from identifying the research topic and clearly defining the research question, which usually is refined with the help of the specialist researcher or team conducting the review.

It is recommended to draw up a project management plan to identify the activities and milestones, decisions needed and to anticipate and mitigate any problems or risks.

5. Consider timing

The length of time an evidence review takes depends on the complexity of the research question, the skills of the team involved and the body of evidence they are dealing with. You will also need to factor stakeholder engagement into the project timelines. A rough approximation of the time needed to conduct each step in the evidence review is provided in section 1.6 of the Evidence Portal Technical Specifications.

6. Consider budget

Total costs for conducting an evidence review can vary greatly and are mostly related to the complexity of the evidence review and the skills of the researcher or organisation delivering it.

How do I identify a research topic?

Many evidence reviews are commissioned in response to the needs of organisations looking to make evidence-informed decisions.

The reviews available on the Evidence Portal have been commissioned in response to priority topics identified in consultation with our stakeholders. Further reviews will continue to be commissioned on a priority basis.

If there is a particular research topic that you would like to see included, you can raise this through your usual communications channels with DCJ (for example, through local district commissioning and planning staff) or via the Evidence Portal mailbox: EvidencePortal@dcj.nsw.gov.au.

Before you commission your own evidence review, it is a good idea to check with the Evidence Portal team that a similar review is not already being conducted or planned.

Defining the Research Question

To get the best possible outcome from your evidence review you will need to have a clear research question and scope. It is important to get this first stage right as a poorly designed research question can lead to problems at a later stage of the evidence review or result in a review that does not deliver on the intended outcome.

The Evidence Portal Technical Specifications provide detailed advice on how to define your research question. They include a Research Question template at Appendix B as well as a completed example at Appendix C.

After you choose your subject matter (topic), you can use the Technical Specifications to help narrow down your research question. The research team you have commissioned to conduct your evidence should then assist you in refining your research question and scope.

What if there is too much literature?

Sometimes the research question will need to be revisited as it is too broad. Initial searches might identify copious amounts of literature that is too time consuming and resource intensive to go through. If this happens, you should review your research question and consider what is in scope and out of scope for the review to narrow down your search.

What is there isn’t enough literature?

Sometimes the research question will need to be revisited it is too narrow. Initial searches might bring up only a small body of evidence, or you might struggle to find relevant and useful papers. If this happens, you should review your research question and consider what is in scope and out of scope to broaden your search. Alternatively, you may just have to make do with a small evidence base and note the limitations of what you’ve found.

Steps for conducting an Evidence Review

The Evidence Portal Technical Specifications identify the fundamental activities needed to conduct a high-quality evidence review. The Technical Specifications outline an eight-step process that should be followed.

Only evidence reviews that have been conducted in line with the Technical Specifications will be eligible for inclusion in the Evidence Portal. However, as mentioned above, there are exceptions to this rule. Certain evidence reviews where the evidence is minimal may still have value in building the evidence for what works. If you have an evidence review in this category, talk to Evidence Portal staff if you wish to include it on the portal.

How to source a researcher/research organisation

Most organisations will commission an evidence review through a tendering process or seeking written proposals from selected providers. Staff from government agencies are required to follow strict procurement processes when commissioning services. See https://buy.nsw.gov.au/buyer-guidance/get-started/procurement-framework.

In any procurement, it is good practice to set out a statement of requirements of what you are seeking. It is recommended that you attach the Technical Specifications so potential suppliers are clear on what they need to deliver.

In selecting your provider, it is beneficial to involve teams of mixed expertise or more than one organisation. For example, research organisations who have expertise and experience in undertaking and adapting electronic database searches and synthesising evidence working in partnership with subject matter experts.

If you are unsure of where to go to seek potential suppliers, a good place to start is the Buy NSW website. The Performance and Management Services scheme for example has a list of prequalified suppliers who supply professional services according to different engagement types. See https://buy.nsw.gov.au/schemes/performance-and-management-services-scheme.

Aboriginal organisations can also be sourced from Supply Nation which is an online directory of verified Indigenous businesses https://supplynation.org.au/.