To configure the settings for Risk of Bias, go to the project and navigate to Settings > Risk of Bias. The Settings page appears.
The Setting Preferences section is a drop-down that allows you to handle the permissions for the selection category and provides the following options:
Allow Judicators to add selection category in Risk of Bias: Select this option if you want to allow the Judicator to add new selection categories while performing the Risk of Bias assessments.
Allow Reviewers to add selection category in Risk of Bias: Select this option if you want to allow the Reviewer to add new selection categories while performing the Risk of Bias assessments.
Additionally, to Configure the Risk of Bias Workflow, navigate to Settings > Setup and scroll to Risk of Bias Workflow on the right:
Single Review: Each record will only need to be assessed by a single Reviewer and once marked complete by that assessor, will be finalized.
Single Review + Quality Check: Each record will need to be assessed by a single Reviewer and once marked complete by that assessor, will move to Pending status. A second Reviewer must review the assessment to complete the record.
Dual Independent Review: Each record will need to be assessed by two Reviewers. After both mark it complete, it will move to pending status. A Judicator is required to resolve discrepancies in the assessment before the record can be marked complete.
PICO Portal Templates #
You can download several Risk of Bias Templates for use in Excel by navigating to Settings > Risk of Bias, clicking on the drop-down menu for Setting Preferences and scrolling down.
You can then upload the completed template to PICO Portal by clicking on the Upload Category button and selecting the Excel document that you want to upload. Alternatively, you can drag and drop the file in the Upload Category box. Click on the Upload button.
You can manually add Categories, in general or to an already uploaded template, by typing them into the Category field. You can add multiple categories by separating them with a comma. Click on Add to save the new categories.
By default, PICO Portal will activate all categories and assign a display order of “1” until modified. Click the Edit icon under Actions to change the display order, add a description, etc. After you make the modifications, click the Save button to save your changes.
Question Types #
Single Select: Only one selection can be made when answering the question, i.e. “Yes” or “No”
Multi Select: Multiple selections can be made when answering the question, i.e. “Who provided funding for the study? Select all that apply. A) Government, B) Pharmaceutical Industry, C) Other Industry, D) No Funding (Explicit), E) Not Reported”
Notes: This is a free-text question where the answers can be copied and pasted verbatim from the article, or summarized. It is similar to having notes enabled on the Single and Multi Select questions where you may want to add additional information, but does not involve any selection criteria.
Instructions: This is not a “question” to be answered by assessors. This question type is descriptor text, most commonly used to present details on how to complete the form. It can also be used to denote a “section” of the form, i.e. used to label a heading/domain.
Add New Domain #
You can add multiple Domains by clicking on the + icon in the domain section. A domain is an entirely new set of questions and a Reviewer only has to extract one domain per record. This is most useful in situations where multiple different Risk of Bias tools may be relevant to a project (e.g., if multiple study types are included). Add the name of the new domain and click Save.
Upload the template by clicking on the Upload Category button and selecting the Excel document that you want to upload. Alternatively, you can drag and drop the file in the Upload Category box. Click on the Upload button.
These can be configured in the same ways as described previously.
Configuring Notes and Mandatory Questions #
You can also configure the categories to have a free-text Notes field enabled or not, or to make certain questions mandatory. Free-text Notes adds a note field for clarifying details and additional information (e.g., providing justification for a Risk of Bias judgement).
To configure free-text notes and mandatory questions, click on the Edit icon under Actions on the category and scroll down to More Settings. You can toggle “Is Required” ON and OFF for both the category and notes. If you want the category/question and notes to be enabled, but skippable, toggle the Enable switch to ON. After you make the modifications, click the Save button to save your changes.
Adding Sets and Tabs #
A Set is a collection of tabs and questions that is different from a Domain: Sets are collections of questions that repeat, nested within a domain. Sets are primarily used to capture reported variations of different outcomes.
Click on Add a Set to create a new set. Add a short name (e.g. Results) and a long name (e.g. Study results and outcomes) for the new set, and set the display order for where it will appear in the template. The set will appear as a drop-down in the categories list and questions can be added directly to it. Existing questions can also be moved into sets.
To use a set of questions, you must add at least one Tab. Tabs are the sets of repeating questions included in the set, for example used to capture the same items for various outcomes or other relevant data, e.g. “Outcome 1”, “Outcome 2”. To add Tabs, type them into the Tab field. You can add multiple tabs by separating them with a comma. Click on Add to save the new tabs.
To add the category/question data to the set, scroll down to Add Questions. Type in the question. You can add multiple questions by separating them with a comma. Click on Add to save the new questions. To configure the questions, find the Set in the display order of categories and click on it to open the drop-down. Click on Modify beside the question and select the Question Type, and then click Save.
For example, the set is titled “Results”. The Tabs for the set are “Outcome 1” and “Outcome 2”. Two questions within the set (across tabs) have been created:
1) “Outcome definition” as Question Type “Notes”, so that the primary and secondary outcome results can be copied verbatim from the text.
2) “Time point” as Question Type “Multiple Select”, so that the relevant time periods for which data are presented can be selected.
Sets of questions can be deleted by clicking the X icon under Actions. This will delete all questions in the set.
Enabling AI Query #
All Risk of Bias question types, other than Instructions, can have AI assist activated for both Categories and Sets. In the Edit Question Information window, select Enable AI Query to activate. PICO Portal generates an AI prompt to answer the question, based on the information that is included in the “Description (Long Name)”. You can edit the prompt yourself, if you would like to make modifications.
Once enabled, all questions for which the AI Query is active will have an Ask AI suggested response to the answer. This will only happen after the system has had a chance to update and process the AI queries on the full-text PDFs that are attached and present for each record.
Preview #
When you upload a completed Risk of Bias template, you can preview how it will look to Reviewers and Judicators when they are performing Risk of Bias on the right side of the screen. Any time you make changes or add new categories, click on the Refresh icon to the right of Preview to update the preview of the form.
