The Setup Project screen allows you to perform the following tasks:
- Update project details, such as the project name, client name, due date, project description and the search criteria.
- Configure workflows (Single, Dual, and Custom) and add due dates for Abstract Screening, Full-Text Review, and Data Extraction. The due date appears on the Reviewer Dashboard for their reference.
You can access the Setup Project screen by navigating to settings Settings > Setup.
Create a New Project #
If you have signed in for the first time, you will be asked to start or join a new project. However, if you have been invited to a project by the project Owner, it will appear on the Manage Projects dashboard once you accept the invitation. In case you have not been invited and have signed up and logged in for the first time, you will be asked to create a new project.
You can create multiple projects based on your review goals. For example, if you want to import all of the articles that relate to “Antithrombotic Treatments in COVID-19” and want the Reviewers to work specifically on this, you can create a new project for this requirement. Similarly, you can create and configure projects as per the review goals.
After you have created your project, you can invite team members, upload articles, add keywords and tags, and start a single or a dual review.
To create a new project:
- Log in to PICO Portal.
- On the Manage Projects dashboard, click the Create Project button in the top right corner. The Create Project page appears.
- In the Project Name field, enter a suitable name for your project. For example, “Antithrombotic Treatments in COVID-19”. This is the only required field while creating a project. You can modify these parameters as needed.
- In the Project Description field, enter a description for the project. We recommend that you add the project description as it will be displayed on the Dashboard. This is a mandatory field.
- In the Search Criteria field, enter the search criteria. We recommend that you add the search criteria for future reference. Search criteria helps PICO Portal understand important keywords that will improve the AI prediction.
Tip: If you do not have the project details at this stage, you can leave the fields blank. The project will be created, and you can always update these fields using the Setup tab available in the project.
Delete a Project #
Deleting a project will delete all the records associated with the project. Once you delete a project, you will not be able to retrieve any records, such as articles, keywords, or tags. We do not recommend deleting a project where you have started screening.
Note: Only the project lead (Owner) can delete a project.
To delete a project:
On the Manage Projects dashboard, click the three-dot icon on the project. The Project Delete Confirmation dialog box will appear. Click on Delete.
Managing Workflow #
Before starting a project, you have to define the project workflow process. Choose the appropriate workflow based on your project requirements. PICO Portal allows you to define workflows as:
- Single Review
- Dual Review
- Dual Review with Principal Reviewer
- Custom
- Custom Review with Principal Reviewer
By default, the workflow is set to Single Review. You can define different workflows for Abstract Screening, Full-Text Review, and Data Extraction. For example, you can choose Single Review for Abstract Screening and Dual Review for Full-Text Review.
Single Review
A Single Review process requires one independent Reviewer to screen the citations and an optional Judicator to provide the final consensus. For example, a Junior Reviewer can be assigned a Reviewer’s role and a Senior Reviewer can be assigned a Judicator’s role. In this scenario, a Reviewer will screen all the citations and the Judicator will review the screening results and determine the final consensus. If the citation was not adjudicated, then the Reviewer’s screening will be the final consensus.
The following image provides a workflow of a Single Review:
A team member can have multiple roles. For example, a team member can be a Reviewer and a Judicator and can perform both functions.
Dual Review
A Dual Review process requires two independent Reviewers to screen the citations and a Judicator to handle disagreements. In a systematic review, the dual independent screening process is considered best practice as it minimizes reviewer bias.
When you configure your project for Dual Review, the citation will be reviewed independently by two Reviewers. This is a blind review where a Reviewer is not aware of the screening decision made by the other Reviewer. If there is a disagreement between the two Reviewers, the Judicator will provide the final consensus. The pending disagreements and other project statistics are available on the Project Dashboard.
For example, Reviewer 1 screens a citation and chooses to include it. However, Reviewer 2 chooses to exclude the citation. In this case, there is a disagreement between both the Reviewers. The Judicator can view the conflict and provide the final consensus. The Judicator’s answer would supersede the answers given by both Reviewers. In the case where Reviewer 1 is also a Judicator, then Reviewer 1 will resolve any disagreements.
Note: In Dual Review, the project Owner and Co-Owner(s) are assigned the role of Reviewer and Judicator by default.
In the following scenario, there are two Reviewers reviewing a single citation. After the review is conducted, the Reviewer is assigned the role of Judicator:
In the following scenario, there are two Reviewers reviewing a single citation and then a separate user for adjudicating the results. After the review is conducted by both the Reviewers, the Judicator views conflicting results and provides final consensus:
Dual Review with Principal Reviewer
Consider a scenario where you have 15+ members in the project and want to ensure that all the citations are reviewed by a Principal Reviewer who has an in-depth knowledge of the subject. In such a case, you can select the “Dual Review with Principal Reviewer” option. PICO Portal will ensure that all the citations are reviewed at least once by a Principal Reviewer.
You can assign the Principal Reviewer role to one or more members of your project. However, you will not be able to assign a Reviewer and a Principal Reviewer role to a single member. The citation will be “In Review” until the Principal Reviewer reviews and provides consensus.
Note: Single Review does not require a Principal Reviewer. In Dual Review with Principal Reviewer, the project Owner and Co-Owner(s) are assigned the role of Principal Reviewer and Judicator by default.
Tip: If a project Owner wants to be a Reviewer and not a Principal Reviewer, then the project Owner can use a secondary email and invite themselves for the specific role.
Custom Review
A Custom Review process allows you to select 3 to 5 Reviewers to independently review the citations in your project. Depending on the number of Reviewers you select, each Reviewer will conduct their reviews independently and a Judicator is assigned to resolve conflicts. In the case of Custom Review, PICO Portal takes the majority and decides the final consensus.
Note: In Custom Review, the project Owner and Co-Owner(s) are assigned the role of Reviewer and Judicator by default.
Custom Review with Principal Reviewer
When you select the Custom Review with Principal Reviewer option, PICO Portal ensures that all of the citations are reviewed by a Principal Reviewer mandatorily.
For example, consider the following scenario:
- There are 10 team members in the project
- 6 are assigned as Reviewers
- 2 are assigned as Principal Reviewers
- 1 is assigned as Judicator
In the above scenario, if you select Custom Review with Principal Reviewer and specify 4 as the option, then PICO Portal will ensure that any three Reviewers should review the citations and any one of the Principal Reviewer should review the citation to mark it as complete. The citation will be “In Review” until the Principal Reviewer reviews and provides consensus.
Note: In Custom Review with Principal Reviewer, the project Owner and Co-Owner(s) are assigned the role of Principal Reviewer and Judicator by default.
Define Criterion #
The Manage Criterion screen allows you to specify descriptions for inclusion and exclusion criteria for the PICOS elements. The Reviewer and Judicator can view these criteria during Abstract Screening and Full-Text Review.
The View Criteria button is located in the top right corner of the screening window.
To view the Manage Criterion screen, click Settings > Criterion. After you have defined the criterion, click the Save button at the bottom of the page.
