PICO Portal is a tool that provides a collaborative systematic reviewing experience to its users through the assignment of one or more specific roles to project team members. For example, depending on the project, a user could be just a Reviewer, a Reviewer and Assistant, a Reviewer and Judicator, etc.
The hierarchy of roles is presented in the following list, with the most capabilities and access being at the top (i.e., Owner) and least at the bottom (i.e., Viewer). If a user has been assigned multiple roles, their permissions and capabilities are equal to the highest role that they have (e.g., a Reviewer and Judicator will have the capabilities of a Judicator, above and beyond those of a regular Reviewer).
User Roles #
- Owner
- Co-Owner
- Judicator
- Principal Reviewer
- Reviewer
- Librarian
- Assistant
- Viewer
The following table provides a list of general permissions allowed for a user based on the assigned role. Some permissions can also be modified in the project setup (e.g., “allowing Reviewers to create tags” can be turned ON). Roles are assigned primarily by the Owner, but some roles can also be assigned by the Co-Owner as well:
Owner #
An Owner is the team member who created the project. By default, the Owner can perform all of the functions and roles of the project, including being both a Reviewer and Judicator. These roles can not be changed. Once this is completed, the Owner will then have the option to perform the responsibilities of a Judicator. To do this, click on the number for the articles listed as Pending Conflict or Pending Judication to begin the adjudication process.
Co-Owner #
A Co-Owner has many of the same privileges as the Owner of the project. However, a Co-Owner does not have the ability to delete the project.
PICO Portal provides access to the following menu options as an Owner and Co-Owner:
Judicator #
A Judicator is a senior Reviewer who can review all citations in the project and provide a final consensus on conflicting decisions made by the Reviewers in both of the screening stages and the Risk of Bias and Data Extraction. A Judicator can perform the following functions:
- Review all citations in the project.
- Include or exclude citations after performing a review. While excluding, select the reason depending on which options have been enabled and customized in the project setup, including: Population (P), Intervention (I), Comparator (C), Outcomes (O), Study type (S), Non-RCT, Duplicate, or any other custom reasons for exclusion. The reviewer can also exclude a citation without specifying a reason.
- Screen conflicting articles in cases where two or more Reviewers do not have identical screening decisions. For example, in a dual review, Reviewer 1 may choose to include a citation, and Reviewer 2 may choose to exclude the same citation based on Population. In such a case, this citation will be marked as a conflicting citation, which needs to be reviewed by a Judicator. In a Full-Text review, if the Reviewers have excluded the citation for different reasons, then it will be considered as a conflict.
- Provide final consensus on conflicting articles. Once the judicator screens the citations and the screening results are saved by the system, the article is considered to be completed and is moved from Pending Judication to Included or Excluded.
- Provide final consensus on Risk of Bias and Data Extraction items where two Reviewers have entered conflicting responses.
PICO Portal provides access to the following menu options to a Judicator:
Principal Reviewer #
A Principal Reviewer is a special role for a Reviewer that is required to look at all records if the relevant flow is selected in setup (e.g., Dual Review with Principal Reviewer). For example, in a set of 500 citations with 6 people on a team with dual review, if one person is a Principal Reviewer, that person’s input/decision is required on all 500 citations, and the other members will contribute to the “second” decision on each citation as they screen on a first-come basis.
If there are multiple Principal Reviewers, only one of them is required to look at a citation. This is a role reserved for content experts or situations where one or more people may be “leading” the screening and want to have a say on each citation.
The role is also applicable to the Risk of Bias and Data Extraction stages if the relevant project setup is selected (e.g., Single review with Quality Check). In this case, the Principal Reviewer will be the Judicator.
Reviewer #
A Reviewer can be added by the Owner or Co-Owner. A Reviewer can perform the following functions:
- Review citations in Abstract Screening and Full-Text Review in a single review, dual review, or a custom review.
- Screen possible duplicate citations.
- Include or exclude citations during the Abstract Screening and Full-Text Review process. While excluding, select the reason depending on which options have been enabled and customized in the project setup, including: Population (P), Intervention (I), Comparator (C), Outcomes (O), Study type (S), non-RCT, Duplicate, or any other custom reason. The Reviewer can also exclude a citation without specifying a reason; by default this is “Exclude – other” (X).
- Modify their review decisions, as long as a final decision has not been made by a Judicator. PICO Portal records the decision when the Reviewer selects the outcome (Include or Exclude) and navigates to the next citation.
- Select a number of citations per set to review. The available options are 40, 60, or 100, where 40 means a set of 40 citations, and so on. Once the screening of the set is completed, the user will be prompted to review a new set of citations.
- If the option has been enabled in the project setup, reviewers are also able to view other Reviewer’s decisions, primarily for training purposes only. By default, Reviewers cannot see other Reviewers’ decisions.
PICO Portal provides access to the following menu options as a Principal Reviewer and Reviewer:
Librarian #
A Librarian is a special role in the project that has no screening, Risk of Bias, or Data Extraction responsibilities. They have a unique dashboard that shows the progress on all project components without being able to access or contribute to them. The only access and options available to a Librarian are uploading new citations for screening, managing duplicates, viewing articles, and uploading full-text PDFs for review.
PICO Portal provides access to the following menu options to a Librarian:
Assistant #
An Assistant is a user who performs administrative tasks for the Owner. They do not have any responsibilities for contributing to the stages of the review in the screening, Risk of Bias, or Data Extraction. The Assistant has a very limited role and can perform the following functions:
- Upload citations from various sources.
- Invite users to the project.
- Add and manage keywords.
- Add and manage tags.
- View articles.
- View PRISMA.
- View export status.
PICO Portal provides access to the following menu options to an Assistant:
Viewer #
A Viewer is a third-party stakeholder, a client, or someone on the team who needs to view all of the citations in the project. A Viewer can only view and filter citations and can review the PRISMA flow chart. Viewers do not have the ability to screen the articles, provide final consensus, or make any other modifications to the project.
PICO Portal provides access to the following menu options to a viewer:
Add Members to a PICO Project #
To add a new member and assign a role:
- On the Manage Projects dashboard, click on the project name, and then click on Settings > Invite. The Project Team Members page appears.
- Click on the Add Member button in the top right. Type the email address of the member you want to invite. You can type multiple email addresses in the field. Note: If you would like each member to have a different role, add them individually as new members instead of in a batch.
- After you have added the required email addresses, go to the Assign Project Roles section and select the appropriate role(s). For example, if you want the member to be a Reviewer in the project, then select the Reviewer check box. If you would like them to be a Reviewer and Judicator, select the Review and the Judicator check boxes.
- Once you have completed both of these steps, click on Invite. The member(s) will now be showing in the Pending Acceptance section.
Once you click Invite, an email is sent to the member to accept or reject the invitation. If the member accepts the invitation, the member is redirected to PICO Portal where they can log in or create a new account as required. Once the member accepts the invitation, the project will be visible to them on their Project Dashboard.
If the member rejects the invitation, the member is redirected to PICO Portal and is prompted to provide a reason for rejecting the invitation.
Pending Acceptance #
The Pending Acceptance section displays a list of all the members whom you have invited and their acceptance or rejection to the project is pending.
You can send a reminder/re-invite or delete the invitation. The Status and the Reason columns are updated when the member accepts or rejects the invitation.
For many email services, automated emails such as the invitation from PICO Portal might go into trash or spam folders. It is recommended to check with newly invited members that they have received the invitation. If they do not receive the automatic invitation, an invitation should be sent manually. To manually send an email to the member, click on the Copy Project Invitation icon in the Pending Acceptance section and copy the invite from the dialog box. Once you copy the invite, you can email the member personally from your email box and they can use the included link to directly join the project.
Manage Members #
If a member accepts the invitation, the member is added to the My Project Team section. If you wish to modify any aspect of a member’s role, you can edit their roles by going to the Project Dashboard and navigating to the Team Members section at the bottom of the page. Click on the Edit icon on the right of their name. You can also suspend a member by selecting the check box in the Suspended column.
Modifying User Roles #
To modify a member’s role(s):
- Navigate to the required project and click Settings > Invite. The Project Team Members page appears.
- Scroll down and expand the My Project Team section. A list of all the existing members appears.
- Click the Edit icon next to the member that you want to modify. The Assign Project Roles dialog box appears.
- If you want to change the user from being a Reviewer to Judicator, then clear the Reviewer check box and select the Judicator check box. If you want to make the user a Co-Owner of the project, toggle the Make this member Co-Owner button.
After you have made the modifications, click Save.
Suspending an Existing Member #
If you want to temporarily suspend a member from the project, you can do so without removing the member from the project.
To suspend a member:
- Navigate to the required project and click Settings > Invite. The Project Team Members page appears.
- Scroll down and expand the My Project Team section. A list of all the existing members appears.
- Select the check box next to the member under the Suspended column.
- In the Suspend My Team Member confirmation box, click Submit to suspend the member.
Once you suspend a member, you will not be able to modify any details of the member. You can unsuspend the member by clearing the check box under the Suspended column. The member will be unsuspended and back in the project.
Deleting an Existing Member #
If you do not want a member to be part of the project, you can delete them by navigating to the Project Team Members page and clicking the Delete icon next to the member in the My Project Team section.
Note: The Owner, Co-Owner, and Assistant are the only roles that can add, modify, or delete members from the project.
