The Evaluation Protocol of

National Research Program for Universities

1
 TABLE OF CONTENTS

1. OBJECTIVE OF THE PROTOCOL

2. EVALUATION OVERVIEW
3. SELECTION OF INDEPENDENT EXTERNAL EVALUATORS
4. FINAL SELECTION
5. GRIEVANCE COMMITTEE

2
1. OBJECTIVES OF THE PROTOCOL
The Protocol for The Evaluation of National Research Program for Universities (NRPU) Proposal
Submissions and The Selection of NRPU Grants provide guidelines for the evaluation of proposals
submitted in response to the Call for Proposals for NRPU and in addition provides guidance to the Higher
Education Commission for selection of funded grant projects based upon the results of the evaluation
process.
Proposals may be submitted by qualified higher education institutions in Pakistan as described in the Call
for Proposals. Finance for the selected proposals will be provided under the recurring budget allocation
for the NRPU.
The Higher Education Commission will select independent expert evaluators charged with objectively
assessing proposals submitted in response to the NRPU Call for Proposals. All proposals must address the
Call for Proposals through a combination of research, postgraduate training and partnerships with
academic, local and sectoral stakeholders.

The Protocol will be used by the independent expert evaluators tasked with assessing the merits of the
submitted proposals, and by the Higher Education Commission tasked with selecting the projects to be
supported following the independent evaluation process. In addition, the protocol will be available to
institutions preparing proposals.
The Protocol provides guidelines for the independent expert evaluators on assessment criteria for the
submitted proposals, as well as the procedures to be followed in the evaluation process. The Protocol also
describes the procedures that the Higher Education Commission will employ for the Selection of NRPU
grantees. The NRPU Call for Proposals Guidance contains information for those institutions submitting
proposals, and the Proposal Template Document describes each of the required sections in a
proposal. The independent expert evaluators and the Higher Education Commission may refer to any of
these documents at any point in the evaluation and selection process.

2. EVALUATION OVERVIEW
The Evaluation Process for the NRPU will be conducted in a single phase termed as Desk Review
comprising of the following steps:
a) The first step of the Evaluation Process is a screening of proposals by the Higher Education
Commission. The purpose of the screening is to ensure that submitted proposals:

i. are eligible for the competition; and

ii. respond to the Call for Proposals
Those proposals that do not meet the above criteria will be declined. Those proposals that meet
the above criteria will be considered for further evaluation.
b) Those proposals that meet the above criteria will have a technical assessment by at least Two
(02) independent expert evaluators with the disciplinary expertise necessary to consider the
proposed project. The technical assessment may be conducted by remote evaluators and/or a
panel convened to evaluate a group of NRPU proposals.

3
 c) Following the review of each proposal, a multidisciplinary Panel of experts will be convened to
synthesize the evaluation reports from the Reviewers and to make recommendations to the
Higher Education Commission for the selection of a cohesive portfolio of NRPU grants that reflect
the goals of the NRPU Program.
DESK REVIEW
In response to the Call for Proposals, interested teams (led by university affiliated researchers) must
submit a Proposal.
The first part of the desk review will be administrative. The Higher Education Commission Program
Manager/ Officer will ensure that the PI/ Institution submitting each proposal is eligible to submit a
proposal. In addition, HEC considers the following factors to screen out proposals, leading to rejection:

i. All the information required on the application form are not provided
ii. Required documents are not attached
iii. The application is not prepared in accordance with the instructions given in the Call
for proposal.
iv. Application is not received by the given deadline as per call for proposal
v. The PI has failed in timely completion/ closure of previously awarded Research grants
by the HEC
vi. The PI and/or Institution has been blacklisted or barred to apply for a certain period
of time by HEC

Those proposals that do not meet the above criteria will be declined and The Program Manager will
provide feedback to each Principal Investigator and the host institution identifying the reasons why the
proposal has not been considered for further evaluation.

The screened out proposals at the initial scrutiny/ administrative review will be forwarded for Desk
Review. The Program Manager will assign each proposal to the respective disciplinary panels. Separate
teams of independent expert evaluators – composed of approximately 4-8 members each will be
convened in each of these Panels. The Evaluation Panels will meet at the Higher Education Commission
over the course of a week to conduct the Desk Review of all submitted proposals. Panel members will
evaluate each submitted proposal.

There will be two rounds of evaluation process a) Shortlisting of Proposals; b) Detailed Review of
shortlisted Proposals

In the first round, the proposals will be shortlisted by the respective review panels to screen out potential
proposals for further detailed review. The review panels will conduct short review of the assigned
proposals to determine whether the proposed research concept aligns with the NRPU program goals and
whether the proposal implementation would lead to a substantive contribution to addressing the socio-
economic development issues of Pakistan. Proposals that are found less competitive to be considered for
detailed review, will be rejected and the panel will provide written comments/ justification for rejection
of the proposals. The Principal Investigator and the submitting institution will receive a letter identifying
the reasons why the proposals will not be considered for detailed evaluation.

In second round detailed review of only shortlisted proposals will be conducted. Shortlisted proposals will
be assigned to at least two reviewers (panel members or external reviewers) having relevant subject

4
expertise for independent evaluation of the proposal. The Desk Review by the independent expert
evaluators will be guided by an Evaluation Rubric made available concurrently to the Call for
Proposals. Therefore, institutions will be able to tailor proposals to the evaluation criteria.

The independent expert evaluators must (independently, and without consultation with any other
member of the Desk Review team) evaluate each proposal assigned to them according to the criteria
specified in the Evaluation Rubric. For the Desk Review, the evaluators must rely upon the information
contained in the written NRPU proposal. In addition to a numerical score, evaluators must provide written
justification for the score in each element of the Evaluation Rubric. The Panel Chair may ask any evaluator
to provide further written justification at any point in the Desk Review process.

Following the submission of all of the assigned Evaluation Rubrics for each proposal in a Panel, the
members of each of the Evaluation Panels will discuss all of the proposals in their Panel. This discussion
is to ensure that each proposal is treated fairly in the Desk Review evaluation process, and to recommend
those proposals which should advance for further consideration by a multidisciplinary Panel.

The final step in the evaluation process is a multidisciplinary Panel – composed of two members of each
of disciplinary Panel – to discuss all of the proposals forwarded by the disciplinary Panels. The evaluation
panel will meet to consolidate the work of each disciplinary Panel and will make recommendations to the
HEC on which proposals should be selected for NRPU funding.

The multidisciplinary Evaluation Panel should seek to recommend a portfolio of NRPU grants that:

 Addresses the major priority areas of national relevance

 Based upon the written proposal, supports projects with a high likelihood of success in alignment
with NRPU objectives
 Results in a portfolio of proposals that reflects the diversity of disciplinary areas
 Results in a portfolio of proposals that reflects junior and senior principal investigators/team
leaders
 Results in a portfolio of proposals that supports the Project goals of female principal
investigators
 The proposal must achieve a scoring threshold as determined by the evaluators in the Desk/Panel
Evaluation to ensure quality.

In conjunction with a list of recommendations for the Higher Education Commission, the Desk Review
Panel should provide a written narrative describing the reasoning for its decisions.

3. SELECTION OF INDEPENDENT EXPERT EVALUATORS

The credibility of the independent expert evaluators is essential to the credibility of the evaluation
process and the NRPU competition. The core expectations and requirements for the independent expert
evaluators are:
 Subject matter experts in one or more subject areas relevant to the proposals submitted in
response to the NRPU Call for Proposals
 Capable of effectively evaluating NRPU proposals related to their area of expertise

5
  Independent of any institution submitting a NRPU proposal

In addition to the core requirements, the team of evaluators will include:

 Experts with knowledge of the Pakistan academic system

 Experts with deep experience in the global academic system
 Members of the Pakistan diaspora
 Experts with an industrial background in sectors identified in the Call for Proposals
The Higher Education Commission is tasked with identifying and inviting the independent expert
evaluators to participate in the NRPU evaluation process. The evaluators will be recruited from
institutions and companies both in Pakistan and from around the world. The identities of the evaluators
for each proposal will not be disclosed.

4. AVOIDANCE OF ANY REAL OR PERCEIVED CONFLICT OF INTEREST

The Higher Education Commission is committed to avoid any Conflict of Interest in the Evaluation and
Selection of NRPU grantees.

Prior to participating in the Desk Review of any proposal, all independent expert evaluators must sign a
Conflict of Interest declaration regarding any proposal they are evaluating.

The evaluators are prohibited from receiving any gifts or favors from the institution or any partners.
Similarly, the institution and any partners may in no way offer gifts or favors. The evaluators are required
to immediately report any offers of gifts and favors to the Higher Education Commission. Similarly, the
institutional team is required to report any requests for gift or favors to the Higher Education Commission.

5. FINAL SELECTION
After the multidisciplinary Review Panel has submitted its final evaluation report to the Program Manager,
the Higher Education Commission will review the Panel findings and recommendations and in its final
selection, the Higher Education Commission may deviate from the recommendations of the Panel,
without, however, changing any evaluation marks of the individual proposals. It may do so based upon an
objective and clearly stated rationale to ensure a reasonable geographical, disciplinary and principal
investigator representation in the final selection. In its Selection, the Higher Education Commission will
ensure that the portfolio of NRPU grantees contributes to the basic and applied research relevant to the
national needs. The Higher Education Commission will formulate its position regarding the evaluation
outcomes in writing in the final selection.

The report of the Desk Review panel, and the Minutes of the Higher Education Commission regarding the
outcomes of the evaluation together form the evaluation results. All institutions that submitted proposals
in response to the NRPU Call for Proposals will receive copies of all Evaluation Rubrics associated with the
proposal.

6. GRIEVANCE COMMITTEE

6
In extremely rare occasions, institutions may raise an objection or grievance regarding the NRPU
Evaluation and Selection process. The objection or grievance must first be raised with the HEC for the
NRPU project within seven (07) working days of the selection results being announced .

The Higher Education Commission will set up a small (at least 3-members) Grievance Committee to which
institutions can submit grievances. The Grievance Committee will seek clarifications from the
institution/principal investigator concerned, from the independent expert evaluators, from the Program
Manager and other relevant entities, and provide recommendations on whether the grievance or appeal
should be accommodated and any proposed modified evaluation/selection decision.