Circular No. 38/2014/TT-BKHCN on the evaluation of scientific and technological organizations

Point 3. Evaluation organization: This is an organization assigned or proposed by the competent authority to conduct the evaluation of scientific and technological organizations.

Point 4. International journals listed in the ISI directory: These are journals listed in the directories of the Institute for Scientific Information (ISI) of the United States.

Point 5. International conference proceedings listed in the ISI directory: These are conference proceedings listed in the directories of the Institute for Scientific Information (ISI) of the United States.

Article 3. Reporting onevaluation results

Point 1. The report on evaluation results shall be publicly announced, stored, utilized for science and technology management work in accordance with regulations.

Point 2. The report on evaluation results serves as the basis for monitoring adjustments to the activities of scientific and technological organizations and provides information for subsequent evaluations.

Point 3. The report on evaluation results serves as information for: Planning the system of scientific and technological organizations; formulating policies for managing and developing science and technology; assisting state management agencies in the process of selecting, directly assigning organizations to head scientific and technological tasks, providing loans, funding, and loan guarantees from funds in the field of science and technology.

Article 4. Forms and frequency of evaluation

Point 1. Self-evaluation: Scientific and technological organizations conduct self-evaluation annually. The self-evaluation result report according to Model No. 1 in Appendix III issued together with this Circular shall be kept at the scientific and technological organization for state management purposes in the field of science and technology.

Point 2. External evaluation: Scientific and technological organizations are evaluated externally every five years by state agencies managing science and technology or through independent evaluation organizations. In some special cases, external evaluation will be conducted upon request of the competent state management authority.

Article 5. Funding for Implementation of Evaluation

1.  The evaluation funding shall be sourced from the state budget for scientific and technological affairs and other lawful funding sources.

2.  Settlement and finalization of the funding shall be carried out in accordance with current regulations.

Chapter II

CRITERIA FOR EVALUATING SCIENTIFIC AND TECHNOLOGICAL ORGANIZATIONS

Article 6. Evaluation Criteria

The evaluation must be based on criteria (groups of criteria) specified from Article 7 to Article 16 and detailed in Appendix I issued together with this Circular.

Article 7. Group of Criterion 1 - Evaluation of Orientation for Development and Activity Plan orientation for development and activity plan

1.  The degree of appropriateness of the orientation for development and the scientific and technological activity plan with the position, function, and scientific and technological tasks of the organization.

2.  The degree of feasibility of the scientific and technological activity plan.

3.  The degree of dissemination of the scientific and technological activity plan to all staff members of the organization.

Article 8. Group of Criterion 2 - Evaluation of Human Resources1.   The degree of appropriateness of the structure of qualifications and age of human resources.

2.   The degree of appropriateness of the allocation of human resources according to organizational structure.

3.   The degree of capability of staff meeting the orientation and activity plan set forth.

4.   The degree of rationality in coordinating work among departments (management, research, administration, technology).

5.   The degree of rationality of the organization's policies in encouraging development and attracting highly qualified and professionally capable personnel to work long-term at the organization.

Article 9. Group of Criterion 3 - Evaluation of Financial Resources

1.  The degree of diversity of financial resources of the organization (from the state budget, domestic and international sponsorship).2.  The degree of growth in financial resources (from the state budget, domestic and international sponsorship).3.  The degree of development of financial resources from applying research results and providing scientific and technological services.

4.  The degree of rationality in allocating and using financial resources.

Article 10. Group of Criterion 4 - Evaluation of Equipment and Infrastructure

1.  The degree of adequacy (in quantity and quality) of equipment and research facilities.

2.  The degree of cooperation with domestic and foreign research and development organizations in sharing the use of equipment and research facilities.

3.  The degree of rationality in using equipment and research facilities.

4.  The degree of adequacy of conditions ensuring operations: laboratory space, office space, office equipment, security, safety, hygiene, and other conditions.

Article 11. Group of Criterion 5 - Evaluation of Information Resources

1.  The degree of adequacy regarding quality of the information and reference material supply system serving research of the organization.

2.  The degree of support provided by the organization for researchers to access external information and reference materials.

3.  The degree of effort in promoting and publicizing the organization's research results to domestic and international users.

Article 12. Group of Criterion 6 - Evaluation of Scientific Results (Publications)

1.  The number and quality of scientific articles published in international journals listed in the ISI index.

2.  The number and quality of scientific articles published in domestic journals.

3.  The number and quality of scientific reports published in proceedings of international conferences listed in the ISI index.

4.  The number and quality of scientific reports published in proceedings of domestic conferences.

5.  The number of books and book chapters that the organization has published or co-published.

3. The quantity and quality of scientific reports published in the proceedings of international conferences listed in the ISI catalog.

4. The quantity and quality of scientific reports published in the proceedings of domestic conferences.

5. The number of books and book chapters that the organization has published or co-published.

Article 13. Group 7 Criteria - Evaluation of Technology Outcomes

1. The quantity and quality of technology outcomes (inventions, utility solutions, integrated circuit layout designs, new plant varieties, and other technology-related results).

2. The number of technologies commercialized, transferred, or applied in practice.

Article 14. Group 8 Criteria - Evaluation of Training and Training Results

1. Doctoral and master's degree training results at the organization.

2. Training results to enhance skills and professional expertise for other organizations.

Article 15. Group 9 Criteria - Evaluation of Research Outcome Utilization for Economic and Social Development

1. The extent to which the organization's research outcomes are applied to contribute to addressing specific economic and social development needs of localities, the country, and internationally.

2. The extent to which scientific and technological services are provided to individuals and other organizations.

Article 16. Group 10 Criteria - Evaluation of cooperation development capacity

1. The extent to which cooperative relationships with domestic organizations are established and maintained.

2. The extent to which cooperative relationships with foreign organizations are established and maintained.

3. The extent to which high-level scientific and technological personnel from other domestic and foreign organizations are attracted to cooperate and work at the organization.

Chapter III

EVALUATION METHODS

Article 17. Evaluation Methods

The evaluation of scientific and technological organizations is conducted based on the combination of two methods: Qualitative assessment and quantitative assessment as stipulated in Articles 18 and 19 of this Circular.

Article 18. Qualitative Assessment

1. Qualitative assessment is a method aimed at describing and analyzing the characteristics of the object, analyzing strengths and weaknesses, and proposing recommendations to improve the activities of scientific and technological organizations.

2. Evaluators provide qualitative assessments according to each evaluation criterion to support quantitative assessment.

Article 19. Quantitative Assessment

1. Quantitative assessment is a scoring method based on determining weights, evaluation levels, and compiling scores according to each criterion and group of criteria.

2. Determining appropriate weights for each group of criteria and for each criterion within each group must be based on the main characteristics of the evaluated scientific and technological organization’s activities. These characteristics include: type of research (basic research; applied research and technology development) and specialization in research fields (natural sciences; social and human sciences; engineering and technology; medical sciences; agricultural sciences) and other specific characteristics.

3. The total weight of all evaluation criteria groups is 100%. The weight of each group of criteria (T) is determined so that the total weight of the groups of criteria does not exceed the following provisions:_{For power plants invested under the Build-Operate-Transfer (BOT) model, n is determined according to the operational period of the power plant stipulated in the BOT contract.}a) The weight of Group 1 criteria regarding the orientation and activity plan of scientific and technological organizations as stipulated in Article 7 of this Circular is 10%;

b) The total weight of Groups 2 to 5 criteria concerning the operational resources of scientific and technological organizations as stipulated in Articles 8 to 11 of this Circular is 30%;

c) The total weight of Groups 6 to 10 criteria concerning the outcomes of scientific and technological organizations' activities as stipulated in Articles 12 to 16 is 60%.

4. Scoring assessment:

a) Evaluators determine the level of achievement at each evaluation criterion (M) according to five assessment levels: Poor, Average, Good, Excellent, and Outstanding with corresponding scores of 1, 2, 3, 4, and 5 respectively. Detailed descriptions of each assessment level corresponding to each evaluation criterion are set out in Appendix I issued together with this Circular;

b) The assessment score (Đ) is determined by multiplying the assessment level (M) by the weight (T) of the criterion._i5. The total assessment score is calculated as follows:

a) The assessment score for each group of criteria (Đ) is the sum of the scores of the criteria within the group (Đ)._iWhere j is the number of criteria in each group of criteria._i) with the corresponding weight of that criterion (W_i):

3. Amend Clause 3 Article 2 as follows:_i = L_ix (1 - kcs)_i

b) The final assessment score for the organization (Đ) is the total score of all ten groups of criteria (Đ).

6. Classification of the organization based on the final assessment score:_{For power plants invested under the Build-Operate-Transfer (BOT) model, n is determined according to the operational period of the power plant stipulated in the BOT contract.}a) Đ ranging from 1 to 1.5: Classified as Poor;_i):

3. Amend Clause 3 Article 2 as follows:_{For power plants invested under the Build-Operate-Transfer (BOT) model, n is determined according to the operational period of the power plant stipulated in the BOT contract.} =

b) Đ ranging from above 1.5 to 2.5: Classified as Average;

c) Đ ranging from above 2.5 to 3.5: Classified as Good;_{organize credit institutions, foreign bank branches are responsible for organizing the implementation of this Circular.}d) Đ ranging from above 3.5 to 4.5: Classified as Excellent;_{For power plants invested under the Build-Operate-Transfer (BOT) model, n is determined according to the operational period of the power plant stipulated in the BOT contract.}):

3. Amend Clause 3 Article 2 as follows:_{organize credit institutions, foreign bank branches are responsible for organizing the implementation of this Circular.} =

e) Đ ranging from above 4.5 to 5: Classified as Outstanding.

7. In cases where evaluators in the Expert Team determine the assessment level for a criterion differs by more than one assessment level, the Expert Team must reassess that criterion or group of criteria._{organize credit institutions, foreign bank branches are responsible for organizing the implementation of this Circular.} EVALUATION PROCEDURES

b) Achieved scores ranging from over 1.5 to 2.5: Average category;_{organize credit institutions, foreign bank branches are responsible for organizing the implementation of this Circular.} c) Achieved scores ranging from over 2.5 to 3.5: Good category;

d) Achieved scores ranging from over 3.5 to 4.5: Excellent category;_{organize credit institutions, foreign bank branches are responsible for organizing the implementation of this Circular.} đ) Achieved scores ranging from over 4.5 to 5: Outstanding category.

d) Achieved within the range from over 3.5 to 4.5: Excellent;_{organize credit institutions, foreign bank branches are responsible for organizing the implementation of this Circular.} đạt được trong khoảng từ trên 3,5 đến 4,5: Xếp loại Tốt;

d) Achieved within the range from over 4.5 to 5: Outstanding._{organize credit institutions, foreign bank branches are responsible for organizing the implementation of this Circular.} đạt được trong khoảng từ trên 4,5 đến 5: Xếp loại Xuất sắc.

7. In cases where the experts in the Expert Group have differing evaluation levels for a criterion by more than one evaluation level, the Expert Group must re-evaluate that criterion or group of criteria.

Chapter IV

EVALUATION PROCESS

Article 20. Self-assessment Procedure

Scientific and technological organizations shall conduct self-assessment according to the following steps:

1.  Develop and approve the plan and budget for the assessment.

2.  Establish the Assessment Expert Team

The head of the scientific and technological organization establishes the Assessment Expert Team consisting of 5 to 7 experts, at least 40% of whom are not regular employees of the assessed scientific and technological organization. The assessors must have appropriate qualifications in the relevant field of the assessed scientific and technological organization and extensive knowledge and experience regarding the activities of the organization.

3.  Determine the weight of the assessment criteria

The head of the scientific and technological organization and the Assessment Expert Team shall agree and decide on the weight for each criterion and group of criteria, and fill out the Assessment Form and Summary Score Sheet according to Model No. 1 and Model No. 2 in Appendix III issued together with this Circular.

4.  Collect information, data, and prepare the Assessment File to be sent to the members of the Assessment Expert Team at least five days before the assessment meeting. The Assessment File includes:

a)  Information Form about the activities of the scientific and technological organization according to the model in Appendix II issued together with this Circular;

b)  The Assessment Form and Summary Score Sheet already filled with appropriate assessment weights as stipulated in Clause 3 of this Article.

5. Conduct and report the assessment results:

a) The Assessment Expert Team conducts the assessment according to the criteria and methods specified in Chapter II and Chapter III of this Circular.

b) The head of the Assessment Expert Team is responsible for preparing the Report on the Assessment Results according to Model No. 3 in Appendix III accompanying this Circular for the head of the scientific and technological organization to review and approve.

Article 21. External Assessment Procedure

The external assessment of scientific and technological organizations shall be carried out according to the following steps:

1.  The assessment organization develops the assessment plan and submits it to the competent authority for approval.

2.  After the assessment plan has been approved, the assessment organization informs the scientific and technological organization to be assessed at least 60 days before the assessment date about the procedures and time of the assessment.

3.  The scientific and technological organization prepares the Assessment File and sends it to the assessment organization at least 15 days before the assessment date. The Assessment File includes:

a)  Information Form about the activities of the scientific and technological organization and accompanying documents according to the model in Appendix II issued together with this Circular;

b)  Self-assessment reports as stipulated in Point b, Clause 5, Article 20 of this Circular during the assessment period.

4.  Establish the Assessment Expert Team

Based on the Assessment File, the assessment organization selects suitable assessors and establishes the Assessment Expert Team.

 5.Determine the weight of the assessment criteria

The agency assigning the task or requesting the assessment, the assessment organization, and the Assessment Expert Team shall agree and decide on the weight for each criterion and group of criteria, and fill out the Assessment Form and Summary Score Sheet according to Model No. 1 and Model No. 2 in Appendix III issued together with this Circular.

6. Conduct the assessment

The Assessment Expert Team conducts the assessment according to the criteria and methods specified in Chapter II and Chapter III of this Circular. The assessment is conducted as follows:

a)  Assess based on the file provided by the scientific and technological organization;

b)  On-site assessment: Based on the results of the file-based assessment, the assessors discuss with the leadership and key staff of the scientific and technological organization about the assessed contents and visit the facilities to complete the conclusions.

7. Prepare the report on the assessment results

The report on the assessment results must be agreed upon by the Assessment Expert Team, the assessment organization, and the assessed scientific and technological organization. The preparation of the report on the assessment results is carried out as follows:

a) After completing the on-site assessment work, the assessors discuss, score the assessment, and compile the scores into the Assessment Form and Summary Score Sheet already filled with appropriate assessment weights as stipulated in Clause 4 of this Article;

b)The Assessment Expert Team agrees on the draft report on the assessment results according to Model No. 3 in Appendix III issued together with this Circular and sends it to the head of the assessment organization;

c) The assessment organization sends the draft report on the assessment results to the assessed scientific and technological organization for comments on the assessment results;

d) Based on the comments from the scientific and technological organization, the Assessment Expert Team completes the report on the assessment results and sends it to the head of the assessment organization;

đ) The assessment organization sends the report on the assessment results to the competent authority for approval.

8. Approve the report on the assessment results

Within thirty days from the date of receipt of the report on the assessment results, the competent authority is responsible for reviewing and approving the report on the assessment results.

Chapter V

IMPLEMENTING PROVISIONS

Article 22. Effectiveness

This Circular takes effect from February 15, 2015.

Article 23.Responsibility for Implementation

1.  The Institute of Science Evaluation and Technology Valuation under the Ministry of Science and Technology is responsible for guiding and organizing the implementation of this Circular.

2.  The Director of the Institute of Science Evaluation and Technology Valuation, the Heads of units under the Ministry, and the Heads of related agencies and organizations are responsible for implementing this Circular.

3.  During the implementation process, if any difficulties arise, agencies, organizations, and individuals are requested to promptly reflect them to the Ministry of Science and Technology for appropriate amendments and supplements./.