Circular No. 18/2019/TT-BKHCN on the assessment of activities and service quality of public non-profit organizations in the field of science and technology

CIRCULAR

Regulations on the assessment of activities and service quality of public institutions in the field of science and technology

______________________

THE MINISTER

MINISTRY OF SCIENCE AND TECHNOLOGY

Pursuant to the Law on Science and Technology dated June 18, 2013;

Pursuant to the Government Decree No. 08/2014/NĐ-CP dated January 27, 2014 detailing and guiding the implementation of certain provisions of the Law on Science and Technology; 0Pursuant to the Government Decree No. 16/2015/NĐ-CP dated February 14, 2015 on the mechanism for autonomy of public institutions;

Pursuant to Decree No. 95/2017/NĐ-CP dated August 16, 2017, of the Government stipulating the functions, tasks, powers, and organizational structure of the Ministry of Science and Technology;

Pursuant to Decision No. 2099/QĐ-TTg dated December 27, 2017 of the Prime Minister promulgating the List of public services using state budget funds under the management of the Ministry of Science and Technology; 0Considering the proposal of the Director of the Institute of Science Evaluation and Technology Valuation and the Head of the Legal Department, The Minister of Science and Technology issues this Circular on the assessment of activities and service quality of public institutions in the field of science and technology.

Pursuant to Decree No. 54/2016/NĐ-CP dated June 14, 2016 of the Government stipulating the self-management mechanism of public scientific and technological organizations;

Implementing This Circular stipulates criteria, methods, and procedures for assessing the activities and service quality of public institutions in the field of science and technology.

1. State administrative agencies, organizations conducting assessments of activities and service quality, and public institutions in the field of science and technology.

2. Organizations and individuals related to the work of assessing activities and service quality of public institutions in the field of science and technology.

PART I

GENERAL PROVISIONS

Article 1. Scope of Regulation

Services of public institutions in the field of science and technology

Article 2. Applicability

include activities serving state administration in: scientific and technological activities; development of scientific and technological potential; intellectual property areas; standardization, measurement, and quality control areas; nuclear energy, radiation safety, and nuclear safety areas; specialized training and professional development in the field of science and technology.

Service quality of public institutions in the field of science and technology

Article 3. Explanation of Terms

In this Circular, the following terms are understood as follows:

1. is the aggregate result of indicators reflecting the level of satisfaction with the services provided by such organizations; is the criterion established to evaluate the results achieved against set targets in the organization's orientation and plan through comparing strengths and weaknesses.

2. Weight of an evaluation criterion is the coefficient indicating the relative importance of this criterion (group of criteria) compared to other criteria (groups of criteria).

3. Evaluation criterion Assessment organization

4. is an organization with appropriate functions and tasks assigned to conduct the assessment of activities and service quality of public institutions in the field of science and technology. Article 4. Principles, frequency, and funding for assessment

51. Organizations and individuals tasked with assessment must be responsible for the accuracy, objectivity, and timeliness of the assessment. 2. The assessment may be conducted simultaneously or separately for each category: activities or service quality of public institutions in the field of science and technology.

3. The assessment shall be carried out once every five years or upon the request of the competent authority to serve state administration and must be performed by organizations with the appropriate functions and tasks for assessing science and technology, selected and decided by the Ministry of Science and Technology.

4. Assessment funding shall be sourced from the science and technology public service budget and other legitimate sources and allocated to the assessment organization appropriately. Settlement of accounts for the funding shall be carried out in accordance with the law.

ASSESSMENT OF ACTIVITIES OF PUBLIC INSTITUTIONS IN THE FIELD OF SCIENCE AND TECHNOLOGY

3. The evaluation shall be conducted every five years or upon the request of the competent authority to serve state management and must be carried out by organizations with the function and task of evaluating science and technology, selected and decided by the Ministry of Science and Technology.

4. The evaluation budget shall be sourced from the state budget for scientific and technological activities, other legitimate funding sources, and allocated to the evaluation organization accordingly. The settlement and finalization of the budget shall be implemented in accordance with the provisions of the law.

Chapter II

EVALUATION OF THE ACTIVITIES OF PUBLIC INSTITUTIONS IN THE FIELD OF SCIENCE AND TECHNOLOGY

Article 5. Criteria for evaluating the activities of public non-profit organizations in the field of science and technology

The evaluation of the activities of public non-profit organizations in the field of science and technology must be based on the provisions regarding the following groups of criteria:

1. Group 1 - Evaluation of development orientation and activity plan, including two criteria implemented according to the guidelines stipulated in Section 1 of Appendix 1a issued together with this Circular:

a) Criterion 1. Clarity, completeness, and the degree of appropriateness of the development orientation and activity plan compared to the organization's position, functions, and tasks.

b) Criterion 2. Feasibility and the degree of appropriateness of the organization's development orientation and activity plan with international trends and national and local strategies in the organization's field of operation.

2. Group 2 - Evaluation of human resources, including two criteria implemented according to the guidelines stipulated in Section 2 of Appendix 1a issued together with this Circular:

a) Criterion 3. The cadre team has a suitable and stable structure to ensure the organization's operations according to its functions and tasks.

b) Criterion 4. The cadre team has sufficient necessary capabilities to meet the organization's development orientation and activity plan.

3. Group 3 - Evaluation of equipment and infrastructure, including three criteria implemented according to the guidelines stipulated in Section 3 of Appendix 1a issued together with this Circular:

a) Criterion 5. The degree of adequacy (in terms of quantity and quality) of equipment to ensure the organization's professional operations according to its functions and tasks.

b) Criterion 6. The degree of rationality in the use and coordination, sharing the use of equipment for professional operations.

c) Criterion 7. The degree of adequacy of basic infrastructure: laboratory space, office space, office equipment, application of information technology, security, hygiene, and other conditions.

4. Group 4 - Evaluation of financial resources, including three criteria implemented according to the guidelines stipulated in Section 4 of Appendix 1a issued together with this Circular:

a) Criterion 8. The degree of diversity and growth of financial resources for the organization's operations (from state budget, enterprises, and other sources within and outside the country).

b) Criterion 9. The degree of development of financial resources from the application of scientific research results and technological development and provision of science and technology services.

c) Criterion 10. The degree of reinvestment by the organization to develop human resources, infrastructure, and technological equipment from non-state budget financial resources.

5. Group 5 - Evaluation of organizational activities, including four criteria implemented according to the guidelines stipulated in Section 5 of Appendix 1a issued together with this Circular:

a) Criterion 11. The degree of rationality in coordinating work among units within the organizational structure.

b) Criterion 12. The degree of dissemination of information to staff (on strategy, policies, and operational plans of the unit; support for accessing relevant literature and information related to the field of science and technology the organization operates in...).

c) Criterion 13. The degree of effort in introducing and promoting products, services, and operational outcomes to potential partners.

d) Criterion 14. Development of cooperative relationships with domestic and foreign organizations.

6. Group 6 - Evaluation of capacity and publication results, including two criteria implemented according to the guidelines stipulated in Section 6 of Appendix 1a issued together with this Circular:

a) Criterion 15. Publication of domestic scientific publications.

b) Criterion 16. Publication of international scientific publications.

7. Group 7 - Evaluation of technological development capacity and implementation results, including two criteria implemented according to the guidelines stipulated in Section 7 of Appendix 1a issued together with this Circular:

a) Criterion 17. Development of new technologies of value.

b) Criterion 18. Implementation of new technologies in management and production.

8. Group 8 - Evaluation of training and service provision capacity and results, including two criteria implemented according to the guidelines stipulated in Section 8 of Appendix 1a issued together with this Circular:

a) Criterion 19. Training capacity and results for science and technology human resources.

b) Criterion 20. Service provision capacity and results.

Article 6. Method for evaluating the activities of public scientific and technological institutions.

The evaluation of scientific and technological organizations shall be based on combining two assessment methods: qualitative and quantitative assessments as follows:

1. Qualitative assessment to describe and analyze the characteristics of the subject, analyze strengths and weaknesses, and propose recommendations to improve the activities of scientific and technological organizations. The assessor provides qualitative assessments according to each evaluation criterion to support quantitative assessment.

2. Quantitative assessment to score evaluations based on determining weightages, evaluation levels, and compiling scores according to each criterion and group of criteria.

3. Determining appropriate weightages for each group of criteria and for each evaluation criterion within each group from 0% upwards must be based on the main operational characteristics of the evaluated organization. These characteristics include: functions, tasks, type of research (basic research; applied research and technology development), and expertise in various fields of study (natural sciences; engineering and technology; medical and pharmaceutical sciences; agricultural sciences; social sciences; and humanities) and other specific characteristics.

4. The total weightage of all groups of evaluation criteria is 100%. The weightage of each group of criteria (Wn) is determined to be suitable and ensure the following principles:

a) The weightage of Group 1 criteria as stipulated in Clause 1, Article 5 of this Circular is 10%.

b) The total weightage of Groups 2 to 5 criteria as stipulated in Clauses 2 to 5, Article 5 of this Circular is 50%.

c) The total weightage of Groups 6 to 8 criteria as stipulated in Clauses 6 to 8 is 40%.

5. Scoring the evaluation

a) The assessor determines the level of achievement at each evaluation criterion (Li) according to five evaluation levels: poor, average, good, very good, and excellent with corresponding scores of 1, 2, 3, 4, and 5. Detailed descriptions of each evaluation level corresponding to each evaluation criterion are carried out according to the guidelines set forth in Appendix 1a issued together with this Circular.

b) The score at each criterion (S_i) is determined by multiplying the evaluation level (L_i) with the corresponding weight of that criterion (W_i):

3. Amend Clause 3 Article 2 as follows:_i = L_i x (1 - kcs)_i

6. Total evaluation score is calculated as follows:

a) The score for each group of criteria (S_{For power plants invested under the Build-Operate-Transfer (BOT) model, n is determined according to the operational period of the power plant stipulated in the BOT contract.}) is the sum of the scores of the criteria (S_i) in that group:

 

Where j is the number of criteria in each group of criteria;

b) The final score for the organization (S_{organize credit institutions, foreign bank branches are responsible for organizing the implementation of this Circular.}) is the total score of all eight groups of criteria (S_{For power plants invested under the Build-Operate-Transfer (BOT) model, n is determined according to the operational period of the power plant stipulated in the BOT contract.}) multiplied by the corresponding weight of each group of criteria (W_{For power plants invested under the Build-Operate-Transfer (BOT) model, n is determined according to the operational period of the power plant stipulated in the BOT contract.}):

7. Classifying the organization based on the final score

a) Excellent: when the total score St reaches 4.5 points or higher and no criterion scores below 4 points.

b) Very Good: when the total score St reaches between 3.5 to less than 4.5 points and no criterion scores below 3 points.

c) Good: when the total score St reaches between 2.5 to less than 3.5 points and no criterion scores below 2 points.

d) Average: when the total score St reaches between 1.5 to less than 2.5 points and no criterion scores below 1 point.

e) Poor: when the total score St is less than 1.5 points.

f) The evaluation results of the organization at points a, b, c, and d of this clause will be reduced to the next lower level if any criterion does not meet the minimum score required at each level as specified.

Article 7. Evaluation Process for Public Service Organizations in the Field of Science and Technology

1. Preparation for Evaluation

a) The evaluation organization shall prepare the Evaluation Plan and submit it to the competent authority (the entity commissioning the evaluation) for approval.

b) After the Evaluation Plan has been approved, the evaluation organization shall notify the evaluated organization to coordinate and provide information for the evaluation process.

c) The evaluated organization shall prepare the initial evaluation dossier and send it to the Evaluation Organization at least 30 days before the evaluation date.

d) The evaluation dossier includes: Report on the activities of the organization, Information Form, and accompanying documents as specified in Appendix 1b issued together with this Circular regarding the Information Form on the activities of public service organizations in the field of science and technology.

2. Establishment of the Evaluation Expert Team

Based on the evaluation dossier, the evaluation organization will select suitable experts and establish an Evaluation Expert Team consisting of 3, 5, or 7 members depending on the scale and characteristics of the evaluated organization.

3. Determination of Evaluation Weights: The evaluation organization and the Evaluation Expert Team shall agree and decide on the weight for each criterion and group of criteria for evaluation and fill out the Evaluation Form and Summary Score Sheet according to Model No. 1.1 and Model No. 1.2 in Appendix 1c issued together with this Circular.

4. Conducting the Evaluation: The evaluation organization shall cooperate with the Evaluation Expert Team to conduct the evaluation according to the criteria and methods stipulated in Articles 5 and 6 of this Circular. The evaluation shall be conducted as follows:

a) The evaluation organization shall prepare an Analysis Report on information and data about the activities of the evaluated organization based on the indicators in each evaluation criterion and simultaneously identify the information and data needed to be researched and supplemented to include in the evaluation dossier provided to the experts.

b) On-site Evaluation: The evaluation expert shall study the evaluation dossier and interview, discuss with the leadership and key staff of the evaluated organization, and survey the physical facilities to obtain sufficient information and data to draw conclusions for the evaluation. The on-site evaluation report shall be added to the evaluation dossier.

c) Evaluation Based on Dossier: Each member of the Evaluation Expert Team shall study the evaluation dossier and provide their views and evaluations on the Evaluation Form according to Model No. 1.1 in Appendix 1c issued together with this Circular.

- The evaluation organization shall compile the evaluation results and fill out the Summary Evaluation Result Form according to Model No. 1.2 in Appendix 1c issued together with this Circular.

- In cases where the evaluation experts in the Evaluation Expert Team determine that the evaluation level for a criterion differs by more than one evaluation level, the Evaluation Expert Team must discuss and re-evaluate that criterion.

5. Preparing the Evaluation Result Report: The Evaluation Result Report shall be agreed upon by the Evaluation Expert Team and the evaluation organization. The preparation of the Evaluation Result Report shall be carried out as follows:

a) Based on the Evaluation Forms and Summary Evaluation Result Forms of the Evaluation Expert Team, the evaluation organization shall draft the Evaluation Report according to Model No. 1.3 in Appendix 1c issued together with this Circular.

b) The Evaluation Expert Team and the evaluation organization shall agree, complete, and be responsible for the accuracy of the Evaluation Report in accordance with the law.

6. The evaluation organization shall submit the Evaluation Report to the entity commissioning the evaluation for approval.

7. Approval of Evaluation Results: Within 30 days from the date of receipt of the Evaluation Report, the entity commissioning the evaluation shall be responsible for reviewing and approving the Evaluation Report.

Chapter III

EVALUATION OF THE QUALITY OF SERVICES PROVIDED BY PUBLIC SERVICE ORGANIZATIONS IN THE FIELD OF SCIENCE AND TECHNOLOGY

Article 8. Criteria for evaluating the quality of services provided by public service organizations in the field of science and technology

The evaluation of the quality of services provided by public service organizations in the field of science and technology must be based on the regulations concerning the following groups of criteria:

1. Group 1 Criteria - Evaluation of service provision capacity, including two criteria implemented according to the guidelines stipulated in Section 1, Appendix 2a attached hereto:

a) Criterion 1. Service structure compliance with the organization's functions and tasks;

b) Criterion 2. Experience in providing services of the unit.

2. Group 2 Criteria - Evaluation of adherence to service standards, including two criteria implemented according to the guidelines stipulated in Section 2, Appendix 2a attached hereto:

a) Criterion 3. Compliance with principles and procedures when performing services;

b) Criterion 4. Development and compliance with technical regulations.

3. Group 3 Criteria - Evaluation of rationality in the service provision process, including three criteria implemented according to the guidelines stipulated in Section 3, Appendix 2a attached hereto:

a) Criterion 5. Rationality of service provision processes;

b) Criterion 6. Ease of coordination among resources ensuring service delivery;

c) Criterion 7. Rational interaction between the service-providing organization and customers.

4. Group 4 Criteria - Evaluation of resources ensuring service delivery, including two criteria implemented according to the guidelines stipulated in Section 4, Appendix 2a attached hereto:

a) Criterion 8. Level of infrastructure, technology, facilities, and equipment support for service activities;

b) Criterion 9. Level of human resource support for service activities.

5. Group 5 Criteria - Evaluation of results and impact, including three criteria implemented according to the guidelines stipulated in Section 5, Appendix 2a attached hereto:

a) Criterion 10. Provision of services as committed;

b) Criterion 11. Customer trust in the organization;

c) Criterion 12. Growth rate in the number of customers served and accessing services.

Article 9. Methodology for Evaluating the Quality of Services Provided by Public Service Organizations in the Field of Science and Technology

The evaluation of the quality of services provided by public service organizations in the field of science and technology is conducted through a scoring method based on determining scores for each criterion, each group of criteria, and compiling the final score.

Scoring at each criterion, each group of criteria, and compiling the final score follows a five-point scale: poor, average, good, very good, and excellent corresponding to scores of 1, 2, 3, 4, and 5 respectively. Specifically as follows:

1. The evaluator determines the level of achievement of each evaluation criterion according to five levels (poor, average, good, very good, and excellent corresponding to scores of 1, 2, 3, 4, and 5) according to the guidelines stipulated in Appendix 2a (basis, indicators, and description of evaluation levels) attached hereto;

2. The score for each group of criteria is calculated as the arithmetic mean of the scores of each criterion within the group;

3. The final score for the quality of services provided by the organization is calculated as the arithmetic mean of the scores of each group of criteria;

4. Service quality ranking is divided into five levels as follows:

a) Excellent: when the total score reaches 4.5 points or higher and no criterion scores below 4 points;

b) Very Good: when the total score reaches from 3.5 to less than 4.5 points and no criterion scores below 3 points;

c) Good: when the total score reaches from 2.5 to less than 3.5 points and no criterion scores below 2 points;

d) Average: when the total score ranges from 1.5 to less than 2.5 points and no criterion scores below 1 point;

e) Poor: when the total score is below 1.5 points;

f) The quality of services provided by the organization at points a, b, c, and d of this clause will decrease to the next lower level if any criterion does not meet the minimum score at each level as prescribed.

Article 10. Quality service evaluation process for public institutions in the field of science and technology

1. Preparation for Evaluation

a) The evaluation organization builds an Evaluation Plan and submits it to the ordering agency for approval.

b) After the Evaluation Plan is approved, the Evaluation Organization notifies the evaluated organization to cooperate and provide information for the evaluation process.

c) The evaluated organization prepares the initial evaluation dossier and sends it to the Evaluation Organization at least 30 days before the evaluation date.

d) The initial evaluation dossier includes: a report on the services provided by the organization, information sheets, and accompanying documents implemented according to Appendix 2b issued together with this Circular.

2. Based on the initial evaluation dossier, the evaluation organization develops a plan and collects information from customers and related parties (reviewing documents, conducting surveys through questionnaires, and direct interviews) to obtain complete information and data according to the indicators and criteria specified in Appendix 2a issued together with this Circular.

3. Establishing the Evaluation Expert Team: based on the evaluation dossier, the evaluation organization establishes criteria and selects suitable experts to form an Evaluation Expert Team consisting of 3, 5, or 7 members, depending on the scale and characteristics of the evaluated organization.

4. Conducting the evaluation: the evaluation organization collaborates with the Evaluation Expert Team to conduct the evaluation according to the criteria and methods stipulated in Articles 8 and 9 of this Circular. The evaluation is carried out as follows:

a) The evaluation organization builds an Information Analysis Report on the quality of services provided by the evaluated organization according to the indicators in each evaluation criterion and simultaneously identifies the necessary information and data to be included in the evaluation dossier for the experts.

b) On-site evaluation: the evaluation expert studies the evaluation dossier and collaborates with the evaluation organization to conduct on-site evaluations.

At the operational site of the service-providing organization and/or at some organizations or units benefiting from the services, representatives of the Evaluation Expert Team and the evaluation organization conduct interviews, survey physical facilities, and gather necessary and relevant information to have sufficient information and data to draw conclusions for the evaluation. The on-site evaluation report is supplemented into the evaluation dossier.

c) Evaluation based on the dossier: each member of the Evaluation Expert Team studies the final evaluation dossier and provides their assessment on the Evaluation Form according to Model 2.1 in Appendix 2c issued together with this Circular;

The evaluation organization compiles the evaluation results and fills in the Summary Evaluation Result Form according to Model 2.2 in Appendix 2c issued together with this Circular;

In case the evaluation experts in the Evaluation Expert Team determine that the evaluation level for a criterion differs by more than one evaluation level, the Evaluation Expert Team must discuss and re-evaluate that criterion.

5. Building the evaluation result report: the evaluation result report must be agreed upon by the Evaluation Expert Team and the evaluation organization. The construction of the evaluation report is carried out as follows:

a) Based on the evaluation forms and the summary evaluation result forms of the Evaluation Expert Team, the evaluation organization drafts the evaluation report according to Model 2.3 in Appendix 2c issued together with this Circular;

b) The Evaluation Expert Team and the evaluation organization agree, finalize, and are responsible for the accuracy of the evaluation report according to the law.

6. The evaluation organization shall submit the Evaluation Report to the entity commissioning the evaluation for approval.

7. Approving the evaluation results: within 30 days from the date of receiving the evaluation report, the competent authority is responsible for reviewing and approving the evaluation report.

Chapter IV

IMPLEMENTING PROVISIONS

Article 11. Effective Date

This Circular takes effect from February 10, 2020.

Circular No. 38/2014/TT-BKHCN dated December 16, 2014, of the Minister of Science and Technology regarding the evaluation of scientific and technological organizations becomes invalid from the date this Circular takes effect.

Article 12. Transitional Provisions

For organizations that are currently implementing evaluations of activities according to Circular No. 38/2014/TT-BKHCN before the effective date of this Circular, they shall continue to follow the provisions of Circular No. 38/2014/TT-BKHCN.

Article 13. Implementation Organization

1. The Institute of Science Evaluation and Technology Valuation under the Ministry of Science and Technology shall develop an overall plan to implement evaluations for organizations as stipulated in this Circular.

2. The Institute of Science Evaluation and Technology Valuation is responsible for organizing training and enhancing knowledge for evaluation experts working on the activities and service quality of public institutions in the field of science and technology as stipulated in this Circular.

1. The Minister, Heads of Ministries equivalent to ministries, Heads of government agencies, Chairmen of provincial People's Committees under the central government, and related agencies, units, and individuals are responsible for implementing this Circular.

1. During implementation, if any issues arise, agencies, organizations, and individuals are requested to promptly reflect them to the Ministry of Science and Technology for guidance or research on amendments and supplements to ensure compliance.

2. The Minister, Heads of Ministries equivalent to Ministries, Heads of Government Agencies, Chairmen of People's Committees of provinces and centrally-administered cities are responsible for organizing the implementation of this Circular./.