Evaluation and prospect of scientific research capacity ranking for community healthcare centres in China

Qian Liu, Jianwei Shi, Dehua Yu, Hua Jin, Xuhua Ge, Hanzhi Zhang, Zhaoxin Wang,5

ABSTRACT The National Health and Family Planning Commission of the People's Republic of China has proposed to improve the medical capacity of general practitioners and the establishment of general practice in recent health reform.For the first time, the ability to conduct scientific research was included in this reform, which demands community healthcare centres (CHCs) to strengthen their research capacity. The evaluation of community scientific research capacity has become an important endeavour to promote the implementation of research in CHCs. Since 2016,our research team has been working on an evaluation system and has published the scientific research capacity ranking for the top 100 CHCs in China. The latest released ranking of scientific research capacities of China CHCs has aroused great attention in the country.

BACKGROuND OF SClENTlFlC RESEARCH CAPAClTY RANKlNG FOR COMMuNlTY HEALTHCARE CENTRES (CHCS) lN CHlNA

In China, the reform and opening- up policy launched in the late 1970s has resulted in the competition between elementary health institutions and tertiary hospitals.1Since 1999, the Chinese government has made great efforts to support community healthcare institutions to increase capacities in providing comprehensive healthcare, especially through the New Health Reform in 2009.2At the same time, stricter and more stringent capacity assessment requirements were imposed on them, which brought reform to chronic disease management among CHCs in China.However, has the health reform improved the ability of CHCs? In some high- income regions in China such as Shanghai, the scientific research ability was included in the capacity assessment for the general practitioners as early as 2002.3In 2015, new requirements and directions were instilled for CHCs so they become the qualified gatekeeper for disease prevention and treatment. Among these requirements, improving the scientific research capacity of CHCs was officially put forward for the first time.45By analysing the papers published by CHCs in the past 20 years, it is found that the total number of articles presents spurt growth. Between 1994 and 1997, 40 scientific research projects were undertaken by community medical institutions in Shanghai, with total funding of six million ¥.6Moreover, as the new healthcare reform in China gradually deepens,more and more highly educated doctors are rooted in the community whose demands for scientific research are more urgent. For community scientific research personnel, it is of great value to turn practical problems into scientific research, which is conducive to promote diagnosis, treatment and servicelevel improvements, as well as to promote holistic development, including research ability of medical workers.

To understand the current scientific research capacity of the CHCs in China, our team constructed an innovative community scientific research ability evaluation system in 2016. In the following consecutive years,we released the ranking lists based on data from CHCs in the whole country.7Since the top 100 list of Chinese CHCs based on scientific research was released, it has drawn great attention from both the government and the CHCs in China.

METHOD AND PROCESS OF COMMuNlTY RESEARCH CAPAClTY EvALuATlON ESTABLlSHMENT Determine the evaluation framework and its index system

Based on Donabedian’s theory,8the assessment framework was preliminarily constructed from three aspects: structure,process and outcome. We developed the framework consisting of four dimensions. The dimension of scientific research input and output included all types of scientific research achievements in China. They remind us not to only focus on the quantity of prize- winning achievements and papers but also to pay attention to the level of prize- winning and the proportion of core journals.Both dimensions can make the community's daily scientific research work more targeted, which provides further guidance for CHCs to transform their input in a more efficient and effective approach. The scientific research management dimension is similar to the process dimension, which aims to measure the quality of the CHCs’ daily scientific research. The dimension of scientific research efficiency aims to measure the efficiency of scientific research funds and the ability of achievement transformation.9-11Furthermore, through brainstorming and focus group discussion, the four factors of the evaluation framework were defined as follows: (1) scientific research input, including scientific research projects, scientific research funds obtained, personnel engaged in the scientific research, scientific research infrastructure and academic conferences and training; (2) scientific research output, including aspects of prize- winning achievements,publication of papers, publication of works, number of patents, consultation reports for governments, earnings from transformation of achievements, talent cultivation,academic part- time jobs and titles of team talents; (3)scientific research management, including management personnel engaged, special management funds, scientific research management system and management mechanism; and (4) scientific research efficiency was measured through the use of funds and the application of results.After three rounds of revision, the evaluation system of the scientific research capacity of CHCs was established and confirmed (table 1).

To determine the index weight in the evaluation system

In our research, the analytical hierarchy process (AHP)was used to establish the weights of the index.12According to AHP, the combined weight of community scientific research comprehensive evaluation index was determined. We invited a total of 48 experts to give their feedback on the index weight. Among these experts, 91.67%were experts with senior professional titles; 18.75% had more than 10 years of relevant general practice experience; and the others had relevant experience for more than 15 years. The detailed processes to determine the index weight were as follows: first, we clarified the evaluation index system and constructed the comparative judgement optimisation matrix. Second, after calculating the weight coefficient of the index, the acceptability of the weight coefficient was tested. Third, we calculate the combined weight coefficient of assessment indicators.Finally, the combined weight of capacity appraisal indicators was determined. For CHCs, the weight of the four dimensions is relatively balanced, indicating that each dimension is crucial to the improvement of scientific research capacity.

Source of data in this evaluation system

Based on the initial CHC screening list and the principle of voluntary participation, the CHCs that ultimately participated in the evaluation were first identified. Later on, a comprehensive questionnaire was sent to the CHCs,including all the evaluation aspects, such as research projects, papers published, scientific research management,scientific research achievements transformation and other issues in the past 2 years. After questionnaire data collection by the CHCs, three researchers would recollect,review, modify and confirm these data by three rounds of checking. In this process, our team would use the Chinese databases CNKI, Wanfang and Wipp and also the international databases Web of Science, Embase, PubMed to examine papers and patents which contained authors working at a CHC in China. Moreover, we searched official websites to confirm whether possible data provided by CHCs were reliable.

LATEST RESuLT OF THlS RANKlNG AND FEEDBACK FROM THE ATTENDED CHCS

Based on the latest data in 2019, we obtained the list of the top 100 CHCs. Seventy- six CHCs in eastern China were ranked in the list, including 60 in Shanghai, 2 in Shandong Province, 1 in Anhui Province, 9 in Zhejiang Province and 4 in Jiangsu Province. In southern China,six CHCs that appeared on the list were all in Guangdong province. In the northern part of China, 12 CHCs were all in Beijing. In the northwest of China, there was only one CHC in Gansu province appearing in the top 100 list. Five CHCs in southwest China were included in the list, including three in Sichuan Province, one in Chongqing Province and one in Yunnan Province.However, there was no CHC included in northeast China. The results showed that CHCs in Beijing and Shanghai accounted for more than 70% of the total; to some extent, this indicates that the degree of economic development is closely related to the capacity of community scientific research.

After the release of the top 100 list, we collected feedback from experts in CHCs, universities and hospitals by the questionnaire survey so as to provide a reference for the optimisation of the evaluation system. By using kappa consistency test, we found our results showed that experts were in good agreement with the evaluation results in Beijing and Shanghai, but the evaluation results of Jiangsu and Zhejiang province and the Pearl River Delta region were unsatisfying, indicating a further improvement in the future. Also, we conducted telephone interviews, video interviews and other forms of research with 20 CHCs in Beijing and Shanghai. According to the survey, six CHCs awarded the medical staff with excellent scientific research performance. At the same time, the indexes were studied by all CHCs with the aim of optimising the management level of scientific research. For the CHCs which did not enter the list of top 100 CHCs,they accepted peer supervision and were encouraged to strengthen the research atmosphere to improve their research capacity in the future.

Table 1 Comprehensive evaluation system of the scientific research capacity of community healthcare centres

Table 1 Continued

RANKlNG OF SClENTlFlC RESEARCH CAPAClTY FOR CHCS DREW GREAT PuBLlC ATTENTlON

Since the results of the evaluation on the scientific research capacity of CHCs were published from 2016 to 2018, more and more CHCs across the country have begun to actively adjust their scientific research practices and have gradually improved their scientific research capacity. Moreover,the project was recognised by many local governments,which played an important role in promoting and guaranteeing future scientific research evaluation. Therefore,the research is expected to stimulate the enthusiasm and initiative of CHCs around the country to strengthen their research capacity, to create a better scientific research environment and to promote the holistic development of community healthcare services.

LlMlTATlONS

The present study should be considered for its limitations. In the present evaluation system, we considered overall input and output to reflect the scale difference.However, CHCs vary in size in different parts of China;thus, it is more reasonable to add indicators of scientific research input and output per capita for further improvement. Moreover, due to limited resources and personnel,questionnaires were issued based on the initial screening sample list, and there was low participation in some areas.Furthermore, as we could not obtain the real non- public data, CHCs in some areas were excluded, which may result in the lack of comprehensiveness of the sample CHCs. Nevertheless, there is a huge scientific resource for community health service centres in China, especially on data related to chronic disease. Our research team has strived to understand the status quo of the scientific research capacity of CHCs in China and to stimulate the enthusiasm of CHCs to promote scientific research around the country.

ContributorsQL and ZW: performed the analysis and wrote the paper. JS and HJ:conceived and designed the analysis, and contributed data or analysis tools. DY, XG and HZ: collected the data and contributed data or analysis tools.

FundingThis study was supported by the National Natural Science Foundation of China (71774116 and 71603182) and the Shanghai Municipal Planning Commission of Science and Research Fund (201740202).

Competing interestsNone declared.

Patient consent for publicationNot required.

Provenance and peer reviewNot commissioned; externally peer reviewed.

Data availability statementData are available in a public, open access repository.Yes.

Open accessThis is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY- NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non- commercially,and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non- commercial. See: http:// creativecommons. org/ licenses/ by- nc/ 4. 0/.