An empirical study on evaluation indexes of scientific research achievements in nursing†

Jin-Lian Cheng*, Xi Tan

aThe First Hospital of Shanxi Medical University, Taiyuan, Shanxi 030001, China

bSchool of Nursing, Shanxi Medical University, Taiyuan, Shanxi 030001, China

1. Introduction

With the rapid development of nursing, the number of scientific findings has been increasing, for which scientific evaluation and recognition are particularly important. The evaluation of nursing research results involves the verification and recognition of the nursing research findings in terms of their scientific approach, creativity,and scientific value, with further evaluation of the quality and the effectiveness of the nursing research results.1As early as in the 1920s, scientific and technological evaluations were being performed in other countries,and the evaluation of the results received a great deal of attention, with legal protection and government funding, but no evaluation was directly provided. The quality of the results is the most important evaluation criterion;quantitative and qualitative evaluations are combined,and the evaluation process is open and transparent.2As reported,3-6the five core nursing periodicals with early start published a total of 14,486 research papers from 2000 to 2015, with the number of nursing research findings increasing year by year. At present, China’s effort to support scientific nursing research is increasing.The “Law of the People’s Republic of China on Promoting the Transformation of Scientific and Technological Achievements”, the “Provisional Regulations of the State Council on Technology Transfer”, the “Notice of the Ministry of Finance on the Issuance of the Interim Measures for the Management of the National Guiding Fund for the Conversion of Scientific and Technological Achievements”, and other related laws and regulations were subsequently promulgated, which provided a policy basis for the promotion, application, and transformation of nursing research findings. The application of nursing research results is conducive to accelerating the implementation of innovation-driven development strategies, promoting the combination of science and technology with the economy, and providing good guidance and services for the nursing work. Therefore, we should constantly improve the evaluation mechanisms for scientific research findings and should attach great importance to the scientific evaluation of nursing research results to optimize and promote the application of clinical nursing results. This empirical study applied evaluation indicators for nursing research results to investigate the findings from clinical nursing research.The results of this evaluation are reported herein.

2. Study objects and methods

2.1 Study objects

The 12 core periodicals of nursing reported in the Chinese Science and Technology Journal Citation Report (Core Edition) in 2015 were used as the objects of this study.The results of funded clinical nursing studies in the period 2012–2016 were searched, and 150 scientific studies were randomly selected for a survey of the authors. The inclusion criteria were as follows: (1) availability of accurate contact information; (2) informed consent provided by the authors and willingness to participate in the survey; and (3) completion of the study by each author.

2.2 Research methods

2.2.1 Research tools

According to the preestablished evaluation indicators for nursing research results,7a questionnaire for general information regarding and application of the nursing research results was developed, which included five parts: (1) introduction, demonstrating the purpose of the questionnaire and guidance for completing the questionnaire; (2) general information of the authors conducting the study; (3) basic information of the nursing research findings; (4) application of the nursing research findings;and (5) factors influencing the application of the nursing research findings.

2.2.2 Evaluation methods

To ensure the scientific approach and accuracy of the evaluation conclusions, the clinical nursing research findings were retrospectively evaluated based on the characteristics of the nursing research results. According to the characteristics of the nursing research results,based on the relevant literature and expert interviews in China and other countries, combined with the preevaluation index of the nursing research achievements previously designed by our research group, the evaluation indexes for the nursing research findings were preliminarily developed after two rounds of expert consultation using the Delphi method.7The evaluation indexes for the nursing research findings were divided into three dimensions: the level of the study, the value of the result, and the effectiveness of the result. The Cronbach coefficient of this evaluation was 0.743,indicating that the internal consistency was good. The indicators were evaluated using the five-level scoring system (scores of 0–4), with total scores of 0–52.The dimension of study level had nine entries, with scores of 0–36; the dimension of result value had two entries, with scores of 0–8; and the dimension of result effectiveness had two entries, with scores of 0–8.

2.2.3 Distribution of the questionnaires

A total of 150 questionnaires were distributed by e-mail and on-site, and 128 were collected, with a recovery rate of 85.33%. Among those questionnaires returned, 122 were valid, with an effective recovery rate of 95.31%.The entries in this questionnaire were simple and easy to understand, and the participants could generally complete the survey in 10–15 minutes.

2.2.4 Statistical analysis

The original data were input into Excel for statistical analysis using SPSS 20.0 software. The qualitative data are described as frequencies and composition ratios. The quantitative data are presented as M±SD(mean±standard deviation) or M±QR (median±quartile range) based on the status of normal distribution. The measurement data between two groups were compared using the t-test, and the measurement data among multiple groups were compared using one-way analysis of variance or approximate variance analysis with the Welch method for correction, with comparisons between groups using the least significant difference or Dunnett’s T3 method. Differences with P ＜ 0.05 were considered statistically significant.

3. Results

3.1 General information on the clinical nursing research results

Regarding the project-level perspectives of the clinical nursing studies, five studies were at the national level(4.10%), 46 were at the provincial level (37.70%), 34 were at the municipal level (27.87%), and 37 were at the hospital or university level (30.33%). From the perspective of the type of the clinical nursing studies, 19 studies were fundamental and theoretical studies (15.57%),11 studies were in the category of soft science (9.02%),and 92 studies were in the category of application of technology (75.41%). Regarding the primary research institutes of the studies, 100 studies were conducted at medical institutions (81.97%), 15 at colleges and universities (12.30%), and seven at other institutes (5.74%).Among the 122 clinical nursing studies, 98 were independent projects (80.33%), and 24 were collaborative projects (19.67%). The 122 clinical nursing studies included 39 originally innovative studies (31.97%) and 83 partially innovative studies (68.03%), among which 37 were award-winning studies (30.33%).

Of the 122 clinical nursing research studies, the authors of 63 (51.64%) studies had received training for the application and transformation of the research results; the authors of 101 (82.79%) studies worked at hospitals, and the authors of 21 (17.21%) studies worked at colleges or universities.

3.2 Distribution of the topics of the clinical nursing studies

The distribution of the topics in the 122 clinical nursing studies were as follows: 45 in specialist nursing technology (36.89%), 16 in new nursing technology (13.11%),10 in psychological nursing (8.20%), 10 in rehabilitation care (8.20%), 10 in Chinese medicine nursing (8.20%),10 in quality of life and personal management of patients(8.20%), nine in clinical nursing theory (7.38%), six in health education (4.92%), three in diet and nutrition(2.46%), and three in extended care (2.46%).

3.3 Promotion of the application of the results from clinical nursing studies

Among the results of the 122 clinical nursing studies,41 were in stable applications, 45 were applied in a small range with a small batch, 15 were in clinical trials,and 21 had not been applied in clinical trials. The scope of application was mostly limited to the author’s own department and hospital.

3.4 Results of the evaluation of the 122 clinical nursing studies

The scores of various dimensions in the evaluation of the clinical nursing studies are shown in Table 1.

3.5 Evaluation scores of the clinical nursing studies in different situations

3.5.1 Evaluation scores of applied and unapplied

clinical nursing studies

A comparison of the evaluation scores of the applied and unapplied clinical nursing studies is shown in Table 2.

3.5.2 Evaluation scores of the clinical nursing studies at different levels

A comparison of the evaluation scores of the clinical nursing studies at different levels is shown in Table 3.

3.5.3 Evaluation scores of the clinical nursing studies with different innovation statuses

Comparison of the evaluation scores of the clinical nursing studies with different innovation statuses is shown in Table 4.

Table 2. Comparison of the evaluation scores of applied and unapplied clinical nursing studies

3.5.4 Evaluation scores of the clinical nursing studies by authors from different types of institutes

Comparison of the evaluation scores of the clinical nursing studies by authors from different types of institutes is presented in Table 5.

3.5.5 Evaluation scores of the clinical nursing studies by authors with and without related training

Comparison of the evaluation scores of the clinical nursing studies by authors with and without related training is presented in Table 6.

Table 3. Comparison of the evaluation scores of the clinical nursing studies at different levels

Table 4. Comparison of the evaluation scores of the clinical nursing studies with different innovation statuses

Table 5. Comparison of the evaluation scores of clinical nursing studies by authors from different types of institutes

Table 6. Comparison of the evaluation scores of clinical nursing studies by authors with and without related training

Table 7. Factors affecting the application of the results of clinical nursing studies

3.6 Factors affecting the application of the results of the clinical nursing studies

Table 7 shows the factors affecting the application of the results of clinical nursing studies.

4. Discussion

4.1 Application of clinical nursing scientific research achievements and their empirical analysis

The goal of nursing studies is to identify nursing problems that need to be investigated in practice.These studies systematically investigate or evaluate nursing problems using scientific methods, and the results should directly or indirectly benefit the nursing practice to improve nursing work and enhance the quality of nursing care.8Transformation of the results of nursing studies is an important part of the development of nursing, as this process can promote nursing studies, improve nursing practice, and scientifically provide quality care for patients.9Therefore, this study investigated the results of clinical nursing studies.Among the clinical nursing studies included in our survey, the results of 101 studies had been promoted in applications (82.79%), but the scope of the applications was mostly limited to the authors’ own department and hospital, with no large-scale promotion of applications.Squires et al.10systematically analyzed 55 papers on the application of nursing research results, among which 38 papers showed that the utilization of the nursing research results was moderate; however, the authors of that analysis also believed that this result was too optimistic. The clinical nursing studies in this survey included 37 award-winning studies, among which 30 studies were promoted in a wider range of applications by the authors after receiving their award,indicating that the science and technology award enhanced the effectiveness of the research results and accelerated their transformation and application.11Thus, science and technology awards and funds for promotion of the application of a study’s findings can improve the scientific research conditions of the care providers, with the ultimate result of promoting excellent clinical nursing research results over a wider range,ideally not limited to the authors’ own department and hospital, in addition to fostering the role of the clinical nursing research results in clinical guidance,promoting the use of scientific approaches in decisionmaking in clinical care, and ensuring the safety and effectiveness of the nursing services. According to our results, the scores of the studies with applied results were higher than those of studies with unapplied results, and the difference was statistically significant(P ＜ 0.05), indicating that high-level and high-quality nursing results were rigorous in their scientific design,with strong practicality and feasibility, as well as high academic and application value, as shown in Table 2.

From the study-level perspective, the overall scores of studies at the provincial level were higher than those at the hospital level, and the difference was statistically significant (P ＜ 0.05). The scores of the national-level projects were also higher than those of studies at the municipal and hospital levels,and the scores of the provincial-level projects were higher than those of studies at the hospital level.Moreover, the scores of the studies describing original innovations were higher than those of studies describing partial innovations, and the differences were statistically significant (P ＜ 0.05), as shown in Tables 3 and 4. These findings may be related to the relatively strict review system for national and provincial projects, with strict requirements in terms of scientific approach, innovation, and technical means of the projects, as well as the personnel composition and the research capabilities of the research team.12These findings suggest that in scientific research management, innovation based on scientific research achievements should receive more attention and that various aspects of projects conducted at the municipal and hospital levels should be strictly controlled to focus on the hotspots of current nursing research and the urgent clinical problems that need to be solved,thereby achieving the intended purpose of research in the context of clinical practice.

Considering the value of the research results, the scores of the clinical research studies conducted by authors from hospitals were higher than those of studies conducted by authors from universities, and the difference was statistically significant (P ＜ 0.05), as shown in Table 5. This finding may be related to the fact that the results of studies conducted by authors from hospitals were based on clinical practice. Nurses identified and investigated the problems in clinical work and then applied the results of the scientific research to clinical practice to solve the problems, thereby verifying the studies’ high academic and application value.

The total scores for the level and effectiveness of studies completed by authors who received related training were significantly higher than those for studies completed by authors who received no related training, and the difference was statistically significant(P ＜ 0.05), as shown in Table 6. This finding suggests that learning and training can not only enhance the awareness, among the nursing staff, of the application of study results but can also help them understand current methods, approaches, and related legal policies for the transformation of study results. Additionally, the nursing staff can learn from previous successful experiences and follow the market, thereby promoting the application of nursing research results.

The practical results demonstrated that the established evaluation indexes of nursing research results can reflect the quality of clinical nursing research studies, which were consistent with actual situations and reflected a certain predictive capacity in the application of clinical nursing research results; this suggests that these indicators can scientifically and effectively evaluate clinical nursing research results, laying a certain foundation for improving the quality of the evaluation of nursing research results.

4.2 Factors influencing the application of clinical nursing research results

The application of scientific research results is a systematic project. From the proposal of the project to the successful application of the results, there are many intermediate steps and influencing factors, such as the maturity and adaptability of the studies; the funds for research, development, and transformation; the supporting technology of the scientific research results; and the environment, national policies, laws, and regulations influencing the application of the results. Problems in any of these areas will affect the smooth implementation of the application of the results. Our survey results showed that the top three factors influencing the application of scientific studies are funding (22.14%), the management (20.05%), and the result (17.48%). The funding factor is important for the promotion and application of clinical nursing research results, as are the lack of funding for the transformation of the results and the lack of venture capital. Publicity and transformation of the results require strong financial support. Therefore, this finding suggests that investment in the transformation of scientific nursing research results should be increased to accelerate their transformation and application. The management factor includes the weak involvement of management in the transformation of results, imperfect management and incentive mechanisms, and insufficient efforts for the propaganda and promotion of the results.Whether the nursing research results can be applied in clinical practice is closely related to the degree of management support.13In the future, management should pay more attention to the transformation of clinical nursing research results and should update nursing procedures and related technologies in a timely manner. The extensive and in-depth publicity of awardwinning scientific and technological achievements and the winners themselves not only functions to widely disseminate and introduce the scientific and technological achievements, as well as the necessary information, for their application but also serves to praise the deeds and contributions of the institution’s scientific and technical personnel, thus encouraging them to work harder in new scientific and technological studies and to strengthen transformations of their scientific and technological achievements.14The result factor is an internal factor that restricts a study’s promotion and application,including the lack of maturity of the findings, the lack of corresponding technical support, an unclear marketing orientation, poor adaptability of the results, and the form of the results. Therefore, the care providers should prepare for the promotion and application of the research results at the stage of the project proposal to make the most of the current clinical needs and should design reasonable and feasible research programs and measures, which will allow the nursing research results to better serve clinical practice.

In addition, the individual subjective factor, the market factor, and the policy factor also have certain impacts on the promotion and application of clinical nursing research results. If nurses are not fully aware of the value of scientific research or do not realize that application is the ultimate goal of scientific research,15the promotion and application of clinical nursing research results will be restricted. In the future, we should strengthen policy support; improve management of scientific research and the evaluation mechanisms for research results; strengthen related training opportunities in scientific research and design,implementation, and achievement transformation;and encourage and support the nursing staff in the transformation of their research results to truly achieve the purpose that scientific research intends to provide for clinical practice.

5. Conclusions

With the continuous development of nursing science and the rapid increase in the number of scientific research achievements, the establishment of scientific and effective evaluation indexes for scientific research findings is necessary. The results of this study show that the evaluation indexes of nursing research findings can scientifically and effectively evaluate the level,value, and effectiveness of clinical nursing research results, which will be beneficial to the optimization of scientific research results and have a certain predictive capacity in their application. Additionally, in the future,we should enhance policy support, increase funding opportunities in nursing research, improve scientific research management and the evaluation mechanisms for research results, pay more attention to personnel training, improve the quality of scientific research, and standardize the promotion and application activities for research results so that these results can be converted into real power to drive the healthy development of nursing.

Conflicts of interest

The author declares no conflicts of interest.

References

1. Cheng JL, Chu YP, Han SF, Li YJ, Tan X. Study and prospect of nursing research results evaluation indicators. Chin Nurs Res. 2016;5:1672-1676. (in Chinese).

2. Werner BM, Souder WE. Measuring R&D performance – U.S. and German practices. Res Tech Manag. 1997;40:28-32.

3. Cheng JL, Han SF, Chu YP, et al. Status quo of analysis of nursing research in 5 types of nursing periodicals from 2013 to 2015. Chin Nurs Res.2016;21:2608-2612. (in Chinese).

4. Cheng JL, Chu YP, Han SF, Kou LH, Sun YM, Ye XJ.Status quo of analysis of nursing research in 5 types of nursing periodicals from 2010 to 2012. Chin Nurs Res. 2015;15:1827-1831. (in Chinese).

5. Cheng JL, Han SF, Lv P, Zhao N. Status quo of analysis of nursing research in 5 types of nursing periodicals from 2007 to 2009. Chin Nurs Res.2010;22:2060-2064. (in Chinese).

6. Cheng JL, Han SF, Sun YM, Wang YQ, Lv P, Li N.Status quo of analysis of nursing research in 5 types of nursing periodicals from 2000 to 2006. Chin Nurs Res. 2008;22:300-305. (in Chinese).

7. Tan X, Cheng JL. Study of nursing research results evaluation indicators. Chin Nurs Res. 2017;31:2773-2774. (in Chinese).

8. Zhang JP, Guo YF. Translational nursing – a new direction of nursing research. J Nurs Sci. 2014;3:84-87. (in Chinese).

9. Zeng N, Yan QY. Progress in the study of the influencing factors for the transformation of nursing research results. Chin Nurs Manag. 2012;9:91-93. (in Chinese).

10. Squires JE, Hutchinson AM, Boström AM, O’Rourke HM, Cobban SJ, Estabrooks CA. To what extent do nurses use research in clinical practice? A systematic review. Implement Sci. 2011;6:21.

11. Wang FT. Analysis of the application and transformation of the results of the national science and technology award. China Sci Tech Forum. 2006;4:32-35(in Chinese).

12. Cheng JL, Tan X, Kou LH. Analysis of characteristics of highly cited papers published in 5 nursing periodicals from 2013 to 2015. Chin Nurs Res.2016;22:2741-2746. (in Chinese).

13. He R, Guan YM, Liu WF, Feng W. Analysis of barriers on application of nursing research findings in clinical practice. J Nurs Sci. 2007;15:58-59.(in Chinese).

14. Wang YK, Ai YM, Huang CH. Status analysis of the transformation of awarded science and technology achievements and policy suggestions to speed up transformation. Sci Res Manag. 1998;2:23-28.(in Chinese).

15. Huo M, Li XH. Problem analysis and strategy on the application of Chinese clinical nursing research based on the communication theory. Chin Nurs Manag.2009;9:68-70. (in Chinese).