Preliminary Expert Consensus on the Scientific Evaluation Standard of Health Works for Public

Youth Science Group of Popular Medical Writing Committee,China Science Writer Association.

National Health Accomplishment Promotion Committee,China Health Culture Association.

Abstract The popularization of health science is an important part of the Plan of Healthy China 2030.It is of great significance to establish the evaluation standard of health works for public in order to solve the problem of mixed quality in the field of health science popularization in China.The scientific evaluation of health science should be based on evidence-based method,according to six aspects:author,evidence selection,evidence application,evidence evaluation,peer review&publishing platform and conflict of interest.

Keywords:Health works for public,Expert consensus,Evidence-based medicine

1.Background of expert consensus

As an important part of Healthy China 2030,health knowledge popularization is of great significance to the implementation of the Healthy China strategy [1].At present,the quality of popular science works is mixed,and the source of information is a major problem in the field of health science popularization in China.As an important measure to solve this problem,the scientific evaluation standard of health works for public is the premise and guarantee for standardizing the practice of popularization of health science and forming a scientific environment for popularizing health knowledge.

Expert consensus on the scientific evaluation criteria for health science (hereinafter referred to as“consensus”)applies to health information which refers to information that is based on scientific and technological knowledge,scientific concepts,scientific methods,and scientific skills in the health field,and is presented and disseminated in a way that is easy for the public to understand,accept,and participate.By popularizing this information,the public can form healthy concepts,adopt healthy behaviors,master health skills,and improve health literacy,thereby maintaining and promoting their own health [2].Therefore,in order to evaluated health works more objectively and accurately,it is vital to follow scientific principles as well as evidence-based thinking in the process of creating health works [3,4],which are further divided into six aspects:evidence selection,evidence evaluation,evidence application,participants,peers review and conflicts of interest.

This consensus applies to all forms of health works for public,including but not limited to health articles,videos,speeches,comics,etc.Limited by text form,examples in this consensus are mainly health articles.(Unless otherwise stated,the cases are quoted from the Voice of Medical Youths,the official platform of the Youth Science Group of Popular Medical Writing Committee,China Science Writer Association [5].)

2.Explanation of glossary and recommended levels in the expert consensus

2.1 Glossary

(1)Selection bias:refers to the various biases that arise when selecting the appropriate literature based on criteria for inclusion and exclusion of the literature,including inclusion of standard bias and selector bias[6].The selection bias in the consensus for health works mainly refers to the bias of the selector bias,that is,the bias of health works due to the subjective will of the creator.

(2)Evidence classification:refers to the classification of evidence based on the quality of evidence in evidence-based medicine.The higher the quality of evidence,the higher the level.Evidence classification can be evaluated according to methods such as grading of recommendations assessment,development and evaluation,GRADE[7].

(3)Peer-reviewed:refers to the activities of some experts in one or more fields to evaluate the health works in this field [8].

(4)Conflict of interest:refers to the interest occur between authors,reviewers or editors which is not fully announced.Such interest might influence their judgment of the published works of popular science.Financial relationships are the most common conflicts of interest,such as employers,shareholders,consultants,and patents.In addition to financial relationships,personal relationships or competition,academic competition,ideology,beliefs,etc.may also cause conflicts of interest [9].

2.2 Description of expert consensus degree level involved in the consensus

In the process of expert consensus formulation,the expert group is consulted by Delphi Method.Experts are required to score the recommendation level foreach item according to the Likert 5 scale.The 1-5 points in turn represent extremely unimportant,unimportant,general,Important and extremely important,respectively.According to the Delphi expert consultation results,the proportion of experts who recommended this item (scores of 4 and 5)was calculated after the three rounds of consultation.The level of expert consensus is determined according to the proportion of experts who recommended this item.The specific correspondence is shown in Table 1.

Table 1.Correspondence between the proportion of recommended entries and the level of expert consensus

3.Evaluation of scientific nature of health works for public

3.1 Creators

Authors are the main creators of health works.Other participants may include editors,reviewers etc.Their professional background and attitude towards the requirements of scientific standard profoundly affect the quality of final work [10].The specific correspondence is shown in Table 2.

3.1.1 Authors of health works should have a professional background in the fields involved in the works and a full understanding of the fields discussed.

The author's professional background is conducive to the author's in-depth understanding of the topic and form an objective opinion to guarantee the scientific nature of health works.Professional background mentioned here include but not limited to:experts or senior practitioners in related fields,and students who have been well trained in this field.

Example:Dietary Fiber is Good for Health,but Babies Should not Eat More

Author:Yu Zhang,Ph.D./Associate Researcher,Deputy director of the Office of Nutrition and Health of the Chinese Center for Disease Control and Prevention,committee member of Student Health Education Branch of China Student Nutrition Promotion Association,lecturer of Beijing Science and Technology University,member of China Nutrition Society and Beijing Talent Association.

Interpretation:The article is followed by a brief introduction of the author,which proves that the author has a professional background.Therefore,this article is recognized as scientific.

3.1.2 Authors of health works should not only have the experience of science popularization writing,but also have a strong ability to transform professional knowledge into science popularization works and retain its scientific nature.

When transforming scientific papers to health popularization works,the author’s experience in the creation of health works is important to guarantee their works to be scientific and avoid ambiguity or lack of logic.

Example:Dietary Fiber is Good for Health,but Babies Should Not Eat More

Author:ZHANG Yu,Nutrition consultant for TV program e.g.CCTV Meet on the Road of Health,Sheng Huo Quan;food safety expert of The Beijing News,Guangming Online etc.Senior nutrition popularization works writer;Speakers for more than 100 public lectures.

Interpretation:A brief introduction of the author is attached to the article.The author has adequate experience in writing health popularization works.Therefore,this article is recognized as scientific.

Table 2.Recommend entries for participants

3.2 Evidence Selection

It is recommended to select evidence based on evidence-based methods objectively and reasonably,which could avoid improper conclusions of health science due to bias in the choice of evidence.At the same time,appropriate evidence selection is beneficial to obtain public’s trust in health science works [11].The specific correspondence is shown in Table 3.

3.2.1 In the selection of health works,the viewpoints related to the issues discussed should be fully presented and the selection bias should be avoided as far as possible.

Lack of control over selection bias may lead to the subjective tendency of health works and lost scientific.Therefore,for the topics discussed,it should be ensured that subjective factors do not lead to bias in the choice of evidence of opposite opinions.

Example:Moderate drinking is good for cardiovascular health? 500,000+national data say no to you!

...Some epidemiological studies in the past also indicated that drink small or moderate amount of alcohol (12.5 to 25.0 g of alcohol per day)can reduces the risk of stroke and cardiovascular disease.However,this causal logic has always been controversial,whether this is the protective effect of alcohol or the nondrinkers have other health problems......

Interpretation:The core point of the article is that drinking does not beneficial to cardiovascular health,but the author does not ignore the evidence contrary to the viewpoint,indicating that the article is less influenced by the author's subjective opinions and is more scientific.

3.2.2 When the core conclusion of health works is discussed,it should be demonstrated with high-level evidence.

The health works is highly scientific with the support of high-level evidence.When choosing evidence support for core viewpoints of core topics,high-level evidence should be selected,expert opinions or case reports should be avoided in particular.

Example:Moderate drinking is good for cardiovascular health? 500,000+national data say no to you!

...On 23th August 2018,The Lancet released a study on drinking and disease,which analyzed 694 data sources on personal and group drinking from 195 countries between 1990 and 2016,and 592 data sources on the risk of alcohol intake.Final result indicates that the safest amount of alcohol consumption is zero,which means that no drinking is the healthiest behavior.Interpretation:The article uses the evidence whose core viewpoints has been proved,which is high-level evidence according to the GRADE classification.Therefore,this article is highly scientific.

3.2.3 Health works should list the sources of evidence cited.

Sources of evidence usually include published papers,statistical data published by authoritative platforms etc.The original source of the cited evidence should be listed in the work to support the authenticity of the evidence.In terms of readability and communicability,sources of evidence could be written in the text or listed as references at the end.

For example:Moderate drinking is good for cardiovascular health? 500,000+national data say no to you!

……References:……Griswold M G,Fullman N,Hawley C,et al.Alcohol use and burden for 195 countries and territories,1990–2016:a systematic analysis for the Global Burden of Disease Study 2016[J].Lancet,2018,392(10152):1015-1035.……

Interpretation:The article lists the references related to the article at the end,which is highly scientific.

3.3 Evaluation of evidence

It is recommended to conduct quality evaluation onthe selected evidence,give priority to high quality evidence,and try to avoid selecting low quality evidence.If low-quality evidence is used,it should be supported with high-quality evidence or indicate potential problems with the evidence (Table 4).

Table 3.Recommended entries for the evidence selection

3.3.1 Controversial evidence should be avoided in health works as much as possible.

When discussing controversial content in health science works,it is important not to mislead the content as academic consensus.Any potential misunderstandings need to be clarified.

Example:Gallbladder stone and gallbladder polyp,must cut gallbladder? Can the gallbladder be preserved?This method is called gallbladder stone removal surgery.Some hospitals do have this operation.However,this type of surgery has not been widely promoted,and there is some controversy exist.Domestic and foreign experts have different opinions on the preservation of gallstones by removing gallbladder.

……Experts who do not support this method believe that:……

……Experts who support this method believe that:……

Interpretation:This article describes the controversy about gallstone removal surgery,states the views of both parties,avoids misunderstanding,and is highly scientific.

3.3.2 Evidence for health works should be representative examples and avoid special cases to represent the general situation to prove the author’s view.

Individual cases are at the lowest level in the classification of evidence,and their scientific has not been further studied.Meanwhile,from the perspective of communication,the use of individual cases can easily arise public empathy and lead to the spread of unscientific rumors.Therefore,if a case is necessary,the reason for choosing the case,the reliability of the case and the particularity of the case or citing other evidence to further prove the scientific nature of the work all should be explained.

Example:Fumigated periodontal disease

I am a member of the smokers.I don't remember when I started smoking,although the daily intake is not a lot but never interrupted.With the increase of smoking age,my teeth,which used to be white and even,began to develop dental calculus and the gap between them widened;It is inevitable that gums are bleeding and retching when brushing my teeth every day,accompanied by bad breath.And every morning after getting up,there will be black,sticky sputum in my mouth.……Smoking is one of the recognized high-risk factors for periodontal disease.The longer the age of smoking,and the more the amount of smoking per day,the periodontal disease will be more serious.Periodontal disease is one of the most common infectious diseases in human.It can not only be manifested as gingival bleeding,periodontal pocket formation and alveolar bone absorption,but also cause loosening and loss of teeth,which seriously affects daily life.

Interpretation:Although the article uses personal experience as a case to attract readers and prove the harm of smoking,it also cites other evidence to prove the point of view and thus can be considered as scientific.

3.3.3 Expert opinions without high-level evidence should be carefully quoted in health works

Expert opinion is more acceptable to the reader than other evidence,but at a lower level of evidence,and more likely to be falsified in science popularization.It should not be used alone,but with the support of higher level of evidence.If it is only the expert's personal opinion,it should be clarified to the public in the work.

Table 4.Recommended entries for evidence evaluation

Example:10 anti-cancer experiences summed up by 10 top cancer experts,recommended collection! [12]

Denian Ba,academician of the Chinese Academy of Sciences,said:I never recommend eating healthcare products that enhance immunity.I have only one secret recipe for cancer,and that is to maintain a happy mood.The study found that when a person was angry,the body produced enough stress hormones to kill the mice.Therefore,"stress hormone" also known as"toxic hormones." If you are restless,angry,nervous,depressed all day,the level of stress hormones will remain high for a long time,and the body's immune system will be suppressed and destroyed,so that cancer has a chance!

Interpretation:The article quotes expert advice to attract readers,then using other evidence to prove the idea,and thus can be considered as more scientific.

3.4 Evidence Application

In the creation of health works,when applying the selected evidence to prove the raised questions,draw corresponding conclusions and put forward appropriate recommendations,the whole process should conform to the scientific logic chain[13].The specific correspondence is shown in Table 5.

3.4.1 The problem of the definition of a health science work should be scientific.At the same time,the evidence of works selection should be able to prove the core issues raised.

The questions raised by health works should have a clear background,purpose,target population,etc.,and can be scientifically proved,which is the premise to ensure the scientific demonstration of health science popularization.The evidence selected in the work should be able to clearly answer the questions raised.

Example:The ones who are barefoot is not afraid of who wear shoes? How do calluses protect your feet?

High heels,flat shoes,hard shoes,soft shoes...The thickness and hardness of the bottom of all kinds of shoes vary,so how do they affect our walking? For people who walk barefoot,what role can calluses play?According to a study published this week in Nature,calluses formed by friction and compression on the sole of the foot can protect people walking barefoot without affecting their sensitivity or walking posture.

Interpretation:At the beginning of this article,health knowledge the author hopes to popularize is put forward,and the problems raised are demonstrated in the following article.Therefore,this article is scientific.

3.4.2 The conclusion of a work should have sufficient evidence,and the process of proving the conclusion should be fully discussed through proper logical proof.

The conclusion of the work should be based on the comprehensive consideration of the arguments put forward,through a clear discussion process.

Example:Although the dietary fiber is good,the baby should not eat more

Nowadays,many Internet health science articles are also propagandizing the benefits of dietary fiber for babies,but they may neglect the problem that excessive dietary fiber intake also has potential harm to babies'health.First of all,for the baby,the stomach capacity and digestive capacity are very limited.Because dietary fiber itself cannot be digested,it cannot provide any nutrition.In addition,dietary fiber has the side effects of hindering and slowing down digestion and nutrient absorption,affecting the digestion of other nutrients such as protein and the absorption of calcium and iron.Therefore,the Chinese Society of Nutrition recommends that the daily suitable amount of cellulose for adults should be 25g,while that for children should be reduced.The European Academy of Pediatrics says school-age children should be around 10g a day.Longterm intake of dietary fiber may result in insufficient intake of other nutrients including energy,protein,iron,calcium.

Table 5.Recommended entries for evidence application

Interpretation:This article proves that infants should not eat more dietary fiber through a detailed and logical argument,which is scientific.

3.4.3 When the article comes to a conclusion and forms a recommendation,it should weigh the advantages and disadvantages comprehensively and consider whether the recommendation has other risks,operability and necessity.

The recommendations from health works should be able to be transformed into life practice,so that people can form scientific and reasonable health concept and health behavior,and effectively improve citizens'health literacy with the support of perfect evidence chain.Therefore,it is necessary to ensure that the recommendation opinions are objective,necessary and operable.

Example:Is rice really a junk food and a fattening tool?Therefore,although rice has some shortcomings in nutrition,as long as the intake is reasonably controlled and matched with other foods,such as cooking rice with miscellaneous grains,and eating it with high protein meat and eggs,it will not become a fertilizer.And for those who have stomach problems,rice is easy to digest,which is really an indispensable member of the staple food.

Interpretation:This article discusses the common view that ‘rice is a tool for fattening’ and presents both advantages and disadvantages of rice when making recommendation,which is scientific.

3.5 Peer Review and Publishing Platform

Table 6.Recommendation entries of peer review and publishing platform

3.5.1 Health works for public shall be reviewedby professional scholars in relevant fields,and the review results shall be reflected in the works.

Reviewers’ professional background should be close related to the topic discussed in health works and reviewers should have a full understanding of the topic,in order to avoid serious scientific problems.During implementation process,the precision of health works should be guaranteed,and the relevant reviewers should be responsible for the scientific nature of the works.

Example:Surgical Consent Form=Unequal Treaties?Author:Mingyu Wang Reviewer:Yimin Fan

Interpretation:This article was reviewed by chief physician of neurosurgery and listed in the article.So,the article is scientific.

3.5.2 Health works published on authoritative platforms are generally more scientific.

Health works published in authoritative platforms are evaluated and assessed by the platform.Therefore,these articles are generally more scientific.In case health works published on multiple platforms,the platform with highest authority shall prevail.

Example:Everything Done After Completing Heart Stent?

…The Voice of Medical Youths is the official platform of the Youth Science Group of Popular Medical Writing Committee,China Science Writer Association.It is a health science popularization brand that integrates science popularization,original health works creation,landing project creation,and health popularization research.

Interpretation:This article was published on the official platform Voice of Medical Youths;thus,it is highly scientific.

3.6 Conflict of Interest

Recommendation Entries of Conflict of Interest wasshown in Table 7.

Table 6.Recommendation entries of peer review and publishing platform

Table 7.Recommendation Entries of Conflict of Interest

3.6.1 Health works should not violate the principle of conflict of interest

Health works that violate conflicts of interest may become the cause of other influencing factors,affecting the author's subjective attitude,and thus affecting the scientific nature of health works.Authors are obligated to disclose potential conflicts of interest when they have certain identity or participate in activities which may cause bias,for the convenience of the public to evaluate the scientific nature of the health works.

Example:Surgical Consent Form=Unequal Treaties?Conflict of interest:The authors declare that there is no conflict of interest to this article.

Interpretation:Conflict of interest has been declared in this article,thus this article is scientific.

4.Potential users

(1)Health works authors.When creating health works,authors could use this consensus as reference to improve scientific quality of the work.

(2)Health works reviewers.When reviewing health science works,reviews could evaluate the scientific quality of health work based on this consensus.

(3)Readers of health works.Readers could use this consensus to judge the scientific quality of health wWORKSorks.

5.Pros and cons of the application of the consensus

Pros:(1)With the increasing awareness of the importance of health knowledge popularization and evidence-based medicine,high-level scientific standard for evaluating health works is required;(2)In China,not all health science works are with high quality and their scientific nature is still the primary problem of health popularization.Expert consensus established based on this problem has extensive demand.

Cons:(1)This consensus is written in Chinese,which may limit the scope of application of the consensus;(2)Popularization of health science is closely related to new media.Due to the fragmentation,quantification and differentiation of new media,the application of consensus may be difficult to cover all users.

6.Limitations of consensus

As there is still a lack of high-quality evidence in the areas covered by this consensus,expert consensus process was used to form the consensus.Therefore,this consensus is less reliable compared with other consensus which is supported by high-quality evidence.The experts involved in the consensus formation are all work in health science professionals in China,which may lead to cultural differences in this consensus.In addition,this consensus also lacks the participation of health science audiences.

List of steering committees

Qin Tang (Science and Technology Popularization Department of Chinese Medical Association);

Suning You (Journal of the Chinese Medical Association)

List of consulting experts

Hongwei Ao (Peking University Hospital of Stomatology);Shicai Chen (Beijing Luhe Hospital,Capital Medical University);Yulin Cui (Propaganda Department Health Promotion Division of National Health Commission);Jing Dong (Party and Government Information Center of People's Daily Online);Jian Fang (Huadu District People's Hospital of Guangzhou);Hongbin Gao (China Research Institute for Science Popularization);Zhongyi Gu (Beijing Dietetic Association);Baorong Hu (The First Affiliated Hospital of Harbin Medical University);Xiangdong Huang (The Newspaper for China’s Physicians/Magazine of China Hospital CEO);Yun Huang(Xiangya Hospital Central South University);Jin Li(Popular Medical Writing Committee,China Science Writer Association);Guole Lin (Peking Union Medical College Hospital);Huanlong Liu (The Second Hospital of Hebei Medical University);Maobo Liu (Fujian Medical University Union Hospital);Pengwei Lv (The First Affiliated Hospital of Zhengzhou University);Haitao Niu (Cancer Hospital Chinese Academy of Medical Sciences);Shuqing Ren (The First Hospital of Hebei Medical University);Guangfeng Ruan (China Food Information Center);Shengtie Ruan (China Health Promotion Foundation);Yang Shi (Science and Technology Development Center of Chinese Pharmaceutical Association);Zhiqiang Song (Peking University Third Hospital);Gang Song (Peking University First Hospital);Baojun Suo (Peking University Third Hospital);Xinying Sun (School of Public Health of Peking University);Qingjun Su (Beijing Chao-yang Hospital,Capital Medical University);Lixiang Wang (the Third Medical Center of PLA General Hospital);Xiujun Wang (Changzhi Medical College);Jinhua Wang (The First Affiliated Hospital of Harbin Medical University);Xing Wang(The 245th Hospital of the people's Liberation Army);Xingguo Wang (Dalian Municipal Central Hospital);Shuxia Wu (Institute of Health Service and Medical Information,Academy of Military Medical Sciences);Lifeng Xie (Peking University Third Hospital);Jing Xu (School of Journalism and communication,Peking University);Lili Xu (Beijing Zhongguancun Hospital);Muying Yang (Fujian Medical University Union Hospital);Jinghui Yao (the Third Affiliated Hospital of Southern Medical University);Detao Yin (the First Affiliated Hospital of Zhengzhou University);Hui Yu (the First Affiliated Hospital of Zhengzhou University);Renwen Yu (the Seventh Medical Center of PLA General Hospital);Chuanhong Yin (Science Popularization Daily);Chun Yu (Science and Technology Communication Center of China Association for Science and Technology);Shiying Yu(Tongji Hospital Affiliated to Tongji Medical College,Huazhong University of Science and Technology);Bo Zhang (Union Hospital,Tongji Medical College,Huazhong University of Science and Technology);Yu Zhang (Chinese Center for Disease Control and Prevention);Zhimin Zhang (China Research Institute for Science Popularization);Jinghui Zheng (Ruikang Hospital Affiliated to Guangxi University of Chinese Medicine);Kai Zhong (China Food Information Center);Daliang Zhou (Harbin First Hospital Affiliated to Harbin Institute of Technology).

Working Committee

Qungang Chang (the First Affiliated Hospital of Zhengzhou University);Liang Chen (Beijing Tiantan Hospital,Capital Medical University);Ran Chen(Nutrition and Health Sub Center of China Knowledge Center for Engineering Sciences and Technology);Shuai Ma (Beijing Chao-yang Hospital,Capital Medical University);Yongchao Song (The First Hospital of Hebei Medical University);Baojun Suo(Peking University Third Hospital);Weijing Tang(Union Hospital,Tongji Medical College,Huazhong University of Science and Technology);Yifeng Tang(the First Affiliated Hospital of Zhengzhou University);Haibin Wei (Guangxi University of Chinese Medicine);Lizhu Weng (Fujian Medical University);Xin Xu(School of Life Sciences,Peking University);Lijie Yang (the First Affiliated Hospital of Harbin Medical University);Yu Zhang (Chinese Center for Disease Control and Prevention).

Appendix 1.Scientific evaluation tools for health works for public (final edition)