A Critical Review of Personal Epistemology of Psychology，Theology and Pharmacy Students：A Comparative Study

【Abstract】This is paper is to critically analyze the journal article “Personal Epistemology of Psychology， Theology and Pharmacy Students： A Comparative Study” by Minna Kaartinen‐Koutaniemi & Sari Lindblom‐Yl?nne （2008） from the aspects of research question， research design， literature review， methodology and methods， as well as prospects， in an attempt to highlight the important elements in academic writing.

【Key words】critical review; journal article; academic writing

【作者簡介】沈燕，女，上海政法学院，讲师，硕士研究生，研究方向：二语习得，英语教育研究。

Ⅰ.Introduction

This paper will critically analyze the journal article “Personal Epistemology of Psychology， Theology and Pharmacy Students： A Comparative Study” published in Studies of Higher Education by Minna Kaartinen‐Koutaniemi & Sari Lindblom‐Yl?nne （2008）. The article aims to examine interdisciplinary differences in students personal epistemology by comparative approach with semi-structured interviews and qualitative content analysis.

Ⅱ.Research questions

Instead of research questions， the article under analysis explicitly states its hypotheses at the end of the Introduction， “the personal epistemology of students firstly evolves from interaction with the nature of the discipline， secondly from the disciplinary environment and curriculum， and thirdly from academic practices and aims modified by university teachers” （p. 180）， narrowed down from its research purposes “to describe and interpret students conceptions of thinking and reasoning based on data gathered by qualitative interview.…to compare the variation in students personal epistemology between three disciplines” （p. 180）. The hypotheses， together with the research purposes， clearly indicate the research will be directed towards a qualitative study with comparative approach， as one can learn from the expressions like “to describe and interpret students conceptions” and to “to compare…between three disciplines”. The phrase “evolve from” also indicates a causal relationship between students personal epistemology and the three aspects under investigation， which can be addressed by research techniques like content analysis employed in this work. “A hypothesis is a predicted answer to a research question” （White， 2016： 193）. Research questions are important in that they equip readers with important information about the direction of the study （Creswell， 2014：128）. By summarizing the issues to be investigated， these hypotheses per se serve their purposes to guide the readers， as well as the research design.

However， it is arguable that these hypotheses in form may not be most suitable in this study. Creswell （2014： 29） holds qualitative studies develop “a detailed understanding of a central phenomenon”， and are best suited to explore unknown variables. By taking the place of research questions， these “predicted answers to research questions” may exclude possibilities for additional findings， thus diminishing the exploring strength of a qualitative study.

Ⅲ.Research design

The design of this study makes clear what a research design is supposed to： the units of analysis， the type of data collection methods， and the methods of data analysis （Trowler， 2016：16）.

Three comparator groups were set up by discipline to examine personal epistemological differences among disciplines. The choice of the three disciplines， the Department of Psychology， the Faculty of Theology and the Faculty of Pharmacy， was based on the Bechers four-dimensional classification of the academic fields and in view of their similar professionally-oriented program goals and the same research-intensive environment， as well as distinctive disciplinary features. By incorporating more similar factors in comparison， it is more reliable to attribute differences between groups to the effects of variables under study （Gorard， 2016： 209）. Careful sampling also improves the likelihood of producing safe research findings （Kelly， 2016： 98）. According to the authors， a fourth hard-pure discipline was not included in their study because those disciplines didnt offer professionally oriented curricula in the case university.

Cases， 52 in total and in similar numbers for each discipline， were randomly selected， which is believed to produce the least bias （Gorard， 2016： 208）， from the final-year master students who were close to complete their masters degrees within each discipline. The researchers made good sense by selecting final-year master students as their cases. Studies show students become members of their disciplinary culture as their studies progress （Becher and Trowler， 2001; Palmer and Marra， 2004）， however， “disciplinary influence is generally weaker than before in the changed context in universities in the 21st century” （Trowler， Suanders & Bamber， 2012： p. ii）. Therefore， final-year master students can be more representative of their respective discipline as the research cases.

Semi-structured interviews are employed to obtain students individual reflections on elements connected to their personal epistemology， and describe and interpret students responses about thinking and reasoning by focusing on “the source and nature of knowledge， the essential process of acquiring knowledge， students views on preferences of learning environments and self-reflection about their own competences and abilities” （p. 182）.

Content analysis was applied to tally， categorize and group the responses on the dimensions that brought meaning， structure and order to the data， and also to uncover the relationships between students personal epistemology and three perspectives under examination. The latter， though， is not explicitly expressed by the authors.

In general， the design serves the research purposes and hypotheses—to “describe and interpret students conceptions of thinking and reasoning”， to compare disciplinary differences and to disclose the interaction between students personal epistemology and “the nature of the discipline”， “the disciplinary environment and curriculum”， and “academic practices and aims modified by university teachers”. However， the authors didnt mention how the three comparator groups were selected， given that there might be other departments that met the selection criteria. They neither reported how the size of cases was determined， nor about the saturation， leaving， again， the inquiry potential of the work in question （Collins， 2016： 284）.

Ⅳ. Literature Review

The article has a very compact literature review in its Introduction， mainly based on Hofers work. It begins with Hofers summary of the existing theories on personal epistemology， which leads to the “core of an individuals personal epistemological theory”-- general dimensions of students thinking （Kaartinen-Koutaniemi & Lindblom-Yl?nne， 2008）. It then quotes Hofer on the common hypothesis of epistemological differences， and the work of others on the interaction between students personal epistemological belief and disciplinary differences. Above all， the authors claimed that this study was based upon Hofers definition of personal epistemology （p. 180）. Literature was referred to， Hofer quoted again， so as to justify the need to study—to include “the measurement of disciplinary differences in personal epistemology” with the application of qualitative methods （p. 180; P. 188）. There are also some dialogues with the literature scattering in the Discussion section as well， mainly for the purpose of discussing and supporting the research results..

So， the brief literature review of this article serves， to some extent， to guide the research design， justify the need to study， define the theme of the study and provide a conceptual framework for the study.

However， its systematicity （Denner， Marsh & Campe， 2016： 145） is in doubt. The review is mainly narrative. There is neither much critical analysis nor appraisal or assessment of the previous literature. There are claims that are not underpinned with evidence. For instance， the article begins that “[p]ersonal epistemology has been explored using several theoretical frameworks”（P.179）， but neither is further explanation made， nor is any literature referred to. This might be accounted for by the limited length of the article， but the trustworthiness is undermined. So is the thoroughness of the review when it shows a distinct influence of Hofer， but bears no indication of the approach to searching the literature or the scope of the search.

Ⅴ.Methodology

Comparative approach is used to uncover the disciplinary differences in students personal epistemology. One of the strengths of comparative studies lie in that they allow large-scale and representative sampling （see ‘research design section above）. However， comparative studies， by collecting data only at one time as in this study， are ineffective in charting developmental change over time or in addressing causal relationships （Cohen， Manion & Morrison， 2011： 273; Kelly， 2016：103）. This limitation， nevertheless， can be offset by the qualitative nature of the study under analysis， by its data analysis technique of content analysis （Cohen， Manion & Morrison， 2011： 572） and by careful sampling （Kelly， 2016：100）. Oddly enough， the authors didnt mention the weakness of comparative studies in finding out causal relations or the strengths of the design in this regard.

Semi-structured interviews enabled the researchers to obtain free， open and in-depth （Marvasti & Freie， 2016： 637） conceptions from the cases of their personal epistemology in relation to the discipline nature， the disciplinary environment and the academic aims and practices. Interview talks in semi-structured interviews are structured around a set of themes; interviewers are “expected to adapt， modify and add to the prepared questions if the flow of the interview talk suggests”， thus producing “multiply layers of meaning” and “rich empirical data about the lives and perspectives of individuals”（Cousin， 2008： 71-72）. However， he also acknowledged that “gathering and representing peoples experiences is fraught with interpretive difficulties” （p. 73）， and “the success of the interview centers on the interviewers ability…”（p.87）. Furthermore， he suggests hypothesis-driven research may be incompatible to the open and exploratory nature of such in-depth interviews （p. 81）. Therefore， it is really hard to explain the researchers intension to use research hypotheses， instead of questions， in a qualitative study.

Content analysis is believed as a research technique for making systematic， objective， valid and replicable inferences from the data （Schreiber & Ferrara， 2016： 830）， and the authors may choose it over thematic analysis， in spite of the more powerful exploratory power of the latter， because frequency can be measured so that the most significant features of students personal epistemology can be identified. Another strength of content analysis worth mentioning in this study is its capability to find causal relationships， which makes up for the limitation of the comparative approach. However， existing literature also argues frequent occurrence does not necessarily indicate greater importance （Vaismoradi， Turunen， and Bondas， 2013; Cohen， Manion & Morrison， 2011：567）. And there are also disagreements among researchers about the exploratory potential of content analysis due to its pre-ordinate nature of coding and categorizing （Cohen， Manion & Morrison， 2011： 573）， which may explain why no new findings are reported in this qualitative study.

Ⅵ.Methods

The research methods adopted for data collection and analysis are in alignment with the project underpinnings， literature review and the wider methodology. There is a good rational for sampling， which facilitates to compare differences and infer causal relationships. However， it is worth mentioning that there is a difference in the proportion of samples between disciplines， with almost half of the Psychology and Pharmacy students selected as samples while only a tenth of the Theology students. So when decisions on sample size and saturation are not explained， it will be hard to judge the reliability.

The interview schedule corresponds to the themes under investigation. Interview questions are claimed to “derive from the previous research” and “modified to fit the current environment” and carefully designed to guarantee authentic answers on personal experiences and conceptions rather than factual knowledge from text books （p. 182）. Crafting good questions can contribute to the validity of the interview （Cousin， 2008： 82）; including an ethical framework in the report increases trustworthiness of the work （p.78）. This article provided information of voluntariness and anonymity， but that of consent was not mentioned.

The content analysis shows its reliability and validity. A pilot study was reported to be run with Psychology students to guarantee appropriate coding and categorization of the work （Schreiner & Ferrar， 2016： 839; Cohen， Manion & Morrison， 2011： ）. Inter-coder reliability was reported to exceed 90% in agreement. The data was electronically analyzed with ATLASti. Computer analysis is believed to be able to help mitigate inconsistency in classification （Cohen， Manion & Morrison， 2011：572）. The level of analysis aligns with the three variables proposed in the research hypotheses. However， the exact coincidence of the categories with the hypothesized variables brings doubt on the exhaustiveness of the categorization， as categories are inferred by researchers， and the more inferences are made， the more reliability and validity are compromised （Cohen， Manion & Morrison， 2011：566）.

Findings were presented together with a table providing clear details of categories， sub-categories， frequencies and percentages. However， a careful examination will find inconsistencies between the table information and the verbal presentation， which undermined the trustworthiness of the findings. Some of the findings were not reported on the basis of consistent statistical criteria. For instance， when the authors claimed interdisciplinary variation in the first category， there was less than 10% inter-group difference while there was an over 10% statistical difference between groups when they claimed the three d isciplines “are similar” in the second category.

Ⅶ.Prospect

The study managed to test its hypotheses that “the personal epistemology of students …evolves from interaction with the nature of the discipline，…the disciplinary environment and curriculum， and ...academic practices and aims modified by university teachers” （P. 180）. But there is a discrepancy between the hypotheses and the findings when the author claimed in the Discussion Section that “[t]he comparisons revealed that students epistemological beliefs interacted with their disciplinary environment” （p.188）. The phrase “evolves from” indicates a one-way， causal relation while the expression “interacted” implies the interplay between the two aspects. However， no evidence can be found in the report that personal epistemology and disciplinary environment affect each other. Such discrepancies may compromise the validity of the findings and the trustworthiness of this article.

This study， though qualitative in nature， confirmed the existing literature on personal epistemology without reporting any new findings. This may be explained that the researchers， when performing content analysis， relied too much on the pre-existing categories derived from previous literature and failed to make modifications to the analytical categories when new themes and interpretations emerge， thus confining the data analysis to testing and confirming the pre-existing knowledge rather than exploring new one （Cohen， Manion & Morrison， 2011： 573）. Or a bold assumption can be made that this work is more method-driven than question-driven， in view of the fact that hypotheses rather than research questions were adopted， and the authors claims that existing literature suggested a need of qualitative studies to “explor[e] disciplinary similarities and differences in personal epistemology”（p.180） and “[t]he multidisciplinary， comparative approach， applied in this study， constitutes a new opening to the study of personal epistemology” （p.188）.

Ⅷ.Conclusion

By doing this review， we are aware of the importance， first of all， of research questions， and then， of the alignment between the research questions， the design and the approaches， to a valid， reliable and trustworthy report of a study. Incidentally， knowledge of techniques like sampling， content analysis， etc. are also incorporated. The paper， therefore， serves to highlight essential elements of good research reports and academic writing.

Acknowledgements：

The author acknowledges the support of academic staff on the Doctoral Programme in Higher Education Research，Evacuation and Enhancement at Lancaster University from which this publication has arisen.

References：

[1]Becher， T.， and Trowler， P. （2001）. Academic tribes and territories： Intellectual enquiry and the culture of disciplines. Buckingham： Open University Press.

[2]Cohen， L.， Manion， L. & Morrison， K. （2011）. Research Methods in Education （7th ed.）. Abingdon： Routledge.

[3]Collins， K. （2016）. Chapter 13 Sampling decisions in educational research. In： Dominic Wyse， et al. （Eds）. The BERA/SAGE Handbook of Educational Research. London： SAGE Publications.

[4]Cousin， G. （2008）. Researching in Higher Education： An Introduction to Contemporary Methods and Approaches. Taylor & Francis E-library.

[5]Creswell， J. （2014）. Educational research： Planning， conducting， and evaluating quantitative and qualitative research （4th ed.）. Essex： Pearson Education Limited.

[6]Denner， J.， Marsh， E. & Campe， S. （2016）. Chapter 6 Approaches to reviewing research in Education. In： Dominic Wyse， et al. （Eds）. The BERA/SAGE Handbook of Educational Research. London： SAGE Publications.

[7]Gorard， S. （2016）. Chapter 9 An introduction to the importance of research design. In： Dominic Wyse， et al. （Eds）. The BERA/SAGE Handbook of Educational Research. London： SAGE Publications.

[8]Kelly， S. （2016）. Chapter 4 Shared principles of causal inference in qualitative and quantitative research. In： Dominic Wyse， et al. （Eds）. The BERA/SAGE Handbook of Educational Research. London： SAGE Publications.

[9]Kaartinen-Koutaniemi， M.， and Lindblom-Yl?nne， S. （2008）. Personal epistemology of psychology， theology and pharmacy students： a comparative study. Studies in Higher Education， 33（2）： 179-191.

[10]Marvasti， M. & Freie， C. （2016）. Chapter 30 Research interviews. In： Dominic Wyse， et al. （Eds）. The BERA/SAGE Handbook of Educational Research. London： SAGE Publications.

[11]Palmer， B.， and Marra， R.M. （2004）. College student epistemological perspectives across knowledge domains： A proposed grounded theory. Higher Education， 47（ 3）： 311–335.

[12]Schreiner， J. B. & Ferrar， L. N. （2016）. Chapter 41 Content analysis. In： Dominic Wyse， et al. （Eds）. The BERA/SAGE Handbook of Educational Research. London： SAGE Publications.

[13]Towler， P. （2016）. Doing Doctoral Research into Higher Education…and getting it right. CreateSpace Independent Publishing Platform.

[14]Trowler， P.， Saunders， M. & Bamber， V. （2012）. Tribes and Territories in the 21st century： Rethinking the significance of disciplines in higher education. Abingdon： Routledge.

[15]Vaismoradi， M.， Turunen， H.， & Bondas， T. （2013）. Content analysis and thematic analysis： Implications for conducting a qualitative descriptive study. Nursing and Health Sciences， 15（3）： 398-405.

[16]White， P. （2016）. Chapter 8 Research questions in education research. In： Dominic Wyse， et al. （Eds）. The BERA/SAGE Handbook of Educational Research. London： SAGE Publications.