Student Engagement and Learning Outcomes in Technology-mediated Learning in Shaanxi Province

陈叶?Shadi Kafi Mallak

【Abstract】This study investigated the impact of subjective task value, self-perceptions, technology use and teaching presence respectively on university students learning outcomes via student engagement in technology-mediated learning (TML) setting. The quantitative empirical research to test the hypothesized relationships was conducted by questionnaire survey on university students who applied technology-mediated learning in Shaanxi province.

【Key words】pilot study; reliability; validity Preliminary Data Analysis

【作者簡介】陈叶，西北政法大学；Shadi Kafi Mallak, 马来西亚科技大学。

1. Preliminary Data Analysis

A pilot is of paramount importance to the reliability, validity and practicability of the questionnaire(Cohen et al., 2018). The six panel of experts were chosen from people who had enough experience and expertise on education. These experts offered concrete suggestions for the improvement of the questionnaire. Based on experts opinion, the scale items were subjected to necessary arrangements of alternate wording, terminology, item format and sequencing. In this study, the questionnaire was pilot-tested by 109 undergraduate students in Shaanxi province. The preliminary data analysis is assessed by Cronbach Alpha coefficient, the total correlation coefficients, KMO coefficient, Bartletts test and the factor loading. Results of this preliminary evaluation on each variable show that the scale of each factor achieves the required reliability.

2. Quality Assessment of Research Data

The formal questionnaire consists of two parts, one is the survey of demographic variables, the other is the items of 5 scales. The targeted population in this research include university students in Shaanxi province. 530 questionnaires were sent to students by Super Star through Wechat and QQ. After the deletion of the invalid questionnaires from the original dataset, 452 questionnaires were used in the formal analysis. Using the data collected, the reliability and validity of the measurement will be further tested. On this basis, structural equation model analysis and regression analysis are used to test the research hypothesis.

In this study, the skewness of each measurement item is between - 0.486 and 0.485, while the kurtosis is between - 0.681 and 0.872. A normal distribution of score can be determined by the values of skewness and kurtosis. At this stage, Kline (2015) argues that when the absolute value of skewness is lower than 3 and the absolute value of kurtosis is lower than 10, the sample basically are shaped as normal distribution. Therefore, it shows that the data mainly obey the normal distribution and can be analyzed in the next step.

As to common method bias, now it is more common to load all measuring latent variables onto a single factor and constrained with no rotation so that the hypothesis that “a single factor explains all the variations” can be tested more accurately (Hair, Black, Babin, Anderson, & Tatham, 1998). In this study, the results of using the Harman single-factor test for survey data shows that the variance explained 38.986% of the total variation, which is less than judgment standard 50% proposed by Hair et al. (1998). Consequently, common method bias does not affect the analysis results.

3. Assessing Reliability and Validity of Each Variable

In this study, based on the analysis of the reliability coefficients, the alpha value for all items in the instrument ranged between of 0.799 and 0.897. As alphas is considered to be correct from 0.70, the results in this study indicated the acceptable inter-item correlation. In the formal analysis, confirmatory factor analysis (CFA) has been used to assess and modify the measurement model of a latent construct (Awang, 2012). For confirmatory factor analysis, before interpreting the analysis results, determining the overall consistency of the measurement model first and the sample data through the fitting index is of necessity, that is, the fitness of the model. Generally speaking, fitting index can be divided into three kinds：absolute fit, incremental fit and parsimonious fit. The fitting indexes of the model are all up to the reference standard, indicating that the model has good adaptability.

4. Correlation Analysis

Correlation analysis is the premise and foundation of regression analysis and structural equation model analysis as well as the preliminary test of research hypotheses. The correlation coefficients between each variable ranged from 0.009 to 0.754. From Table 4.1, it shows that task value, teaching presence, self-perception and technology use are significantly related to student engagement and learning outcomes.

5. Conclusion

The above are the summary of current data analysis process. The pilot study, quality assessment of research data, reliability and validity of each variable and correlation analysis have made a solid foundation for further study.

Table 4.1

Descriptive Statistics and Correlation Matrix of All Variables

1 2 3 4 5 6 7

1. Task Value 1

2. Teaching

Presence .578** 1

3. Self-Perception .557** .645** 1

4. Technology Use .342** .478** .570** 1

5. Student

Engagement .645** .691** .663** .449** 1

6. Learning

Outcomes .625** .729** .754** .577** .710** 1

7. Self-Discipline .011 -.009 -.061 -.253** .071 -.091 1

Mean 3.777 3.981 3.890 3.619 3.901 3.877 2.698

Standard

Deviation 0.630 0.613 0.565 0.667 0.584 0.614 0.821

References：

[1]Cohen, L., Manion, L., & Morrison, K. Research Method in Education[J]. London and New York: Routledge, 2018.

[2]Kline, R. B. Principles and practice of structural equation modeling: Guilford publications[J]. 2015.

[3]Hair, J. F., Black, W. C., Babin, B. J., Anderson, R. E., & Tatham, R. L[J]. Multivariate data analysis (Vol. 5): Prentice hall Upper Saddle River, NJ, 1998.