1
VIETNAM NATIONAL UNIVERSITY, HANOI
UNIVERSITY OF LANGUAGES AND INTERNATIONAL STUDIES
FACULTY OF POST-GRADUATE STUDIES
 BÙI THỊ THU GIANG

EVALUATION OF THE EFFECTIVENESS OF ONLINE LEARNING PROGRAM AS
PART OF THE ENGLISH COURSE TO SECOND-YEAR NON-ENGLISH MAJOR
STUDENTS AT HANOI UNIVERSITY OF INDUSTRY
(Đánh giá hiệu quả của chương trình tiếng Anh trực tuyến như một phần
của chương trình tiếng Anh đối với sinh viên không chuyên năm thứ hai
trường Đại học Công nghiệp Hà Nội)

M.A. Minor Programme Thesis

THESIS SUMMARY

Field: English Language Teaching Methodology
Code: 60 14 10
Supervisor: Khoa Anh Việt, M.A. HA NOI – 2012



1
2.3. Subjects 17
2.3.1. Questionnaire respondents 17
2.3.2. Interview informants 17
2.4. Data collection instruments 18
2.4.1. Questionnaire 18
2.4.2. Interview 18
2.4.3. Pre-test, post-test and students‟ online learning reports 18
2.5. Data analysis procedures 19
CHAPTER 3: DATA ANALYSIS AND DISCUSSION 20
3.1. Data analysis 20
3.1.1. Questionnaire 20
3.1.2. Interview 29
3.1.3. Pre-test, post-test and online learning reports 31
3.2. Major findings 36
3.2.1. Students‟ and teachers‟ opinions and attitudes toward the OLP 36
3.2.2. The effectiveness of the OLP on students‟ English learning 37
3.2.3. Some suggestions to maximize the effectiveness of the OLP 37
PART C: CONCLUSION 39
1. Summary of the study 39
2. Limitations and suggestions for further study 40
REFERENCES 41
APPENDIX 1 (Student questionnaire – English version) I
APPENDIX 2 (Student questionnaire – Vietnamese version) IV
APPENDIX 3 (Teacher interview) VII
APPENDIX 4 (Pre-test) VIII
APPENDIX 5 (Post-test) XIII
1
LIST OF FIGURES, TABLES AND CHARTS

Figure 1: Evaluation as central to practice, from Gustafson & Branch (2002)
Figure 2: Conceptual framework applied to the analysis of PBJ tutorial evaluation
Table 1: Students‟ opinions on the three aspects of the OLP
Chart 1: The importance of learning English
Chart 2: The importance of Online Learning Program
Chart 3: The benefits of the OLP to students
Chart 4: The results of pre-test of CĐT2
Chart 5: The results of pre-test of CK1
Chart 6: The results of pre-test of QTDLKS1
Chart 7: The results of post-test of CĐT2
Chart 8: The results of post-test of CK1
Chart 9: The results of post-test of QTDLKS1
Chart 10: The results of pre-test and post-test of CĐT2
Chart 11: The results of pre-test and post-test of CK1
Chart 12: The results of pre-test and post-test of QTDLKS1
Chart 13: The results of pre-test and post-test
more than 10 faculties and departments. This fact results in the demand of
innovation in teaching and learning style of which online learning is a choice.
However, because this is a very new way to be applied, the challenges as well as
shortcomings are unavoidable. On carrying out the study, the researcher hopes that
it can be a big help in deciding whether or not or in what way to teach and learn
English more effectively with the adaptation of online learning.

1
Another important reason for the study is the decreasing excitement in
students‟ learning. When online learning program was introduced into the course,
most students seemed interested and hard working with the subject. They generally
finished the lessons assigned on time in the very first weeks. However, about some
weeks later, they seemed to neglect their study both online and in class. Many of
students submitted the lessons late and blamed mainly for the technical mistakes.
Their study in class became a bit less exciting than that of the first weeks. Each time
students were asked about online lessons, many of them did not remember anything
about the content and how they finished the lessons. Basing on these problems, the
researcher decided to carry out a study on the effectiveness of Online Learning
Program (OLP) to students who took part in it.
2. Aims and objectives of the study
The study is designed to bring a new look to the OLP as integrated to the
main English courses at Hanoi University of Industry. In other words, this program
is part of the blended learning course in training English for non-major students.
The study, therefore, aims at evaluating the effectiveness of the OLP to second-year
non-major students. In particular, it firstly aims at investigating students‟ and
teachers‟ attitudes towards the OLP. Secondly, it is intended to explore how
effective the OLP as part of the course is on students‟ learning through figuring out
the benefits and the learning results they had during and after its implementation.
Providing some suggestions by students and teachers to maximize the educational
outcomes of this program is the final purpose of this study.

attitudes to the OLP in general as well as its effectiveness to their students. Lastly,
students‟ online learning reports and results from pre-test and post-test were
collected to evaluate their learning progress after the introduction of the OLP.
5. Design of the study
The study consists of 3 parts:
- Part A: Introduction:
This part consists of rationale, aims, scope, method, and design of the study.
- Part B: Development:

1
This part is the main content including 3 chapters:
+ Chapter 1: is the literature review which deals with the concepts of online
learning, evaluation of online learning program, and online learning at Hanoi
University of Industry
+ Chapter 2: is about the methodology of the study
+ Chapter 3: is about the data analysis, discussion and major findings
Part C: Conclusion:
This part presents the summary of the study, limitations and the suggestions for
further study
number of institutions and organizations all over the world in a wide range of fields.
According to Wang and Ip (2010), recent pervasive social, technological and
economic changes have significantly influenced educational contexts, and thus
require radical rethinking the delivery of higher education (Rigmor, et al. 2000), and
e-learning has currently been particularly attractive for educational purposes in
recent years because of its worldwide accessibility, capabilities, and interactive
function (Li & Hart, 1996; Lin, 2006). The Sloan Consortium is a leading
researcher of online learning. In a survey of more than 2,500 US universities and
colleges (2007), they find that over 3.9 million students were taking at least one

1
online course during the fall 2007 term - a 12 percent increase over the number
reported the previous year. They concluded: “online enrollments have continued to
grow at rates far in excess of the total higher education student population, with the
most recent data demonstrating no signs of slowing.” There has been a significant
rise in the number of universities and colleges making online courses available in
the last few years. 96 percent of the very largest institutions (more than 15,000 total
enrollments) have some online offerings, as outlined in the report „Making the
Grade: Online Education in the United States‟ by the Sloan Consortium in 2006.
1.1.2. Reasons for online learning
Because of the increasing popularity over the world, online learning proves
many undeniable benefits. These are the main reasons to use online learning
provided by Shank and Sitze (2004):
First of all, it can improve access and flexibility. People can log in
everywhere, every time to complete a lesson or refer to learning materials.
Secondly, the delivery speed is faster and it is cost-saving. For organizations
that need to convey targeted information that quickly becomes outdated, online
modules are almost always faster and cheaper than flying trainers across the country
and requiring learners to sit in a classroom for a set number of hours.
Thirdly, control is improved and standardized. Online learning presents a

It is clearly seen that online learning plays a crucial part in education; it is,
however, not to say that there are no challenges faced by teachers and students who
engaged in the course. Current research has turned the attention from focusing on
the linguistic and affective characteristics of computer-assisted discussion in single
classrooms to long-distance collaboration which is challenging to arrange, because
it involves diverse learners with different cultural backgrounds, communicative
expectations, and rhetorical frameworks (Kern, Ware, & Warschauer, 2004). As for
Belz (2002), institutional differences in computer access, academic calendars and
accreditation systems are also important factors for learners‟ participation levels and
interaction. O‟Down and Eberbach (2004) point out that the roles teachers play in
organizing telecollaboration to promote greater learner autonomy or intercultural

1
competence has been generally neglected (as cited in Wang & Ip, 2010). Actually,
the learning process of teachers, according to Richards (2005), to integrate ICT
tools and programs effectively and confidently can be frustrating and often is not
achieved without adequate support. Wang and Ip (2010) also mentioned factors
affecting online learning, i.e., regarding computer literacy, successful skills and
strategies for face-to-face learning will not be suffice for online learning because
learners need a basic-medium level of ICT competence (Hara & Kling, 1999).
What‟s more, in order to have good outcomes, students should have social
interaction, computer literacy, computer self-efficacy, and positive online learner
qualities. Otherwise, their learning will be poor with predetermined pace of
learning, poorly functioning learner experiences, dissatisfaction, and negative online
learner qualities. They also emphasized that some of the aforementioned factors are
related to student skills and attitudes while others are related to educational design
which teachers should take good care of (Pillay, Irving & Tones, 2007). Faculty
Focus (2009) stressed two of the most common frustrations of online learning
which should be minimized, i.e., confusing course organization (how course
elements are structured within the course) and unclear navigation (what links or

not carried out:
 A participant may continue to take courses in which they fail to learn.

1
 Course modifications may not be based on participant outcome data and
changes may cause effective training activities to be replaced by ineffective
activities.
 Training may be viewed as a wasteful use of resources.
To sum up, evaluation is a necessary process and an important feedback tool to
ensure the development and improvement of a program.
1.2.2. Types of evaluation
There are two types of evaluation: summative and formative. According to
Russ-Eft and Preskill (2001, as cited in Brill & Park, 2011), “summative evaluation
is implemented for the purpose of determining the merit, worth or value of the
evaluand in a way that leads to a final evaluative judgement, usually regarding
ongoing use, while formative evaluation is for the purpose of developing or
improving the evaluand”. The evaluation of the OLP at HaUI applied both types.
The evaluation results are hoped to enable the Faculty of Foreign Languages and the
provider to make summative judgments regarding the effectiveness of the OLP, and
therefore make improvements and ongoing investment in it.
1.2.3. Evaluation of online learning program and the conceptual framework
Although the benefits as well as challenges in online learning are obvious, it
is not easy to evaluate an online course. According to Son (2005) the online course
which is carried out on a website is evaluated in terms of its purpose, accuracy,
currency, authority, loading speed, usefulness, organization, navigation, reliability,
authenticity, interactivity, feedback, multimedia, communication and integration.
Trần Thanh Điện and Esichaikul (2007) in their case study about evaluating online
learning environment provide three models of evaluation: objectives-oriented
evaluation, CIPP model, and Kirkpatrick‟s evaluation model. Objectives-oriented
evaluation is to compare learning results to online learning objectives. From this

considered in evaluation. Other scholars also suggest their own evaluation model
sharing similar characteristics with Kirkpatrick‟s to some extent. Thair (2008)
mentions Treadway Parker‟s model with four types of evaluation: job performance,
group performance, participant satisfaction and participant knowledge gained.

1
According to Parker, most evaluation studies focus on the last two categories.
Jackson and Kulp (1979, as cited in Thair, 2008) develop a slightly different model
with four levels of program results: reaction outcomes, capability outcomes,
application outcomes and worth outcomes among which „reaction outcomes‟
presents the participants‟ opinions of the program as a whole or as specific
components such as program content, materials, methods or activities, „capability
outcomes‟ covers what participants are expected to know, think, do or produce by
the end of the program, „application outcomes‟ refers to what the participants know,
think, do or produce in the real-world settings, and „worth outcomes‟ represents the
extent to which an organization benefits from training in terms of money, time,
effort, or resources invested.
In his study, Thair (2008) proposes an adaptation of Kirkpatrick‟s model
which can be applied to e-learning environments. The proposed model contains
three areas:
+ Interaction
+ Learning
+ Results
The interaction phase investigates the utility of the e-learning interface, its
aesthetic qualities, user satisfaction and interaction as well as the ease at which the
interface facilitated learning.
The learning phase measures the actual learning as the direct result of the e-
course. It is to assess whether the learner has learned the information or acquired the
skills necessary to apply to the pertinent area.
The results phase examines the benefit of the knowledge acquired, the ability

has paid great attention to teaching and learning English. New technological and
educational achievements are always priorities to be applied to have better teaching
and learning outcomes. Therefore, online learning received considerate
consideration from leaders of the Faculty to be adopted as part of two English
courses for non-English major students with the hope that their English knowledge
as well as skills will be highly improved. This is the first time online learning has
been applied into teaching and learning English at Hanoi University of Industry in
cooperation with GK Group-an online learning provider.
1
CHAPTER 2: METHODOLOGY
2.1. The setting of the research
The research is conducted at the Faculty of Teacher Training and Tourism

students‟ grammar, vocabulary and three skills: reading, listening and writing. The
program does not support speaking skill in forms of direct conversations or
discussions. Some activities of indirect forms and pronunciation activities are
replaced instead.
Each teacher in charge of teaching these courses and each student were given
accounts to get access to the website: . When
entering the website, after confirming registration information and some
installations, students could start learning each unit. Each time students learned on
this website was recorded on another website to be only used by the teachers who
taught them. Because this program focused on individual‟s learning achievements,
there was little interaction among learners. Students finished the activities in each
lesson and their answers were recorded to be automatically marked by the program.
Students then could know whether they had right or wrong answers and correct
them until they are right. Each time students finished the activities, they knew the
results.
Each teacher participated in these courses had 5 periods (equivalent to 225
minutes) to help students in their learning. They could answer questions related to
the lessons or knowledge related to the course. Teachers and students could arrange
time themselves provided that it is at their convenience. Other technical troubles
would be solved by GK Group via emails or on telephone. Besides having an online
account, teachers also had a common account to another website:
to check their students‟ learning progress. Every
week, students were assigned a lesson of each unit which was parallel with the unit
learned in class. One week later, teacher exported the report telling their students‟
learning process. From this report, teacher could know which students had not
finished their lessons. Because this is an integrated part to the main course, the mark

1
students got from it was just one of two progress marks among four marks needed
for the course.

2.4.1. Questionnaire
The purpose of the questionnaire is to explore students‟ attitudes and
opinions towards online learning as part of the course, the benefits students get from
online learning, the challenges they come up with during the course, and their
suggestions for better use of online learning in their learning of English. The survey
questionnaire was translated into Vietnamese to help them understand the questions
clearly and give correct answers.
2.4.2. Interview
Structured interviews were carried out to 10 teachers so as to investigate
their opinions, attitudes and the suggestions for maximum use of online learning.
The interviews were carried out when it was convenient, either at Faculty of
Foreign Languages or during the break time between two-period classes in the staff
room. The interview lasted from 10-15 minutes for each teacher.
2.4.3. Pre-test, post-test and students’ online learning reports
 Pre-test and post-test: To evaluate students‟ progress during the
implementation of the OLP in the term, two tests were designed. The pre-test
was taken at the beginning of the course (in the second week of the term),
and the post-test was carried out by the end of the course (in the 14
th
week of
the term). The content of the two tests were mainly taken from the online
materials with familiar types of exercises to students. These tests include
grammar and vocabulary, reading, listening, writing and speaking and were
carried out offline (in class).
 Students’ online learning reports: Every week during the term, students‟
online learning reports were exported by their teachers from the website for
learning management. The data in each weekly report show students‟
completion of the lesson (%), the score they got (on the scale of 100) and the
time they spent learning that lesson. At the end of the term, the final report
was exported to show students‟ completion of all lessons (%), the score they