- 1 -
CHAPTER 1: INTRODUCTION
1.1 Rationale for choosing this topic
English has already played a specially important role in the increasing development
of science, technology and international relations, which has resulted in the growing needs
for English language learning and teaching in many parts of the world. English has become a
compulsory subject in national education in many countries, among which Vietnam has
considered learning and teaching English as a major strategic tool to develop human
resources, as a way to keep up with other countries. Therefore, in any level of education,
from primary to university or postgraduate degree, learners must learn or want to learn
English as a compulsory subject or their target to access to information technology and to
find a good job. It is true that English teaching/ learning is essential for job training.
Fully aware of the importance of the English language, the University of
Technology, Ho Chi Minh National University has encouraged and required their students to
learn it as a compulsory subject during the first three academic years. Therefore, English has
been taught at the University of Technology since it was established, aiming at equipping the
students with an essential tool to go deeper into the world. However, to evaluate how
students acquire when they learn a foreign language, how well they use what they have been
taught and at which level of English they are standing is not paid much attention to. The
evaluation only counts for calculating the percentage of the number of students who pass
English tests, which ; therefore, doesn’t say anything about the validity, reliability or
discrimination of the tests. The results of English test are not successfully and completely
employed. In addition, during the time I have worked as a teacher of English at the
University of Technology, I have heard teachers and learners complaining about the English
achievement test in terms of its content, its structure. As a result, the English section has
decided to implement the renewal of the item bank in order to make it more valid and more
reliable.
Seeing the point, the author is encouraged to undertake this study entitled
“Evaluating the Reliability and Validity of an English Achievement Test for Third-year
Non- major students at the University of Technology, Ho Chi Minh National University
and some suggestions for changes” with the intention to find out how valid and reliable the

year students in terms of its validity and
reliability.
Then, quantitative methodology was used to collect and analyze data. After
collecting data, the author employed statistic software to interpret it and to present suggested
findings.
- 3 -
1.5 Research questions
This study is implemented to find answers to the following research questions:
1. Is the achievement test for third-year non-English major students at the
University of Technology, Ho Chi Minh National University reliable?
2. Is the achievement test for third-year non-English major students at the
University of Technology, Ho Chi Minh National University valid?
3. Is it necessary to make some changes to the test? If yes, what are the changes?
1.6 Design of study
The thesis is organized into four major chapters:
Chapter 1- Introduction presents such basic information as: the rationales, the aims, the
method, the research questions and the design of the study.
Chapter 2- Literature Review reviews theoretical backgrounds on evaluating a test, which
includes language testing, criteria of good tests and theoretical ideas on test reliability and
validity as well as achievement tests.
Chapter 3- The study is the main part of the thesis showing the context of the study and the
detailed results obtained from collected tests and findings in response to the research
questions.
Chapter 4- Conclusion offers conclusions and practical implications for the test
improvement. In this part, the author also proposes some suggestions for further research on
the topic.
- 4 -
CHAPTER 2: LITERATURE REVIEW
This chapter provides an overview of the theoretical background of the study. It
includes four main sections. Section 2.1 discusses the importance of testing in education.

themselves”.
Shortly speaking, it is undeniable that testing is an integrative part of teaching and it
can be separated from the program or from the course goals. Testing has both positive and
negative impact on teaching. Testing provides the teacher with information on how effective
his teaching has been, or the teacher can use tests to diagnose his own efforts as well as those
of his students.
Testing and Learning
Testing is a tool to “pinpoint strengths and weaknesses in the learned abilities of the
student” (Henning, 1987: 1). That is, through testing, learners can find out at which level
they are standing and what difficulties they have faced up with. As a result, they can adjust
their learning; explore more effective ways of learning. At the same time, the teacher can
rely on the result of tests to understand better learners’ ability and then can improve his
methods of teaching or revise knowledge. Thus, Read (1982: 2) said that “a test can help
both teachers and learners to clarify what the learners really need to know”. It is clear that
not only the teacher but also learners may achieve the benefits through testing.
To sum up, tests can benefit students, teachers and even administrators by confirming
progress that has been made and showing how they can best redirect their future efforts.
Tests can help. In addition, good tests can sustain or enhance class morale and aid learning.
2.2 Language Testing
Language testing is one of the forms of testing and it is also one form of
measurements. Its importance in English learning is reviewed as: “properly made English
tests can help create positive attitudes toward instruction by giving students a sense of
accomplishment and a feeling that the teacher’s evaluation of them matches what he has
taught them. Good English tests also help students learn the language by requiring them to
study hard, emphasizing course objectives, and showing them where they need to improve”
(Davies, 1996: 5).
Mc Namara (2000) presented three main roles of language testing, which is applied
not only in education but in other fields as well. Firstly, language testing is considered as a
key to succeed as language testing is a decisive way in recruitment. Secondly, it serves
educational goals. According to Mc Namara, tests are used to place learners in a suitable

Bachman and Palmer (1996) consider the criteria as qualities of test usefulness rather
than individual factors. Their idea of usefulness can be visually presented as in Figure 2.1:
Usefulness = reliability + validity +impact + authenticity +
- 7 -
interactiveness + practicality

Figure 2.1 Usefulness
(Bachman and Palmer, 1996)
Henning (1987) added more test characteristics and he summarized in the form of the
table called A checklist for Test Evaluation. The checklist is for rating of the adequacy of a
test for any given purpose.
Test
usefulness
Practicality
Reliability
Validity
Authenticity
Interactiveness

Impact
- 8 -
Table 2.1 A checklist for test evaluation
Name of test ________________________________
Purpose Intended ____________________________
Test characteristic Rating (0 = highly inadequate, 10 = highly adequate)
1. Validity _______________________
2. Difficulty _______________________
3. Reliability _______________________
4. Applicability _______________________
5. Relevance _______________________

be unreliable.
Henning (1987: 10) claimed that all tests are subject to inaccuracies. The ultimate
scores gained by the test-takers only provide approximate estimations of their true abilities.
While some measurement error is unavoidable, it is possible to quantify and greatly
minimize the presence of measurement error. A test on which the scores obtained are
generally similar when it is administered to the same students with the same ability, but at a
different time is said to be a reliable test. And since test reliability is related to test length, so
that the longer tests tend to be more reliable than shorter tests, knowledge of the importance
of the decision to be based on examination results can lead us to use tests with different
numbers of test items.
Test reliability is considered as “a quality of test score” by Bachman (1990: 24). He
makes a further point that if a student receives a low score on a test one day and high score
on the same test two days later, the test doesn’t yield consistent results, and the score cannot
be considered reliable indicator of the individual’s ability.
Reliability can also be viewed as an indicator of the absence of random error when
the test is administered. When random error is minimal, scores can be expected to be more
consistent from administration to administration.
Sources of Error
According to Bachman (1990, 165), there are four factors that affect language test
scores. The effects of these various factors on a test score can be illustrated as in Figure 2.2.
- 10 -
Figure 2.2 Factors that affect language test scores
We can infer from the figure that a score in a language test is indicated by
communicative language ability. Also, the language test is affected by factors other than
communicative language ability. They are:
 Test method facets: are systematic to the extent that is uniform from one test
administration to another (Appendix 1).
 Personal attributes: include individual characteristics such as cognitive style,
knowledge of particular content areas and group characteristics such as: sex, race,
and ethnic background. It is also systematic.

Thus, the higher the reliability coefficient is, the lower the standard error is. The
lower the standard error is, the more reliable the test scores are.
Types of reliability estimates
According to Henning (1987), there are several types of reliability estimates, each
influenced by different sources of measurement error, which may arise from bias of item
selection, from bias due to time of testing or from examiner bias. These three major sources
of bias may be addressed by corresponding methods of reliability estimate:
a. Selection of specific items:
- Parallel Form Reliability
- Internal Consistency Reliability estimates (Split Half Reliability)
- Rational equivalence
b. Time of testing:
- Test-retest Method
c. Examiner bias
- Inter-rater Reliability
- 12 -
Parallel form reliability: indicates how consistent test scores are likely to be if a person
takes two or more forms of a test. A high parallel form reliability coefficient indicates that
the different forms of the test are very similar which means that it makes virtually no
difference which version of the test a person takes. On the other hand, a low parallel form
reliability coefficient suggests that the different forms are probably not comparable; they
may be measuring different things and therefore, cannot be used interchangeably.
A formular for this method may be expressed as follows:
(2). R
tt
= r
A,B
(Henning, 1987)
(In which, R
tt

: the correlation of the
scores from one half of the test with those from the other half).
Rational equivalence is another method which provides us with coefficient of internal
consistency without having to compute reliability estimates for every possible split half
combination. This method focuses on the degree to which the individual items are correlated
with each other.
(4).
( )








−
−
=
∑
2
22
1
t
tt
tt
s
ss
n
n

,
,
11
−+
=
(In which R
tt:
Inter-rater reliability, N: the number of raters whose combined estimates form
the final mark for the examinees, r
A, B
: the correlation between the raters, or the average
correlation among the raters if there are more than two).
To improve the reliability of a test is to become aware of test characteristics that may
affect reliability. Among these characteristics are test difficulty, discriminability, item
quality, etc.
Test difficulty: is calculated by the following formular:
(7)
(In which, p: difficulty, Cr: sum of correct responses, N: number of examinees)
According to Heaton (1988: 175), the scale for the test difficulty is as follows:
p: 0.81-1: very easy (the percentage of correct responses is 81%-100%)
p: 0.61-0.8: easy (the percentage of correct responses is 61%-80%)
p: 0.41-0.6: acceptable (the percentage of correct responses is 41%-60%)
p: 0.21-0.4: difficult (the percentage of correct responses is 21%-40%)
p: 0-0.2: very difficult (the percentage of correct responses is 0-20%)
Discriminability
The formula for item discriminability is given as follows:
- 14 -
(8)
(In which, D: discriminability, Hc: number of correct responses in the high group, Lc:
number of correct responses in the low group).

the basis of the aim of the test. For instance, a listening test with written multiple-choice
options may lack validity if the printed choices are so difficult to read that the exam actually
measures reading comprehension as much as it does listening comprehension.
- 15 -
Validity is classified into such subtypes as:
Content validity
This is a non-statistical type of validity that involves “the systematic examination of
the test content to determine whether it covers a representative sample of the behavior
domain to be measured” (Anastasi & Urbina, 1997: 114). A test has content validity built
into it by careful selection of which items to include. Items are chosen so that they comply
with the test specification which is drawn up through a thorough examination of the subject
domain. Content validity is very important in evaluating the validity of the test in terms of
that “the greater a test’s content validity, the more likely it is to be an accurate measure of
what is supposed to measure” (Hughes, 1989: 22).
Construct validity
A test has construct validity if it demonstrates an association between the test scores
and the prediction of a theoretical trait. Intelligence tests are one example of measurement
instruments that should have construct validity. Construct validity is viewed from a purely
statistical perspective in much of the recent American literature Bachman and Palmer
(1981a). It is seen principle as a matter of the posterior statistical validation of whether a test
has measured a construct that has a reality independence of other constructs.
To understand whether a piece of research has construct validity, three steps should
be followed. First, the theoretical relationships must be specified. Second, the empirical
relationships between the measures of the concepts must be examined. Third, the
empirical .evidence must be interpreted in terms of how it clarifies the construct validity of
the particular measure being tested (Carmines & Zeller, 1991: 23).
Face validity
A test is said to have face validity if it looks as if it measures what it is supposed to
measure. Anastasi (1982: 136) pointed out that face validity is not validity in technical sense;
it refers, not to what the test actually measures, but to what it appears superficially measure.