VIETNAM NATIONAL UNIVERSITY, HANOI
UNIVERSITY OF LANGUAGES AND INTERNATIONAL STUDIES
FACULTY OF POSTGRADUATE STUDIES
------------------------

NGUYỄN THỊ OANH

A STUDY ON THE CONSTRUCTION OF A MARKING SCHEME FOR
END-OF-SEMESTER ENGLISH ORAL TESTS FOR 10th GRADE
STUDENTS AT CAM GIANG HIGH SCHOOL IN HAI DUONG.
Nghiên cứu xây dựng bảng đánh giá cho bài kiểm tra nói cuối kỳ môn
Tiếng Anh cho học sinh lớp 10 trường THPT Cẩm Giàng, Hải Dương.

M.A MINOR PROGRAMME THESIS
Field: English Teaching Methodology
Code: 60140111

Hanoi – 2017


VIETNAM NATIONAL UNIVERSITY, HANOI
UNIVERSITY OF LANGUAGES AND INTERNATIONAL STUDIES
FACULTY OF POSTGRADUATE STUDIES
------------------------

NGUYỄN THỊ OANH

A STUDY ON THE CONSTRUCTION OF A MARKING SCHEME FOR
END-OF-SEMESTER ENGLISH ORAL TESTS FOR 10TH GRADE
STUDENTS AT CAM GIANG HIGH SCHOOL IN HAI DUONG.
Nghiên cứu xây dựng bảng đánh giá cho bài kiểm tra nói cuối kỳ môn

constructive feedback throughout this thesis. She has guided me in searching for
relevant theory to my thesis and has also assisted in collecting data. Consequently, I
have learnt a lot about the assessment of English oral tests and construction of a
marking scheme for oral tests.
Secondly, I would like to thank Ms. Bui Thien Sao, an expert of the Center for
Language Testing and Assessment of the University of Languages and International
Studies for her invaluable assistance during the research time.
Thirdly, this thesis would not have been possible without the enthusiastic
participation of six English teachers and 150 students at Cam Giang High School
where the research was carried out.
Lastly, I would like to thank my family whose love and support help me complete
this thesis.

ii


ABSTRACT
This thesis was conducted at Cam Giang High School. The participants consisted of
150 tenth-grade students and six English teachers at Cam Giang High School. Two
experts in the Center for Language Testing and Assessment of the University of
Languages and International Studies also participated in this study. The research
aims at constructing a marking scheme for the end-of-semester English oral test of
tenth-grade students. Constructing a marking scheme for oral tests is a complex
process. In this paper, a combination of three methods: intuitive, qualitative and
quantitative are employed by the researcher. The research started with writing the
draft of marking scheme. Then, the researcher obtained the judgments from experts
and other teachers. Next, the marking scheme was piloted with 150 tenth-grade
students. Based on analyzing the students‟ scores of the oral tests, the researcher
examined how well the marking scheme works. The findings revealed that the
marking scheme can be used by the teachers effectively although there is still a need

1.1.1 Communicative competence in the CEFR ..........................................................6
1.2 What is speaking?..................................................................................................9
1.2.1 Assessing speaking ...........................................................................................10
1.3 Marking scheme ..................................................................................................10
1.3.1 What is a marking scheme? ..............................................................................10
1.3.2 Approach to construct a marking scheme ........................................................12
1.3.3 Steps to construct a marking scheme ...............................................................13
1.3.4 Types of marking schemes ................................................................................15
1.3.5 Structure of a marking scheme .........................................................................17
1.3.6 Available speaking marking schemes ...............................................................18
1.3.7 Previous studies ................................................................................................20

iv


CHAPTER 2: METHODOLOGY ............................................................................22
2.1 Setting of the study..............................................................................................22
2.2 Participants ..........................................................................................................23
2.3 Description of the end-of- semester oral test ......................................................24
2.4 Research design ...................................................................................................24
2.4.1 Rationale for using a multiple-method approach .............................................24
2.4.2 Research procedure ..........................................................................................25
2.5 Data collection instruments .................................................................................28
2.5.1 The interview with the teachers........................................................................28
2.5.2 Sample oral test ................................................................................................29
2.5.3 Data collection procedure ................................................................................30
2.6 Data analysis method ..........................................................................................30
2.6.1 Descriptive technique .......................................................................................30
2.6.2 Statistical technique .........................................................................................30
2.6.3 Data analysis procedure ..................................................................................31

Table 2: Pearson Correlation on vocabulary ............................................................. 39
Table 3: Pearson Correlation on pronunciation ........................................................ 39
Table 4: Pearson Correlation on fluency and coherence ........................................... 40
Table 5: Pearson Correlation on Sum ....................................................................... 40

vii


PART A: INTRODUCTION
This part is offered to introduce the rationale of the study, the problem to be
addressed in the study, the aims and objectives of the study, and the research
questions to be answered. It will also present the scope of the study, significance of
the study, an overview of the employed methods and the design of the study.
1. Rationale
English is the most popular official language in the world and the primary language
of global trade and commerce. It is an international means of interaction and
communication in almost all countries. Proficiency in English is seen as a desirable
goal for a lot of people in the world. In many countries including Vietnam, English
is taught as a compulsory subject at school and it is included in many exams. Of all
four skills: reading, speaking, listening and writing; speaking is generally thought to
be the most important due to the fact that a lot of learners have spent years studying
English but they still cannot speak it fluently. In order to speak a foreign language,
the learners must master the sound system of the language, use appropriate
vocabulary and be able to put words together intelligibly with minimal hesitation.
Moreover, they also need to understand what is being said to them and respond
appropriately to maintain good–natured relation to achieve communication goal
(Luoma, 2004). Although students‟ speaking skills are often practiced and
developed, it is not extensively assessed. Comparing to other skills, speaking is the
most difficult language skill to assess the reliability. The student‟ speaking ability is
usually judged during a face-to-face interaction, in real time between the teacher

attempts have been made to do a study titled “A study on the construction of a
marking scheme for end-of-semester English oral tests for 10th grade students at
Cam Giang High School in Hai Duong”. Hopefully, the study, to which I will
devote all my efforts, will make a contribution to the English teaching and learning
of the teachers and students at Cam Giang High School.
2. Aims and objectives of the study
The study is aimed at constructing a marking scheme for end-of-semester English
oral tests for 10th grade students at Cam Giang High School in Hai Duong with
hope to find an effective tool for assessing students‟ English speaking competence.

2


In order to achieve this aim, the following objectives will be addressed in the study:
+ To find out the marking criteria and descriptors of the target marking scheme
+ To investigate the feasibility of the proposed marking scheme
3. Research question
The study aims at answering the following questions:
1. What are the marking criteria of the marking scheme for end-of-semester
English oral test for 10th grade students at Cam Giang High School?
2. What are the descriptors for each criterion of the marking scheme for end-of
-semester English oral test for 10th grade students at Cam Giang High School?
4. Scope of the study
The scope of the research has been made quite clear from the title: “A study on the
construction of a marking scheme for end-of-semester English oral tests for 10th
grade students at Cam Giang High School in Hai Duong”.
Firstly, the study only focuses on oral testing, namely the construction of a marking
scheme. To be more specific, it refers to the process to find out the marking criteria
and descriptors for a marking scheme which helps teachers at Cam Giang High
School assess students‟ oral test. This will help teachers at Cam Giang High School

consists of the context of the study, the information of the subjects, instruments of
data collection, procedures of data collection and methods of data analysis.
Chapter 3 (Results and Discussion) reports the statistical results and the analysis of
the data.
Part III (Conclusion) closes the study by summarizing the whole study with
concluding remarks and offering some limitations and suggestions for further
studies.

4


PART B: DEVELOPMENT
CHAPTER 1: LITERATURE REVIEW
This chapter provides an overview of the literature on the field of communicative
competence, assessing speaking and marking scheme. It discusses communicative
competence, communicative competence in the CEFR, definitions of speaking,
assessing speaking, definitions of marking scheme, approaches to construct a
marking scheme, types of marking scheme and the structure of a marking scheme. A
review of the available marking schemes and previous studies is also presented.
1.1 Communicative competence
During the past few years, the concept of communicative competence has been
discussed

and

redefined by many researchers

and authors. The

term

organizational knowledge (grammatical and textual knowledge) and pragmatic
knowledge (illocutionary competence and sociolinguistic competence). Strategic
competence involves the ability to assess whether the situation is practicable and to
plan for the next movement (Luoma, 2004). Bachman and Palmer (1996) claimed
that their model can be used as a checklist for developing language tests.
1.1.1 Communicative competence in the CEFR
The CEFR is the Common European Framework for language learning, teaching
and assessment, which focuses on the nature of language use and the language user
and the implications for learning and teaching (Council of Europe, 2001). The
Common European Framework defines levels of proficiency which allow learners‟
progress to be measured at each stage of learning and provides criteria for assessing
four English skills: reading, speaking, listening and writing. Of all skills, speaking
is very important and within communication the learners can learn and apply
various skills. The Framework consist of three proficiency levels which are referred
to respectively as Basic User: A1, A2, Independent User: B1, B2, and Proficiency
User: C1, C2 (Council of Europe, 2001)
Communicative competence in the narrower sense consists of three components,
linguistic competences, sociolinguistic competence and pragmatic competences
(Council of Europe, 2001, p. 108). In the model of communicative competence of
Canale and Swain (1980), there are also three components, grammatical
competence, sociolinguistic competence and strategic competence. Although the
CEFR and Canale and Swain label these categories slightly differently, they use
similar categories to describe competences. In the following paper, the categories
from the CEFR are particular relevance for the assessment of speaking will be
discussed.

6


Linguistic competence is considered the core of the model of communicative



Sociolinguistic competence is “concerned with the knowledge and skill required to
deal with the social dimension of language use” (Council of Europe, 2001, p. 118).
Sociolinguistic competences include linguistic markers of social relations,
politeness conventions, expressions of folk-wisdom, register differences, and dialect
and accent (Council of Europe, 2001, p. 119). Canale and Swain defined
sociolinguistic competence as “the ability to communicate appropriately in a variety
of contexts, this includes both verbal and non-verbal communication” (Canale and
Swain, 1980).
Linguistic markers of social relations vary from language to language. They include
use of choice of greeting, use and choice of address forms, conventions for
turn-taking in conversations and the use of choice of expletive (Council of Europe,
2001, p. 119).
Politeness conventions vary from one culture to another and are a frequent source of
inter-ethnic misunderstanding when polite expressions are literally interpreted
(Council of Europe, 2001, p. 119).
Expressions of folk wisdom are fixed formulas about daily life, often used in
newspaper headlines. The expressions include proverbs, idioms, and expressions for
beliefs, attitudes and values and are often used in graffiti and on T-shirt slogans
(Council of Europe, 2001, p. 120).
Register differences refer to “systematic differences between varieties of language
used in different contexts” (Council of Europe, 2001, p. 120). Register differences
express differences in level of formality: frozen, formal, neutral, informal, familiar
and intimate (Council of Europe, 2001, p 120).
Dialects and accents perform people‟s origin, sociolinguistic competences include the
ability to distinguish between various social classes, regional provenances, national
origins, ethnicities and occupational groups (Council of Europe, 2001, p 121).
Pragmatic competences include discourse competence and functional competence.
“Discourse competence is the ability of a user/ learner to arrange sentences in

to express oneself. In the production stage, the learner uses the knowledge about
pronunciation and communication strategies to produce language (Bygate, 1987).
The CEFR has distinguished clearly between interaction (spontaneous) and
production (prepared) of language. Interaction activities are mainly spontaneous and

9


are carried out throughout conversation and more or less informal discussion.
Production activities are mainly prepared and rehearsed in advance (Council of
Europe, 2001, p. 178).
1.2.1 Assessing speaking
The categories for oral assessent is enormous and deciding which criteria to use for
assessment is a relatively dificult work. Assessor should determine the most
appropriate criteria that could not only be used to assess students‟ speaking, but also
be relevant to the objective of the course/ lesson, etc (Knight , 1992).
The CEFR classifies 12 qualitative categories relevant to oral assessment. The
CEFR has also developed illustrative scales for assessment and each scale describes
the level of proficiency. There are a number of categories relevant to assess
speaking such as turn-taking strategies, co-operating strategies, asking for
clarification, fluency, flexibility, coherence, thematic development, precision,
sociolinguistic competence, general range, vocabulary range, grammatical accuracy,
vocabulary control and phonological control (Council of Europe, 2001). However, it
is impossible to assess all criteria simultaneously. The assessers need to make
choices for each assessment situation and select only several criteria that are
relevant to the particular context. Choosing no more than 4 or 5 criteria in each
testing situation guarantees feasibility as well as reliability of the assessment
( Council of Europe, 2001, pp: 192-193).
1.3 Marking scheme
1.3.1 What is a marking scheme?

not in themselves test instruments and need to be used in
conjunction with tests appropriate to the population and test
purpose. Raters or judges are normally trained in the use of
proficiency scales so as to ensure the measure’s reliability (David
et all, 1999, pp: 153-154)
From above definition, it can be seen that a rating scale includes both the fields to
be assessed (construct) and the alignment between examiner‟s performance and the
predetermined levels of behavior descriptions. Thus, it is important to consider two
above components when constructing a rating scale. In addition, for different
purposes the construction of scale can be different. Alderson (1991) and Pollitt and
Murray (1996) classify different purposes that rating scales could serve:
user-oriented, constructor-oriented and assessor-oriented. A user-oriented scale is
“designed to communicate information about typical of likely test taker behaviors at
a given level” (Taylor, 2011, p. 190). Constructor-oriented scale “guides test writer
in their choice of tasks to include in a test” (Taylor, 2011, p. 190). Assessor oriented
scale “guides the rating process, focusing on the quality of performances expected”
(Taylor, 2011, p. 190). Because one rating scale is rarely appropriate for all the
purposes above, the purpose compatible with the scale must be prioritized to
measure the sample language elicited from learner‟s performance in particular
testing situation (Nakatsuhara, 2007).
In the present paper, the researcher focuses on constructing an assessor-oriented

11


speaking marking scheme which teachers at Cam Giang high school can utilize to
assess students‟ oral tests.
1.3.2 Approach to construct a marking scheme
Marking scheme construction is recognized to be a complex process (Brindley,
1998, Fulcher, 2003, North, 2000). Traditionally, the design and construction of

There are a number of approaches to marking scheme construction. The best
methods for marking scheme construction are said to combine all three approaches
including intuitive, quantitative and qualitative approaches in “a complementary
and cumulative process” (Council of Europe 2001, 207). Therefore, in constructing
a marking scheme for assessing student s‟ oral test at Cam Giang High School, a
combination of intuitive, qualitative and quantitative approaches was used.
1.3.3 Steps to construct a marking scheme
According to Mertler (2001) building a marking scheme includes following seven steps:
Step 1: Re-examine the learning objectives to be addressed by the task. This
helps to match the teacher‟s scoring guide with objectives and actual
instruction.
Step 2: Identify specific observable attributes that the students demonstrate in
their product, process, or performance.
Step 3: Brainstorm characteristics that describe each attribute. Identify ways
to describe above average, average, and below average performance for
each observable attribute identified in Step 2.
Step 4a: For holistic rubrics, write thorough narrative descriptions for
excellent work and poor work incorporating each attribute into the
description. Describe the highest and lowest levels of performance
combining the descriptors for all attributes.
Step 4b: For analytic rubrics, write thorough narrative descriptions for
excellent work and poor work for each individual attribute. Describe the
highest and lowest levels of performance using the descriptors for each
attribute separately.
Step 5a: For holistic rubrics, complete the rubric by describing other levels on
the continuum that ranges from excellent to poor work for the collective
attributes. Write descriptions for all inter mediate levels of performance.

13



14


+ Propose the criteria
+ Determine the number of scales
+Develop the descriptors
+Discuss with other teachers and revise
+Trail the scale
+ Stabilize the scale -> Version 1
Phase 2: Qualitative
External expert reviewing the existing scale
+ Rank the descriptors in order of difficulty
+ Trail the scale
+ Discuss and revise the scale
+ Stabilize the scale -> Version 2
Phase 3: Quantitative
Standard –setting phase
+ Trail the raters
+ Trail the scale
+ Analyze the scores
+ Stabilize the scale -> version 3

(Taylor, 2011, p. 195)

In this research, the researcher follows a three - phase process suggested by Taylor
(2011) for some reasons. Firstly, according to the CEFR (2001, p. 207) the best
methods for rating scale development are said to take advantage of the strengths of
a range of intuitive, quantitative and qualitative approaches. The best scale, the
CEFR (2001, p. 207) suggests, combine all three approaches in a “complementary

Each criterion is assessed separately and using different descriptive marking scheme
(Taylor, 2011, p 179). Analytic marking scheme results initially in many scores,
followed by a summary total score representing an assessment on a
multidimensional level (Mertler, 2001). Using an analytic marking scheme helps to
focus rater judgments more narrowly which contribute to rater agreement and rating
reliability (Weir, 1990). Scoring tends to be more consistent across students and
grades. It provides students meaningful and specific feedback on area of strength
and weakness (Nitko, 2001). Besides these advantages, using analytic marking
schemes also has some disadvantages. The use of an analytic marking scheme can

16