Assessing the Relationship between the Vocabulary Level Test ( VLT ) and Reading Comprehension

It has been considered crucial to have adequate vocabulary knowledge in order to support EFL reading comprehension. This research was done to find out if there was any relationship between the performance of Indonesian students in the Vocabulary Level Test (VLT) and their comprehension of their English reader textbook. This pilot research describes the implications for profiling books used in teaching English (EFL) especially for reading comprehension. It also provides information on how to measure the lexical level of textbooks used for reading or for other similar subjects that involve reading. A number of 30 undergraduate students from Universitas Putera Batam participated in the VLT. Then their English reader textbook, Mosaic 1, was profiled to measure its lexical vocabulary level. The results showed that only 1% of the participants had mastered the 2,000-word level, this meant that the vocabulary level of the textbook was still far above the vocabulary knowledge of the students. Thus, they had difficulties to comprehend their reading textbook. Some additional activities have therefore been recommended to be done before, during and after studying reading comprehension.


INTRODUCTION
Vocabulary in foreign language learning has attracted more and more interest from both researchers and language teachers because of its role as a fundamental and complex component in first or second language learning.Schmitt et al (2001) have stated that there are two parts of vocabulary knowledge: they are breadth or size, and depth of vocabulary knowledge.Breadth is regarded as the size of a person's vocabulary and depth is concerned with the level of understanding of various aspects of a given word (Teng, 2014).The importance of the breadth to language learners is when they have to comprehend meaning whilst reading a text.The number of words they know will influence their reading comprehension ability.A number of researchers have investigated the correlation between breadths of vocabulary and reading comprehension performances (Milton, 2009;Nation, 2001;Zhang & Anual, 2008).In most of these cases the researchers have shown that breadth correlates with reading comprehension performance.
The Vocabulary Level Test (VLT) is typically used to measure receptive vocabulary knowledge at the lexical level.The test refers to word class category levels, such as 2000, 3000, 5000, etc., academic words.Each level contains 30 items (Xing & Fulcher, 2007).The VLT is a validated and reliable test (Nation, 2001).Some studies have reported the reliability, e.g.0.94 or 0.91 (Liu, 2016;Read, 2009) and this test has been examined by other experts (Gyllstad, 2007;Teng, 2015;Webb & Sasao, 2013).As the measurement of VLT correlates with the level of reading comprehension, it is used to assess the reading texts chosen for undergraduate students and is the initial input to the teachers about the level of their students' ability before their lessons begins.
It is rather difficult to find results from VLTs done in Indonesia.The only relevant research in foreign language learning comes from research done in other countries: China, Singapore, and Sweden (Gyllstad, 2007;Liu, 2016;Teng, 2014Teng, , 2015;;Zhang & Anual, 2008).Therefore, it is a pioneering work to do research to measure the VLT of students and to find out their vocabulary level compared to the profile of their reading textbook in Indonesia.This kind of research can be considered as ground-breaking.Moreover, understanding the vocabulary skills of students studying EFL is expected to shine light on one of the base issues in reading comprehension.If there are too many 'difficult' words, beyond the vocabulary capability and capacity of the students, it might hinder their understanding of the reading text, which could be a point to be investigated.The initial information to get is: to know the level of students' vocabulary and to understand how far they can go with the reading text assigned.This leads to the main reason why this research is being done, to see the implications for pedagogy therein.

LITERATURE REVIEW
There are two parts in this section; the first is related to vocabulary and the second to the correlation between vocabulary size and reading comprehension.

English Vocabulary
English vocabulary plays both a significant and complex role, not only in learning a first language but also in second language acquisition.When it comes to what English vocabulary means, some specialist terminology needs to be introduced to achieve a better understanding of it, such as tokens, types, lemmas and word families.English vocabulary refers to the set of words known and used by a particular person within a language (Schmitt, 2013).Let's look at the following example of a sentence: 'The man takes the wallet'.In this sentence, there are five separate words.This means the sentence has five tokens.A token is a separate word in a text.When deciding the number of word types in the example above, it is the number of different words in the sentence that counts rather than the number of separate words.It is quite useful, as it helps learners figure out how many words there are in a reading text, how many words they have written for an essay and how large a corpus is.
From the stated example above, though it has five tokens there are only four types as the word 'the' occurs twice, but the other words in the sentence occur only once.When measuring the vocabulary size of a foreign language learner, the number of types of words is more significant than the number of tokens.This is due to the fact that more attention is paid to the number of types of words that a particular learner can produce, rather than their repetition (Moghadam et al, 2012).All of its inflections, a lemma refers to the head or root of a word, but the part of speech of this word must not be changed.For example, in English, the lemma of the word 'take' would include 'taking', 'takes', 'took' and 'taken', but 'taker' does not belong to its lemma because 'taker' is a noun, not a verb.A word family usually includes the base word and all of its inflections and derivatives regardless of its part of speech (Schmitt, 2013), so the word family of 'bring' includes 'bring ', 'brings', 'bringing', and 'brought' and of 'drink' includes 'drink' and 'drinks', both nouns and verbs, 'drinking', 'drank' and 'drunk' where they refer to imbibing 'drinks' but not to 'drunk' and 'drunken' where they refer to a person.

Vocabulary Size and Reading Comprehension Relationship
Reading is an important part of all, even the most elementary vocabulary programs (Schmitt, 2013).Vocabulary knowledge is quite necessary for language learners in reading comprehension as a learner needs to know most of the words in a running text in order to comprehend that text.Vocabulary is basic to the reader, yet inadequate vocabulary will undoubtedly influence second language learners' reading performance in a negative way.Even though the number or percentage has not been established, some research shows a significant relationship between vocabulary size and reading comprehension ability.Nation (2001) declares that learners with a vocabulary size of more than 3000 words can read with ease because in most situations it gives coverage of at least 95% of a running text.English language learners need 98% vocabulary mastery to acquire proper reading comprehension (Qian, 2002).Zhang and Anual (2008), through a research study with secondary school students in Singapore using VLT, found that the students' vocabulary knowledge at the 2000 and the 3000 word levels correlated with their reading comprehension ability.Some research has been done on the role of vocabulary size in reading comprehension, yet only a few studies correlated participants' performance on VLT with textbook profiling to investigate whether some pedagogy implications could be found.
Liu (2016) reveals the result of VLT and English textbook profiling with Chinese undergraduate students at Jiangsu University of Technology in PRC, China.Her research showed that 81% of the participants had mastered a 2000-word vocabulary, hence most of the students had no significant difficulties in understanding the texts used.
The aforementioned research findings indicate that second language learners' vocabulary size is highly correlated with their reading comprehension ability.In the case of a text that contains too many difficult words, beyond the language learners' vocabulary knowledge and ability, this could hinder their understanding of the text and hence negatively impact to their language learning processes.

METHODOLOGY
In this section, the research method, the participants, the materials used and the research procedure followed in this study are described.

Research Setting
This research was done with students of reading from the English Department at Putera Batam University in Batam Island, Indonesia.In the last four years, this English Department has looked at the availability of textbooks for reading.The students' vocabulary capability is a main part of their considerations.The reading comprehension classes use the text book chosen by the campus library and are seeking a new, recent edition book.Therefore, this research was done to find out the actual size of the vocabulary of the English students at particular lexical levels, using the VLT by Schmitt (2013) to see whether the current English textbook fits the current vocabulary level of the students.

Participants
The participants were 30 first-year English major students studying at Putera Batam University.The 17 male and 13 female participants ranged from 18 to 22 years of age.The Mosaic 1 Reading book, 4 th Edition, which was used, has been used as the reading text book for the past four years.

Research Materials
Two primary sources of materials were used in this research.The first was the English reading text book (see 3.2) used by the students.The second was the VLT.

Vocabulary Levels Test (VLT)
Schmitt et al ( 2001) developed the Vocabulary Levels Test which is designed to give an estimation of the vocabulary size of second language (L2) learners of general or academic English.The VLT was used because of its practicality.Regardless of the participants' learning background, the examiner can administer and interpret this test to show the general vocabulary knowledge of the participants.For example, knowledge of the most frequent 2000 words in English provides lexical resources that are required for basic everyday verbal communication.Schmitt et al (2001) have then indicated that the next 1000 words would provide additional material for spoken discourse, 3000 words is the threshold which should allow learners to begin to read real texts.Some researchers (e.g.Milton, 2013;Schmitt et al, 2001;Teng, 2015) indicate that knowledge of the most frequent 5000 words is needed to provide enough vocabulary to enable learners to read real texts.There might be some words still unknown, but that level of knowledge should allow learners to infer the meaning of any unknown words from their context, and to understand over 90% of the communicative content of a text.
In the case of the academic section had to be fitted somewhere between the frequency levels on the basis of the results from this particular group of examinees, the above mean scores would best place it between the 2000 and the 3000 word levels.
Referring to Schmitt et al (2001), the academic level of vocabulary sits in a broad range between the 2000-word level and the 10000-word level.Read (2009) also states it is useful to calculate a realistic minimum vocabulary size for students.Some scholars work on the assumption that in order to read independently, learners should know at least 95% of the running words in a text.Hsueh-Chao and Nation (2000) say that a reader needs to know up to 98% of the words in a text to achieve adequate comprehension.This means, that from each 50 words in a running text, the reader may only miss the meaning of one word as a maximum to be able to adequately comprehend the text.
The VLT derives from the fact that separate sections measure learners' knowledge of words from a number of distinct frequency levels (Schmitt et al, 2001).In this way, it facilitates a profile of a learner's vocabulary rather than a single-figure estimate of overall vocabulary size.The levels commonly addressed are the 2000 and the 3000 word levels.Schmitt et al (2001) state that at these levels of the test, the words are taken by stratified sampling from the reading, with reference to frequency data from the General Service List (GSL) (West, 1953).The only exception to this is the section of the VLT below 2000, where words from the 1000 level and the 2000 level are sampled at a 1:2 ratio.The first thousand words of the General Service List (GSL) are usually those with a frequency higher than 332 occurrences per 5 million words, plus months, days of the week, numbers, titles: Mr, Mrs, Miss, Ms, Mister, plus frequent greetings e.g.'Hello', 'Hi', etc.The words in the academic section are sampled from the University Word List.Reflecting the distribution of these word classes in English, the words from the stratified sample tended to fall into a 3 (noun): 2(verb): 1 (adjective) ratio.This ratio was maintained in the test, with each section containing three noun clusters, two verb clusters and one adjective cluster.Appendix A illustrates the format of the vocabulary skill test.
Some important aspects of the VLT level test have been proposed by Schmitt et al (2001).The first is about format options.Format options are words instead of definitions.The second is about definitions.The definitions are kept short; hence there is a minimum of reading, allowing for more items to be taken within a given period of time.The third is words.They are learned incrementally, and tests should aim to tap into partial lexical knowledge.The Levels Test was designed to do this.The option words in each cluster are chosen so that they have very different meanings.Therefore, even if students have only a minimal impression of a target word's meaning, they should be able to make the correct match.The fourth are clusters.They are designed to minimize aids to guessing.The target words are in alphabetical order, selected randomly and the definitions are in order of length.The fifth is that words used in the definitions are always more frequent than the target words.The sixth is that word counts from which the target words were sampled typically give base forms.The seventh is each level produces a reliability figure above 0.90.The last is that the target words in each cluster begin with different letters and do not have similar orthographic forms.
Considering the reliability and arguments from previous results, this research is designed to measure students' VLT results at the level of the 2000 to 3000 word classes.These two word classes are grouped as the high frequency range classes (Gyllstad, 2007).Assessing the participants' vocabulary through these two levels is considered the basic minimum requirement for participants' reading comprehension referring to the Mosaic reader at intermediate level.Moreover, as Schmitt et al (2001) state, if the test is shortened, the reliability index is likely to become lower as well.The test can be finished in the range of 15 to 60 minutes for a total of 4 levels (2K, 3K, 5K and 10K); therefore, for the 2 levels the time needed would be in the range of 15 to 30 minutes.Considering the vocabulary ability of the participants is still not high, these two levels can still fulfill the minimum measurement expectation for efficiency.Gyllstad (2007) states the VLT reliability coefficient in groups tested varied from 0.81 to 0.95.The VLT has been used in some research to measure the level of vocabulary correlated with the Vocabulary Learning Strategy (Gyllstad, 2007;Teng, 2015;Webb & Sasao, 2013).Considering the relevant research, when using the VLT, the researcher considers that X > 0.8 reliability is an acceptable level.
In Figure 1, at each level, there are sets of words with three definitions on the right side and six words on the left.In this test, a total of 5 sets or 30 words were tested at each level.Participants must choose the right word that goes with each meaning.The examinee must write the number of that word next to its meaning.An example is set out below: Then the score on each level of VLT would indicate the number of words known at that particular level, e.g.: if an examinee gets 10 words correct that means he knows 33% of the words at that level.Having 24 words (80%) or more correct at a level of VLT indicates mastery of that particular level (Laufer & Nation, 1999).

The Textbook Used
The textbook used in this study was Mosaic 1 by (Wegmann & Knezevic, 2001).This book has twelve chapters, and for this research, two chapters were chosen for the profiling.

Procedures
First, the students were asked to do the VLT managed by their English lecturer in class.The students were given 30 minutes to complete the entire test and once the test was done, they were scored by the researcher.Then the second step, the students' English textbook, Mosaic 1 was profiled using online software, a so-called VP-Compleat, which can be found from the website: www.lextutor.com,developed by Paul Nation.This website is well known around the world and quite popular among researchers in the field of language teaching-learning.This software was designed to analyze all the vocabulary in Text A and Text B (which were chosen randomly) and as a result the lexical level of all the vocabulary in the chapters chosen from the textbook was found.Finally, based on the results of the students' performance on the VLT and the profiling of the chapters from the textbook, discussions were held to determine 1. copy (_ _) _ end or highest point 2. event 3. motor (_ _) _this moves a car 4. pity 5. profit (_ _) _thing made to be like another 6.tip whether the new textbook fits the students' current vocabulary level along with some pedagogical implications associated with the choice.The assessment of the participants mastering of the VLT was based on the minimum of 24 words correct at that particular level (Laufer & Nation, 1999).Moreover, the mean score percentage shows the average level of the participants word knowledge in the test.

FINDINGS
This section describes the VLT performance and the reading text book used by these English major students.

Students' Performance on VLT
The participants two VLT levels at the 2000 and the 3000-word levels.From Table 1, it can be seen that the participants did not do very well on the VLT.The overall mastery at the 2K level was 10%, while only 6% reached the 3K level.The mean score at the 2K level was 60%, and at the 3K level it was 47%.

Profiling of the Textbook
The English textbook, Mosaic 1 has eight units.Two reading texts were scanned and profiled by using the software VP-Compleat on its website.Figure 1 below shows the lexical distribution of the participants' textbook using the Classic (GSL/AWL) profiling model.As shown in Figure 2, the vocabulary in the participants' textbook was mostly high-frequency words with 81% from the 1000 and 2000 words lists and only 4% from the 3000.The assessment was done with two readings from the text, Mosaic 1, as per the details in Appendix B.

DISCUSSION
Based on the participants' performance in the VLT, the results suggest that most of the participants had not achieved mastery at the most frequent 2000-word level.Table 1 shows that only 3 (10%) of the 30 participants were categorized as a master at the 2000-word level and only 2 (6%) at the 3000 word-level.Considering that about 81% of the textbook vocabulary consists of words at the K-1 and K-2 word levels, most of the students might find it difficult to understand the texts.Based on the results in Table 2, K-3 words up only 4% of the total vocabulary.Although the percentage of K-3 words is not great since participants did not do very well at the 2000 word level of the VLT, this might negatively impact their understanding of the textbook.According to Zhang and Anual (2008), K-2 and K-3 words are highly correlated to reading comprehension skills.Therefore, most of the participants were not likely to understand the textbook.The vocabulary level is correlated to the reading comprehension.Additionally, based on the results shown in Figure 1, words at the K-1, K-2 and K-3 word levels cover 85.3% of all words in the book which is close to the findings of Nation (2001).K-1, K-2 and K-3 words, in most situations, give coverage of 95% of running text.Those students with cover 95% of vocabulary are expected to comprehend the reading text than those below.
Although only a small number of samples of this research study restrict the generalizability of the findings, yet some pedagogical implications can still be drawn for tertiary students of the same proficiency level using similar textbooks.First, as most of the students had not mastered words at the K-2 word level of the VLT, it is suggested that teachers should still focus on the K-1 to K-2 level words by exposing their students to these words in their daily teaching.Besides, extra vocabulary training and practice should be provided to the group of students who scored less than 24 (80%), in this case the majority.Second, since the students' performance on the K-3 levels was not yet satisfactory, they might not be able to comprehend more complicated articles or texts.Textbooks with coverage of 85.3% K-1 to K-3 words can help them build up their vocabulary to the K-3 level.Therefore, the teachers should make good use of this textbook, putting emphasis on those frequently used words from the K-1 to K-3 levels, guiding the students with constant reviews and using those words in context until they fully grasp the meanings of them.Meanwhile, teachers should carefully select readings from various reading texts which fit the students' needs and give them to the students as extra reading, thus accommodating students' sensitivity to understand words at the 2000 and 3000 levels and even higher level such as 5000 to 10000 (Zhang & Anual, 2008).The after reading evaluation could also be the base for choosing the next reading text to be used to achieve higher level.

CONCLUSIONS
The present study showed that all but two of the students had not yet achieved mastery at the 2000 and 3000-word levels.As the performance of the students at these levels was not satisfactory, the results showed that these students needed more training and practice at the frequently used 2000 to 3000 word levels first before they moved up to the next level.The lexis of their textbook was normally distributed, with the 2000 common words accounting for about 81%.As the textbook was in excess of the students' current level they could not easily use that book for their English learning.These results showed that the students' VLT level was lower than that which had been found in previous studies.Liu (2016), for example, declares that 81% of the participants in his study had mastered the 2000-word level.
It is suggested that teachers should not only make good use of their textbook but also deliberately find extra material for enlarging the students' vocabulary, for example, they should find suitable material from other reading texts such as from the internet sources and find ways to add more common words to the vocabulary of their students.Vocabulary knowledge, which has proved to be a predictor of learners' language proficiency, plays a significant role in language learning.Language teachers should pay special attention to students' vocabulary learning, assisting them with appropriate and effective ways to learn and enlarge their vocabulary to at least meet the minimum requirement for reading at the level of their texts.To consider the vocabulary level of their students when teaching is crucial for successful teaching-learning of ESL.This research, by making an attempt to link learners' vocabulary size at particular levels to the profiling of the textbook, might serve as a good method for choosing teaching materials and designing a teaching syllabus for a second or foreign language context, and as input for teachers as to how to deal with reading texts.Although the sample is relatively small, which limits the generalizability of the findings, this study represents a group of undergraduates who are at the same English proficiency level in an Indonesian university context, where in fact their major is English.Based on the findings of this research, future research could concentrate on enlarging the vocabulary of participants at other levels and in other faculties and with various backgrounds.

Figure 1 .
Figure 1.A set of words from a Vocabulary Levels Test.

Figure 2 .
Figure 2. The lexical distribution of the participants' textbook, Mosaic 1.

Table 1 .
Results of students' performance on Vocabulary Level Test (VLT).
Some extra words are in the list to make the test more difficult.You do not have to find a meaning for those words.In the example above, those words are business, clock and shoe.If you have no idea about the meaning of a word, do not guess.But if you think you know the meaning, and then you should try to pick the right answer.