Non-Native English Speaker Readability Metric: Reading Speed and Comprehension

This paper presents an investigation to evaluate the reading speed and reading comprehension of non-native English speaking students by presenting a simple analytical model. For this purpose, various readability softwares were used to estimate the average grade level of the given texts. The relationship between the score obtained by the students and their reading speed under average grade level 9 and 14 using font size 12 and 14 is presented. The experimental results show that the reading speed and the score versus the students may be explained by a linear regression. Reading speed decreases as the score decreases. The students with a higher magnitude of reading speed scored better marks. More importantly, we find that the reading speed of our students is lower than the native English speakers. This approach of modeling the readability in linear form significantly simplifies the readability analysis.

• Readability
• Reading Speed
• Software Readability
• Flesch-Kincaid Grade Level
• Gunning-Fog Score
• Coleman-Liau Index
• Smog Index
• Automatic Readability Index
• Linear Regression

1. Introduction

Language is an essential tool to convert our ideas into words. Language is a form of communication which has two important parts: 1) The expressions or experiences which we want to express or share and 2) the suitable words that we use to convey these ideas or experiences. English as a Lingua Franca is an existing common language used for communication between speakers of different languages. When we learn English language we aim to develop all four major skills of the language: Listening, Speaking, Reading and Writing.

All four skills are either receptive or transmittal skills.

Reading is an active and diligent process. To establish a successful and productive communication between a reader and a writer, the prerequisite is the clarity of thoughts and correct usage of the words in the text. The relevance of writing is unattainable unless the reader understands and comprehends the text in its actual context. Reading involves the understanding of words, and their meanings to have an accurate interpretation of the text. When reader reads a text he or she should be able to connect the words with the given situation for better understanding.

Readability is defined as the ease with which a written text can be understood by a reader. The readability of a particular text depends on both: The complexity of its vocabulary and syntax.

To measure readability, various computer-based formulas have been proposed that include only two factors [1] - [13] : 1) the number of syllables or (letters) in a word and 2) the number of words in a sentence. The results generated by these various formulas are not accurate and not always in a good agreement.

In the present paper, we will examine the reading speed V_RS and reading comprehension text shown as a black text color on a plain background for non- native English speaking undergraduate students of our institution. Different presentations of text with different font sizes were assigned to the students. The obtained results are compared with those of similar investigations. Correlation between performance goals such as success rate, time to complete tasks versus students will be presented.

2. Methodology, Experimental Result & Discussion

In order to check the readability a planned survey was conducted in two phases. In the first phase, a drafted text was chosen with a few direct questions to evaluate the students’ reading speed. The difficulty level of the text was checked thoroughly using readability software [1] - [11] before it was given to the students. The students were asked to read the text carefully for a few minutes and write the answer in the space provided after each paragraph. The time they took to read the text was noted by the instructor. The purpose of this investigation was just to see whether the students had understood the text by considering the time it took to complete the tasks. In the second phase, after monitoring the reading speed, a reading comprehension was given to the students. An elaborated and structured text having variety of questions as multiple choice questions was given to the students. The purpose of this survey was to check the understanding of the students. The total time taken by the students to complete the text was also noted by the instructor.

The sample subjects of this study are 70 students both male and female, 60% male and 40% female. In term of language proficiency, we used a placement test with average level 9 and 14, calculated using the readability software indicated in Table 1 [1] - [11] . The selected subjects were employed and non-employed students

Table 1

Grade level and readability score [ 1 ] - [ 11 ]

	Text for comprehension	Text for reading speed
Readability formula	Grade	Grade
Flesch-Kincaid grade level	12.6	8.7
Gunning-Fog Score	16.2	11
Fry-Graph	14.1	8.9
Coleman-Liau index	16.8	8.3
Smog index	11.7	8.7
Automatic readability index	13.2	9.1
Average grade level	14.1	8.7
Character count	1.621	2.168
syllable	534	711
Word count	293	517
Sentence count	17	25
Characters per word	5.5	4.2
Syllabus per word	1.8	1.4
Words per sentence	17.2	20.7
Readability Formula	score	score
Flesch-Kincaid grade level	35.2	68.4

with different age ranges between 20 and 35. Two different reading tests were used to evaluate the students’ performance. Speed of reading test (V_RS) and comprehension test with different presentation of font sizes text assigned to the subjects divided into two groups.

For the speed of reading test, the average grade level and score of the text are 8.7 and 68.4, respectively. The details about the grade level texts are indicated in Table 1 and can be calculated using Fry graph software Figure 1. Scores calculated by readability software are different. The scores obtained by the students usually range between 0 and 100. A higher score indicates easier readability. Score readability in the range between [90 - 100] is too easily understood by an average 11 old year student, while score in the range between [60 - 70] can be easily understood by 13 - 15 old year student. Score in the range between [0 - 30] is best understood by university student. Based on the USA education system, a grade level is equivalent to the number of years of education a person has had. Scores over 22 should generally be taken to mean graduate level text.

The reading speed text contained 516 words. The subjects were allowed to do the test after they confirmed that they had understood about how to answer the questions. They were also told to be a little fast because of the time constraint.

Presenting text in darker colors and larger size with more pixel aids helped students in performing better. Black text on a plain background has been found to yield faster reading than black text on a medium textured background [14] , up to 32 percent faster than reading light text on a dark background [14] . It

seems that larger text with more pixel draws more attention than smaller ones. In this investigation, for rapid reading and understanding we present black text on white page with high contrast backgrounds. The font size is either 12 or 14 points.

Long sentences using unnecessary words are often used to express more than one idea in a sentence. Research indicates that brief and simple sentences are easily and readily understood than long sentences. Sentences over 20 words in length cause a loss in reading comprehension [15] . In addition to adequate contrast between text and its background, it is also recommended that the number of sentences in a paragraph should not exceed six as indicated in Table 1 which is provided in this investigation.

The comprehension reading text contained 295 words. The test was supposed to be consistent with the information items, to be tested with intermediate learners. The average grade level and score of reading comprehension text are 14.1 and 35.2, respectively. For the reading speed text, the average grade level and score are 9 and 68.4, respectively. The grade level and score are indicated in Table 1 and can be calculated using Raygor graph software Figure 2. The score is the number of items the subject completes accurately within the time limit indicated in Figures 3-8.

Data are represented using a plot called a scatter plot or x − y scatter diagram plot. During analysis we try to find the equation of a line that fits the data. This is called the regression line. Points are (x = student, y =reading speed or score) pairs can be plotted on the Cartesian coordinate system. From the study of correlation when the slope of the regression line is positive the value of y increases

as the value of x increases. This is called a positive correlation. When the slope of the regression line is negative the value of y decreases as x increases. The strength of these relationships is given by the correlation coefficient R which can be calculated. Regression analysis is used to predict the value of the variable based on the value of a second variable which is controlled by the experimenter. Results may be plotted on a scatter plot. Correlation is used to give information about the relationship between x and y. When the regression equation is calculated, the correlation results indicate the nature and the strength of the relationship. The correlation coefficient, R, indicates this relationship between x and y.

Values of R range from −1 to +1. A correlation coefficient of 0 means that there is no relationship. A value of −1 is a perfect negative coefficient and a correlation value of +1 indicates a perfect positive correlation. Another value of use in correlation analysis is the coefficient of determination which is represented as R², and varies between 0 and 1.

In Figure 3 and Figure 4, after collecting the data, the sample for V_RS and students’ score are sorted in ascending order. Then, the sorted data of V_RS and score (y-axis) versus the corresponding students (x-axis) are displayed as a scatter plot. The x-axis coordinates x₁ corresponds to the first student’s name point where x_n corresponds to the n^th student’s name point.

The variation in V_RS using font size 12 associated with the student scores versus student names are illustrated in Figure 3 and Figure 4, for two student groups, group1 and group 2, respectively. The experimental results show that the students with a higher magnitude of V_RS scored better marks. On the other hand, lower mark is observed when the V_RS decreases by approximately from 120 to 70 [words/min]. More importantly, each group shows a linear relationship between V_RS or score versus the reader. The average slopes obtained for the group 1 and 2 are given in Equation (1.1), (1.2) and (2.1), (2.2), respectively with a slight difference (a_rs1 = −0.015, a_score1 = −3.6759) and (a_rs2 = −0.088, a_score2 = −1.87). The test for the reading speed under the average level 9, shows approximately the same extracted parameters for group 1 (a_rs1 = −0.015, b_rs1 = 1.0034) and group 2 (a_rs2 = −0.0088, b_rs2 = 0.8427), respectively. Based on these extracted results, it seems that these groups perform quite similarly. On other hand, with higher average grade level 14, for the text comprehension, the extracted parameters for group 1 (a_score1 = −3.6759, b_score1 = 104.82) and for group 2 (a_score2 = −1.87, b_score1 = 78.911) show that the readability performance is quite different. Note that the x-axis is the name of the student and not the number of the student, and the graph was obtained after sorting the data from the smallest to the highest score.

Linear regression for reading speed and score obtained by the students for group 1 (Figure 3) are given in Equations (1.1) and (1.2), respectively.

y = − 0.015 x + 1.0034       ( readings   peed ) (1.1)

y = − 3.6759 x + 104.82       ( Score ) (1.2)

Linear regression for reading speed and score versus the students for group 2 (Figure 4), are given in Equations (2.1) and (2.2), respectively.

y = − 1.87 x + 0.78911       ( readings   peed ) (2.1)

y = − 0.088 x + 0.8427       ( Score ) (2.2)

R-squared is a statistical measure of how close the data are to the fitted regression line. The reading speed data indicates a very strong positive correlation (R² > 0.77) between V_RS and student. While, the score graph versus the student shows very weak correlation in the range of R 2 = [ 0.1055 − 0 , 1958 ] .

Figure 5 and Figure 6 show the score obtained by the student versus V_RS using font size text 12. When V_RS changes approximately from 30 to 130 [words/ min], the score increases linearly by approximately from 70% to 120%. The relationship between V_RS and its corresponding score may be described by linear regression given by the Equations (3.1) and (3.2), respectively. With an average slope and -intercept (a_SS1 = 0.0038, b_ss1 = 0.5931) and (a_SS2 = 0.0039, b_ss2 = 0.59097) for the group 1 and 2, respectively. The two samples appear radically the same.

Linear regression score versus reading speed for the group 1 (Figure 5) and group 2 (Figure 6) are given in Equations (3.1) and (3.2), respectively. The value of the coefficient of determination R² is less than 0.2. This indicates very weak correlation.

y = 0.0 38 0 x + 0. 5931 (3.1)

y = 0.0 39 x + 0. 5 0 97 (3.2)

For reading comprehension under different size font 12 and 14, the relationship between the students and their scores is illustrated in Figure 7 and Figure 8 for the group 1 and 2, respectively. This correlation can be described by linear relationship indicated by the equations. 4.1 and 4.2 for group 1 and 2, respectively. Note that the x-axis is the name of the student and not the number of the student, and the graph was obtained after sorting the

data from the lowest to the highest score. In this investigation, the subjects were told that there was no time limitation and they could do this part at their own pace, but they were not supposed to take a lot of time. The value of the coefficient of determination under this condition is larger than 0.84, which indicates very strong correlation.

y = 0.0 544 x + 1 . 1492 (4.1)

y = − 0.0 1 x + 1 .00 23 (4.2)

For the same group, the score obtained under font 12 and 14 can also be described by linear relationship. The average slopes for group 1 (a = −0.0544, Figure 7) and for group 2 (a = −0.01, Figure 8) are different. The minimum value scored for group 1, is about 30%, while for the group 2 the minimum value is 87%.

The survey indicated that more than 90% of the students do not read. In comparison with the native speaker, it seems that non English speaking students do not spend more time in reading. This could be explained by many factors such as, difficulty in reading, lack of motivation, environment, etc.

The coefficient of determination gives an indication of the contribution of the factor being studied in the regression analysis to the relationship between reading speed, score and student. In the case of reading speed data (Figure 3 and Figure 4) and score data (Figure 7, Figure 8), the value of the coefficient of determination, R² is larger than 0.77. This indicates a very strong correlation between the student and their performance. On other hand, the coefficient of determination, the score obtained by the student under controlled time shows very weak correlation (Figure 3 and Figure 4) with R² magnitude less than 0.2. This indicates that that the time is an important factor changing the magnitude of the coefficient of determination R² from 0.2 (Figure 3, Figure 4 score data) to more than 0.84 (Figure 7, Figure 8).

It is extremely important for a reader to get enough motivation to read and then using his skills to comprehend and understand the text. Motivation can be instilled through consistent encouragement by the educator and parents. But interest can only be developed through regular reading. The complexity of the text affects the reader’s motivation. In order to develop interest, it is essential that text should not be very complex to understand. If the text has difficult vocabulary and complex syntax then the reader loses interest in the very first phase of reading.

Reading skill, once developed, can be most easily maintained at a high level by the students. If the students have poor receptive skills then they will find reading monotonous. As a result they will never interpret the text in the right context. They will face multiple problems to comprehend and understand English especially in their academic courses where English Language is most commonly used. Due to lack of student’s proficiency in the language, to bridge the language gap, it requires a lot of collaborative efforts and coordination from the student and the teacher to overcome the difficulties in the process of learning. Higher complexity requires more learning and results in less efficient human performance.

In addition to the effect of the environment, the difference of reading speed and performance may be explained by individual differences and basic skills. We all differ in learning abilities and in task completion [16] .

Knowledge, experience, environment, and familiarity will help to remember and increase the readability speed. For example, it seems that most native English-speaking people remember English words easily than non-native speaker. Reading in one’s native language is easier than reading something in a foreign language. A non-native speaker who uses English or learns English as a second language faces many difficulties in the process of learning English. Reading doesn’t come easily. To enjoy reading and achieve proficiency, interest should be created in reading independently without facing any difficulties. Reading should be developed as a habit so that the reader enjoys what he reads.

Learning is the process of encoding in long-term memory information that is contained in short-term memory. It is a complex process that requires consistent efforts. Learning process is improved through repetition and deep analysis, only if the information being transferred from short-term memory, has structure and is meaningful and familiar. Based on above learning processes, it can be ascertained that high readability requires high skill. In case of sample students due to lack of adequate reading, their process of encoding is likely to be reduced which results in less efficient performance indicated by the slow reading speed and low score due to the low degree of familiarity and deep analysis.

The essence of skill is in the performance of actions characterized by consistency and economy of effort. Given enough time, people can improve their performance in almost any task. Usability goals versus performance in the form of measurable objectives must also be established. Performance goals such as the time it takes to complete tasks versus success rates must be defined. In performance, research indicates that a greater working memory is positively related to increase reading comprehension, reasoning skill, and learning technical information [17] . In addition, information stored within working memory is variously thought to last from 5 to 30 seconds, with estimates of working memory storage capacity size is about of 3 to 4 items [14] . Based on the above theories, it seems that low reading speed can be explained by the minimum information stored time with low capacity storage.

Lind et al., reported that there are two levels of information processing [18] . Both levels function simultaneously. The highest level used for reading and understanding, which consists of consciousness and working memory, performing reasoning and problem solving. This level is limited, slow, and sequential. In contrast to the lowest level, perceiving the physical form of information sensed, it processes familiar information rapidly. It seems that the process of both levels of our students is likely to be reduced.

3. Conclusion

Through our study we made an attempt to investigate the degree of readability of non-native English speakers. The study demonstrates that speed of reading, which is an indicator of assessing a non-native English speaker’s readability, is lower than that of a native English speaker. More importantly, the results show that the relationship between the reading speed and score versus the reader can be described by linear regression with very strong correlation. The magnitude of extracted parameters namely the slope and y-intercept maybe used as guideline to asses and evaluate the readability. In our future work, to clarify the weak correlation obtained in the present investigation, experiments will be carried under separated gender, controlled time and different grade level.

Authors’ Contribution

The co-authors (Dr. Muktisharma and Dr. Deepti Sharma) prepared the drafted text that was given to the students to carry out the survey with sharing contribution writing introduction, methodology and conclusion by all the authors. While the experimental results analysis, modelling, discussion, and full citations of references in the press manuscript are the contributions of the main author (Dr. Aissa Boudjella).

Acknowledgements

The first author would like to acknowledge the Wikipedia foundation to make available the free and open source software used in this investigation.

References

1. Flesch, R. (1948) A New Readability Yardstick. Journal of Applied Psychology, 32, 221-233. https://doi.org/10.1037/h0057532

2. Klare, G.R. (1963) The Measurement of Readability. University of Iowa Press, Ames, IA.

3. Chall, J.S. (1958) Readability: An Appraisal of Research and Application. Bureau of Educational Research, Ohio State University, Columbus, OH.

4. Farr, J.N., Jenkins, J.J. and Paterson, D.G. (1951) Simplification of the Flesch Reading Ease Formula. Journal of Applied Psychology, 35, 333-357.https://doi.org/10.1037/h0062427

5. Dale, E. and Chall, J.S. (1948) A Formula for Predicting Readability. Educational Research Bulletin, 27, 1-20, 37-54.

6. Gunning, R. (1952) The Technique of Clear Writing. McGraw-Hill, New York.

7. Coleman, M. and Liau, T.L. (1975) A Computer Readability Formula Designed for Machine Scoring. Journal of Applied Psychology, 60, 283-284.https://doi.org/10.1037/h0076540

8. Fry, E.B. (1963) Teaching Faster Reading. Cambridge University Press, London.

9. Fry E.B. ,et al. (1968)A Readability Formula That Saves Time Journal of Reading 11, 513-516.

10. McLaughlin G.H. ,et al. (1969)SMOG Grading: A New Readability Formula Journal of Reading 22, 639-664.

11. Senter, R.J. and Smith, E.A. (1967) Automated Readability Index. Wright-Patterson Air Force Base. p. 3. AMRL-TR-6620.

12. DuBay, W.H. (200) Smart Language: Readers, Readability, and the Grading of Text. Impact Information, Costa Mesa.

13. Caylor, J.S., Stitch, T.G., Fox, L.C. and Ford, J.P. (1973) Methodologies for Determining Reading Requirements of Military Occupational Specialties: Technical Report No. 73-5. Human Resources Research Organization, Alexander.

14. Koyani, S.J., Bailey, R.W. and Nall, J.R. (2004) Research-Based Web Design and Usability Guidelines. National Cancer Institute.

15. Galitz, W.O. (2002) The Essential Guide to User Interface Design an Introduction to GUI Design Principles and Techniques. John Wiley & Sons, Inc., Hoboken.

16. Nielsen, J. (2006) Ten Best Intranets of 2006. www.useit.com/Alertbox

17. Baddeley A. ,et al. (1992)Working Memory Science 255, 556-559.

18. Lind, N.E., Johnson, M. and Sandblad, B. (1992) The Art of the Obvious. CHI 95. Monterey, 3-7 May 1992.