A Standardized Test for Communicative Koine Greek (Needed Resources)

About the Series
This is part of an ongoing series where I discuss resources that should be developed to aid teachers and students in acquiring Koine Greek communicatively. In this installment, I address the need for a standardized test for Koine Greek.

I think that once the communicative Koine movement gets off the ground in colleges and seminaries, it would be helpful to develop a standardized test for Koine Greek. Why have a standardized test? Because it provides a standard, of course! This has several benefits. The most significant, I think, is that a standardized test sets the standard. That is, it expresses the consensus of language teachers as to what constitutes fluency, proficiency or competency in the language. Thus, simply having a standardized examination that tests for reading proficiency (not translating/parsing) or communicative competency in Greek establishes those skills as the goal.

Second, and closely related to the first, because standardized testing gives us empirical data, it produces accountability for teachers. It doesn’t allow us to hide behind the curtain anymore, claiming that we teach students to read Greek, when we’ve really just taught them to memorize a few charts, some English glosses, and the translation of a bunch of practice sentences that we’ll recycle on exams. If your students are making A’s and B’s in Greek I/II but can’t get even a low-passing grade on a standardized test, we know you’ve got some ‘splaining to do, Lucy!

Ricky finds out that Lucy has not been using a communicative pedagogy

Third, the empirical data provided by a standardized test can be used to demonstrate the effectiveness (or ineffectiveness) of the communicative method. The data can also help teachers determine whether a certain communicative strategy actually produces increases in proficiency; e.g. does explicit teaching of grammar actually translate into a better score on a reading comprehension test?

Fourth, the data from the test results give students a way to measure their progress in the language. Is the student at a point where she is ready to participate in exegetical classes (conducted entirely in Greek, of course)? What is the student’s reading level—is he ready to read Josephus, or does he need to stick with 1 John for now? Such a test would also be useful for placing students who wished to transfer from one school’s Greek program to another’s.

Fifth, a standardized language test is monolingual. This means that the same standardized test (e.g. in French) can be taken regardless of the student’s first language. Similarly, a standardized Greek test would allow a student in Hong Kong to measure himself against a student in Paris, London, or Dallas. No one would have an advantage, since the test is language-neutral.

An objection is often raised that standardized testing produces teachers who “teach to the test.” This may be a problem in subjects like history, literature, etc., but not as much in language learning. If the test is designed to measure proficiency, then “teaching to the test” is perfectly legitimate, since it will mean training students to be proficient readers and language users.

Foreign Language Tests – What are the Standards?
Let’s not reinvent the wheel. There are already a lot of well-established standardized language tests and certification systems out there for modern languages. Why not take our cue from them and see what they test for and how they test for it? For example, the Test of English as a Foreign Language (TOEFL) has 4 parts:

  1. Reading Comprehension – read 3-4 academic-level texts and answer questions about them.
  2. Listening Comprehension – listen to some brief lectures, discussions and conversations, then answer questions about them.
  3. Speaking (the written version of the TOEFL substitutes Structure and Written Expression) – talk about a certain topic, respond to questions, etc.
  4. Writing – write essays in response to reading and listening tasks

Prefers translation tests.

Please note: The entire test is in English. No, seriously, there are really some people who think that an English test should be in English. Obviously, they need to talk to some Greek profs and learn what it means to really know a language!

What Our TOGFL Should Look Like
Now, I’d like to see all of these sections on our test of Greek as a foreign language (TOGFL), but what if we set out sights really low and just started with a reading comprehension test? What would it look like?

  • How long would the test be? The TOEFL reading comprehension test allows 60-80 minutes.
  • How many passages? TOEFL requires the student to read 4 passages and answer around 50 questions altogether. See a sample here. Note that each passage is ca. 35 lines; so altogether the student would be required to read for comprehension around 140 lines of text, plus all the questions and answers. That sounds about right, although it’s more reading than most Greek students probably do in their first two years of Greek! I ran the text from the sample TOEFL question through a reading level scoring program and it came in at a 12th grade reading level. I think Josephus would be a good Greek analogue. In terms of length, 140 lines would be 7 pages of Greek in a Loeb edition. The point is not to test speed reading, but comprehension. Nevertheless, it requires a certain amount of speed. Don’t forget that you would have to go back and reread the text in order to answer some of the comprehension questions.
  • Would it test translation? All questions on the TOEFL are in English, and all possible answers (it’s a multiple choice test) are in English. Similarly, our test would be entirely in Greek. There would be no translation.
  • What would it test for? As the name implies, it would test for reading comprehension. Again, if you’ve taught Greek/Hebrew for too long to understand what reading comprehension means (as opposed to translation/analysis/parsing) :), please look at the sample from the TOEFL. It means that you would actually approach Greek as a means of communication.
  • How would the passages be chosen? This is the tough part of test-making! I think the best bet would be to have some easier passages along with some more difficult ones. Similarly, some of the comprehension questions could be easier and some could be harder. That would allow for a lot of gradation in scores. Probably a better strategy, though, would be to have multiple tests, each designed for a certain level—thus, a Level I test might cover Gospel of John type texts, while a Level 5 might cover something more along the lines of Plutarch. This is how many foreign language certification programs work. It allows the student to progress through a certification program level by level, with tests taken to verify that you have actually achieved that level of proficiency. Greek passages could also be composed de novo, though I can’t see any reason why that would be necessary except perhaps at the lowest levels, where it would be hard to find authentic Greek that’s easy enough.

What’s Available Now For Greek?
Here’s the problem: there’s simply nothing out there even remotely like this. There are, however, two National Greek Exams. Let’s examine both of them (see what I did there? pretty clever, eh?).

First is the American Classical League’s National Greek Exam. There are several exams, each pitched at a different level (introductory, beginning, intermediate, Attic prose, Attic tragedy, and Homeric). Here’s how the test works:

  • It’s all in English, except for the passage itself.
  • Vocabulary aids and grammatical aids are provided. I.e., they actually gloss and parse some of the words for you!
  • Length: 40 questions (multiple choice) on one passage of up to 23 lines. Yes, that appears to be the entire test, about 1/7 of the length of the TOEFL Reading Comprehension section.
  • Content: Most of the questions (20+) will deal with morphology, grammar, and syntax: parse this, analyze this construction, etc. Around 15 will deal with reading comprehension (though I’m not sure what they mean by that), 2 with English derivatives (!), 1 with meter, and 1-2 with questions about general Greek literature or culture. Again, ALL of the questions and answers are in English.

Hopefully you can see how this test is about as far as you can possibly get from testing proficiency in a language.

Your second option at the moment is the National Biblical Greek Exam, which is a service of Logos Bible Software company. Unlike the ACL test, the NBGE tests knowledge of “Biblical” Greek. It’s conveniently administered online. Like the ACL test, the focus is on morphology, grammar, and syntax. It’s one hour and has 50 multiple choice questions (randomly selected from a question database; they also have an adaptive version). Their questions ask you to a) translate single words, b) identify elements of a sentence (e.g. identify the subject or d.o.), c) choose the best word to plug into a sentence, or d) identify the function of a word (e.g. what type of infinitive, participle, etc).

Their home page claims that they have reading comprehension questions, but I did not see any in the sample questions I surveyed. I also found a lot of typos and unusual word choices in their questions. Like the ACL test, it’s all in English, and primarily tests knowledge of metalanguage and ability to translate into Biblish. Indeed, one of the problems with such a test is that it is not metalanguage-neutral, meaning that if you learned Greek grammar from a book which used terminology different from the NGBE’s then you’re out of luck.

I think it’s pretty clear that we don’t presently have what we need (and, I’d like to have a speaking/listening/writing component, too, ideally!). And, when we compare tests for modern languages to tests for ancient Greek, we see how shamefully low our standards are. What do you think? Would a TOGFL be something worth developing? Would traditional Greek programs be open to such a test, especially if it were limited solely to reading comprehension? Would Greek teachers be able to pass such a test? Sound off in the comments!

This entry was posted in Greek Pedagogy and tagged , , , , , , . Bookmark the permalink.

8 Responses to A Standardized Test for Communicative Koine Greek (Needed Resources)

  1. Stephen Hill says:

    I think you’re right that a universal assessment measure would be a good thing, but I’m not convinced that a standardized language test is the way to go. One problem with the TOEFL is that it’s more about test-taking skill than actual language competency, as evidenced by the fact that native English speakers don’t always get high scores on it. TOEFL preparation classes are taught rather differently than regular conversational ESL/EFL classes. One presupposes a certain level of English and then teaches you strategies to do well on the exam; the other focuses exclusively on your language competency and not on your test-taking skills.

    As an alternative, I think something like the European Common Framework of Reference for Languages might be helpful (http://en.wikipedia.org/wiki/Common_European_Framework_of_Reference_for_Languages). It’s not a standardized test, but rather a description of different levels of language ability. A1, for example, the lowest level, requires that the learner “Can understand and use familiar everyday expressions and very basic phrases aimed at the satisfaction of needs of a concrete type,” etc. Using something like that would make it clear very quickly that our standards of Greek competency are preposterously low.

    • Stephen, you are probably right that the TOEFL is not the greatest test, though I’m not sure the fact that native speakers can do poorly on it proves that. I know a lot of native speakers whose language skills are quite poor–speaking and listening, but especially writing and reading. On the other hand, I have taught a lot of international students who have met a certain TOEFL standard and are not really able to follow lectures or engage in discussion. We need to realize though, that any examination is going to require test-taking skills to some extent–I think that’s unavoidable, though it should be minimized as much as possible.
      I too like the ECFRL. I believe the Goethe tests that I linked to employ that Framework. Nevertheless, the TOEFL is a helpful example in that it gets people to think about what a monolingual, reading comprehension test would look like in terms of content and length. Once everyone agrees on fluency as the goal, and on basic testing standards (monolingual, metalanguage-neutral, comprehension-based), then we can argue about how best to measure fluency.

  2. I wish that a) this test already existed, and b) that I could find someone who would be interested in doing well on it.

    Part of the problem is that grammar/translation is so much easier and faster to acquire. I evangelize for reading comprehension and fluency whenever I talk to other Greek students, and most look as though they are restraining themselves from shushing me. “Shh! My Greek professor might hear you…!”

    I audited an ‘intermediate’ level Greek class this past summer (focused on exegesis, actually) and the professor had enough trouble getting the students to even read their Greek New Testaments at all because the students hadn’t really been convinced that it was worth it to struggle through when they had a perfectly good English translation sitting next to them. I almost threw my own copy at them!

    Back to the topic at hand, though. If you decide to put such a test together please let me know and I will provide whatever help I can.

    I have been putting together simple ‘Reading Aids’ that are designed for people who are at the present ‘intermediate’ level in their Greek studies but have little real reading comprehension. The idea is to give them something that will allow them to stay in the text as much as possible, while at the same time encouraging them to wean themselves off the aid itself so that they are eventually able to read the text with understanding on their own. While I’m not sure that I have really accomplished my goal of avoiding glosses and English as much as possible, they’re getting there. I hope to eventually cut English out of the aids entirely. In the meantime, this has worked well for me as I attempt to develop reading comprehension/fluency as an independent learner.

    Here’s a link: http://www.greekingout.com/2011/11/new-resources-greek-reading-aids/

    • Yes, I saw those, Refe. They look great! I think they’re exactly the kind of thing we need. I think your experience with the intermediate class is pretty typical–lots of linguistics (under the guise of “exegesis”), little reading.

  3. Here’s my attempt to do an easy matching component of such a test:

    1. ἡ πίναξ α. οἰκεῖ ἐν θαλάσσῃ.
    2. ὁ ποιήτης β. τάυτην οἱ ἵπποι ἐλαύνουσι.
    3. ἡ κλίμαξ γ. πεινῶν ἐσθίω τοῦτο.
    4. τὸ ἱμάτιον δ. ἄρτος τίθεται ἐπὶ ταύτην.
    5. στρατηγός ε. τοῦτο καλοῦμαι.
    6. ἰχθύς ζ. οὖτος ἄγει ἀνθρώπους κατὰ τὸν πόλεμον.
    7. τὸ σῖτον η. φορῶ τοῦτο.
    8. οἱ ῶμοι θ. δυο ἔχει ἄνθρωπος.
    9. ἡ ἅμαξα ι. ἀναβαίνομεν καὶ καταβαίνομεν.
    10 τὸ ὄνομα κ. ποίημα ποιεῖ.

  4. And here’s some true or false:

    ἀληθὲς ἣ ψευδές?
    1. ὁ ἀλἐκτωρ πετεινόν ἐστιν. ἀληθὲς ἣ ψευδές?
    2. τὸ πὺρ θερμόν ἐστιν. ἀληθὲς ἣ ψευδές?
    3. ὄρη μεγάλα ἐστιν. ἀληθὲς ἣ ψευδές?
    4. τἠν σελήνην οἱ ἄνθρωποι ἐσθίουσιν. ἀληθὲς ἣ ψευδές?
    5. κωφὸς λαλεῖν δύναται. ἀληθὲς ἣ ψευδές?
    6. ἡλος ὀξύς εστιν. ἀληθὲς ἣ ψευδές?
    7. ὁ ἡλιος τῆς νυκτὸς φλέγει. ἀληθὲς ἣ ψευδές?
    8. φῶς σκοτεινόν ἐστιν. ἀληθὲς ἣ ψευδές?
    9. τοῖς ἵπποις λούονται οἱ ἄνθρωποι. ἀληθὲς ἣ ψευδές?
    10. σφαῖρα ἔχει γωνίας. ἀληθὲς ἣ ψευδές?

  5. Dmitry Gusarov says:

    There is one another option – AS and A Level GCE Test. It’s UK version of Advanced Placement Test.

  6. The is problem is easily resolved on a technical with (1) The development of a test battery of questions. (2) Entry of the information LMS authoring software (e.g., Adobe Captivate), (3) hosting the work on a server with LMS delivery capabilities (e.g., Adobe, Moodle, etc.).

Leave a Reply to Mark Lightman Cancel reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s