(This posting includes a handout which you are welcome to use with your colleagues.)*

At first, all the teachers wanted to administer the oral test for placing students into one of the four levels of conversation classes. But that enthusiasm waned once they discovered what this commercially-made placement test would entail.

Two major problems with many speaking placement tests (commercial and in-house)

1)  The testing process in labor intensive. The scoring rubrics are onerous, ineffective and require time-consuming training.

2)  Rather than just focusing on the skills being developed in speaking/conversation classes, the interviewers have to evaluate several peripheral aspects of speaking at the same time.

A Speaking Placement-Testing Process That Addresses Those Problems.

In general, these two skills distinguish students at various levels:

Production: When talking to someone during a conversation/discussion:

• the types of topics they are able to talk about.
• the amount of details they can tell.
• the amount of hesitations they use when talking.

Reception: When listening to someone else talk during a conversation/discussion:

• the quality of understanding responses they can use when listening to someone else.
• the ability to ask follow-up questions.
• how well they can understand their interlocutor when s/he speaks at a faster speed.

How commercially-made speaking placement tests fail to address those essential characteristics.

Weakness 1) They evaluate only production skills, which are only half of a conversation. They don’t evaluate students’ “reception” ability to respond to someone with understanding responses, for example, “I see” ; “Really?” ; “Can you repeat that?” And they don’t assess the important skill of extending the conversation by asking follow-up questions or asking for more details.

Weakness 2) Their scoring rubrics require the (teachers) examiners to focus on unnecessary aspects of a conversation that do not help them accurately ascertain the students’ proper levels.

For example, here are the categories in a typical commercially-made placement-test rubric that illustrates those weaknesses

Weakness 1: The categories being evaluated:

First, in this rubric, the only useful category is “Details.” How many details a student can give will most often correlate closely with their clarity of pronunciation, vocabulary, grammar and fluidity. For example, except for a students with a speech impediment, students who are advanced enough to talk with many details will have had enough experience using English that they would have cleared up pronunciation problems that would have caused confusion. Any remaining pronunciation problems would be best dealt with one-on-one with a teacher/tutor rather than placing the student at a lower-level than his/her speaking skills. At the same time, we would not place (the rare) student who had very good pronunciation, grammar and vocabulary at a higher level if s/he could only speak with a few details or about simple topics. In sum, for the vast majority of students, their level of vocabulary and grammar are usually reflected in the quality of details that they can tell.

Second, notice that there is no assessment in the rubric of student reception skills (e.g. asking questions, giving understanding responses, asking for clarifications).

Weakness 2: To fill out the rubric, the evaluator needs to focus on various descriptions that are, in many ways, irrelevant for the purpose of determining a placement level.  Here are the descriptions for three of the top levels of the rubric mentioned above.

Now, imagine you only focused on the fifth column about “details” during the evaluation session. Any student who is at Level 5 with “additional details beyond the required” will NOT be at Level 3 with “little variety in structures.”  Even if s/he has a Level 4 Pronunciation “slight search for words,” that would not affect his/her placement into Level 5. In sum, on this rubric, the only column the evaluator needs to focus on for accurate placement is the fifth one “Details.”

A Rubric-Free Speaking Placement Test

For this, the evaluation is built into the “conversation” process. As soon as the students have reached their speaking/conversational-skill level, the test stops. Here is an overview. (I’ll add a link below to an attachment for the “overview” and “questions” on a Word document.  Feel free to download and customize them for your program.)

Speaking/Conversation Placement-Test Interview

Overview

Step 1) The evaluator begins by asking Questions 1-3 at a slow speed, and if possible, asks follow-up questions.

• If the student seems challenged by Step 1, the evaluator continues with Questions 4-7 and then stops the test. We can be confident that that student will benefit from Level 1.

Step 2) If the student can answer Questions 1-3 and the follow-up questions to those, the evaluator skips Questions 4-7. She jumps to Questions 8 & 9 (Level 2) and asks them a bit faster.

• If the student seems challenged by Step 2, the evaluator continues with Questions 10-12 and then stops the test. We can be confident that that student will benefit from Level 2.

Step 3) If the student can answer Questions 8 & 9 and the follow-up questions, the evaluator skips Questions 10-12. She jumps to Questions 13-14 (Level 3) and asks them at natural speed. Also, the roles are reversed with the student asking the evaluator questions about a topic.

This pattern continues for the rest of the evaluation process.

Speaking/Conversation Placement Test

Interviewer Questions

 Elementary Level 1 Questions: Ask at a slow speech. Ask any follow-up questions slowly.

1. Where do you live now?
2. Do you have any brothers and sisters?
3. What will you do tomorrow?

(If the student can answer #1 – 3 comfortably, skip to Question # 7. If #1 – 3 seem challenging, continue with # 4 – 6.)

4. Do you have a pet?
5. What is your favorite subject to study?
6. What is your favorite season?

(End interview if student continues to seem challenged to talk about Questions 4-6. Student will benefit from Elementary Level 1. If student seemed to comfortably answer Questions 4-6, go to Question #7.)

Low-intermediate Level 2 Questions: Ask at a bit faster speed. Ask follow-ups.

7. Why did you choose to come to (this town)? (Or … choose this program?)
8. Can you tell me about your family?

(If the student can answer #7 & 8 comfortably with some details, skip to Question # 13. If #7 & 8 are challenging for the student, continue with #9 & 10.)

9. What do you like to do during your free time?
10. What were you doing last year at this time?

(End interview if student continues to seem challenged with Questions 9 & 10. Student will benefit from Low-Intermediate Level 2. If student seemed to comfortably answer Questions 9 & 10, go to Question #11.)

Intermediate Level 3 Questions: Ask at natural speed. Ask follow-ups.

11. What are your plans for the future?
12. If I plan to visit your country, what advice would you give me?
13. Do you often use a cell phone or computer?

Change roles

“Now, we’ll have a conversation, and you will ask me questions like in a conversation. So I want you to choose a topic and ask me some questions.” (Note: This is an opportunity for a natural conversational exchange. I.e., the evaluator/teacher can also ask the student questions, ask for clarification and give understanding response.)

Topic choices on a piece of paper or on a board:

• travel        • exercise        • movies              • friends       • high school days

(End interview if answering #11-13 and if asking questions are challenging. The student will benefit from Intermediate Level 3.  If the student can answer #11-13 comfortably and ask questions naturally, go to Question #14.)

Advanced Level 4 Questions:  Ask at natural speed. Ask follow-ups. Introduce this next level of questions by saying, “I have three more questions I’d like to ask. Let’s have a conversation about these. You can answer them and ask me questions about them too.

14. Can you tell me about a problem in your country, or in the world, that you have read or heard about?
15. Tell me about someone whom you respect.
16. Are you worried about anything or feel stress about anything these days?

Option: Change roles. If the student hasn’t asked any questions to keep the conversation going during Questions 14-16, say, “Now I’d like you again to ask me some questions like in a conversation.  First choose a topic and then start the conversation.”

Topics:

• global warming               • money         • music        • happiness      • languages

(End of interview.  If answering Questions # 17-19 and if asking questions is challenging, the student will benefit from Advanced Level 4.)

Here is the handout of the above overview and interview questions that you can use with your staff: Speaking placement testing process rubric free

One simple criterion for evaluator/teacher to consider during the interviews: Can the student comfortable answer the questions with sufficient details, and can they ask conversational questions?

As we evaluate those two aspects, we can observe how smoothly the student speaks, how many hesitations there are, how clear the pronunciation is, how difficult it is to understand what they are trying to say. We do all those holistically. In other words, we don’t break down each aspect and evaluate them separately.

Then we decide which Speaking/Conversation class will best help them develop the skills needed to comfortably engage in conversations/discussions. To make this decision, it’s important for evaluators/teachers to have a norming session before conducting the interviews.

Most important step for preparing evaluators/teachers: Norming

The evaluators/teachers need to know when a student seems challenged trying to answer the questions at a certain level and feel confident that the student will benefit by studying at that level. They also need to know if a student seems to have the ability to try to answer questions at the next level during the interview. Those are the reasons for norming sessions. In other words, the evaluators need to clearly understand what the norms are for students in each level.

Unlike the norming session for placement testing that involves complex rubrics (see sample of commercially-made test above), this rubric-free norming process is not only relatively stress-free and less time-consuming, but it also carries great benefits for the program overall. Here is how it works:

1) Before the norming session, the “norming leader” gathers some interviewer/student recordings of previous-terms’ speaking placement testing sessions. The recordings should include students who were placed at different levels. (The first time we tried this rubric-free testing process twenty years ago, we asked current students at various levels in our program to volunteer “for fun” to pretend to be new students taking the placement test  while we recorded them. We used those recording for our first norming session.)

2) For the norming sessions, it’s very important that teachers with experience teaching the various levels attend, even if they will not be interviewers.

3) The norming session starts with all the interviewers and experienced teachers present. The norming leader plays the recording of the first interview. After hearing the first question and student’s response, the experienced teachers can tell their impressions of how comfortably the student seemed to answer. Everyone then continue to listen and discuss the complete first interview.  After listening to it, they come to an agreement about what level would be best for that student. Novice evaluators have many chances to clarify with the experienced teachers specifically what characteristics of the student stood out.

4) During actual placement test interviews, the interviews are recorded. After all the interviews are completed, each novice evaluator/teacher listens to the interviews with an experienced instructor and confirm the correct level for the student. Similarly, experienced interviewers have the option of getting advice from another teacher about the proper level of a student if they feel uncertain.

The many benefits from these norming sessions.

• It makes clear that the focus of the testing interviews is on skills: understanding questions, answering questions and asking questions.
• It becomes clear what the expectations (e.g. answering with details, clarity of ideas, smoothness when talking, and formulating questions) are for students entering a specific level. No rubric is necessary.
• Teachers become familiar with what types of students in general are in all the levels in a program and what skills are developed in those levels.

Post-testing follow- up

• In the programs in which I have been a coordinator, we had this policy. After two or three days in a speaking/conversation class in the term, if a NEW student feels s/he may have been misplaced, s/he can petition to be re-tested. To tell the truth, this rarely happens, but if someone does, it merely means that the teacher at the next level has an “evaluation conversation” with the student. That teacher can then explain to the student what important skills s/he will be developing in his/her present class, and thus, confirms that the student has been properly placed, or the teacher can move the student up a level.
• For future reference, after the second week and at the end of a term, teachers provide the placement test coordinator with their impression of the new students in the speaking classes.

In an interesting article in the International Journal of English Linguistics titled “Investigating the Validity of a University-Level ESL Speaking Placement Test via Mixed Methods Research,” the authors compared the effectiveness of a commercially-made vs. in-house placement test. They also included data on the impressions that students and teachers had concerning how effective the tests were in proper placement of the students. This well-researched article gives an interesting perspective on this topic.

David Kehe

*About the free-download materials. During my 40 years of teaching ESL, I have had many colleagues who were very generous with their time, advice and materials. These downloads are my way of paying it forward.