Constanze,
We have captured foreign language proficiency a number of different ways over the years (assessments, can-do scales, Likert-type items, BARS, etc.). We were even involved in a project where a sample of English speech at each proficiency level was provided as an anchors to help participants make the self-rating. I think you should consider linking the self-assessment of English proficiency to an actual description of proficiency from a functional perspective (what the individual can do), not just a general item referring to "language proficiency" with a Likert scale. There are a number of easy to use can-do type scales out there.
Language proficiency in Europe is typically described using the Common European Framework of Reference (CEFR) which scales proficiency into levels (A1, A2, B1, etc.), and most students who have studied language in German should be familiar with the CEFR. CEFR contains a number of rating scales (e.g., can-do statements) for proficiency in different modalities (e.g., spoken production) that can be applied to any language and are available in the CEFR book. We have used these scales in a study in the Netherlands (presented in Germany and published in a volume edited by Erwin Tschirner). See below. Also, many of the Germans may have tested in English so they may have an actual CEFR score to report.
We have also used can-do statements based on the American Council on Teaching of Foreign Languages (ACTFL) scale and the ILR scale used by the US Government. The ILR can-do statements are available on the ILR website. So, there are many choices. We did a meta-analysis looking at proficiency self-ratings and actual proficiency (Surface et al., 2012 March) and the correlation between can-do measures and actual proficiency assessment scores is in the .5-.6 range.
I hope this is helpful.
Best,
Eric
Slagter, P. J., Surface, E. A., & Watson, A. M. (2010, July). CEFR and ACTFL scale comparability through perceived and actual proficiency assessments. Symposium presented at the ACTFL CEFR Alignment Conference 2010, Leipzig, Germany. [Download]
http://www.amazon.de/Aligning-Frameworks-Reference-Language-Testing/dp/3860576216
Surface, E. A., Nelson, K. J., DuVernet, A., & Thornhill, D. (2012, March). Perceived versus actual productive proficiency measures: A meta-analytic investigation of the relationship between self-reported and assessed speaking and writing. Paper presented at the American Association for Applied Linguistics 2012 Annual Conference, Boston, MA.
Council of Europe (2011). Common European Framework of Reference for : Learning, Teaching, Assessment. Council of Europe.
Dr. Eric A. Surface
President and Principal I/O Psychologist
SWA Consulting Inc.
311 S. Harrington Street
Suite 200
Raleigh, NC 27603
919-480-2751 (main office line)
888-418-4979 (Fax)
esurface@swa-consulting.com
This email, including any attachments, is only for the intended addressee. It is subject to copyright, is confidential and may be the subject of legal or other privilege, none of which is waived or lost by reason of this transmission. If the receiver is not the intended addressee, please accept our apologies, notify us by return, delete all copies and perform no other act on the email. Unfortunately, we cannot warrant that the email has not been altered or corrupted during transmission. Thank you.
From: Organizational Behavior Division Listserv [mailto:OB@AOMLISTS.PACE.EDU] On Behalf Of Guerrero, Laura
Sent: Thursday, September 04, 2014 11:02 AM
To: OB@AOMLISTS.PACE.EDU
Subject: Re: [OB-LIST] using short measures for testing the english level of participants in a field study
Constanze,
I have used a similar scale (only used the first three questions) but the answers ranged from 1 – 10. This turned out to be important because most of our respondents were fluent but we still found variance. I don't think we would have found variance on a scale from 1 – 5. Our variable was not a control variable so this was important for us.
Guerrero, L. & Rothstein, M. G. (2012). Antecedents of underemployment: Job search of skilled immigrants in Canada. Applied Psychology: An International Review, 61(2), 323-346.
The length of the language scale was not an issue for reviewers. In retrospect, it may have been because despite high levels of self-reported fluency, we still found an impact of fluency on other variables.
I hope this helps.
Regards,
Dr. Laura Guerrero
Assistant Professor of Management
Marketing & Management Department
College of Business Administration
University of Texas at El Paso
500 W. University Avenue
El Paso, TX 79968-0539
915-747-5014
lguerrero5@utep.edu
From: Organizational Behavior Division Listserv [mailto:OB@AOMLISTS.PACE.EDU] On Behalf Of constanze dostal
Sent: Thursday, September 04, 2014 1:11 AM
To: OB@AOMLISTS.PACE.EDU
Subject: [OB-LIST] using short measures for testing the english level of participants in a field study
I am part of a research project, focusing on the effects of fair/unfair treatment of employees at work, for which we will collect data from university students in Austria and Germany.
To avoid validation issues, we would like to do the survey in English. In these two countries it is quite common to attend lectures in English, and even take the exams in English. Based on a small pilot study, we know that the majority of students has no problem to answer the survey, however, we would like to control the effect of English competence/proficiency. To that end, we would like to use a short measure, such as the one used by Turban, Lau, Ngo, Chow, and Si (2001) in a paper published in Journal of Applied Psychology.
"How proficient are you in speaking English?" (5-point- Likert scale)
"How proficient are you in reading English?" (5-point- Likert scale)
"How proficient are you in writing English?" (5-point- Likert scale)
"Have you passed a standardized English language examination administered by a university? (yes - no)
Thus, our question is, would you say that this could become an issue later on, when we submit the paper for review?
So, if there is anybody who could help, or give us any advice in this concern, me and my colleagues would kindly appreciate it.
Turban, D. B., Lau, C. M., Ngo, H. Y., Chow, I. H., & Si, S. X. (2001). Organizational attractiveness of firms in the People's Republic of China: A person–organization fit perspective. Journal of Applied Psychology, 86(2), 194.