This is a very crucial point, and an excellent conversation.
Classic psychometric theory sometimes steers us wrong by placing an emphasis
on internal consistency. Thinking about the applied history of the use of
psychometrics (e.g., educational placements, job selection, etc.) and the
relative "cost" of developing and validating a scale (before personal
computers, SPSS...), it make sense why building a water-tight, internally
consistent scale was so highly valued.
But, this approach assumes that we're thinking about latent variables within
reflective/indicator models (i.e. responses on test items 1-10 are all
caused by construct X). Internal consistency makes much less sense when we
think about composite constructs within formative models, wherein the
aggregate of the items creates the latent construct.
As example, Lee and Mitchell's job embeddedness scale (a composite measure)
should have relatively LOW internal consistency; while serving on multiple
committees at work and owning my own home may all lead to embeddedness,
these individual items are theoretically only weakly related. In fact,
especially high internal consistency on this type of scale might imply
common method bias (preferring one end of the scale over the other on
similarly worded questions), or spurious correlation (tenure within a
company or age might be related both to owning a home and being on a lot of
committees).
Internal reliability is important, and there should be a theoretical
argument for its absence. But reviewers should also be less dogmatic about
ALWAYS expecting high coefficient alphas.
--Keith
-----Original Message-----
From: Organizational Behavior Division Listserv
[mailto:
OB@AOMLISTS.PACE.EDU] On Behalf Of Ben Schneider
Sent: Monday, March 29, 2010 10:50 AM
To:
OB@AOMLISTS.PACE.EDU
Subject: Re: [OB-LIST] Responses to request for Low alpha scores of Big 5
TIPI scale----note on reliability and validity
Hi All,
Several comments in this exchange inlcuded the idea that relaibility puts a
limit on valididty; this is not true for internal relaiaiblity, the topic of
the e-mails.
Think about it this way: Multiple regression takes uncorrelated variables to
establish the prediction equation making the prediction equation unrelaible
from an intenal consistency standpoint. Or, consider the use of
criterion-keyed measures (like bio-data) where the predcitive power comes
from combining items that are not internally comsistent.
I have seen people say this for my entire career and what is true is that
re-test relaibility puts a limit on prediction since a variable can't
predict another variable better than the square root of itself.
Ben
Benjamin Schneider, Ph.D.
Senior Research Fellow, VALTERA
Professor Emeritus, University of Maryland
1363 Caminito Floreo, Suite G
La Jolla, CA 92037
tel/fx: 858-488-7594
bschneider@valtera.com
VALTERA R
Better Organizations Through Better Science R
www.valtera.com
Chicago Corporate Office:
Valtera Corporation
1701 Golf Rd., 2-1100
Rolling Meadows, IL 60008-4257
www.valtera.com
This email and attachments, if included, may contain material that is
confidential. This material is intended for the sole use of the individual
or entity to whom it is addressed. If you received this message in error,
please contact the sender and delete all copies.
-----Original Message-----
From: Organizational Behavior Division Listserv
[mailto:
OB@AOMLISTS.PACE.EDU] On Behalf Of Stefan Volk
Sent: Monday, March 29, 2010 3:55 AM
To:
OB@AOMLISTS.PACE.EDU
Subject: [OB-LIST] Responses to request for Low alpha scores of Big 5 TIPI
scale
Thanks to everyone who responded to my request concerning low alpha
scores of scales with small numbers of items. I've attached a Word
document that has the summary.
Thanks again!
>
>
> On Tue, Mar 23, 2010 at 5:33 AM, Stefan Volk
> <
stefan.volk@uni-tuebingen.de
> <mailto:
stefan.volk@uni-tuebingen.de>> wrote:
>
> Dear all,
>
> we used the Gosling et al. (2003) 10-item personality
> inventory (TIPI) and obtained low Cronbach's alpha scores. Sam
> Gosling provides an explanation on his website indicating that
> alphas are misleading when calculated on scales with small
> numbers of items. I was wondering if someone could provide me
> with or point me to some more arguments for reviewers apart
> from the explanation given by Sam, in the ideal case something
> that has been published. I see once in a while that authors do
> not report alphas if they use two-item scales. What is the
> theoretical argument of not reporting alphas, if scales
> consist of only two items?
>
> Many thanks in advance,
> Stefan
>