Hi Andrew:
Interesting questions. As for the first question, I guess that the ability of factors to predict incremental variance is the best evidence of discriminant validity. Though we often hear reviewers say "well, your variables correlate at .80; they are redundant"! I think that statements like that are not helpful; nor is it really helpful to test whether the correlation between two variables is different from zero (the power of the test is of course important). Many things are very highly correlated but conceptually unique.
Suppose for model (where all exogenous variables and the disturbance "e" are normally distributed with mean zero and SD1):
y = b0 + b1x + b2z + 2*e
the test of b2=0 is, really the best test. Whether x and z correlate highly is not of issue; what if, for arguments sake, they correlate at .80? Does that matter? Not really. What matters most if if there is enough information to tease out z's unique effects (and that both variables are exogenous--see below). Enough information will, at constant correlation depend entirely on sample size. For instance, let's take the above model and, to spice it up, set the correlation between x and z at .90, and vary the sample size, and Monte Carlo it 1,000 times. What % of time will b2 be significant:
N % sig.
20 .145
40 .297
60 .383
80 .476
100 .594
150 .767
250 .927
500 .998
1000 1.000
1500 1.000
One will detect a significant effect with reasonable power at samples of 150 or more!
What if the distrubance is 3*e (i.e., more error in the model), then we need a much larger sample (close to 500):
N % sig.
20 .087
40 .155
60 .195
80 .250
100 .319
150 .418
250 .612
500 .881
1000 .999
1500 1.000
So, the point is that if the sample size is sufficiently large it is highly probably that b2 is significant--and if it is, well, then there is no need to worry. I demonstrated this point in a recent publication, where we used Monte Carlo study to show we were not underpowered despite high correlations among the independent variables--and there we had many variables and lots of colinearity (see Study 4):
Antonakis, J., & House, R. J. (2014). Instrumental leadership: Measurement and extension of transformational-transactional leadership theory. The Leadership Quarterly, 25(4) 746-771.
http://dx.doi.org/10.1016/j.leaqua.2014.04.005 Note too that high correlations could be due to endogeneity bias (explained by an omitted variable)--thus, it is important to be sure of the exogeneity of the variables (we talked about that too in the above paper). If not, then corrective procedures are required like two-stage least squares estimation.
As for your second question, well it is hard to answer that. If you are not aware of other variables that may correlate with your variable, then you have a problem--and that is a problem of exogeneity (related to the above issue). Is your target variable exogenous? If it is not then you will have a huge problem to even obtain a consistent estimate (i.e., one that converges to the true population value). You may be interested in the following two papers:
Antonakis, J., Bendahan, S., Jacquart, P., & Lalive, R. (2010). On making causal claims: A review and recommendations. The Leadership Quarterly, 21(6). 1086-1120.
http://dx.doi.org/10.1016/j.leaqua.2010.10.010 There is a prequel to that paper:
Antonakis, J., Bendahan, S., Jacquart, P., & Lalive, R. (2014). Causality and endogeneity: Problems and solutions. In D.V. Day (Ed.), The Oxford Handbook of Leadership and Organizations.
http://www.hec.unil.ch/jantonakis/Causality_and_endogeneity_final.pdf (and a podcast that goes with that on Youtube
http://www.youtube.com/watch?v=dLuTjoYmfXs).
Hope this helps.
John.
__________________________________________ John Antonakis Professor of Organizational Behavior Director, Ph.D. Program in Management Faculty of Business and Economics (HEC) University of Lausanne Internef #618 CH-1015 Lausanne-Dorigny Switzerland Tel ++41 (0)21 692-3438 Fax ++41 (0)21 692-3305 http://www.hec.unil.ch/people/jantonakis Associate Editor: The Leadership Quarterly Organizational Research Methods __________________________________________
On 20.10.2014 16:28, Li, Andrew wrote: