Discussion: View Thread

  • 1.  Adapting existing scales

    Posted 10-14-2015 21:15
    Many studies adapt existing measures. The reasons are various, including making it more appropriate for the study's sample, changing the level of analysis (e.g., referent shift), or altering it due to translation or cultural issues.

    My question is: Are there published guidelines for adapting existing scales? I'm familiar with Tim Hinkin's articles on scale development, but that deals with creating new measures, not adapting existing ones. I've searched the literature but haven't found anything, but it's hard to figure out exactly what the right search terms would be.

    Michael Johnson
    Associate Professor of Organizational Behavior
    The Boeing Company Professor of Business Management
    Foster School of Business, University of Washington
    544 Paccar Hall, Box 353226
    Seattle, WA 98195
    (206) 616-2756
    mdj3@uw.edu



  • 2.  Adapting existing scales

    Posted 10-15-2015 02:14
    IHi Michael,

    I am sorry that I cannot come up with some articles that propose particular
    guidelines but actually I think that the things to consider are the same as
    when you create a new scale:

    a) consider your own ontological stance regarding the construct and its
    measure - that is: do you have a realist stance and think of the construct in terms
    of ONE existing entity (or several) or do you think that the scale is some
    empirical means that makes up or constitutes the construct (i.e., constructivist
    approach). In this regard, I found the paper by Denny Borsboom enlightening:

    Borsboom, D., Mellenbergh, G. J., & van Heerden, J. (2003). The theoretical status
    of latent variables. Psychological Review, 110(2), 203–219.

    b) If you think of the construct in terms of a constructivist perspective, issues
    of conent validity become salient. Then the question arises if the elements
    captured by the construct (i.e. which in turn are captured by the items of the scale)
    if your adapted items/facets have the same understanding than the original scale.
    Here, cognitive interviewing can be helpful:

    Willis, G. B. (2005). Cognitive interviewing: A tool for Improving questionnaire design.
    Thousand Oaks, CA: Sage.

    If you - in constrast - adopt the realist view, the issue of a measurement model
    becomes important that connects the items with an underlying common factor.
    Here, it is essential to understand what the common factor model is really about:
    Namely it proposes that an underlying latent variable causes the items responses
    and explains the correlations among the items. A common factor IS NOT a "theme",
    "domain" are "summary" - it is a singular/specific entity "out there" emerging
    causal influence.

    Most CFAs fail because authors a) have a constructivist view but apply the common
    factor model or b) the scales are heterogeneous and the factor model is to simplicistic.
    As aforementioned: Issues of multidimensionality belong to the theoretical level and
    should be considered in terms of a set of latent variables instead of bringing the
    multidimensionality in by using the backdoor (i.e. the measurement level, heterogenous
    items).

    In both cases (scales as composites vs. reflective indicators of a latent variables) issues
    of validity become (again) salient: That is you have to test the structure of the latent
    variable - item-connections (e.g, in a CFA or better full SEM) as well as analyze
    connections between the construct and other criteria.

    My 2 c

    Best,
    Holger
    _______________________________________ Holger Steinmetz University of Paderborn Department of Management Technologiepark 11 33100 Paderborn  Postal address Warburger Str. 100 33098 Paderborn  Phone: +49 5251 604930 Fax:   +49 5251 602077 Profile: http://scholar.google.de/citations?user=4au1va0AAAAJ&hl=de&oi=ao _______________________________________
    Am 15.10.2015 um 03:15 schrieb Michael Johnson:
    3F9B93D3-BC5B-475B-BCC4-3EA8F1A2D627@u.washington.edu" type="cite">
    Many studies adapt existing measures. The reasons are various, including making it more appropriate for the study's sample, changing the level of analysis (e.g., referent shift), or altering it due to translation or cultural issues.

    My question is: Are there published guidelines for adapting existing scales? I'm familiar with Tim Hinkin's articles on scale development, but that deals with creating new measures, not adapting existing ones. I've searched the literature but haven't found anything, but it's hard to figure out exactly what the right search terms would be.

    Michael Johnson
    Associate Professor of Organizational Behavior
    The Boeing Company Professor of Business Management
    Foster School of Business, University of Washington
    544 Paccar Hall, Box 353226
    Seattle, WA 98195
    (206) 616-2756
    mdj3@uw.edu




  • 3.  Adapting existing scales

    Posted 10-15-2015 09:28

    Michael,

     

    The following article offers recommendations about this and related issues (see pp. 588-590):

     

    ·       Aguinis, H., & Vandenberg, R. J. 2014. An ounce of prevention is worth a pound of cure: Improving research quality before data collection. Annual Review of Organizational Psychology and Organizational Behavior, 1: 569-595. [available at http://mypage.iu.edu/~haguinis/pubs.html]

     

    As noted on page 589, "...once a scale has been created, any alteration such as changing the wording for some items or eliminating others from the scale means that the resulting scale may no longer be accurately measuring the underlying construct that it was originally created to assess."

     

    Also, pages 148-152 in the following article include additional relevant material and guidelines:

     

    ·       Aguinis, H., & Edwards, J. R. 2014. Methodological wishes for the next decade and how to make wishes come true. Journal of Management Studies, 51: 143-174. [available at http://mypage.iu.edu/~haguinis/pubs.html]

     

    I hope this helps!

     

    All the best,

     

    --Herman.

     

    Herman Aguinis

    John F. Mee Chair of Management

    Professor of Organizational Behavior and Human Resources

    Founding and Managing Director, Institute for Global Organizational Effectiveness

     

    Indiana University

    Kelley School of Business

    http://mypage.iu.edu/~haguinis/

     

    GO FROM MOMENT TO MOMENTUM

     

    Am 15.10.2015 um 03:15 schrieb Michael Johnson:

    Many studies adapt existing measures. The reasons are various, including making it more appropriate for the study's sample, changing the level of analysis (e.g., referent shift), or altering it due to translation or cultural issues.

     

    My question is: Are there published guidelines for adapting existing scales? I'm familiar with Tim Hinkin's articles on scale development, but that deals with creating new measures, not adapting existing ones. I've searched the literature but haven't found anything, but it's hard to figure out exactly what the right search terms would be.

     

    Michael Johnson
    Associate Professor of Organizational Behavior
    The Boeing Company Professor of Business Management

    Foster School of Business, University of Washington
    544 Paccar Hall, Box 353226
    Seattle, WA 98195
    (206) 616-2756
    mdj3@uw.edu

     

     



  • 4.  Adapting existing scales

    Posted 10-15-2015 10:53

    Herman,

    I like to compliment this warning about changing words in a scale.

    Isn't this the whole point...it is the comprehension of words and statements that construct meaning in context. Changing any elements of this content in context changes what is being measured. As obvious as these statements are, as is the original statement, changes to these items may better measure the underlying construct.

    Any change calls for a renewal of the validation study/studies done for the new scale and construct, and review of the underlying construct may be warranted or indicated too.

    There are alternative strategies suitable for certain research scenarios such as using alternative scales and shorter versions where there are acceptable validity metrics available.

    Dr. John Read

    On 15 Oct 2015 5:36 pm, "Aguinis, Herman" <haguinis@indiana.edu> wrote:

    Michael,

     

    The following article offers recommendations about this and related issues (see pp. 588-590):

     

    ·       Aguinis, H., & Vandenberg, R. J. 2014. An ounce of prevention is worth a pound of cure: Improving research quality before data collection. Annual Review of Organizational Psychology and Organizational Behavior, 1: 569-595. [available at http://mypage.iu.edu/~haguinis/pubs.html]

     

    As noted on page 589, "...once a scale has been created, any alteration such as changing the wording for some items or eliminating others from the scale means that the resulting scale may no longer be accurately measuring the underlying construct that it was originally created to assess."

     

    Also, pages 148-152 in the following article include additional relevant material and guidelines:

     

    ·       Aguinis, H., & Edwards, J. R. 2014. Methodological wishes for the next decade and how to make wishes come true. Journal of Management Studies, 51: 143-174. [available at http://mypage.iu.edu/~haguinis/pubs.html]

     

    I hope this helps!

     

    All the best,

     

    --Herman.

     

    Herman Aguinis

    John F. Mee Chair of Management

    Professor of Organizational Behavior and Human Resources

    Founding and Managing Director, Institute for Global Organizational Effectiveness

     

    Indiana University

    Kelley School of Business

    http://mypage.iu.edu/~haguinis/

     

    GO FROM MOMENT TO MOMENTUM

     

    Am 15.10.2015 um 03:15 schrieb Michael Johnson:

    Many studies adapt existing measures. The reasons are various, including making it more appropriate for the study's sample, changing the level of analysis (e.g., referent shift), or altering it due to translation or cultural issues.

     

    My question is: Are there published guidelines for adapting existing scales? I'm familiar with Tim Hinkin's articles on scale development, but that deals with creating new measures, not adapting existing ones. I've searched the literature but haven't found anything, but it's hard to figure out exactly what the right search terms would be.

     

    Michael Johnson
    Associate Professor of Organizational Behavior
    The Boeing Company Professor of Business Management

    Foster School of Business, University of Washington
    544 Paccar Hall, Box 353226
    Seattle, WA 98195
    (206) 616-2756
    mdj3@uw.edu

     

     



  • 5.  Adapting existing scales

    Posted 10-15-2015 09:54

    Dear Michael,


    At the Academy of Management conference in Vancouver this year, Lin, Dalal, and Withrow presented a workshop on "Efficient Survey Measurement: Generating Short Form Surveys" (PDW, program session #174). Please find a description of the workshop below. I attended said workshop and will send you the materials I received from the organizers to your personal email address.


    Best of luck with your research!


    Christian


    -- 

    Christian Kiewitz, PhD
    Professor in Management
    Management/Marketing Department
    University of Dayton
    300 College Park
    Dayton, OH 45469-2271  USA

    Phone +1.937.229.2046


    +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

    Session Type: PDW Workshop

    Program Session: 174 | Submission: 14012 | Sponsor(s): (RM, OB, HR)

    Scheduled: Friday, Aug 7 2015 3:30PM 5:

    30PM at The Fairmont Hotel

    Vancouver in Galiano Island

    Efficient Survey Measurement:

    Generating Short Form Surveys

    Short Form Surveys

     

    Organizer: Bing Chun Lin, Koç U.

    Organizer: Dev K Dalal, U. of Connecticut

    Organizer: Scott Withrow, Koç U.

     

    Survey measurement is the predominant means by which key behavioral/psychological constructs are measured across fields of research (Sackett & Larson, 1990). Spurred by a variety of reasons (e.g., popularization of experience sampling designs), however, the need to develop short-form measures has increased. In spite of the demand, few guidelines have been offered, resulting in varied approaches to shorting survey scales with mixed results. Compounded with inconsistent reporting of shortening procedures in the Management literature, many researchers are left confused about how to most effectively shorten self-report scales. The co-organizers hope to address this gap by educating researchers on scale shortening best practices, and to raise field-wide awareness of the importance of systematizing scale-shortening procedures. The 2 hour and 20 minute PDW will be divided into five activities. First, Dr. Lin will educate attendees on the value of using short measures, circumstances wherein short measures should be used, and the dangers inherent in using short measures. Second, Dr. Dalal will present methods on shortening surveys without prior respondent data. Third, to ensure comprehension, we will guide attendees through two scale-shortening exercises: one scale prepared by the organizers, and a second scale provided by select attendees. Fourth, Dr. Withrow will then present Classical Test Theory and Item Response Theory methods of shortening scales when prior respondent data for measures is available. The PDW will close with a Question-and-Answer panel discussion including Drs. Dalal and Withrow and moderated by Dr. Lin.

    We will present two methods of shortening surveys (with and without prior response data), and engage audience members in two measure shortening exercises. We will guide all attendees through measure shortening procedures for one survey measure provided by the organizers. We will then have attendees work in groups to shorten one measure per group (measures will be provided by volunteering attendees). If you are interested in having your measure shortened, please contact the co-Organizer, Bing Chun Lin, bclin@ku.edu.tr.

    +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

     


    On Wed, Oct 14, 2015 at 9:15 PM, Michael Johnson <mdj3@uw.edu> wrote:
    Many studies adapt existing measures. The reasons are various, including making it more appropriate for the study's sample, changing the level of analysis (e.g., referent shift), or altering it due to translation or cultural issues.

    My question is: Are there published guidelines for adapting existing scales? I'm familiar with Tim Hinkin's articles on scale development, but that deals with creating new measures, not adapting existing ones. I've searched the literature but haven't found anything, but it's hard to figure out exactly what the right search terms would be.

    Michael Johnson
    Associate Professor of Organizational Behavior
    The Boeing Company Professor of Business Management
    Foster School of Business, University of Washington
    544 Paccar Hall, Box 353226
    Seattle, WA 98195
    (206) 616-2756
    mdj3@uw.edu




  • 6.  Adapting existing scales

    Posted 10-15-2015 13:15

    Michael, you might also find the following references pertaining to the construction of short versions of scales to be useful:

    1.               Bono & McNamara, 2011, AMJ, 657-660

    2.               Keller & Dansereau 2001, Organizational Research Methods.

    3.               Credé et al., 2012, JPSP

    4.               Smith et al., 2000, Psychological Assessment

    Bob

     

    Robert C. Liden

    Professor of Management

    Associate Dean for CBA Ph.D. Programs; Coordinator of the OB/HR Doctoral Program

    Department of Managerial Studies (M/C 243) Room 2232

    University of Illinois at Chicago

    601 S. Morgan Street

    Chicago, IL 60607-7123

     

     

     

    From: Organizational Behavior Division Listserv [mailto:OB@AOMLISTS.PACE.EDU] On Behalf Of Christian Kiewitz
    Sent: Thursday, October 15, 2015 8:54 AM
    To: OB@AOMLISTS.PACE.EDU
    Subject: Re: [OB-LIST] Adapting existing scales

     

    Dear Michael,

     

    At the Academy of Management conference in Vancouver this year, Lin, Dalal, and Withrow presented a workshop on "Efficient Survey Measurement: Generating Short Form Surveys" (PDW, program session #174). Please find a description of the workshop below. I attended said workshop and will send you the materials I received from the organizers to your personal email address.

     

    Best of luck with your research!

     

    Christian

     

    -- 

    Christian Kiewitz, PhD

    Professor in Management

    Management/Marketing Department

    University of Dayton

    300 College Park

    Dayton, OH 45469-2271  USA

     

    Phone +1.937.229.2046

     

    +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

    Session Type: PDW Workshop

    Program Session: 174 | Submission: 14012 | Sponsor(s): (RM, OB, HR)

    Scheduled: Friday, Aug 7 2015 3:30PM 5:

    30PM at The Fairmont Hotel

    Vancouver in Galiano Island

    Efficient Survey Measurement:

    Generating Short Form Surveys

    Short Form Surveys

     

    Organizer: Bing Chun Lin, Koç U.

    Organizer: Dev K Dalal, U. of Connecticut

    Organizer: Scott Withrow, Koç U.

     

    Survey measurement is the predominant means by which key behavioral/psychological constructs are measured across fields of research (Sackett & Larson, 1990). Spurred by a variety of reasons (e.g., popularization of experience sampling designs), however, the need to develop short-form measures has increased. In spite of the demand, few guidelines have been offered, resulting in varied approaches to shorting survey scales with mixed results. Compounded with inconsistent reporting of shortening procedures in the Management literature, many researchers are left confused about how to most effectively shorten self-report scales. The co-organizers hope to address this gap by educating researchers on scale shortening best practices, and to raise field-wide awareness of the importance of systematizing scale-shortening procedures. The 2 hour and 20 minute PDW will be divided into five activities. First, Dr. Lin will educate attendees on the value of using short measures, circumstances wherein short measures should be used, and the dangers inherent in using short measures. Second, Dr. Dalal will present methods on shortening surveys without prior respondent data. Third, to ensure comprehension, we will guide attendees through two scale-shortening exercises: one scale prepared by the organizers, and a second scale provided by select attendees. Fourth, Dr. Withrow will then present Classical Test Theory and Item Response Theory methods of shortening scales when prior respondent data for measures is available. The PDW will close with a Question-and-Answer panel discussion including Drs. Dalal and Withrow and moderated by Dr. Lin.

    We will present two methods of shortening surveys (with and without prior response data), and engage audience members in two measure shortening exercises. We will guide all attendees through measure shortening procedures for one survey measure provided by the organizers. We will then have attendees work in groups to shorten one measure per group (measures will be provided by volunteering attendees). If you are interested in having your measure shortened, please contact the co-Organizer, Bing Chun Lin, bclin@ku.edu.tr.

    +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

     

     

    On Wed, Oct 14, 2015 at 9:15 PM, Michael Johnson <mdj3@uw.edu> wrote:

    Many studies adapt existing measures. The reasons are various, including making it more appropriate for the study's sample, changing the level of analysis (e.g., referent shift), or altering it due to translation or cultural issues.

     

    My question is: Are there published guidelines for adapting existing scales? I'm familiar with Tim Hinkin's articles on scale development, but that deals with creating new measures, not adapting existing ones. I've searched the literature but haven't found anything, but it's hard to figure out exactly what the right search terms would be.

     

    Michael Johnson
    Associate Professor of Organizational Behavior
    The Boeing Company Professor of Business Management

    Foster School of Business, University of Washington
    544 Paccar Hall, Box 353226
    Seattle, WA 98195
    (206) 616-2756
    mdj3@uw.edu

     

     



  • 7.  Adapting existing scales

    Posted 10-15-2015 15:37
    In cases where range restriction corrections are desirable and there exist data on population variance (IQ measures with a standard deviation of 15), shortening scales damages your ability to correct for range restriction because you no longer have an estimate of the population variance.

    Mike

    Michael A. McDaniel, Ph.D.,
    Professor and National Treasure, Department of Management, 
    Virginia Commonwealth University 301 West Main Street, PO Box 844000, 
    Richmond, VA 23284-4000
    voice: 804.827.0209     e-mail: MAMcDani@vcu.edu
    Research Professor, VCU Department of Psychology


    On Thu, Oct 15, 2015 at 1:14 PM, Bob Liden <bobliden@uic.edu> wrote:

    Michael, you might also find the following references pertaining to the construction of short versions of scales to be useful:

    1.               Bono & McNamara, 2011, AMJ, 657-660

    2.               Keller & Dansereau 2001, Organizational Research Methods.

    3.               Credé et al., 2012, JPSP

    4.               Smith et al., 2000, Psychological Assessment

    Bob

     

    Robert C. Liden

    Professor of Management

    Associate Dean for CBA Ph.D. Programs; Coordinator of the OB/HR Doctoral Program

    Department of Managerial Studies (M/C 243) Room 2232

    University of Illinois at Chicago

    601 S. Morgan Street

    Chicago, IL 60607-7123

     

     

     

    From: Organizational Behavior Division Listserv [mailto:OB@AOMLISTS.PACE.EDU] On Behalf Of Christian Kiewitz
    Sent: Thursday, October 15, 2015 8:54 AM
    To: OB@AOMLISTS.PACE.EDU
    Subject: Re: [OB-LIST] Adapting existing scales

     

    Dear Michael,

     

    At the Academy of Management conference in Vancouver this year, Lin, Dalal, and Withrow presented a workshop on "Efficient Survey Measurement: Generating Short Form Surveys" (PDW, program session #174). Please find a description of the workshop below. I attended said workshop and will send you the materials I received from the organizers to your personal email address.

     

    Best of luck with your research!

     

    Christian

     

    -- 

    Christian Kiewitz, PhD

    Professor in Management

    Management/Marketing Department

    University of Dayton

    300 College Park

    Dayton, OH 45469-2271  USA

     

     

    +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

    Session Type: PDW Workshop

    Program Session: 174 | Submission: 14012 | Sponsor(s): (RM, OB, HR)

    Scheduled: Friday, Aug 7 2015 3:30PM 5:

    30PM at The Fairmont Hotel

    Vancouver in Galiano Island

    Efficient Survey Measurement:

    Generating Short Form Surveys

    Short Form Surveys

     

    Organizer: Bing Chun Lin, Koç U.

    Organizer: Dev K Dalal, U. of Connecticut

    Organizer: Scott Withrow, Koç U.

     

    Survey measurement is the predominant means by which key behavioral/psychological constructs are measured across fields of research (Sackett & Larson, 1990). Spurred by a variety of reasons (e.g., popularization of experience sampling designs), however, the need to develop short-form measures has increased. In spite of the demand, few guidelines have been offered, resulting in varied approaches to shorting survey scales with mixed results. Compounded with inconsistent reporting of shortening procedures in the Management literature, many researchers are left confused about how to most effectively shorten self-report scales. The co-organizers hope to address this gap by educating researchers on scale shortening best practices, and to raise field-wide awareness of the importance of systematizing scale-shortening procedures. The 2 hour and 20 minute PDW will be divided into five activities. First, Dr. Lin will educate attendees on the value of using short measures, circumstances wherein short measures should be used, and the dangers inherent in using short measures. Second, Dr. Dalal will present methods on shortening surveys without prior respondent data. Third, to ensure comprehension, we will guide attendees through two scale-shortening exercises: one scale prepared by the organizers, and a second scale provided by select attendees. Fourth, Dr. Withrow will then present Classical Test Theory and Item Response Theory methods of shortening scales when prior respondent data for measures is available. The PDW will close with a Question-and-Answer panel discussion including Drs. Dalal and Withrow and moderated by Dr. Lin.

    We will present two methods of shortening surveys (with and without prior response data), and engage audience members in two measure shortening exercises. We will guide all attendees through measure shortening procedures for one survey measure provided by the organizers. We will then have attendees work in groups to shorten one measure per group (measures will be provided by volunteering attendees). If you are interested in having your measure shortened, please contact the co-Organizer, Bing Chun Lin, bclin@ku.edu.tr.

    +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

     

     

    On Wed, Oct 14, 2015 at 9:15 PM, Michael Johnson <mdj3@uw.edu> wrote:

    Many studies adapt existing measures. The reasons are various, including making it more appropriate for the study's sample, changing the level of analysis (e.g., referent shift), or altering it due to translation or cultural issues.

     

    My question is: Are there published guidelines for adapting existing scales? I'm familiar with Tim Hinkin's articles on scale development, but that deals with creating new measures, not adapting existing ones. I've searched the literature but haven't found anything, but it's hard to figure out exactly what the right search terms would be.

     

    Michael Johnson
    Associate Professor of Organizational Behavior
    The Boeing Company Professor of Business Management

    Foster School of Business, University of Washington
    544 Paccar Hall, Box 353226
    Seattle, WA 98195
    (206) 616-2756
    mdj3@uw.edu

     

     




  • 8.  Adapting existing scales

    Posted 10-18-2015 03:13
    Here is one more article I didn't see cited in this thread yet:

    Stanton, J. M., Sinar, E. F., Balzer, W. K., & Smith, P. C. (2002). Issues and Strategies for Reducing the Length of Self Report Scales. Personnel Psychology, 55(1), 167-194. 


    _________________
    Tara Wernsing, Ph.D.
    Associate Professor
    Organizational Behavior & Leadership
    IE Business School
    Alvarez de Baena 4-1º
    28006 Madrid, Spain
    phone: (+34) 91 568 9659


    From: Organizational Behavior Division Listserv on behalf of Michael A McDaniel
    Reply-To: Organizational Behavior Division Listserv
    Date: Thursday, October 15, 2015 at 21:36
    To: "OB@AOMLISTS.PACE.EDU"
    Subject: Re: [OB-LIST] Adapting existing scales

    In cases where range restriction corrections are desirable and there exist data on population variance (IQ measures with a standard deviation of 15), shortening scales damages your ability to correct for range restriction because you no longer have an estimate of the population variance.

    Mike

    Michael A. McDaniel, Ph.D.,
    Professor and National Treasure, Department of Management, 
    Virginia Commonwealth University 301 West Main Street, PO Box 844000, 
    Richmond, VA 23284-4000
    voice: 804.827.0209     e-mail: MAMcDani@vcu.edu
    Research Professor, VCU Department of Psychology


    On Thu, Oct 15, 2015 at 1:14 PM, Bob Liden <bobliden@uic.edu> wrote:

    Michael, you might also find the following references pertaining to the construction of short versions of scales to be useful:

    1.               Bono & McNamara, 2011, AMJ, 657-660

    2.               Keller & Dansereau 2001, Organizational Research Methods.

    3.               Credé et al., 2012, JPSP

    4.               Smith et al., 2000, Psychological Assessment

    Bob

     

    Robert C. Liden

    Professor of Management

    Associate Dean for CBA Ph.D. Programs; Coordinator of the OB/HR Doctoral Program

    Department of Managerial Studies (M/C 243) Room 2232

    University of Illinois at Chicago

    601 S. Morgan Street

    Chicago, IL 60607-7123

     

     

     

    From: Organizational Behavior Division Listserv [mailto:OB@AOMLISTS.PACE.EDU] On Behalf Of Christian Kiewitz
    Sent: Thursday, October 15, 2015 8:54 AM
    To: OB@AOMLISTS.PACE.EDU
    Subject: Re: [OB-LIST] Adapting existing scales

     

    Dear Michael,

     

    At the Academy of Management conference in Vancouver this year, Lin, Dalal, and Withrow presented a workshop on "Efficient Survey Measurement: Generating Short Form Surveys" (PDW, program session #174). Please find a description of the workshop below. I attended said workshop and will send you the materials I received from the organizers to your personal email address.

     

    Best of luck with your research!

     

    Christian

     

    -- 

    Christian Kiewitz, PhD

    Professor in Management

    Management/Marketing Department

    University of Dayton

    300 College Park

    Dayton, OH 45469-2271  USA

     

     

    +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

    Session Type: PDW Workshop

    Program Session: 174 | Submission: 14012 | Sponsor(s): (RM, OB, HR)

    Scheduled: Friday, Aug 7 2015 3:30PM 5:

    30PM at The Fairmont Hotel

    Vancouver in Galiano Island

    Efficient Survey Measurement:

    Generating Short Form Surveys

    Short Form Surveys

     

    Organizer: Bing Chun Lin, Koç U.

    Organizer: Dev K Dalal, U. of Connecticut

    Organizer: Scott Withrow, Koç U.

     

    Survey measurement is the predominant means by which key behavioral/psychological constructs are measured across fields of research (Sackett & Larson, 1990). Spurred by a variety of reasons (e.g., popularization of experience sampling designs), however, the need to develop short-form measures has increased. In spite of the demand, few guidelines have been offered, resulting in varied approaches to shorting survey scales with mixed results. Compounded with inconsistent reporting of shortening procedures in the Management literature, many researchers are left confused about how to most effectively shorten self-report scales. The co-organizers hope to address this gap by educating researchers on scale shortening best practices, and to raise field-wide awareness of the importance of systematizing scale-shortening procedures. The 2 hour and 20 minute PDW will be divided into five activities. First, Dr. Lin will educate attendees on the value of using short measures, circumstances wherein short measures should be used, and the dangers inherent in using short measures. Second, Dr. Dalal will present methods on shortening surveys without prior respondent data. Third, to ensure comprehension, we will guide attendees through two scale-shortening exercises: one scale prepared by the organizers, and a second scale provided by select attendees. Fourth, Dr. Withrow will then present Classical Test Theory and Item Response Theory methods of shortening scales when prior respondent data for measures is available. The PDW will close with a Question-and-Answer panel discussion including Drs. Dalal and Withrow and moderated by Dr. Lin.

    We will present two methods of shortening surveys (with and without prior response data), and engage audience members in two measure shortening exercises. We will guide all attendees through measure shortening procedures for one survey measure provided by the organizers. We will then have attendees work in groups to shorten one measure per group (measures will be provided by volunteering attendees). If you are interested in having your measure shortened, please contact the co-Organizer, Bing Chun Lin, bclin@ku.edu.tr.

    +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

     

     

    On Wed, Oct 14, 2015 at 9:15 PM, Michael Johnson <mdj3@uw.edu> wrote:

    Many studies adapt existing measures. The reasons are various, including making it more appropriate for the study's sample, changing the level of analysis (e.g., referent shift), or altering it due to translation or cultural issues.

     

    My question is: Are there published guidelines for adapting existing scales? I'm familiar with Tim Hinkin's articles on scale development, but that deals with creating new measures, not adapting existing ones. I've searched the literature but haven't found anything, but it's hard to figure out exactly what the right search terms would be.

     

    Michael Johnson
    Associate Professor of Organizational Behavior
    The Boeing Company Professor of Business Management

    Foster School of Business, University of Washington
    544 Paccar Hall, Box 353226
    Seattle, WA 98195
    (206) 616-2756
    mdj3@uw.edu

     

     




  • 9.  Adapting existing scales

    Posted 10-18-2015 10:29
    Hi Michael,

    You might also consider the following article:

    Maloney, P., Grawitch, M. J., & Barber, L. K. (2011). Strategic item selection to reduce survey length: Reduction in validity? Consulting Psychology Journal: Practice and Research, 63(3), 162-175.

    Cheers,

    Allen

    C. Allen Gorman, PhD
    Assistant Professor & Associate Chair
    Department of Management & Marketing
    [cid:image001.jpg@01CF649C.8C8584B0]
    128 Sam Wilson Hall
    PO Box 70625
    Johnson City, TN 37614
    Office: 423-439-5592
    Fax: 423-439-5661
    E-mail: gormanc@etsu.edu<mailto:gormanc@etsu.edu>
    ________________________________
    From: Organizational Behavior Division Listserv [OB@AOMLISTS.PACE.EDU] on behalf of Tara Wernsing [Tara.Wernsing@IE.EDU]
    Sent: Sunday, October 18, 2015 3:13 AM
    To: OB@AOMLISTS.PACE.EDU
    Subject: Re: [OB-LIST] Adapting existing scales

    Here is one more article I didn’t see cited in this thread yet:

    Stanton, J. M., Sinar, E. F., Balzer, W. K., & Smith, P. C. (2002). Issues and Strategies for Reducing the Length of Self Report Scales. Personnel Psychology, 55(1), 167-194.

    _________________
    Tara Wernsing, Ph.D.
    Associate Professor
    Organizational Behavior & Leadership
    IE Business School
    Alvarez de Baena 4-1º
    28006 Madrid, Spain
    phone: (+34) 91 568 9659
    tara.wernsing@ie.edu

    From: Organizational Behavior Division Listserv on behalf of Michael A McDaniel
    Reply-To: Organizational Behavior Division Listserv
    Date: Thursday, October 15, 2015 at 21:36
    To: "OB@AOMLISTS.PACE.EDU<mailto:OB@AOMLISTS.PACE.EDU>"
    Subject: Re: [OB-LIST] Adapting existing scales

    In cases where range restriction corrections are desirable and there exist data on population variance (IQ measures with a standard deviation of 15), shortening scales damages your ability to correct for range restriction because you no longer have an estimate of the population variance.

    Mike

    Michael A. McDaniel, Ph.D.,
    President, International Society of Intelligence Research<http://www.isironline.org/>
    Advisory Board Member: metaBUS: You’re either omnibus or off the bus<http://metabus.org>
    Professor and National Treasure, Department of Management,
    Virginia Commonwealth University 301 West Main Street, PO Box 844000,
    Richmond, VA 23284-4000
    http://www.people.vcu.edu/~mamcdani/
    voice: 804.827.0209 e-mail: MAMcDani@vcu.edu<mailto:MAMcDani@vcu.edu>
    Research Professor, VCU Department of Psychology


    On Thu, Oct 15, 2015 at 1:14 PM, Bob Liden <bobliden@uic.edu<mailto:bobliden@uic.edu>> wrote:
    Michael, you might also find the following references pertaining to the construction of short versions of scales to be useful:
    1. Bono & McNamara, 2011, AMJ, 657-660
    2. Keller & Dansereau 2001, Organizational Research Methods.
    3. Credé et al., 2012, JPSP
    4. Smith et al., 2000, Psychological Assessment
    Bob

    Robert C. Liden<http://business.uic.edu/home-uic-business/faculty-departments-research/department-of-managerial-studies/faculty/robert-liden>
    Professor of Management
    Associate Dean for CBA Ph.D. Programs; Coordinator of the OB/HR Doctoral Program
    Department of Managerial Studies (M/C 243) Room 2232
    University of Illinois at Chicago
    601 S. Morgan Street
    Chicago, IL 60607-7123
    [cid:66CFD763-0E55-49A3-8B79-7E6EF2F4B0C8@uic.edu]



    From: Organizational Behavior Division Listserv [mailto:OB@AOMLISTS.PACE.EDU<mailto:OB@AOMLISTS.PACE.EDU>] On Behalf Of Christian Kiewitz
    Sent: Thursday, October 15, 2015 8:54 AM
    To: OB@AOMLISTS.PACE.EDU<mailto:OB@AOMLISTS.PACE.EDU>
    Subject: Re: [OB-LIST] Adapting existing scales

    Dear Michael,

    At the Academy of Management conference in Vancouver this year, Lin, Dalal, and Withrow presented a workshop on “Efficient Survey Measurement: Generating Short Form Surveys” (PDW, program session #174). Please find a description of the workshop below. I attended said workshop and will send you the materials I received from the organizers to your personal email address.

    Best of luck with your research!

    Christian

    --
    Christian Kiewitz, PhD
    Professor in Management
    Management/Marketing Department
    University of Dayton
    300 College Park
    Dayton, OH 45469-2271 USA

    Phone +1.937.229.2046<tel:%2B1.937.229.2046>
    kiewitz@udayton.edu<mailto:kiewitz@udayton.edu>

    +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
    Session Type: PDW Workshop
    Program Session: 174 | Submission: 14012 | Sponsor(s): (RM, OB, HR)
    Scheduled: Friday, Aug 7 2015 3:30PM 5:
    30PM at The Fairmont Hotel
    Vancouver in Galiano Island
    Efficient Survey Measurement:
    Generating Short Form Surveys
    Short Form Surveys

    Organizer: Bing Chun Lin, Koç U.
    Organizer: Dev K Dalal, U. of Connecticut
    Organizer: Scott Withrow, Koç U.

    Survey measurement is the predominant means by which key behavioral/psychological constructs are measured across fields of research (Sackett & Larson, 1990). Spurred by a variety of reasons (e.g., popularization of experience sampling designs), however, the need to develop short-form measures has increased. In spite of the demand, few guidelines have been offered, resulting in varied approaches to shorting survey scales with mixed results. Compounded with inconsistent reporting of shortening procedures in the Management literature, many researchers are left confused about how to most effectively shorten self-report scales. The co-organizers hope to address this gap by educating researchers on scale shortening best practices, and to raise field-wide awareness of the importance of systematizing scale-shortening procedures. The 2 hour and 20 minute PDW will be divided into five activities. First, Dr. Lin will educate attendees on the value of using short measures, circumstances wherein short measures should be used, and the dangers inherent in using short measures. Second, Dr. Dalal will present methods on shortening surveys without prior respondent data. Third, to ensure comprehension, we will guide attendees through two scale-shortening exercises: one scale prepared by the organizers, and a second scale provided by select attendees. Fourth, Dr. Withrow will then present Classical Test Theory and Item Response Theory methods of shortening scales when prior respondent data for measures is available. The PDW will close with a Question-and-Answer panel discussion including Drs. Dalal and Withrow and moderated by Dr. Lin.
    We will present two methods of shortening surveys (with and without prior response data), and engage audience members in two measure shortening exercises. We will guide all attendees through measure shortening procedures for one survey measure provided by the organizers. We will then have attendees work in groups to shorten one measure per group (measures will be provided by volunteering attendees). If you are interested in having your measure shortened, please contact the co-Organizer, Bing Chun Lin, bclin@ku.edu.tr<mailto:bclin@ku.edu.tr>.
    +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++


    On Wed, Oct 14, 2015 at 9:15 PM, Michael Johnson <mdj3@uw.edu<mailto:mdj3@uw.edu>> wrote:
    Many studies adapt existing measures. The reasons are various, including making it more appropriate for the study's sample, changing the level of analysis (e.g., referent shift), or altering it due to translation or cultural issues.

    My question is: Are there published guidelines for adapting existing scales? I’m familiar with Tim Hinkin’s articles on scale development, but that deals with creating new measures, not adapting existing ones. I’ve searched the literature but haven’t found anything, but it’s hard to figure out exactly what the right search terms would be.

    Michael Johnson
    Associate Professor of Organizational Behavior
    The Boeing Company Professor of Business Management
    Foster School of Business, University of Washington
    544 Paccar Hall, Box 353226
    Seattle, WA 98195
    (206) 616-2756<tel:%28206%29%20616-2756>
    mdj3@uw.edu<mailto:mdj3@uw.edu>
    faculty.washington.edu/mdj3/mjohnson<http://faculty.washington.edu/mdj3/mjohnson>