Discussion: View Thread

Dear OB list, longtime listener, first time caller.

We've seen an increase in the sophistication of methods for longitudinal analysis (see: McArdle (2009), Chan (1998), among others). But I haven't found a similar amount of writing on frequent repeated measurement of psychological constructs, with the aim of detecting intraindividual variability. Perhaps someone here is aware of literature I've overlooked? I'm trying to find out how one should go about measuring a psychological construct longitudinally over time, when the sampling time period is very close together... even minutes apart. An added problem is that the constructs are complex, e.g.: trust, felt-responsibility for a coworker/partner, or the components of self-regulation.

The closest literature I've been able to find uses ecological momentary assessment (EMA) (e.g., Shiffman, Stone, and Hufford, 2008), or mood measurement using potentiometers.

Kuppens et al. (2007) used an EMA sampling method where a PDA would beep and the subject would pick a point on a two-dimensional graph indicating their core affect at that moment. They found they could do this up to every 17 minutes over a number of weeks. But the burden was so high they could only ask that single question.

In a similar EMA study, Ceja and Navarro (2011) asked one open ended question (what activity are you performing) and 5 scale questions, but they only did this 6 times a day over 3 weeks.

In a study measuring mood change over a short period of time, Stanley and Isaacowitz (2011) asked their subjects to perform a task (viewing faces on a computer) and gave them a potentiometer. They were asked to move the potentiometer between 0 - 100 depending on how they felt during the task (0 = sad, 100 = happy).

In the negotiation literature the closest I've come is finding is Ferrin et al (2008) who ask a 7-item trust scale 4 times in-between rounds of a negotiation task.

Yet, all of these studies make no mention of measurement issues with frequent longitudinal repeated measures. Are there any? Or is the main problem fatigue/burden on the respondent?

I can imagine issues of:

-          practice or carryover effects,

-          test-retest reliability (the higher the test-retest reliability, the lower the chance of detecting change over time--a paradox of sorts for longitudinal change research, isn't it?),

-          recall, memory, priming, and/or anchoring effects (after all, the same measure is being repeated within minutes),

-          regression to the mean (actual change is moderated or completely hidden by regression),

-          acquiecense (similar to burden--over time, the subject begins to enter the same answers to every question)

To decrease burden and acquiecense, can a complex multi-item scale be simplified into a single item slider/scale? But, would this not prevent the use of latent growth curve modeling? Chan (1998) argued that longitudinal growth modeling requires measurement invariance which can only be assessed if multiple indicators are used to measure the construct. If that's the case, what then of the single item measurements used for core-affect or mood above? E.g., is it completely unreliable to use a single measure of trust? (How much do you trust your partner? 1 – 7)

Any help would be appreciated, and I can summarize the response back to the list for those interested.

Thanks!

Chris.

References

Ceja, L., & Navarro, J. (2011). Dynamic patterns of flow in the workplace: Characterizing within-individual variability using a complexity science approach. Journal of Organizational Behavior, 32(4), 627-651.

Chan, D. (1998). The Conceptualization and Analysis of Change Over Time: An Integrative Approach Incorporating Longitudinal Mean and Covariance Structures Analysis (LMACS) and Multiple Indicator Latent Growth Modeling (MLGM). Organizational Research Methods, 1(4), 421-483.

Ferrin, D. L., Bligh, M. C., & Kohles, J. C. (2008). It takes two to tango: An interdependence analysis of the spiraling of perceived trustworthiness and cooperation in interpersonal and intergroup relationships. Organizational Behavior and Human Decision Processes, 107(2), 161-178.

Kuppens, P., Van Mechelen, I., Nezlek, J. B., Dossche, D., & Timmermans, T. (2007). Individual differences in core affect variability and their relationship to personality and psychological adjustment. Emotion, 7(2), 262-274.

McArdle, J. J. (2009). Latent Variable Modeling of Differences and Changes with Longitudinal Data Annual Review of Psychology (Vol. 60, pp. 577-605).

Shiffman, S., Stone, A. A., & Hufford, M. R. (2008). Ecological Momentary Assessment. Annual Review of Clinical Psychology, 4(1), 1-32.

Stanley, J. T., & Isaacowitz, D. M. (2011). Age-related differences in profiles of mood-change trajectories. Developmental Psychology, 47(2), 318-330.

Christopher Poile, PhD
Assistant Professor, Human Resources and Organizational Behaviour
Edwards School of Business | University of Saskatchewan
PotashCorp Centre   |  25 Campus Drive  |  Saskatoon, SK  S7N 5A7
website | T  306.966.2491  |  F  306.966.2514

Chris,

I suggest you look at a superb article by Curran and Bauer on the disaggregation of within-person and between-person effects in longitudinal models.

Curran, P.J. & Bauer (2011). The disaggregation of within-person and between-person effects in longitudinal models of change. Annual Review of Psychology, 62, 583-619.

Best Wishes,

Pat

Dear OB list, longtime listener, first time caller.

We've seen an increase in the sophistication of methods for longitudinal analysis (see: McArdle (2009), Chan (1998), among others). But I haven't found a similar amount of writing on frequent repeated measurement of psychological constructs, with the aim of detecting intraindividual variability. Perhaps someone here is aware of literature I've overlooked? I'm trying to find out how one should go about measuring a psychological construct longitudinally over time, when the sampling time period is very close together... even minutes apart. An added problem is that the constructs are complex, e.g.: trust, felt-responsibility for a coworker/partner, or the components of self-regulation.

The closest literature I've been able to find uses ecological momentary assessment (EMA) (e.g., Shiffman, Stone, and Hufford, 2008), or mood measurement using potentiometers.

Kuppens et al. (2007) used an EMA sampling method where a PDA would beep and the subject would pick a point on a two-dimensional graph indicating their core affect at that moment. They found they could do this up to every 17 minutes over a number of weeks. But the burden was so high they could only ask that single question.

In a similar EMA study, Ceja and Navarro (2011) asked one open ended question (what activity are you performing) and 5 scale questions, but they only did this 6 times a day over 3 weeks.

In a study measuring mood change over a short period of time, Stanley and Isaacowitz (2011) asked their subjects to perform a task (viewing faces on a computer) and gave them a potentiometer. They were asked to move the potentiometer between 0 - 100 depending on how they felt during the task (0 = sad, 100 = happy).

In the negotiation literature the closest I've come is finding is Ferrin et al (2008) who ask a 7-item trust scale 4 times in-between rounds of a negotiation task.

Yet, all of these studies make no mention of measurement issues with frequent longitudinal repeated measures. Are there any? Or is the main problem fatigue/burden on the respondent?

I can imagine issues of:

-          practice or carryover effects,

-          test-retest reliability (the higher the test-retest reliability, the lower the chance of detecting change over time--a paradox of sorts for longitudinal change research, isn't it?),

-          recall, memory, priming, and/or anchoring effects (after all, the same measure is being repeated within minutes),

-          regression to the mean (actual change is moderated or completely hidden by regression),

-          acquiecense (similar to burden--over time, the subject begins to enter the same answers to every question)

To decrease burden and acquiecense, can a complex multi-item scale be simplified into a single item slider/scale? But, would this not prevent the use of latent growth curve modeling? Chan (1998) argued that longitudinal growth modeling requires measurement invariance which can only be assessed if multiple indicators are used to measure the construct. If that's the case, what then of the single item measurements used for core-affect or mood above? E.g., is it completely unreliable to use a single measure of trust? (How much do you trust your partner? 1 – 7)

Any help would be appreciated, and I can summarize the response back to the list for those interested.

Thanks!

Chris.

References

Ceja, L., & Navarro, J. (2011). Dynamic patterns of flow in the workplace: Characterizing within-individual variability using a complexity science approach. Journal of Organizational Behavior, 32(4), 627-651.

Chan, D. (1998). The Conceptualization and Analysis of Change Over Time: An Integrative Approach Incorporating Longitudinal Mean and Covariance Structures Analysis (LMACS) and Multiple Indicator Latent Growth Modeling (MLGM). Organizational Research Methods, 1(4), 421-483.

Ferrin, D. L., Bligh, M. C., & Kohles, J. C. (2008). It takes two to tango: An interdependence analysis of the spiraling of perceived trustworthiness and cooperation in interpersonal and intergroup relationships. Organizational Behavior and Human Decision Processes, 107(2), 161-178.

Kuppens, P., Van Mechelen, I., Nezlek, J. B., Dossche, D., & Timmermans, T. (2007). Individual differences in core affect variability and their relationship to personality and psychological adjustment. Emotion, 7(2), 262-274.

McArdle, J. J. (2009). Latent Variable Modeling of Differences and Changes with Longitudinal Data Annual Review of Psychology (Vol. 60, pp. 577-605).

Shiffman, S., Stone, A. A., & Hufford, M. R. (2008). Ecological Momentary Assessment. Annual Review of Clinical Psychology, 4(1), 1-32.

Stanley, J. T., & Isaacowitz, D. M. (2011). Age-related differences in profiles of mood-change trajectories. Developmental Psychology, 47(2), 318-330.

Christopher Poile, PhD
Assistant Professor, Human Resources and Organizational Behaviour
Edwards School of Business | University of Saskatchewan
PotashCorp Centre   |  25 Campus Drive  |  Saskatoon, SK  S7N 5A7
website | T  306.966.2491  |  F  306.966.2514