Discussion: View Thread

  • 1.  "crowdsourcing" management journal rankings

    Posted 01-18-2011 16:19
    [Apologies for cross-postings]

    Greetings:

    As many have noted, journal impact factors and various other ranking
    measures have significant problems.  To provide additional information
    about journals, we have set up an effort to "crowdsource" management
    journal rankings, at "all our ideas" (a technology platform set up by
    Matt Salganik @ Princeton).

    If you have a few minutes, rank the journals (currently there are
    30,000+ pair-wise votes):

    RANK HERE: http://www.allourideas.org/management

    Best,

    Teppo


  • 2.  "crowdsourcing" management journal rankings

    Posted 01-18-2011 18:10
    This is all very interesting, Tepo. But there seems to be an underlying assumption here that collective subjective impressions should trump objective data. The citation data tell us that, compared to other journals, our colleagues cite fewer articles in, for example, ASQ and Org. Science; but it appears many of us continue to believe nonetheless that these are leading journals in the management field. So what is more important: objective data or subjective impression? I believed we taught evidence-based principles. Am I mistaken?

    Cheers
    Neal Ashkanasy

    -----Original Message-----
    From: Organizational Behavior Division Listserv [mailto:OB@AOMLISTS.PACE.EDU] On Behalf Of Teppo Felin
    Sent: Wednesday, 19 January 2011 7:19 AM
    To: OB@AOMLISTS.PACE.EDU
    Subject: [OB-LIST] "crowdsourcing" management journal rankings

    [Apologies for cross-postings]

    Greetings:

    As many have noted, journal impact factors and various other ranking
    measures have significant problems.  To provide additional information
    about journals, we have set up an effort to "crowdsource" management
    journal rankings, at "all our ideas" (a technology platform set up by
    Matt Salganik @ Princeton).

    If you have a few minutes, rank the journals (currently there are
    30,000+ pair-wise votes):

    RANK HERE: http://www.allourideas.org/management

    Best,

    Teppo


  • 3.  "crowdsourcing" management journal rankings

    Posted 01-19-2011 11:23
    With regards to Neal's comments, I note that I also prefer objective indicators such as impact factors. However, objective impact factors do not necessarily agree with each other although I would suspect that they share a common factor. There is a literature on using consensus ratings to identify the best answer when there is no clearly best answer. 2 + 2 has a clear best answer . However, the emotion expressed by a face item (a picture of a face) in an alleged "emotional intelligence" test does not have an answer that is as clearly defendable as 2 + 2. Likewise, the scoring key for a situational judgment test or an alleged "practical intelligence" test also is less clear than 2 + 2.

    I look forward to seeing the pooled judgment ratings on journals and an analysis of how they compare to objective indicators.

    Best wishes,

    Mike



    Michael A. McDaniel, Ph.D.
    Professor - Human Resources and
    Organizational Behavior
    Department of Management, Research Professor, Department of Psychology
    Virginia Commonwealth University
    301 West Main Street, , PO Box 844000
    Richmond, VA 23284-4000

    http://www.people.vcu.edu/~mamcdani/
    voice: 804.827.0209
    e-mail:
    MAMcDani@vcu.edu
    skype: MichaelAMcDaniel

    Doctoral Program in Management

    The Management Department of the VCU School of Business offers a Ph.D. in Business. Participating faculty with research interests in OB and HR include: Ron Humphrey, Sven Kepes, Michael McDaniel, In-Sue Oh, Doug Pugh, & Anson Seers.

    Students with interest in the doctoral program, should contact Anson Seers.





    From: Neal Ashkanasy <n.ashkanasy@BUSINESS.UQ.EDU.AU>
    To: <OB@AOMLISTS.PACE.EDU>
    Date: 01/18/2011 08:51 PM
    Subject: Re: [OB-LIST] "crowdsourcing" management journal rankings
    Sent by: Organizational Behavior Division Listserv <OB@AOMLISTS.PACE.EDU>





    This is all very interesting, Tepo.  But there seems to be an underlying assumption here that collective subjective impressions should trump objective data.  The citation data tell us that, compared to other journals, our colleagues cite fewer articles in, for example, ASQ and Org. Science; but it appears many of us continue to believe nonetheless that these are leading journals in the management field.  So what is more important: objective data or subjective impression?  I believed we taught evidence-based principles.  Am I mistaken?

    Cheers
    Neal Ashkanasy

    -----Original Message-----
    From: Organizational Behavior Division Listserv [
    mailto:OB@AOMLISTS.PACE.EDU] On Behalf Of Teppo Felin
    Sent: Wednesday, 19 January 2011 7:19 AM
    To: OB@AOMLISTS.PACE.EDU
    Subject: [OB-LIST] "crowdsourcing" management journal rankings

    [Apologies for cross-postings]

    Greetings:

    As many have noted, journal impact factors and various other ranking
    measures have significant problems.  To provide additional information
    about journals, we have set up an effort to "crowdsource" management
    journal rankings, at "all our ideas" (a technology platform set up by
    Matt Salganik @ Princeton).

    If you have a few minutes, rank the journals (currently there are
    30,000+ pair-wise votes):

    RANK HERE:
    http://www.allourideas.org/management

    Best,

    Teppo



  • 4.  "crowdsourcing" management journal rankings

    Posted 01-19-2011 13:18

    I have a pragmatic view on journal rankings. JAP, AMJ etc. usually provides good empirical articles to be used in teching and research communication, but I seldomly become inspired or surprised by reading them, while journals such as Org. Science and ASQ more often publish articles that make me think and come up with new research ideas. Having said that, individual articles tell their own story, independent on where they are published. Number of cites may certainly be viewed as objective, but the question of what the number of cites actually indicates is another story (timing, meta-analysis with large samples, narrative reviews, etc?)

    Best, Bård
    Bård Kuvaas, Dr. Oecon/PhD
    Professor of Organizational Psychology
    Department of Leadership and Organization Management
    BI Norwegian School of Management
    Nydalsveien 37, 0484 Oslo, Norway
    Telephone: +47 06600
    Dial direct: + 47 4641 0731
    Telefax: +47 4641 0701
    Home telephone: +47 6301 0478
    E-mail: bard.kuvaas@bi.no
    Home page: http://www.bi.no/en/Research/Academic-homepage/?ansattid=FGL90032



    Michael A McDaniel/AC/VCU <mamcdani@VCU.EDU>
    Sent by: Organizational Behavior Division Listserv <OB@AOMLISTS.PACE.EDU>

    19.01.2011 18:02

    Please respond to
    Organizational Behavior Division Listserv <OB@AOMLISTS.PACE.EDU>

    To
    <OB@AOMLISTS.PACE.EDU>
    cc
    Subject
    Re: [OB-LIST] "crowdsourcing" management journal rankings





    With regards to Neal's comments, I note that I also prefer objective indicators such as impact factors. However, objective impact factors do not necessarily agree with each other although I would suspect that they share a common factor. There is a literature on using consensus ratings to identify the best answer when there is no clearly best answer. 2 + 2 has a clear best answer . However, the emotion expressed by a face item (a picture of a face) in an alleged "emotional intelligence" test does not have an answer that is as clearly defendable as 2 + 2. Likewise, the scoring key for a situational judgment test or an alleged "practical intelligence" test also is less clear than 2 + 2.

    I look forward to seeing the pooled judgment ratings on journals and an analysis of how they compare to objective indicators.


    Best wishes,


    Mike



    Michael A. McDaniel, Ph.D.
    Professor - Human Resources and
    Organizational Behavior
    Department of Management, Research Professor, Department of Psychology
    Virginia Commonwealth University
    301 West Main Street, , PO Box 844000
    Richmond, VA 23284-4000

    http://www.people.vcu.edu/~mamcdani/
    voice: 804.827.0209
    e-mail:
    MAMcDani@vcu.edu
    skype: MichaelAMcDaniel

    Doctoral Program in Management

    The Management Department of the VCU School of Business offers a Ph.D. in Business. Participating faculty with research interests in OB and HR include: Ron Humphrey, Sven Kepes, Michael McDaniel, In-Sue Oh, Doug Pugh, & Anson Seers.

    Students with interest in the doctoral program, should contact Anson Seers.




    From: Neal Ashkanasy <n.ashkanasy@BUSINESS.UQ.EDU.AU>
    To: <OB@AOMLISTS.PACE.EDU>
    Date: 01/18/2011 08:51 PM
    Subject: Re: [OB-LIST] "crowdsourcing" management journal rankings
    Sent by: Organizational Behavior Division Listserv <OB@AOMLISTS.PACE.EDU>






    This is all very interesting, Tepo.  But there seems to be an underlying assumption here that collective subjective impressions should trump objective data.  The citation data tell us that, compared to other journals, our colleagues cite fewer articles in, for example, ASQ and Org. Science; but it appears many of us continue to believe nonetheless that these are leading journals in the management field.  So what is more important: objective data or subjective impression?  I believed we taught evidence-based principles.  Am I mistaken?

    Cheers
    Neal Ashkanasy

    -----Original Message-----
    From: Organizational Behavior Division Listserv [
    mailto:OB@AOMLISTS.PACE.EDU] On Behalf Of Teppo Felin
    Sent: Wednesday, 19 January 2011 7:19 AM
    To: OB@AOMLISTS.PACE.EDU
    Subject: [OB-LIST] "crowdsourcing" management journal rankings

    [Apologies for cross-postings]

    Greetings:

    As many have noted, journal impact factors and various other ranking
    measures have significant problems.  To provide additional information
    about journals, we have set up an effort to "crowdsource" management
    journal rankings, at "all our ideas" (a technology platform set up by
    Matt Salganik @ Princeton).

    If you have a few minutes, rank the journals (currently there are
    30,000+ pair-wise votes):

    RANK HERE:
    http://www.allourideas.org/management

    Best,

    Teppo




  • 5.  "crowdsourcing" management journal rankings

    Posted 01-19-2011 17:07
    Dear All, the rankings/citation game is both well known and well played. The rankings game can best be summarized by the late Larry Cummings, who once told me, "where you publish will be held against you." It is best played by those whose works are published in our most highly cited journals, but whose scholarship is not well cited. They tend to stresss the quantity of articles published in "A" journals. On the other hand are those whose work is highly cited, but may not appear necessarily in "A." Think of Michael Porter. Now certainly he has published in "A" journals, but I submit his major citations and fame come from his books, and not his journal articles. A down side to counting the citations attributed to one's work, no matter where published, is that a lot of citations may not reflect that ones work was a major intellectual force behind the research citing it, but it has become a ceremonial or requiste cite for publication, such as DiMaggio and Powell's work became. This greatly inflates citations to a work. My own take is that a citation count of 50-100 suggest it is a significant piece of scholarship, >100 that it is a major work, >500 that it belongs in doctoral seminars on the topic. >1000, that it has become a ceremonial cite and a career maker, much like winning the Oscar.

    Of course, I have not even discussed which index we use, Thompson Web of Science, 9generally viewed as the gold standard, or Ann Wil-harzing's publish or perish, based on Google Scholar. In general, looking at the two, I find a 1:3 ratio between being cited in Thompson versus Google Scholar. The most prestigious schools typically only use Thompson, and only count the citations to articles in "A" journals. While schools differ on which journals belong, primary discipline journals usually (always0 count, e.g., Psy Review, Psy Bul. ASR, AJS, MS. In the field of management typical journals would include ASQ, Org Sci, JAP, OBHPD, AMJ, AMR, SMJ, and P Psych. Others runninng a close second would include JOM and specialty journals, eg. JIBS, LQ, JBV.

    Whatever game we choose to play, usually only results in a temporary victory. The number of scholars whose work has lasted 20+ years is very small. New topics emerge, new theories replace other theories, new research drives out old research. All we can do is pursue the topics we are passionate about, and hope our research partially answers the 'hot" questions of the time. After 20+ years, people will either know you or they won't as one who contributed to our collective jopurnal to know.

    Regards to all those who have run the race, who have fought the good fight.

    Kim Boal
    ________________________________________
    From: Organizational Behavior Division Listserv [OB@AOMLISTS.PACE.EDU] On Behalf Of Bard Kuvaas [bard.kuvaas@BI.NO]
    Sent: Wednesday, January 19, 2011 12:18 PM
    To: OB@AOMLISTS.PACE.EDU
    Subject: Re: [OB-LIST] "crowdsourcing" management journal rankings

    I have a pragmatic view on journal rankings. JAP, AMJ etc. usually provides good empirical articles to be used in teching and research communication, but I seldomly become inspired or surprised by reading them, while journals such as Org. Science and ASQ more often publish articles that make me think and come up with new research ideas. Having said that, individual articles tell their own story, independent on where they are published. Number of cites may certainly be viewed as objective, but the question of what the number of cites actually indicates is another story (timing, meta-analysis with large samples, narrative reviews, etc?)

    Best, Bård
    Bård Kuvaas, Dr. Oecon/PhD
    Professor of Organizational Psychology
    Department of Leadership and Organization Management
    BI Norwegian School of Management
    Nydalsveien 37, 0484 Oslo, Norway
    Telephone: +47 06600
    Dial direct: + 47 4641 0731
    Telefax: +47 4641 0701
    Home telephone: +47 6301 0478
    E-mail: bard.kuvaas@bi.no
    Home page: http://www.bi.no/en/Research/Academic-homepage/?ansattid=FGL90032


    Michael A McDaniel/AC/VCU <mamcdani@VCU.EDU>
    Sent by: Organizational Behavior Division Listserv <OB@AOMLISTS.PACE.EDU>

    19.01.2011 18:02
    Please respond to
    Organizational Behavior Division Listserv <OB@AOMLISTS.PACE.EDU>




    To
    <OB@AOMLISTS.PACE.EDU>
    cc

    Subject
    Re: [OB-LIST] "crowdsourcing" management journal rankings







    With regards to Neal's comments, I note that I also prefer objective indicators such as impact factors. However, objective impact factors do not necessarily agree with each other although I would suspect that they share a common factor. There is a literature on using consensus ratings to identify the best answer when there is no clearly best answer. 2 + 2 has a clear best answer . However, the emotion expressed by a face item (a picture of a face) in an alleged "emotional intelligence" test does not have an answer that is as clearly defendable as 2 + 2. Likewise, the scoring key for a situational judgment test or an alleged "practical intelligence" test also is less clear than 2 + 2.

    I look forward to seeing the pooled judgment ratings on journals and an analysis of how they compare to objective indicators.

    Best wishes,

    Mike


    Michael A. McDaniel, Ph.D.
    Professor - Human Resources and
    Organizational Behavior
    Department of Management, Research Professor, Department of Psychology
    Virginia Commonwealth University
    301 West Main Street, , PO Box 844000
    Richmond, VA 23284-4000

    http://www.people.vcu.edu/~mamcdani/
    voice: 804.827.0209
    e-mail: MAMcDani@vcu.edu<mailto:MAMcDani@vcu.edu>
    skype: MichaelAMcDaniel
    Doctoral Program in Management

    The Management Department of the VCU School of Business offers a Ph.D. in Business. Participating faculty with research interests in OB and HR include: Ron Humphrey, Sven Kepes, Michael McDaniel, In-Sue Oh, Doug Pugh, & Anson Seers.

    Students with interest in the doctoral program, should contact Anson Seers<mailto:aseers@vcu.edu>.




    From: Neal Ashkanasy <n.ashkanasy@BUSINESS.UQ.EDU.AU>
    To: <OB@AOMLISTS.PACE.EDU>
    Date: 01/18/2011 08:51 PM
    Subject: Re: [OB-LIST] "crowdsourcing" management journal rankings
    Sent by: Organizational Behavior Division Listserv <OB@AOMLISTS.PACE.EDU>



    ________________________________



    This is all very interesting, Tepo. But there seems to be an underlying assumption here that collective subjective impressions should trump objective data. The citation data tell us that, compared to other journals, our colleagues cite fewer articles in, for example, ASQ and Org. Science; but it appears many of us continue to believe nonetheless that these are leading journals in the management field. So what is more important: objective data or subjective impression? I believed we taught evidence-based principles. Am I mistaken?

    Cheers
    Neal Ashkanasy

    -----Original Message-----
    From: Organizational Behavior Division Listserv [mailto:OB@AOMLISTS.PACE.EDU] On Behalf Of Teppo Felin
    Sent: Wednesday, 19 January 2011 7:19 AM
    To: OB@AOMLISTS.PACE.EDU
    Subject: [OB-LIST] "crowdsourcing" management journal rankings

    [Apologies for cross-postings]

    Greetings:

    As many have noted, journal impact factors and various other ranking
    measures have significant problems. To provide additional information
    about journals, we have set up an effort to "crowdsource" management
    journal rankings, at "all our ideas" (a technology platform set up by
    Matt Salganik @ Princeton).

    If you have a few minutes, rank the journals (currently there are
    30,000+ pair-wise votes):

    RANK HERE: http://www.allourideas.org/management

    Best,

    Teppo