Discussion: View Thread

Dear colleagues:

Regarding the debate about method/sources/rigor/relevance of research
conducted by Google researchers on social sciences, let me share my
experience in dealings with such organization for research purposes.

I worked in Silicon Valley out of college, I therefore have contacts who are
now at the Google, Facebook, etc., some in charge of massive data collection
and analyses with a social science spin.

We therefore explored the possibility of a sabbatical for me to come and
work with them. The exploratory discussion confirmed what is public
knowledge, that they have truly astounding data, orders of magnitude better
than what armies of human PhDs will ever get.

And their data mining capabilities are interesting beyond the fact that we
can envision a future where they could test any quirky relationship (which
they will). More interestingly, their algorithm will probably allow complete
path analysis with large # of variables that would put to shame any
structural equation modeling and endogeneity control ever done in our
published papers.

I know some of you will cringe, challenging the validity and apparently
naïve hopes of such statements. Let's not debate this, let's remember these
are stunning new tools, and as scientists we *should* have naïve attraction
at any new tool to uncover and crack data ;)

So, where does that lead us? Well, it is difficult for them to collaborate
with us researchers from public institutions (i.e., not their employees),
and even to disclose in any manner what they actually do! So (our) ignorance
will remain the norm for the years to come!

Let's just take a "public" event, the disclosure of a study that Facebook
conducted where it manipulated the emotions of 600k+ users. And observe the
public outcry at a manipulation that would have got the vetting of any IRB
at the most conservative school
http://www.theguardian.com/technology/2014/jun/29/facebook-users-emotions-ne
ws-feeds

Even though IRB would have vetted this in a traditional lab (give me a
break, re-arranging order of news about your friends, is the harm
substantial?), we should, as citizens, worry about such experiment at
FB/GOO/MS/... as they reflect on the stunning public manipulation
capabilities of those firms.

However, as scientist and alumnus of Santa Fe Institute, I would not resist
the possibility to go and conduct research with such tools!

So such research are tricky endeavors as their disclosure could taint those
firms in the public eye and their corporate objectives put their brand above
everything. It implies that the secrecy level maintained by those firms on
their research methods/data will remain at par with the ones maintained by
some labs on their research on crypto, nuclear, biological weapons, etc.

Bottom line: the Silicon Valley firms that have massive datasets (most have
;) are selectively hiring people from econ, some from org sciences (you know
who you are ;) but disclosure is fuzzy and the publication output is at best
tangential.

Concluding it lacks rigor just because one does not see much of their pubs,
or worse, just because it is relevant, is sad . It would also spell trouble
for the PhD students we currently train as they will have to deal with such
methods at some point in the future.

The good news is that those firms collaborate discreetly with some
universities. If you are from one of those schools, enjoy mingling with the
future of data science. As far as I'm concerned, since there must be a
penalty to be in France (because there are many advantages, trust me), it
has been difficult to consider that sabbatical so far as my institution is
not (yet?) part of that select group of mainly US schools whose researchers
can be vetted to peek behind the veil. But I'm not losing hope ;)

Best,

Fabrice Cavarretta
Associate Professor of
Leadership and Entrepreneurship
ESSEC Business School
Mob : +33 6 09 59 46 74
@fcavarrettaEN
Author: "Oui! La France est un paradis pour entrepreneurs"

-----Original Message-----
From: Rob Briner [mailto:R.B.Briner@BATH.AC.UK]
Sent: Saturday, February 27, 2016 11:27 AM
Subject: Re: FW: NYTimes: What Google Learned From Its Quest to Build the
Perfect Team

Hi Neal

Thanks for this. I don't know about you or anyone else but I find all these
stories from Google (and some other organizations) about how wonderful they
are at using evidence and data a bit perplexing. It always seems impossible
to see, as it is in this article too, exactly what scientific findings they
looked at, how they reviewed them, how they identified their quality and
relevance, how they summarized or aggregated the evidence, and finally how
and if they actually used it in their work. Also, in other articles and
interviews, Google imply they completely ignore published scientific
evidence and only rely on their own (big) data.

I guess any story where any organization is saying how great it is at doing
something needs to be taken with quite a large pinch of salt unless they are
transparent and detailed about what they are doing and are open about their
failures as well as successes. Also, what is Google's motive for telling us
this?

I'd be really interested to know if anyone has seen any transparent and
detailed and critical account of what Google do around data and scientific
evidence - it's doesn't seem to be in any of the public accounts I've seen
but I could well be missing something.

Cheers

Rob

Rob B Briner | Professor of Organizational Psychology | School of Management
| University of Bath Scientific Director | Center for Evidence-Based
Management (www.cebma.org) Twitter @Rob_Briner

-----Original Message-----
From: Organizational Behavior Division Listserv
[mailto:OB@AOMLISTS.PACE.EDU] On Behalf Of Neal Ashkanasy
Sent: 27 February 2016 00:29
To: OB@AOMLISTS.PACE.EDU
Subject: [OB-LIST] FW: NYTimes: What Google Learned From Its Quest to Build
the Perfect Team

Dear OB colleagues

It's good to see that someone is reading our research and applying our
findings!

NY Times: New research reveals surprising truths about why some work groups
thrive and others falter.
http://www.nytimes.com/2016/02/28/magazine/what-google-learned-from-its-ques
t-to-build-the-perfect-team.html?smprod=nytcore-iphone&smid=nytcore-iphone-s
hare

Cheers
Neal M. Ashkanasy, PhD
UQ Business School
The University of Queensland
Brisbane, Qld 4072, Australia
Phone: +617 3346-8006
Fax: +617 3346-8188
e-mail: n.ashkanasy@uq.edu.au
https://www.business.uq.edu.au/staff/details/neal-ashkanasy

Good discussion. There are literally thousands of I/O psychologists working in organizations and in consulting who every day do excellent work that contributes to their organizations' effectiveness. These folks of necessity speak the language of management--as does Duhigg--and translating what they do into theoretical academese is almost impossible. Right now academics tend to work with conceptual/theoretical problems while those in practice work with practical problems and bridging that gap to get published in research-focused journals is daunting to say the least. Some of us continue to try but the days of publishing some data/findings because they are interesting and potentially useful is now gone. Ben

Sent from my iPhone

> On Feb 28, 2016, at 8:05 AM, Fabrice Cavarretta <cavarretta@ESSEC.EDU> wrote:
>
> Dear colleagues:
>
> Regarding the debate about method/sources/rigor/relevance of research
> conducted by Google researchers on social sciences, let me share my
> experience in dealings with such organization for research purposes.
>
> I worked in Silicon Valley out of college, I therefore have contacts who are
> now at the Google, Facebook, etc., some in charge of massive data collection
> and analyses with a social science spin.
>
> We therefore explored the possibility of a sabbatical for me to come and
> work with them. The exploratory discussion confirmed what is public
> knowledge, that they have truly astounding data, orders of magnitude better
> than what armies of human PhDs will ever get.
>
> And their data mining capabilities are interesting beyond the fact that we
> can envision a future where they could test any quirky relationship (which
> they will). More interestingly, their algorithm will probably allow complete
> path analysis with large # of variables that would put to shame any
> structural equation modeling and endogeneity control ever done in our
> published papers.
>
> I know some of you will cringe, challenging the validity and apparently
> naïve hopes of such statements. Let's not debate this, let's remember these
> are stunning new tools, and as scientists we *should* have naïve attraction
> at any new tool to uncover and crack data ;)
>
> So, where does that lead us? Well, it is difficult for them to collaborate
> with us researchers from public institutions (i.e., not their employees),
> and even to disclose in any manner what they actually do! So (our) ignorance
> will remain the norm for the years to come!
>
> Let's just take a "public" event, the disclosure of a study that Facebook
> conducted where it manipulated the emotions of 600k+ users. And observe the
> public outcry at a manipulation that would have got the vetting of any IRB
> at the most conservative school
> http://www.theguardian.com/technology/2014/jun/29/facebook-users-emotions-ne
> ws-feeds
>
> Even though IRB would have vetted this in a traditional lab (give me a
> break, re-arranging order of news about your friends, is the harm
> substantial?), we should, as citizens, worry about such experiment at
> FB/GOO/MS/... as they reflect on the stunning public manipulation
> capabilities of those firms.
>
> However, as scientist and alumnus of Santa Fe Institute, I would not resist
> the possibility to go and conduct research with such tools!
>
> So such research are tricky endeavors as their disclosure could taint those
> firms in the public eye and their corporate objectives put their brand above
> everything. It implies that the secrecy level maintained by those firms on
> their research methods/data will remain at par with the ones maintained by
> some labs on their research on crypto, nuclear, biological weapons, etc.
>
> Bottom line: the Silicon Valley firms that have massive datasets (most have
> ;) are selectively hiring people from econ, some from org sciences (you know
> who you are ;) but disclosure is fuzzy and the publication output is at best
> tangential.
>
> Concluding it lacks rigor just because one does not see much of their pubs,
> or worse, just because it is relevant, is sad . It would also spell trouble
> for the PhD students we currently train as they will have to deal with such
> methods at some point in the future.
>
> The good news is that those firms collaborate discreetly with some
> universities. If you are from one of those schools, enjoy mingling with the
> future of data science. As far as I'm concerned, since there must be a
> penalty to be in France (because there are many advantages, trust me), it
> has been difficult to consider that sabbatical so far as my institution is
> not (yet?) part of that select group of mainly US schools whose researchers
> can be vetted to peek behind the veil. But I'm not losing hope ;)
>
> Best,
>
> Fabrice Cavarretta
> Associate Professor of
> Leadership and Entrepreneurship
> ESSEC Business School
> Mob : +33 6 09 59 46 74
> @fcavarrettaEN
> Author: "Oui! La France est un paradis pour entrepreneurs"
>
> -----Original Message-----
> From: Rob Briner [mailto:R.B.Briner@BATH.AC.UK]
> Sent: Saturday, February 27, 2016 11:27 AM
> Subject: Re: FW: NYTimes: What Google Learned From Its Quest to Build the
> Perfect Team
>
> Hi Neal
>
> Thanks for this. I don't know about you or anyone else but I find all these
> stories from Google (and some other organizations) about how wonderful they
> are at using evidence and data a bit perplexing. It always seems impossible
> to see, as it is in this article too, exactly what scientific findings they
> looked at, how they reviewed them, how they identified their quality and
> relevance, how they summarized or aggregated the evidence, and finally how
> and if they actually used it in their work. Also, in other articles and
> interviews, Google imply they completely ignore published scientific
> evidence and only rely on their own (big) data.
>
> I guess any story where any organization is saying how great it is at doing
> something needs to be taken with quite a large pinch of salt unless they are
> transparent and detailed about what they are doing and are open about their
> failures as well as successes. Also, what is Google's motive for telling us
> this?
>
> I'd be really interested to know if anyone has seen any transparent and
> detailed and critical account of what Google do around data and scientific
> evidence - it's doesn't seem to be in any of the public accounts I've seen
> but I could well be missing something.
>
> Cheers
>
> Rob
>
> Rob B Briner | Professor of Organizational Psychology | School of Management
> | University of Bath Scientific Director | Center for Evidence-Based
> Management (www.cebma.org) Twitter @Rob_Briner
>
> -----Original Message-----
> From: Organizational Behavior Division Listserv
> [mailto:OB@AOMLISTS.PACE.EDU] On Behalf Of Neal Ashkanasy
> Sent: 27 February 2016 00:29
> To: OB@AOMLISTS.PACE.EDU
> Subject: [OB-LIST] FW: NYTimes: What Google Learned From Its Quest to Build
> the Perfect Team
>
> Dear OB colleagues
>
> It's good to see that someone is reading our research and applying our
> findings!
>
> NY Times: New research reveals surprising truths about why some work groups
> thrive and others falter.
> http://www.nytimes.com/2016/02/28/magazine/what-google-learned-from-its-ques
> t-to-build-the-perfect-team.html?smprod=nytcore-iphone&smid=nytcore-iphone-s
> hare
>
> Cheers
> Neal M. Ashkanasy, PhD
> UQ Business School
> The University of Queensland
> Brisbane, Qld 4072, Australia
> Phone: +617 3346-8006
> Fax: +617 3346-8188
> e-mail: n.ashkanasy@uq.edu.au
> https://www.business.uq.edu.au/staff/details/neal-ashkanasy

Yes Ben. It's sad those days are gone. It's not clear to me that academic research in our field has improved by not collaborating with practitioners or not publishing what are interesting or novel data. I suspect we need different forms of communication (not journals or journal articles) to develop this sort of knowledge.

Cheers

Rob

Rob B Briner
Professor of Organizational Psychology | School of Management | University of Bath
Scientific Director | Center for Evidence-Based Management (www.cebma.org)
Twitter @Rob_Briner


-----Original Message-----
From: Organizational Behavior Division Listserv [mailto:OB@AOMLISTS.PACE.EDU] On Behalf Of Benjamin Schneider
Sent: 28 February 2016 17:49
To: OB@AOMLISTS.PACE.EDU
Subject: Re: [OB-LIST] NYTimes & Google group study: difficulty to evaluate what is not disclosed...

Good discussion. There are literally thousands of I/O psychologists working in organizations and in consulting who every day do excellent work that contributes to their organizations' effectiveness. These folks of necessity speak the language of management--as does Duhigg--and translating what they do into theoretical academese is almost impossible. Right now academics tend to work with conceptual/theoretical problems while those in practice work with practical problems and bridging that gap to get published in research-focused journals is daunting to say the least. Some of us continue to try but the days of publishing some data/findings because they are interesting and potentially useful is now gone. Ben

Sent from my iPhone

> On Feb 28, 2016, at 8:05 AM, Fabrice Cavarretta <cavarretta@ESSEC.EDU> wrote:
>
> Dear colleagues:
>
> Regarding the debate about method/sources/rigor/relevance of research
> conducted by Google researchers on social sciences, let me share my
> experience in dealings with such organization for research purposes.
>
> I worked in Silicon Valley out of college, I therefore have contacts
> who are now at the Google, Facebook, etc., some in charge of massive
> data collection and analyses with a social science spin.
>
> We therefore explored the possibility of a sabbatical for me to come
> and work with them. The exploratory discussion confirmed what is
> public knowledge, that they have truly astounding data, orders of
> magnitude better than what armies of human PhDs will ever get.
>
> And their data mining capabilities are interesting beyond the fact
> that we can envision a future where they could test any quirky
> relationship (which they will). More interestingly, their algorithm
> will probably allow complete path analysis with large # of variables
> that would put to shame any structural equation modeling and
> endogeneity control ever done in our published papers.
>
> I know some of you will cringe, challenging the validity and
> apparently naïve hopes of such statements. Let's not debate this,
> let's remember these are stunning new tools, and as scientists we
> *should* have naïve attraction at any new tool to uncover and crack
> data ;)
>
> So, where does that lead us? Well, it is difficult for them to
> collaborate with us researchers from public institutions (i.e., not
> their employees), and even to disclose in any manner what they
> actually do! So (our) ignorance will remain the norm for the years to come!
>
> Let's just take a "public" event, the disclosure of a study that
> Facebook conducted where it manipulated the emotions of 600k+ users.
> And observe the public outcry at a manipulation that would have got
> the vetting of any IRB at the most conservative school
> http://www.theguardian.com/technology/2014/jun/29/facebook-users-emoti
> ons-ne
> ws-feeds
>
> Even though IRB would have vetted this in a traditional lab (give me a
> break, re-arranging order of news about your friends, is the harm
> substantial?), we should, as citizens, worry about such experiment at
> FB/GOO/MS/... as they reflect on the stunning public manipulation
> capabilities of those firms.
>
> However, as scientist and alumnus of Santa Fe Institute, I would not
> resist the possibility to go and conduct research with such tools!
>
> So such research are tricky endeavors as their disclosure could taint
> those firms in the public eye and their corporate objectives put their
> brand above everything. It implies that the secrecy level maintained
> by those firms on their research methods/data will remain at par with
> the ones maintained by some labs on their research on crypto, nuclear, biological weapons, etc.
>
> Bottom line: the Silicon Valley firms that have massive datasets (most
> have
> ;) are selectively hiring people from econ, some from org sciences
> (you know who you are ;) but disclosure is fuzzy and the publication
> output is at best tangential.
>
> Concluding it lacks rigor just because one does not see much of their
> pubs, or worse, just because it is relevant, is sad . It would also
> spell trouble for the PhD students we currently train as they will
> have to deal with such methods at some point in the future.
>
> The good news is that those firms collaborate discreetly with some
> universities. If you are from one of those schools, enjoy mingling
> with the future of data science. As far as I'm concerned, since there
> must be a penalty to be in France (because there are many advantages,
> trust me), it has been difficult to consider that sabbatical so far as
> my institution is not (yet?) part of that select group of mainly US
> schools whose researchers can be vetted to peek behind the veil. But
> I'm not losing hope ;)
>
> Best,
>
> Fabrice Cavarretta
> Associate Professor of
> Leadership and Entrepreneurship
> ESSEC Business School
> Mob : +33 6 09 59 46 74
> @fcavarrettaEN
> Author: "Oui! La France est un paradis pour entrepreneurs"
>
> -----Original Message-----
> From: Rob Briner [mailto:R.B.Briner@BATH.AC.UK]
> Sent: Saturday, February 27, 2016 11:27 AM
> Subject: Re: FW: NYTimes: What Google Learned From Its Quest to Build
> the Perfect Team
>
> Hi Neal
>
> Thanks for this. I don't know about you or anyone else but I find all
> these stories from Google (and some other organizations) about how
> wonderful they are at using evidence and data a bit perplexing. It
> always seems impossible to see, as it is in this article too, exactly
> what scientific findings they looked at, how they reviewed them, how
> they identified their quality and relevance, how they summarized or
> aggregated the evidence, and finally how and if they actually used it
> in their work. Also, in other articles and interviews, Google imply
> they completely ignore published scientific evidence and only rely on their own (big) data.
>
> I guess any story where any organization is saying how great it is at
> doing something needs to be taken with quite a large pinch of salt
> unless they are transparent and detailed about what they are doing and
> are open about their failures as well as successes. Also, what is
> Google's motive for telling us this?
>
> I'd be really interested to know if anyone has seen any transparent
> and detailed and critical account of what Google do around data and
> scientific evidence - it's doesn't seem to be in any of the public
> accounts I've seen but I could well be missing something.
>
> Cheers
>
> Rob
>
> Rob B Briner | Professor of Organizational Psychology | School of
> Management
> | University of Bath Scientific Director | Center for Evidence-Based
> Management (www.cebma.org) Twitter @Rob_Briner
>
> -----Original Message-----
> From: Organizational Behavior Division Listserv
> [mailto:OB@AOMLISTS.PACE.EDU] On Behalf Of Neal Ashkanasy
> Sent: 27 February 2016 00:29
> To: OB@AOMLISTS.PACE.EDU
> Subject: [OB-LIST] FW: NYTimes: What Google Learned From Its Quest to
> Build the Perfect Team
>
> Dear OB colleagues
>
> It's good to see that someone is reading our research and applying our
> findings!
>
> NY Times: New research reveals surprising truths about why some work
> groups thrive and others falter.
> http://www.nytimes.com/2016/02/28/magazine/what-google-learned-from-it
> s-ques
> t-to-build-the-perfect-team.html?smprod=nytcore-iphone&smid=nytcore-ip
> hone-s
> hare
>
> Cheers
> Neal M. Ashkanasy, PhD
> UQ Business School
> The University of Queensland
> Brisbane, Qld 4072, Australia
> Phone: +617 3346-8006
> Fax: +617 3346-8188
> e-mail: n.ashkanasy@uq.edu.au
> https://www.business.uq.edu.au/staff/details/neal-ashkanasy

Hi thanks very much for this Fabrice

There's lots of interesting stuff here. And sounds like you have some really directly relevant experience. I'm certainly not concluding it lacks rigor. I'm concluding we just don't know as it's not open, transparent, etc. All we get I success stories. This has some parallels with the pharmaceutical industry: Would we just believe a drug company that told us they had lots of great studies and brilliant data which provided their processes and products were highly effective with no side-effects? In that case this has led to demands for greater transparency and accountability which possibly aren't completely out of place here.

Of course, big companies can say whatever they want about their 'big data' and 'awesome' data analytic techniques but if we can't see exactly what was done then we are in no position to make our own judgements: We just don't know. And, again, it may be these people do highly detailed transparent presentations of all this - I'm just talking about what I can access in writing/news/websites etc. As it's the Oscars today, I'm wondering if sometimes academics and others are a little 'star-struck' by companies like Google.

Cheers

Rob

Rob B Briner
Professor of Organizational Psychology | School of Management | University of Bath
Scientific Director | Center for Evidence-Based Management (www.cebma.org)
Twitter @Rob_Briner

-----Original Message-----
From: Organizational Behavior Division Listserv [mailto:OB@AOMLISTS.PACE.EDU] On Behalf Of Fabrice Cavarretta
Sent: 28 February 2016 14:57
To: OB@AOMLISTS.PACE.EDU
Subject: Re: [OB-LIST] NYTimes & Google group study: difficulty to evaluate what is not disclosed...

Dear colleagues:

Regarding the debate about method/sources/rigor/relevance of research conducted by Google researchers on social sciences, let me share my experience in dealings with such organization for research purposes.

I worked in Silicon Valley out of college, I therefore have contacts who are now at the Google, Facebook, etc., some in charge of massive data collection and analyses with a social science spin.

We therefore explored the possibility of a sabbatical for me to come and work with them. The exploratory discussion confirmed what is public knowledge, that they have truly astounding data, orders of magnitude better than what armies of human PhDs will ever get.

And their data mining capabilities are interesting beyond the fact that we can envision a future where they could test any quirky relationship (which they will). More interestingly, their algorithm will probably allow complete path analysis with large # of variables that would put to shame any structural equation modeling and endogeneity control ever done in our published papers.

I know some of you will cringe, challenging the validity and apparently naïve hopes of such statements. Let's not debate this, let's remember these are stunning new tools, and as scientists we *should* have naïve attraction at any new tool to uncover and crack data ;)

So, where does that lead us? Well, it is difficult for them to collaborate with us researchers from public institutions (i.e., not their employees), and even to disclose in any manner what they actually do! So (our) ignorance will remain the norm for the years to come!

Let's just take a "public" event, the disclosure of a study that Facebook conducted where it manipulated the emotions of 600k+ users. And observe the public outcry at a manipulation that would have got the vetting of any IRB at the most conservative school http://www.theguardian.com/technology/2014/jun/29/facebook-users-emotions-ne
ws-feeds

Even though IRB would have vetted this in a traditional lab (give me a break, re-arranging order of news about your friends, is the harm substantial?), we should, as citizens, worry about such experiment at FB/GOO/MS/... as they reflect on the stunning public manipulation capabilities of those firms.

However, as scientist and alumnus of Santa Fe Institute, I would not resist the possibility to go and conduct research with such tools!

So such research are tricky endeavors as their disclosure could taint those firms in the public eye and their corporate objectives put their brand above everything. It implies that the secrecy level maintained by those firms on their research methods/data will remain at par with the ones maintained by some labs on their research on crypto, nuclear, biological weapons, etc.

Bottom line: the Silicon Valley firms that have massive datasets (most have
;) are selectively hiring people from econ, some from org sciences (you know who you are ;) but disclosure is fuzzy and the publication output is at best tangential.

Concluding it lacks rigor just because one does not see much of their pubs, or worse, just because it is relevant, is sad . It would also spell trouble for the PhD students we currently train as they will have to deal with such methods at some point in the future.

The good news is that those firms collaborate discreetly with some universities. If you are from one of those schools, enjoy mingling with the future of data science. As far as I'm concerned, since there must be a penalty to be in France (because there are many advantages, trust me), it has been difficult to consider that sabbatical so far as my institution is not (yet?) part of that select group of mainly US schools whose researchers can be vetted to peek behind the veil. But I'm not losing hope ;)

Best,

Fabrice Cavarretta
Associate Professor of
Leadership and Entrepreneurship
ESSEC Business School
Mob : +33 6 09 59 46 74
@fcavarrettaEN
Author: "Oui! La France est un paradis pour entrepreneurs"

-----Original Message-----
From: Rob Briner [mailto:R.B.Briner@BATH.AC.UK]
Sent: Saturday, February 27, 2016 11:27 AM
Subject: Re: FW: NYTimes: What Google Learned From Its Quest to Build the Perfect Team

Hi Neal

Thanks for this. I don't know about you or anyone else but I find all these stories from Google (and some other organizations) about how wonderful they are at using evidence and data a bit perplexing. It always seems impossible to see, as it is in this article too, exactly what scientific findings they looked at, how they reviewed them, how they identified their quality and relevance, how they summarized or aggregated the evidence, and finally how and if they actually used it in their work. Also, in other articles and interviews, Google imply they completely ignore published scientific evidence and only rely on their own (big) data.

I guess any story where any organization is saying how great it is at doing something needs to be taken with quite a large pinch of salt unless they are transparent and detailed about what they are doing and are open about their failures as well as successes. Also, what is Google's motive for telling us this?

I'd be really interested to know if anyone has seen any transparent and detailed and critical account of what Google do around data and scientific evidence - it's doesn't seem to be in any of the public accounts I've seen but I could well be missing something.

Cheers

Rob

Rob B Briner | Professor of Organizational Psychology | School of Management
| University of Bath Scientific Director | Center for Evidence-Based
Management (www.cebma.org) Twitter @Rob_Briner

-----Original Message-----
From: Organizational Behavior Division Listserv [mailto:OB@AOMLISTS.PACE.EDU] On Behalf Of Neal Ashkanasy
Sent: 27 February 2016 00:29
To: OB@AOMLISTS.PACE.EDU
Subject: [OB-LIST] FW: NYTimes: What Google Learned From Its Quest to Build the Perfect Team

Dear OB colleagues

It's good to see that someone is reading our research and applying our findings!

NY Times: New research reveals surprising truths about why some work groups thrive and others falter.
http://www.nytimes.com/2016/02/28/magazine/what-google-learned-from-its-ques
t-to-build-the-perfect-team.html?smprod=nytcore-iphone&smid=nytcore-iphone-s
hare

Cheers
Neal M. Ashkanasy, PhD
UQ Business School
The University of Queensland
Brisbane, Qld 4072, Australia
Phone: +617 3346-8006
Fax: +617 3346-8188
e-mail: n.ashkanasy@uq.edu.au
https://www.business.uq.edu.au/staff/details/neal-ashkanasy

Lacking transparency and publicizing only the successes - as opposed to the academic research literature that...... Oh wait - never mind

E. Kevin Kelloway, PhD
Canada Research Chair in Occupational Health Psychology
Professor of Psychology
President, Canadian Psychological Association

> On Feb 28, 2016, at 3:36 PM, Rob Briner <R.B.Briner@BATH.AC.UK> wrote:
>
> Hi thanks very much for this Fabrice
>
> There's lots of interesting stuff here. And sounds like you have some really directly relevant experience. I'm certainly not concluding it lacks rigor. I'm concluding we just don't know as it's not open, transparent, etc. All we get I success stories. This has some parallels with the pharmaceutical industry: Would we just believe a drug company that told us they had lots of great studies and brilliant data which provided their processes and products were highly effective with no side-effects? In that case this has led to demands for greater transparency and accountability which possibly aren't completely out of place here.
>
> Of course, big companies can say whatever they want about their 'big data' and 'awesome' data analytic techniques but if we can't see exactly what was done then we are in no position to make our own judgements: We just don't know. And, again, it may be these people do highly detailed transparent presentations of all this - I'm just talking about what I can access in writing/news/websites etc. As it's the Oscars today, I'm wondering if sometimes academics and others are a little 'star-struck' by companies like Google.
>
> Cheers
>
> Rob
>
> Rob B Briner
> Professor of Organizational Psychology | School of Management | University of Bath
> Scientific Director | Center for Evidence-Based Management (www.cebma.org)
> Twitter @Rob_Briner
>
> -----Original Message-----
> From: Organizational Behavior Division Listserv [mailto:OB@AOMLISTS.PACE.EDU] On Behalf Of Fabrice Cavarretta
> Sent: 28 February 2016 14:57
> To: OB@AOMLISTS.PACE.EDU
> Subject: Re: [OB-LIST] NYTimes & Google group study: difficulty to evaluate what is not disclosed...
>
> Dear colleagues:
>
> Regarding the debate about method/sources/rigor/relevance of research conducted by Google researchers on social sciences, let me share my experience in dealings with such organization for research purposes.
>
> I worked in Silicon Valley out of college, I therefore have contacts who are now at the Google, Facebook, etc., some in charge of massive data collection and analyses with a social science spin.
>
> We therefore explored the possibility of a sabbatical for me to come and work with them. The exploratory discussion confirmed what is public knowledge, that they have truly astounding data, orders of magnitude better than what armies of human PhDs will ever get.
>
> And their data mining capabilities are interesting beyond the fact that we can envision a future where they could test any quirky relationship (which they will). More interestingly, their algorithm will probably allow complete path analysis with large # of variables that would put to shame any structural equation modeling and endogeneity control ever done in our published papers.
>
> I know some of you will cringe, challenging the validity and apparently naïve hopes of such statements. Let's not debate this, let's remember these are stunning new tools, and as scientists we *should* have naïve attraction at any new tool to uncover and crack data ;)
>
> So, where does that lead us? Well, it is difficult for them to collaborate with us researchers from public institutions (i.e., not their employees), and even to disclose in any manner what they actually do! So (our) ignorance will remain the norm for the years to come!
>
> Let's just take a "public" event, the disclosure of a study that Facebook conducted where it manipulated the emotions of 600k+ users. And observe the public outcry at a manipulation that would have got the vetting of any IRB at the most conservative school http://www.theguardian.com/technology/2014/jun/29/facebook-users-emotions-ne
> ws-feeds
>
> Even though IRB would have vetted this in a traditional lab (give me a break, re-arranging order of news about your friends, is the harm substantial?), we should, as citizens, worry about such experiment at FB/GOO/MS/... as they reflect on the stunning public manipulation capabilities of those firms.
>
> However, as scientist and alumnus of Santa Fe Institute, I would not resist the possibility to go and conduct research with such tools!
>
> So such research are tricky endeavors as their disclosure could taint those firms in the public eye and their corporate objectives put their brand above everything. It implies that the secrecy level maintained by those firms on their research methods/data will remain at par with the ones maintained by some labs on their research on crypto, nuclear, biological weapons, etc.
>
> Bottom line: the Silicon Valley firms that have massive datasets (most have
> ;) are selectively hiring people from econ, some from org sciences (you know who you are ;) but disclosure is fuzzy and the publication output is at best tangential.
>
> Concluding it lacks rigor just because one does not see much of their pubs, or worse, just because it is relevant, is sad . It would also spell trouble for the PhD students we currently train as they will have to deal with such methods at some point in the future.
>
> The good news is that those firms collaborate discreetly with some universities. If you are from one of those schools, enjoy mingling with the future of data science. As far as I'm concerned, since there must be a penalty to be in France (because there are many advantages, trust me), it has been difficult to consider that sabbatical so far as my institution is not (yet?) part of that select group of mainly US schools whose researchers can be vetted to peek behind the veil. But I'm not losing hope ;)
>
> Best,
>
> Fabrice Cavarretta
> Associate Professor of
> Leadership and Entrepreneurship
> ESSEC Business School
> Mob : +33 6 09 59 46 74
> @fcavarrettaEN
> Author: "Oui! La France est un paradis pour entrepreneurs"
>
> -----Original Message-----
> From: Rob Briner [mailto:R.B.Briner@BATH.AC.UK]
> Sent: Saturday, February 27, 2016 11:27 AM
> Subject: Re: FW: NYTimes: What Google Learned From Its Quest to Build the Perfect Team
>
> Hi Neal
>
> Thanks for this. I don't know about you or anyone else but I find all these stories from Google (and some other organizations) about how wonderful they are at using evidence and data a bit perplexing. It always seems impossible to see, as it is in this article too, exactly what scientific findings they looked at, how they reviewed them, how they identified their quality and relevance, how they summarized or aggregated the evidence, and finally how and if they actually used it in their work. Also, in other articles and interviews, Google imply they completely ignore published scientific evidence and only rely on their own (big) data.
>
> I guess any story where any organization is saying how great it is at doing something needs to be taken with quite a large pinch of salt unless they are transparent and detailed about what they are doing and are open about their failures as well as successes. Also, what is Google's motive for telling us this?
>
> I'd be really interested to know if anyone has seen any transparent and detailed and critical account of what Google do around data and scientific evidence - it's doesn't seem to be in any of the public accounts I've seen but I could well be missing something.
>
> Cheers
>
> Rob
>
> Rob B Briner | Professor of Organizational Psychology | School of Management
> | University of Bath Scientific Director | Center for Evidence-Based
> Management (www.cebma.org) Twitter @Rob_Briner
>
> -----Original Message-----
> From: Organizational Behavior Division Listserv [mailto:OB@AOMLISTS.PACE.EDU] On Behalf Of Neal Ashkanasy
> Sent: 27 February 2016 00:29
> To: OB@AOMLISTS.PACE.EDU
> Subject: [OB-LIST] FW: NYTimes: What Google Learned From Its Quest to Build the Perfect Team
>
> Dear OB colleagues
>
> It's good to see that someone is reading our research and applying our findings!
>
> NY Times: New research reveals surprising truths about why some work groups thrive and others falter.
> http://www.nytimes.com/2016/02/28/magazine/what-google-learned-from-its-ques
> t-to-build-the-perfect-team.html?smprod=nytcore-iphone&smid=nytcore-iphone-s
> hare
>
> Cheers
> Neal M. Ashkanasy, PhD
> UQ Business School
> The University of Queensland
> Brisbane, Qld 4072, Australia
> Phone: +617 3346-8006
> Fax: +617 3346-8188
> e-mail: n.ashkanasy@uq.edu.au
> https://www.business.uq.edu.au/staff/details/neal-ashkanasy

Great note Kevin! Companies have higher standards for research than a less-than 40 percent chance of replication!

Sent from my iPhone

> On Feb 28, 2016, at 12:55 PM, Kevin Kelloway <Kevin.Kelloway@SMU.CA> wrote:
>
> Lacking transparency and publicizing only the successes - as opposed to the academic research literature that...... Oh wait - never mind
>
> E. Kevin Kelloway, PhD
> Canada Research Chair in Occupational Health Psychology
> Professor of Psychology
> President, Canadian Psychological Association
>
>> On Feb 28, 2016, at 3:36 PM, Rob Briner <R.B.Briner@BATH.AC.UK> wrote:
>>
>> Hi thanks very much for this Fabrice
>>
>> There's lots of interesting stuff here. And sounds like you have some really directly relevant experience. I'm certainly not concluding it lacks rigor. I'm concluding we just don't know as it's not open, transparent, etc. All we get I success stories. This has some parallels with the pharmaceutical industry: Would we just believe a drug company that told us they had lots of great studies and brilliant data which provided their processes and products were highly effective with no side-effects? In that case this has led to demands for greater transparency and accountability which possibly aren't completely out of place here.
>>
>> Of course, big companies can say whatever they want about their 'big data' and 'awesome' data analytic techniques but if we can't see exactly what was done then we are in no position to make our own judgements: We just don't know. And, again, it may be these people do highly detailed transparent presentations of all this - I'm just talking about what I can access in writing/news/websites etc. As it's the Oscars today, I'm wondering if sometimes academics and others are a little 'star-struck' by companies like Google.
>>
>> Cheers
>>
>> Rob
>>
>> Rob B Briner
>> Professor of Organizational Psychology | School of Management | University of Bath
>> Scientific Director | Center for Evidence-Based Management (www.cebma.org)
>> Twitter @Rob_Briner
>>
>> -----Original Message-----
>> From: Organizational Behavior Division Listserv [mailto:OB@AOMLISTS.PACE.EDU] On Behalf Of Fabrice Cavarretta
>> Sent: 28 February 2016 14:57
>> To: OB@AOMLISTS.PACE.EDU
>> Subject: Re: [OB-LIST] NYTimes & Google group study: difficulty to evaluate what is not disclosed...
>>
>> Dear colleagues:
>>
>> Regarding the debate about method/sources/rigor/relevance of research conducted by Google researchers on social sciences, let me share my experience in dealings with such organization for research purposes.
>>
>> I worked in Silicon Valley out of college, I therefore have contacts who are now at the Google, Facebook, etc., some in charge of massive data collection and analyses with a social science spin.
>>
>> We therefore explored the possibility of a sabbatical for me to come and work with them. The exploratory discussion confirmed what is public knowledge, that they have truly astounding data, orders of magnitude better than what armies of human PhDs will ever get.
>>
>> And their data mining capabilities are interesting beyond the fact that we can envision a future where they could test any quirky relationship (which they will). More interestingly, their algorithm will probably allow complete path analysis with large # of variables that would put to shame any structural equation modeling and endogeneity control ever done in our published papers.
>>
>> I know some of you will cringe, challenging the validity and apparently naïve hopes of such statements. Let's not debate this, let's remember these are stunning new tools, and as scientists we *should* have naïve attraction at any new tool to uncover and crack data ;)
>>
>> So, where does that lead us? Well, it is difficult for them to collaborate with us researchers from public institutions (i.e., not their employees), and even to disclose in any manner what they actually do! So (our) ignorance will remain the norm for the years to come!
>>
>> Let's just take a "public" event, the disclosure of a study that Facebook conducted where it manipulated the emotions of 600k+ users. And observe the public outcry at a manipulation that would have got the vetting of any IRB at the most conservative school http://www.theguardian.com/technology/2014/jun/29/facebook-users-emotions-ne
>> ws-feeds
>>
>> Even though IRB would have vetted this in a traditional lab (give me a break, re-arranging order of news about your friends, is the harm substantial?), we should, as citizens, worry about such experiment at FB/GOO/MS/... as they reflect on the stunning public manipulation capabilities of those firms.
>>
>> However, as scientist and alumnus of Santa Fe Institute, I would not resist the possibility to go and conduct research with such tools!
>>
>> So such research are tricky endeavors as their disclosure could taint those firms in the public eye and their corporate objectives put their brand above everything. It implies that the secrecy level maintained by those firms on their research methods/data will remain at par with the ones maintained by some labs on their research on crypto, nuclear, biological weapons, etc.
>>
>> Bottom line: the Silicon Valley firms that have massive datasets (most have
>> ;) are selectively hiring people from econ, some from org sciences (you know who you are ;) but disclosure is fuzzy and the publication output is at best tangential.
>>
>> Concluding it lacks rigor just because one does not see much of their pubs, or worse, just because it is relevant, is sad . It would also spell trouble for the PhD students we currently train as they will have to deal with such methods at some point in the future.
>>
>> The good news is that those firms collaborate discreetly with some universities. If you are from one of those schools, enjoy mingling with the future of data science. As far as I'm concerned, since there must be a penalty to be in France (because there are many advantages, trust me), it has been difficult to consider that sabbatical so far as my institution is not (yet?) part of that select group of mainly US schools whose researchers can be vetted to peek behind the veil. But I'm not losing hope ;)
>>
>> Best,
>>
>> Fabrice Cavarretta
>> Associate Professor of
>> Leadership and Entrepreneurship
>> ESSEC Business School
>> Mob : +33 6 09 59 46 74
>> @fcavarrettaEN
>> Author: "Oui! La France est un paradis pour entrepreneurs"
>>
>> -----Original Message-----
>> From: Rob Briner [mailto:R.B.Briner@BATH.AC.UK]
>> Sent: Saturday, February 27, 2016 11:27 AM
>> Subject: Re: FW: NYTimes: What Google Learned From Its Quest to Build the Perfect Team
>>
>> Hi Neal
>>
>> Thanks for this. I don't know about you or anyone else but I find all these stories from Google (and some other organizations) about how wonderful they are at using evidence and data a bit perplexing. It always seems impossible to see, as it is in this article too, exactly what scientific findings they looked at, how they reviewed them, how they identified their quality and relevance, how they summarized or aggregated the evidence, and finally how and if they actually used it in their work. Also, in other articles and interviews, Google imply they completely ignore published scientific evidence and only rely on their own (big) data.
>>
>> I guess any story where any organization is saying how great it is at doing something needs to be taken with quite a large pinch of salt unless they are transparent and detailed about what they are doing and are open about their failures as well as successes. Also, what is Google's motive for telling us this?
>>
>> I'd be really interested to know if anyone has seen any transparent and detailed and critical account of what Google do around data and scientific evidence - it's doesn't seem to be in any of the public accounts I've seen but I could well be missing something.
>>
>> Cheers
>>
>> Rob
>>
>> Rob B Briner | Professor of Organizational Psychology | School of Management
>> | University of Bath Scientific Director | Center for Evidence-Based
>> Management (www.cebma.org) Twitter @Rob_Briner
>>
>> -----Original Message-----
>> From: Organizational Behavior Division Listserv [mailto:OB@AOMLISTS.PACE.EDU] On Behalf Of Neal Ashkanasy
>> Sent: 27 February 2016 00:29
>> To: OB@AOMLISTS.PACE.EDU
>> Subject: [OB-LIST] FW: NYTimes: What Google Learned From Its Quest to Build the Perfect Team
>>
>> Dear OB colleagues
>>
>> It's good to see that someone is reading our research and applying our findings!
>>
>> NY Times: New research reveals surprising truths about why some work groups thrive and others falter.
>> http://www.nytimes.com/2016/02/28/magazine/what-google-learned-from-its-ques
>> t-to-build-the-perfect-team.html?smprod=nytcore-iphone&smid=nytcore-iphone-s
>> hare
>>
>> Cheers
>> Neal M. Ashkanasy, PhD
>> UQ Business School
>> The University of Queensland
>> Brisbane, Qld 4072, Australia
>> Phone: +617 3346-8006
>> Fax: +617 3346-8188
>> e-mail: n.ashkanasy@uq.edu.au
>> https://www.business.uq.edu.au/staff/details/neal-ashkanasy

Publication and other biases run through both practitioner and academic research. This is not good for science.
Hannah R. Rothstein, Ph.D.
Editor-in-Chief, Research Synthesis Methods
Department of Management
Zicklin School of Business
Baruch College--CUNY
1 Bernard Baruch Way
New York, NY 10010
USA

Visit Research Synthesis Methods
The official journal of the Society for Research Synthesis Methodology
at www.researchsynthesismethods.com

________________________________________
From: Organizational Behavior Division Listserv [OB@AOMLISTS.PACE.EDU] on behalf of Benjamin Schneider [Benj262@OUTLOOK.COM]
Sent: Sunday, February 28, 2016 4:14 PM
To: OB@AOMLISTS.PACE.EDU
Subject: Re: [OB-LIST] NYTimes & Google group study: difficulty to evaluate what is not disclosed...

Great note Kevin! Companies have higher standards for research than a less-than 40 percent chance of replication!

Sent from my iPhone

> On Feb 28, 2016, at 12:55 PM, Kevin Kelloway <Kevin.Kelloway@SMU.CA> wrote:
>
> Lacking transparency and publicizing only the successes - as opposed to the academic research literature that...... Oh wait - never mind
>
> E. Kevin Kelloway, PhD
> Canada Research Chair in Occupational Health Psychology
> Professor of Psychology
> President, Canadian Psychological Association
>
>> On Feb 28, 2016, at 3:36 PM, Rob Briner <R.B.Briner@BATH.AC.UK> wrote:
>>
>> Hi thanks very much for this Fabrice
>>
>> There's lots of interesting stuff here. And sounds like you have some really directly relevant experience. I'm certainly not concluding it lacks rigor. I'm concluding we just don't know as it's not open, transparent, etc. All we get I success stories. This has some parallels with the pharmaceutical industry: Would we just believe a drug company that told us they had lots of great studies and brilliant data which provided their processes and products were highly effective with no side-effects? In that case this has led to demands for greater transparency and accountability which possibly aren't completely out of place here.
>>
>> Of course, big companies can say whatever they want about their 'big data' and 'awesome' data analytic techniques but if we can't see exactly what was done then we are in no position to make our own judgements: We just don't know. And, again, it may be these people do highly detailed transparent presentations of all this - I'm just talking about what I can access in writing/news/websites etc. As it's the Oscars today, I'm wondering if sometimes academics and others are a little 'star-struck' by companies like Google.
>>
>> Cheers
>>
>> Rob
>>
>> Rob B Briner
>> Professor of Organizational Psychology | School of Management | University of Bath
>> Scientific Director | Center for Evidence-Based Management (www.cebma.org)
>> Twitter @Rob_Briner
>>
>> -----Original Message-----
>> From: Organizational Behavior Division Listserv [mailto:OB@AOMLISTS.PACE.EDU] On Behalf Of Fabrice Cavarretta
>> Sent: 28 February 2016 14:57
>> To: OB@AOMLISTS.PACE.EDU
>> Subject: Re: [OB-LIST] NYTimes & Google group study: difficulty to evaluate what is not disclosed...
>>
>> Dear colleagues:
>>
>> Regarding the debate about method/sources/rigor/relevance of research conducted by Google researchers on social sciences, let me share my experience in dealings with such organization for research purposes.
>>
>> I worked in Silicon Valley out of college, I therefore have contacts who are now at the Google, Facebook, etc., some in charge of massive data collection and analyses with a social science spin.
>>
>> We therefore explored the possibility of a sabbatical for me to come and work with them. The exploratory discussion confirmed what is public knowledge, that they have truly astounding data, orders of magnitude better than what armies of human PhDs will ever get.
>>
>> And their data mining capabilities are interesting beyond the fact that we can envision a future where they could test any quirky relationship (which they will). More interestingly, their algorithm will probably allow complete path analysis with large # of variables that would put to shame any structural equation modeling and endogeneity control ever done in our published papers.
>>
>> I know some of you will cringe, challenging the validity and apparently naïve hopes of such statements. Let's not debate this, let's remember these are stunning new tools, and as scientists we *should* have naïve attraction at any new tool to uncover and crack data ;)
>>
>> So, where does that lead us? Well, it is difficult for them to collaborate with us researchers from public institutions (i.e., not their employees), and even to disclose in any manner what they actually do! So (our) ignorance will remain the norm for the years to come!
>>
>> Let's just take a "public" event, the disclosure of a study that Facebook conducted where it manipulated the emotions of 600k+ users. And observe the public outcry at a manipulation that would have got the vetting of any IRB at the most conservative school http://www.theguardian.com/technology/2014/jun/29/facebook-users-emotions-ne
>> ws-feeds
>>
>> Even though IRB would have vetted this in a traditional lab (give me a break, re-arranging order of news about your friends, is the harm substantial?), we should, as citizens, worry about such experiment at FB/GOO/MS/... as they reflect on the stunning public manipulation capabilities of those firms.
>>
>> However, as scientist and alumnus of Santa Fe Institute, I would not resist the possibility to go and conduct research with such tools!
>>
>> So such research are tricky endeavors as their disclosure could taint those firms in the public eye and their corporate objectives put their brand above everything. It implies that the secrecy level maintained by those firms on their research methods/data will remain at par with the ones maintained by some labs on their research on crypto, nuclear, biological weapons, etc.
>>
>> Bottom line: the Silicon Valley firms that have massive datasets (most have
>> ;) are selectively hiring people from econ, some from org sciences (you know who you are ;) but disclosure is fuzzy and the publication output is at best tangential.
>>
>> Concluding it lacks rigor just because one does not see much of their pubs, or worse, just because it is relevant, is sad . It would also spell trouble for the PhD students we currently train as they will have to deal with such methods at some point in the future.
>>
>> The good news is that those firms collaborate discreetly with some universities. If you are from one of those schools, enjoy mingling with the future of data science. As far as I'm concerned, since there must be a penalty to be in France (because there are many advantages, trust me), it has been difficult to consider that sabbatical so far as my institution is not (yet?) part of that select group of mainly US schools whose researchers can be vetted to peek behind the veil. But I'm not losing hope ;)
>>
>> Best,
>>
>> Fabrice Cavarretta
>> Associate Professor of
>> Leadership and Entrepreneurship
>> ESSEC Business School
>> Mob : +33 6 09 59 46 74
>> @fcavarrettaEN
>> Author: "Oui! La France est un paradis pour entrepreneurs"
>>
>> -----Original Message-----
>> From: Rob Briner [mailto:R.B.Briner@BATH.AC.UK]
>> Sent: Saturday, February 27, 2016 11:27 AM
>> Subject: Re: FW: NYTimes: What Google Learned From Its Quest to Build the Perfect Team
>>
>> Hi Neal
>>
>> Thanks for this. I don't know about you or anyone else but I find all these stories from Google (and some other organizations) about how wonderful they are at using evidence and data a bit perplexing. It always seems impossible to see, as it is in this article too, exactly what scientific findings they looked at, how they reviewed them, how they identified their quality and relevance, how they summarized or aggregated the evidence, and finally how and if they actually used it in their work. Also, in other articles and interviews, Google imply they completely ignore published scientific evidence and only rely on their own (big) data.
>>
>> I guess any story where any organization is saying how great it is at doing something needs to be taken with quite a large pinch of salt unless they are transparent and detailed about what they are doing and are open about their failures as well as successes. Also, what is Google's motive for telling us this?
>>
>> I'd be really interested to know if anyone has seen any transparent and detailed and critical account of what Google do around data and scientific evidence - it's doesn't seem to be in any of the public accounts I've seen but I could well be missing something.
>>
>> Cheers
>>
>> Rob
>>
>> Rob B Briner | Professor of Organizational Psychology | School of Management
>> | University of Bath Scientific Director | Center for Evidence-Based
>> Management (www.cebma.org) Twitter @Rob_Briner
>>
>> -----Original Message-----
>> From: Organizational Behavior Division Listserv [mailto:OB@AOMLISTS.PACE.EDU] On Behalf Of Neal Ashkanasy
>> Sent: 27 February 2016 00:29
>> To: OB@AOMLISTS.PACE.EDU
>> Subject: [OB-LIST] FW: NYTimes: What Google Learned From Its Quest to Build the Perfect Team
>>
>> Dear OB colleagues
>>
>> It's good to see that someone is reading our research and applying our findings!
>>
>> NY Times: New research reveals surprising truths about why some work groups thrive and others falter.
>> http://www.nytimes.com/2016/02/28/magazine/what-google-learned-from-its-ques
>> t-to-build-the-perfect-team.html?smprod=nytcore-iphone&smid=nytcore-iphone-s
>> hare
>>
>> Cheers
>> Neal M. Ashkanasy, PhD
>> UQ Business School
>> The University of Queensland
>> Brisbane, Qld 4072, Australia
>> Phone: +617 3346-8006
>> Fax: +617 3346-8188
>> e-mail: n.ashkanasy@uq.edu.au
>> https://www.business.uq.edu.au/staff/details/neal-ashkanasy

Dear Neal, Rob, Ben et al.,

I have been watching this conversation with great interest and, as an academic, I agree with the sentiments and positions expressed.

Governments, funding bodies and universities are increasingly requesting (demanding?) information on the Impact of our research.

At QUT we are being asked to write "Impact Cases" about how people are using our research. It will not be easy to provide evidence based information unless you work directly with the organisation using your research, for example using Andy Van Der Ven's Engaged Research or free consulting in exchange for information and access on the impacts. I am sure we can all imagine the challenges ahead.

These challenges could be mitigated if Practitioners and Scholars can publish together, but most Academics are not rewarded for publishing in anything but peer reviewed, ranked journals.

I propose for discussion answering calls for impact with requests for recognition of publications in practitioner journals, i.e. Industry journals. These will by nature be "Success Stories". And we can wtite them with our Practitioner colleagues.

What think thee?

Cheers,
Roxanne

Dr. Roxanne Zolin | Associate Professor | School of Management
QUT Business School | Queensland University of Technology | www.qut.edu.au/business
Phone: + 61 7 3138 5095 | Mobile: 0433 400 113 | Email: r.zolin@qut.edu.au | CRICOS No. 00213J

________________________________________
From: Organizational Behavior Division Listserv <OB@AOMLISTS.PACE.EDU> on behalf of Hannah Rothstein <Hannah.Rothstein@BARUCH.CUNY.EDU>
Sent: Tuesday, 1 March 2016 12:25 AM
To: OB@AOMLISTS.PACE.EDU
Subject: Re: [OB-LIST] NYTimes & Google group study: difficulty to evaluate what is not disclosed...

Publication and other biases run through both practitioner and academic research. This is not good for science.
Hannah R. Rothstein, Ph.D.
Editor-in-Chief, Research Synthesis Methods
Department of Management
Zicklin School of Business
Baruch College--CUNY
1 Bernard Baruch Way
New York, NY 10010
USA

Visit Research Synthesis Methods
The official journal of the Society for Research Synthesis Methodology
at www.researchsynthesismethods.com

________________________________________
From: Organizational Behavior Division Listserv [OB@AOMLISTS.PACE.EDU] on behalf of Benjamin Schneider [Benj262@OUTLOOK.COM]
Sent: Sunday, February 28, 2016 4:14 PM
To: OB@AOMLISTS.PACE.EDU
Subject: Re: [OB-LIST] NYTimes & Google group study: difficulty to evaluate what is not disclosed...

Great note Kevin! Companies have higher standards for research than a less-than 40 percent chance of replication!

Sent from my iPhone

> On Feb 28, 2016, at 12:55 PM, Kevin Kelloway <Kevin.Kelloway@SMU.CA> wrote:
>
> Lacking transparency and publicizing only the successes - as opposed to the academic research literature that...... Oh wait - never mind
>
> E. Kevin Kelloway, PhD
> Canada Research Chair in Occupational Health Psychology
> Professor of Psychology
> President, Canadian Psychological Association
>
>> On Feb 28, 2016, at 3:36 PM, Rob Briner <R.B.Briner@BATH.AC.UK> wrote:
>>
>> Hi thanks very much for this Fabrice
>>
>> There's lots of interesting stuff here. And sounds like you have some really directly relevant experience. I'm certainly not concluding it lacks rigor. I'm concluding we just don't know as it's not open, transparent, etc. All we get I success stories. This has some parallels with the pharmaceutical industry: Would we just believe a drug company that told us they had lots of great studies and brilliant data which provided their processes and products were highly effective with no side-effects? In that case this has led to demands for greater transparency and accountability which possibly aren't completely out of place here.
>>
>> Of course, big companies can say whatever they want about their 'big data' and 'awesome' data analytic techniques but if we can't see exactly what was done then we are in no position to make our own judgements: We just don't know. And, again, it may be these people do highly detailed transparent presentations of all this - I'm just talking about what I can access in writing/news/websites etc. As it's the Oscars today, I'm wondering if sometimes academics and others are a little 'star-struck' by companies like Google.
>>
>> Cheers
>>
>> Rob
>>
>> Rob B Briner
>> Professor of Organizational Psychology | School of Management | University of Bath
>> Scientific Director | Center for Evidence-Based Management (www.cebma.org)
>> Twitter @Rob_Briner
>>
>> -----Original Message-----
>> From: Organizational Behavior Division Listserv [mailto:OB@AOMLISTS.PACE.EDU] On Behalf Of Fabrice Cavarretta
>> Sent: 28 February 2016 14:57
>> To: OB@AOMLISTS.PACE.EDU
>> Subject: Re: [OB-LIST] NYTimes & Google group study: difficulty to evaluate what is not disclosed...
>>
>> Dear colleagues:
>>
>> Regarding the debate about method/sources/rigor/relevance of research conducted by Google researchers on social sciences, let me share my experience in dealings with such organization for research purposes.
>>
>> I worked in Silicon Valley out of college, I therefore have contacts who are now at the Google, Facebook, etc., some in charge of massive data collection and analyses with a social science spin.
>>
>> We therefore explored the possibility of a sabbatical for me to come and work with them. The exploratory discussion confirmed what is public knowledge, that they have truly astounding data, orders of magnitude better than what armies of human PhDs will ever get.
>>
>> And their data mining capabilities are interesting beyond the fact that we can envision a future where they could test any quirky relationship (which they will). More interestingly, their algorithm will probably allow complete path analysis with large # of variables that would put to shame any structural equation modeling and endogeneity control ever done in our published papers.
>>
>> I know some of you will cringe, challenging the validity and apparently naïve hopes of such statements. Let's not debate this, let's remember these are stunning new tools, and as scientists we *should* have naïve attraction at any new tool to uncover and crack data ;)
>>
>> So, where does that lead us? Well, it is difficult for them to collaborate with us researchers from public institutions (i.e., not their employees), and even to disclose in any manner what they actually do! So (our) ignorance will remain the norm for the years to come!
>>
>> Let's just take a "public" event, the disclosure of a study that Facebook conducted where it manipulated the emotions of 600k+ users. And observe the public outcry at a manipulation that would have got the vetting of any IRB at the most conservative school http://www.theguardian.com/technology/2014/jun/29/facebook-users-emotions-ne
>> ws-feeds
>>
>> Even though IRB would have vetted this in a traditional lab (give me a break, re-arranging order of news about your friends, is the harm substantial?), we should, as citizens, worry about such experiment at FB/GOO/MS/... as they reflect on the stunning public manipulation capabilities of those firms.
>>
>> However, as scientist and alumnus of Santa Fe Institute, I would not resist the possibility to go and conduct research with such tools!
>>
>> So such research are tricky endeavors as their disclosure could taint those firms in the public eye and their corporate objectives put their brand above everything. It implies that the secrecy level maintained by those firms on their research methods/data will remain at par with the ones maintained by some labs on their research on crypto, nuclear, biological weapons, etc.
>>
>> Bottom line: the Silicon Valley firms that have massive datasets (most have
>> ;) are selectively hiring people from econ, some from org sciences (you know who you are ;) but disclosure is fuzzy and the publication output is at best tangential.
>>
>> Concluding it lacks rigor just because one does not see much of their pubs, or worse, just because it is relevant, is sad . It would also spell trouble for the PhD students we currently train as they will have to deal with such methods at some point in the future.
>>
>> The good news is that those firms collaborate discreetly with some universities. If you are from one of those schools, enjoy mingling with the future of data science. As far as I'm concerned, since there must be a penalty to be in France (because there are many advantages, trust me), it has been difficult to consider that sabbatical so far as my institution is not (yet?) part of that select group of mainly US schools whose researchers can be vetted to peek behind the veil. But I'm not losing hope ;)
>>
>> Best,
>>
>> Fabrice Cavarretta
>> Associate Professor of
>> Leadership and Entrepreneurship
>> ESSEC Business School
>> Mob : +33 6 09 59 46 74
>> @fcavarrettaEN
>> Author: "Oui! La France est un paradis pour entrepreneurs"
>>
>> -----Original Message-----
>> From: Rob Briner [mailto:R.B.Briner@BATH.AC.UK]
>> Sent: Saturday, February 27, 2016 11:27 AM
>> Subject: Re: FW: NYTimes: What Google Learned From Its Quest to Build the Perfect Team
>>
>> Hi Neal
>>
>> Thanks for this. I don't know about you or anyone else but I find all these stories from Google (and some other organizations) about how wonderful they are at using evidence and data a bit perplexing. It always seems impossible to see, as it is in this article too, exactly what scientific findings they looked at, how they reviewed them, how they identified their quality and relevance, how they summarized or aggregated the evidence, and finally how and if they actually used it in their work. Also, in other articles and interviews, Google imply they completely ignore published scientific evidence and only rely on their own (big) data.
>>
>> I guess any story where any organization is saying how great it is at doing something needs to be taken with quite a large pinch of salt unless they are transparent and detailed about what they are doing and are open about their failures as well as successes. Also, what is Google's motive for telling us this?
>>
>> I'd be really interested to know if anyone has seen any transparent and detailed and critical account of what Google do around data and scientific evidence - it's doesn't seem to be in any of the public accounts I've seen but I could well be missing something.
>>
>> Cheers
>>
>> Rob
>>
>> Rob B Briner | Professor of Organizational Psychology | School of Management
>> | University of Bath Scientific Director | Center for Evidence-Based
>> Management (www.cebma.org) Twitter @Rob_Briner
>>
>> -----Original Message-----
>> From: Organizational Behavior Division Listserv [mailto:OB@AOMLISTS.PACE.EDU] On Behalf Of Neal Ashkanasy
>> Sent: 27 February 2016 00:29
>> To: OB@AOMLISTS.PACE.EDU
>> Subject: [OB-LIST] FW: NYTimes: What Google Learned From Its Quest to Build the Perfect Team
>>
>> Dear OB colleagues
>>
>> It's good to see that someone is reading our research and applying our findings!
>>
>> NY Times: New research reveals surprising truths about why some work groups thrive and others falter.
>> http://www.nytimes.com/2016/02/28/magazine/what-google-learned-from-its-ques
>> t-to-build-the-perfect-team.html?smprod=nytcore-iphone&smid=nytcore-iphone-s
>> hare
>>
>> Cheers
>> Neal M. Ashkanasy, PhD
>> UQ Business School
>> The University of Queensland
>> Brisbane, Qld 4072, Australia
>> Phone: +617 3346-8006
>> Fax: +617 3346-8188
>> e-mail: n.ashkanasy@uq.edu.au
>> https://www.business.uq.edu.au/staff/details/neal-ashkanasy

Yes, I agree universities and funding bodies are trying a little harder which is good and a start.

However, I have major concerns with the notion of 'impact' as Universities typically think of it. In particular, single pieces of research from one university don't really matter in the scheme of things. What's important is what the whole body of evidence suggests about a problem or issue. Second, at least in the UK, the impact agenda is not about genuinely increasing the broader institutional links between research evidence and practice but rather incentivising particular research groups' interactions and research with particular organizations. This is fine, of course, but does nothing to help fix the wider problems which cause the 'gap'. Third, these indicators are being used as metrics to help rank universities and individuals (in the UK academics with impact cases are much sought after and may receive monetary rewards) with the inevitable consequence that they will be gamed - again detracting from the broader purpose. Also, I think looking to areas like medicine or policy making where systematic reviews (not new empirical research) as seen as the most important way for research to have 'impact' gives us some clues about how to really maximize the chances of the body of evidence we've generated over time gets used most effectively.

Woah - enough points already. Sorry.

Cheers

Rob

Rob B Briner | Professor of Organizational Psychology | School of Management | University of Bath
Scientific Director | Center for Evidence-Based Management (www.cebma.org)
Twitter @Rob_Briner

-----Original Message-----
From: Organizational Behavior Division Listserv [mailto:OB@AOMLISTS.PACE.EDU] On Behalf Of Roxanne Zolin
Sent: 29 February 2016 21:42
To: OB@AOMLISTS.PACE.EDU
Subject: Re: [OB-LIST] NYTimes & Google group study: difficulty to evaluate what is not disclosed...

Dear Neal, Rob, Ben et al.,

I have been watching this conversation with great interest and, as an academic, I agree with the sentiments and positions expressed.

Governments, funding bodies and universities are increasingly requesting (demanding?) information on the Impact of our research.

At QUT we are being asked to write "Impact Cases" about how people are using our research. It will not be easy to provide evidence based information unless you work directly with the organisation using your research, for example using Andy Van Der Ven's Engaged Research or free consulting in exchange for information and access on the impacts. I am sure we can all imagine the challenges ahead.

These challenges could be mitigated if Practitioners and Scholars can publish together, but most Academics are not rewarded for publishing in anything but peer reviewed, ranked journals.

I propose for discussion answering calls for impact with requests for recognition of publications in practitioner journals, i.e. Industry journals. These will by nature be "Success Stories". And we can wtite them with our Practitioner colleagues.

What think thee?

Cheers,
Roxanne

Dr. Roxanne Zolin | Associate Professor | School of Management QUT Business School | Queensland University of Technology | www.qut.edu.au/business
Phone: + 61 7 3138 5095 | Mobile: 0433 400 113 | Email: r.zolin@qut.edu.au | CRICOS No. 00213J

________________________________________
From: Organizational Behavior Division Listserv <OB@AOMLISTS.PACE.EDU> on behalf of Hannah Rothstein <Hannah.Rothstein@BARUCH.CUNY.EDU>
Sent: Tuesday, 1 March 2016 12:25 AM
To: OB@AOMLISTS.PACE.EDU
Subject: Re: [OB-LIST] NYTimes & Google group study: difficulty to evaluate what is not disclosed...

Publication and other biases run through both practitioner and academic research. This is not good for science.
Hannah R. Rothstein, Ph.D.
Editor-in-Chief, Research Synthesis Methods Department of Management Zicklin School of Business Baruch College--CUNY
1 Bernard Baruch Way
New York, NY 10010
USA

Visit Research Synthesis Methods
The official journal of the Society for Research Synthesis Methodology at www.researchsynthesismethods.com

________________________________________
From: Organizational Behavior Division Listserv [OB@AOMLISTS.PACE.EDU] on behalf of Benjamin Schneider [Benj262@OUTLOOK.COM]
Sent: Sunday, February 28, 2016 4:14 PM
To: OB@AOMLISTS.PACE.EDU
Subject: Re: [OB-LIST] NYTimes & Google group study: difficulty to evaluate what is not disclosed...

Great note Kevin! Companies have higher standards for research than a less-than 40 percent chance of replication!

Sent from my iPhone

> On Feb 28, 2016, at 12:55 PM, Kevin Kelloway <Kevin.Kelloway@SMU.CA> wrote:
>
> Lacking transparency and publicizing only the successes - as opposed
> to the academic research literature that...... Oh wait - never mind
>
> E. Kevin Kelloway, PhD
> Canada Research Chair in Occupational Health Psychology Professor of
> Psychology President, Canadian Psychological Association
>
>> On Feb 28, 2016, at 3:36 PM, Rob Briner <R.B.Briner@BATH.AC.UK> wrote:
>>
>> Hi thanks very much for this Fabrice
>>
>> There's lots of interesting stuff here. And sounds like you have some really directly relevant experience. I'm certainly not concluding it lacks rigor. I'm concluding we just don't know as it's not open, transparent, etc. All we get I success stories. This has some parallels with the pharmaceutical industry: Would we just believe a drug company that told us they had lots of great studies and brilliant data which provided their processes and products were highly effective with no side-effects? In that case this has led to demands for greater transparency and accountability which possibly aren't completely out of place here.
>>
>> Of course, big companies can say whatever they want about their 'big data' and 'awesome' data analytic techniques but if we can't see exactly what was done then we are in no position to make our own judgements: We just don't know. And, again, it may be these people do highly detailed transparent presentations of all this - I'm just talking about what I can access in writing/news/websites etc. As it's the Oscars today, I'm wondering if sometimes academics and others are a little 'star-struck' by companies like Google.
>>
>> Cheers
>>
>> Rob
>>
>> Rob B Briner
>> Professor of Organizational Psychology | School of Management |
>> University of Bath Scientific Director | Center for Evidence-Based
>> Management (www.cebma.org) Twitter @Rob_Briner
>>
>> -----Original Message-----
>> From: Organizational Behavior Division Listserv
>> [mailto:OB@AOMLISTS.PACE.EDU] On Behalf Of Fabrice Cavarretta
>> Sent: 28 February 2016 14:57
>> To: OB@AOMLISTS.PACE.EDU
>> Subject: Re: [OB-LIST] NYTimes & Google group study: difficulty to evaluate what is not disclosed...
>>
>> Dear colleagues:
>>
>> Regarding the debate about method/sources/rigor/relevance of research conducted by Google researchers on social sciences, let me share my experience in dealings with such organization for research purposes.
>>
>> I worked in Silicon Valley out of college, I therefore have contacts who are now at the Google, Facebook, etc., some in charge of massive data collection and analyses with a social science spin.
>>
>> We therefore explored the possibility of a sabbatical for me to come and work with them. The exploratory discussion confirmed what is public knowledge, that they have truly astounding data, orders of magnitude better than what armies of human PhDs will ever get.
>>
>> And their data mining capabilities are interesting beyond the fact that we can envision a future where they could test any quirky relationship (which they will). More interestingly, their algorithm will probably allow complete path analysis with large # of variables that would put to shame any structural equation modeling and endogeneity control ever done in our published papers.
>>
>> I know some of you will cringe, challenging the validity and
>> apparently naïve hopes of such statements. Let's not debate this,
>> let's remember these are stunning new tools, and as scientists we
>> *should* have naïve attraction at any new tool to uncover and crack
>> data ;)
>>
>> So, where does that lead us? Well, it is difficult for them to collaborate with us researchers from public institutions (i.e., not their employees), and even to disclose in any manner what they actually do! So (our) ignorance will remain the norm for the years to come!
>>
>> Let's just take a "public" event, the disclosure of a study that
>> Facebook conducted where it manipulated the emotions of 600k+ users.
>> And observe the public outcry at a manipulation that would have got
>> the vetting of any IRB at the most conservative school
>> http://www.theguardian.com/technology/2014/jun/29/facebook-users-emot
>> ions-ne
>> ws-feeds
>>
>> Even though IRB would have vetted this in a traditional lab (give me a break, re-arranging order of news about your friends, is the harm substantial?), we should, as citizens, worry about such experiment at FB/GOO/MS/... as they reflect on the stunning public manipulation capabilities of those firms.
>>
>> However, as scientist and alumnus of Santa Fe Institute, I would not resist the possibility to go and conduct research with such tools!
>>
>> So such research are tricky endeavors as their disclosure could taint those firms in the public eye and their corporate objectives put their brand above everything. It implies that the secrecy level maintained by those firms on their research methods/data will remain at par with the ones maintained by some labs on their research on crypto, nuclear, biological weapons, etc.
>>
>> Bottom line: the Silicon Valley firms that have massive datasets
>> (most have
>> ;) are selectively hiring people from econ, some from org sciences (you know who you are ;) but disclosure is fuzzy and the publication output is at best tangential.
>>
>> Concluding it lacks rigor just because one does not see much of their pubs, or worse, just because it is relevant, is sad . It would also spell trouble for the PhD students we currently train as they will have to deal with such methods at some point in the future.
>>
>> The good news is that those firms collaborate discreetly with some
>> universities. If you are from one of those schools, enjoy mingling
>> with the future of data science. As far as I'm concerned, since there
>> must be a penalty to be in France (because there are many advantages,
>> trust me), it has been difficult to consider that sabbatical so far
>> as my institution is not (yet?) part of that select group of mainly
>> US schools whose researchers can be vetted to peek behind the veil.
>> But I'm not losing hope ;)
>>
>> Best,
>>
>> Fabrice Cavarretta
>> Associate Professor of
>> Leadership and Entrepreneurship
>> ESSEC Business School
>> Mob : +33 6 09 59 46 74
>> @fcavarrettaEN
>> Author: "Oui! La France est un paradis pour entrepreneurs"
>>
>> -----Original Message-----
>> From: Rob Briner [mailto:R.B.Briner@BATH.AC.UK]
>> Sent: Saturday, February 27, 2016 11:27 AM
>> Subject: Re: FW: NYTimes: What Google Learned From Its Quest to Build
>> the Perfect Team
>>
>> Hi Neal
>>
>> Thanks for this. I don't know about you or anyone else but I find all these stories from Google (and some other organizations) about how wonderful they are at using evidence and data a bit perplexing. It always seems impossible to see, as it is in this article too, exactly what scientific findings they looked at, how they reviewed them, how they identified their quality and relevance, how they summarized or aggregated the evidence, and finally how and if they actually used it in their work. Also, in other articles and interviews, Google imply they completely ignore published scientific evidence and only rely on their own (big) data.
>>
>> I guess any story where any organization is saying how great it is at doing something needs to be taken with quite a large pinch of salt unless they are transparent and detailed about what they are doing and are open about their failures as well as successes. Also, what is Google's motive for telling us this?
>>
>> I'd be really interested to know if anyone has seen any transparent and detailed and critical account of what Google do around data and scientific evidence - it's doesn't seem to be in any of the public accounts I've seen but I could well be missing something.
>>
>> Cheers
>>
>> Rob
>>
>> Rob B Briner | Professor of Organizational Psychology | School of
>> Management
>> | University of Bath Scientific Director | Center for Evidence-Based
>> Management (www.cebma.org) Twitter @Rob_Briner
>>
>> -----Original Message-----
>> From: Organizational Behavior Division Listserv
>> [mailto:OB@AOMLISTS.PACE.EDU] On Behalf Of Neal Ashkanasy
>> Sent: 27 February 2016 00:29
>> To: OB@AOMLISTS.PACE.EDU
>> Subject: [OB-LIST] FW: NYTimes: What Google Learned From Its Quest to
>> Build the Perfect Team
>>
>> Dear OB colleagues
>>
>> It's good to see that someone is reading our research and applying our findings!
>>
>> NY Times: New research reveals surprising truths about why some work groups thrive and others falter.
>> http://www.nytimes.com/2016/02/28/magazine/what-google-learned-from-i
>> ts-ques
>> t-to-build-the-perfect-team.html?smprod=nytcore-iphone&smid=nytcore-i
>> phone-s
>> hare
>>
>> Cheers
>> Neal M. Ashkanasy, PhD
>> UQ Business School
>> The University of Queensland
>> Brisbane, Qld 4072, Australia
>> Phone: +617 3346-8006
>> Fax: +617 3346-8188
>> e-mail: n.ashkanasy@uq.edu.au
>> https://www.business.uq.edu.au/staff/details/neal-ashkanasy

Yeah quite. Was going to add that as a punchline but thought it might be too much. But indeed. If we can't trust academic research. And we can't trust commercial research then. Well. We've got a lot of work to do clean up our acts...

Rob B Briner
Professor of Organizational Psychology | School of Management | University of Bath
Scientific Director | Center for Evidence-Based Management (www.cebma.org)
Twitter @Rob_Briner


-----Original Message-----
From: Organizational Behavior Division Listserv [mailto:OB@AOMLISTS.PACE.EDU] On Behalf Of Kevin Kelloway
Sent: 28 February 2016 20:43
To: OB@AOMLISTS.PACE.EDU
Subject: Re: [OB-LIST] NYTimes & Google group study: difficulty to evaluate what is not disclosed...

Lacking transparency and publicizing only the successes - as opposed to the academic research literature that...... Oh wait - never mind

E. Kevin Kelloway, PhD
Canada Research Chair in Occupational Health Psychology Professor of Psychology President, Canadian Psychological Association

> On Feb 28, 2016, at 3:36 PM, Rob Briner <R.B.Briner@BATH.AC.UK> wrote:
>
> Hi thanks very much for this Fabrice
>
> There's lots of interesting stuff here. And sounds like you have some really directly relevant experience. I'm certainly not concluding it lacks rigor. I'm concluding we just don't know as it's not open, transparent, etc. All we get I success stories. This has some parallels with the pharmaceutical industry: Would we just believe a drug company that told us they had lots of great studies and brilliant data which provided their processes and products were highly effective with no side-effects? In that case this has led to demands for greater transparency and accountability which possibly aren't completely out of place here.
>
> Of course, big companies can say whatever they want about their 'big data' and 'awesome' data analytic techniques but if we can't see exactly what was done then we are in no position to make our own judgements: We just don't know. And, again, it may be these people do highly detailed transparent presentations of all this - I'm just talking about what I can access in writing/news/websites etc. As it's the Oscars today, I'm wondering if sometimes academics and others are a little 'star-struck' by companies like Google.
>
> Cheers
>
> Rob
>
> Rob B Briner
> Professor of Organizational Psychology | School of Management |
> University of Bath Scientific Director | Center for Evidence-Based
> Management (www.cebma.org) Twitter @Rob_Briner
>
> -----Original Message-----
> From: Organizational Behavior Division Listserv
> [mailto:OB@AOMLISTS.PACE.EDU] On Behalf Of Fabrice Cavarretta
> Sent: 28 February 2016 14:57
> To: OB@AOMLISTS.PACE.EDU
> Subject: Re: [OB-LIST] NYTimes & Google group study: difficulty to evaluate what is not disclosed...
>
> Dear colleagues:
>
> Regarding the debate about method/sources/rigor/relevance of research conducted by Google researchers on social sciences, let me share my experience in dealings with such organization for research purposes.
>
> I worked in Silicon Valley out of college, I therefore have contacts who are now at the Google, Facebook, etc., some in charge of massive data collection and analyses with a social science spin.
>
> We therefore explored the possibility of a sabbatical for me to come and work with them. The exploratory discussion confirmed what is public knowledge, that they have truly astounding data, orders of magnitude better than what armies of human PhDs will ever get.
>
> And their data mining capabilities are interesting beyond the fact that we can envision a future where they could test any quirky relationship (which they will). More interestingly, their algorithm will probably allow complete path analysis with large # of variables that would put to shame any structural equation modeling and endogeneity control ever done in our published papers.
>
> I know some of you will cringe, challenging the validity and
> apparently naïve hopes of such statements. Let's not debate this,
> let's remember these are stunning new tools, and as scientists we
> *should* have naïve attraction at any new tool to uncover and crack
> data ;)
>
> So, where does that lead us? Well, it is difficult for them to collaborate with us researchers from public institutions (i.e., not their employees), and even to disclose in any manner what they actually do! So (our) ignorance will remain the norm for the years to come!
>
> Let's just take a "public" event, the disclosure of a study that
> Facebook conducted where it manipulated the emotions of 600k+ users.
> And observe the public outcry at a manipulation that would have got
> the vetting of any IRB at the most conservative school
> http://www.theguardian.com/technology/2014/jun/29/facebook-users-emoti
> ons-ne
> ws-feeds
>
> Even though IRB would have vetted this in a traditional lab (give me a break, re-arranging order of news about your friends, is the harm substantial?), we should, as citizens, worry about such experiment at FB/GOO/MS/... as they reflect on the stunning public manipulation capabilities of those firms.
>
> However, as scientist and alumnus of Santa Fe Institute, I would not resist the possibility to go and conduct research with such tools!
>
> So such research are tricky endeavors as their disclosure could taint those firms in the public eye and their corporate objectives put their brand above everything. It implies that the secrecy level maintained by those firms on their research methods/data will remain at par with the ones maintained by some labs on their research on crypto, nuclear, biological weapons, etc.
>
> Bottom line: the Silicon Valley firms that have massive datasets (most
> have
> ;) are selectively hiring people from econ, some from org sciences (you know who you are ;) but disclosure is fuzzy and the publication output is at best tangential.
>
> Concluding it lacks rigor just because one does not see much of their pubs, or worse, just because it is relevant, is sad . It would also spell trouble for the PhD students we currently train as they will have to deal with such methods at some point in the future.
>
> The good news is that those firms collaborate discreetly with some
> universities. If you are from one of those schools, enjoy mingling
> with the future of data science. As far as I'm concerned, since there
> must be a penalty to be in France (because there are many advantages,
> trust me), it has been difficult to consider that sabbatical so far as
> my institution is not (yet?) part of that select group of mainly US
> schools whose researchers can be vetted to peek behind the veil. But
> I'm not losing hope ;)
>
> Best,
>
> Fabrice Cavarretta
> Associate Professor of
> Leadership and Entrepreneurship
> ESSEC Business School
> Mob : +33 6 09 59 46 74
> @fcavarrettaEN
> Author: "Oui! La France est un paradis pour entrepreneurs"
>
> -----Original Message-----
> From: Rob Briner [mailto:R.B.Briner@BATH.AC.UK]
> Sent: Saturday, February 27, 2016 11:27 AM
> Subject: Re: FW: NYTimes: What Google Learned From Its Quest to Build
> the Perfect Team
>
> Hi Neal
>
> Thanks for this. I don't know about you or anyone else but I find all these stories from Google (and some other organizations) about how wonderful they are at using evidence and data a bit perplexing. It always seems impossible to see, as it is in this article too, exactly what scientific findings they looked at, how they reviewed them, how they identified their quality and relevance, how they summarized or aggregated the evidence, and finally how and if they actually used it in their work. Also, in other articles and interviews, Google imply they completely ignore published scientific evidence and only rely on their own (big) data.
>
> I guess any story where any organization is saying how great it is at doing something needs to be taken with quite a large pinch of salt unless they are transparent and detailed about what they are doing and are open about their failures as well as successes. Also, what is Google's motive for telling us this?
>
> I'd be really interested to know if anyone has seen any transparent and detailed and critical account of what Google do around data and scientific evidence - it's doesn't seem to be in any of the public accounts I've seen but I could well be missing something.
>
> Cheers
>
> Rob
>
> Rob B Briner | Professor of Organizational Psychology | School of
> Management
> | University of Bath Scientific Director | Center for Evidence-Based
> Management (www.cebma.org) Twitter @Rob_Briner
>
> -----Original Message-----
> From: Organizational Behavior Division Listserv
> [mailto:OB@AOMLISTS.PACE.EDU] On Behalf Of Neal Ashkanasy
> Sent: 27 February 2016 00:29
> To: OB@AOMLISTS.PACE.EDU
> Subject: [OB-LIST] FW: NYTimes: What Google Learned From Its Quest to
> Build the Perfect Team
>
> Dear OB colleagues
>
> It's good to see that someone is reading our research and applying our findings!
>
> NY Times: New research reveals surprising truths about why some work groups thrive and others falter.
> http://www.nytimes.com/2016/02/28/magazine/what-google-learned-from-it
> s-ques
> t-to-build-the-perfect-team.html?smprod=nytcore-iphone&smid=nytcore-ip
> hone-s
> hare
>
> Cheers
> Neal M. Ashkanasy, PhD
> UQ Business School
> The University of Queensland
> Brisbane, Qld 4072, Australia
> Phone: +617 3346-8006
> Fax: +617 3346-8188
> e-mail: n.ashkanasy@uq.edu.au
> https://www.business.uq.edu.au/staff/details/neal-ashkanasy