Discussion: View Thread

  • 1.  distributing questionnaires online using social networks

    Posted 02-07-2010 14:07
    Dear colleagues

    A student of mine intends to use a web based survey and an idea was
    mentioned to distribute the survey on one of the social networks (such
    as Facebook etc) to increase the number of responses. He targets the
    general population and the questionnaire is anonymous. Has anyone used
    this method before? what are the expected problems?
    any feedback or advice is appreciated.

    E.

    --
    ________________________
    Prof. Eran Vigoda-Gadot, Head
    School of Political Sciences,
    Center for Public Management & Policy (CPMP),
    The University of Haifa
    Haifa 31905 Israel

    __________________________
    Voice: 972-4-8240709
    Fax: 972-4-8257785
    e-mail: eranv@poli.haifa.ac.il
    eranv@research.haifa.ac.il
    Home Page: http://poli.haifa.ac.il/~eranv/
    CPMP-Center for Public Management & Policy: http://cpmp.hevra.haifa.ac.il/
    MLGA Program: http://poli.haifa.ac.il/~eranv/material_vigoda/2ND.html


  • 2.  distributing questionnaires online using social networks

    Posted 02-07-2010 18:29

    Hi Eran,

    This is a great question and I am looking forward to hearing from others on the listserve that are collecting data through social network sites.

    Last summer I used Facebook and Linkedin to collect data from alumni groups.  The first question I would consider is if your student intends on posting a link on a groups page or sending direct emails to people through the site. Response rate will be low if you post it in a general area, but there are more IRB issues if you send direct emails.  Also if you send direct emails Facebook thinks you are sending spam and will not let you send more than 25 at a time.  I would be happy to talk to your student about my experience and things to look out for.

     

    Best of luck,

    Emily Amdurer


    Emily Amdurer
    Department of Organizational Behavior
    Case Western Reserve University, Weatherhead School of Management
    E-mail: emilyamdurer@gmail.com; emily.amdurer@case.edu
    Tel: (917) 623-5214


    On Sun, Feb 7, 2010 at 2:06 PM, Eran Vigoda-Gadot <eranv@poli.haifa.ac.il> wrote:
    Dear colleagues

    A student of mine intends to use a web based survey and an idea was mentioned to distribute the survey on one of the social networks (such as Facebook etc) to increase the number of responses. He targets the general population and the questionnaire is anonymous. Has anyone used this method before? what are the expected problems?
    any feedback or advice is appreciated.

    E.

    --
    ________________________
    Prof. Eran Vigoda-Gadot, Head
    School of Political Sciences,
    Center for Public Management & Policy (CPMP),
    The University of Haifa
    Haifa 31905 Israel

    __________________________
    Voice: 972-4-8240709
    Fax: 972-4-8257785
    e-mail: eranv@poli.haifa.ac.il
             eranv@research.haifa.ac.il
    Home Page: http://poli.haifa.ac.il/~eranv/
    CPMP-Center for Public Management & Policy: http://cpmp.hevra.haifa.ac.il/ MLGA Program: http://poli.haifa.ac.il/~eranv/material_vigoda/2ND.html






  • 3.  distributing questionnaires online using social networks

    Posted 02-07-2010 18:40
    All-

    My students and I explored this possibility. When you get into the fine print of the user agreements on Facebook it becomes very clear that it is against their rules to use the site for this purpose. We have sought volunteers through Craigslist, however, with considerable success.

    Linda Skitka


    On Feb 7, 2010, at 1:06 PM, Eran Vigoda-Gadot wrote:

    Dear colleagues

    A student of mine intends to use a web based survey and an idea was mentioned to distribute the survey on one of the social networks (such as Facebook etc) to increase the number of responses. He targets the general population and the questionnaire is anonymous. Has anyone used this method before? what are the expected problems?
    any feedback or advice is appreciated.

    E.

    --
    ________________________
    Prof. Eran Vigoda-Gadot, Head
    School of Political Sciences,
    Center for Public Management & Policy (CPMP),
    The University of Haifa
    Haifa 31905 Israel

    __________________________
    Voice: 972-4-8240709
    Fax: 972-4-8257785
    e-mail: eranv@poli.haifa.ac.il
             eranv@research.haifa.ac.il
    Home Page: http://poli.haifa.ac.il/~eranv/
    CPMP-Center for Public Management & Policy: http://cpmp.hevra.haifa.ac.il/ MLGA Program: http://poli.haifa.ac.il/~eranv/material_vigoda/2ND.html

    <eranv.vcf>




    ________________________________________
    Linda J. Skitka, Ph.D., Professor of Psychology
    Social and Personality Division Chair
    University of Illinois at Chicago
    Department of Psychology (m/c 285)
    1007 W. Harrison St.
    Chicago, IL   60607-7137

    Phone: (312) 996-4464
    Fax: (312) 413-4122
    ISJR Homepage:  Http://www.isjr.org






  • 4.  distributing questionnaires online using social networks

    Posted 02-07-2010 20:36
    I will suggest you to read the following two posts regarding online survey
    (and be prepared for a good laugh). The second one (after page 3) has some
    very insightful observations regarding pitfalls of collecting data via
    online survey.


    http://chronicle.com/forums/index.php/topic,59061.msg1233365.html#msg1233365

    http://chronicle.com/forums/index.php/topic,59065.msg1233493.html#msg1233493

    Best,

    Bing

    --------------------------------------------
    Bing Ran
    Assistant Professor of Public Administration
    School of Public Affairs
    Penn State Harrisburg
    777 W. Harrisburg Pike
    Middletown, PA 17057
    Office: W160 A Olmsted Bldg.
    Phone: +1 717 948 6057
    Fax: +1 717 948 6320
    Email: bingran@psu.edu




    -----Original Message-----
    From: Organizational Behavior Division Listserv
    [mailto:OB@AOMLISTS.PACE.EDU] On Behalf Of Eran Vigoda-Gadot
    Sent: Sunday, February 07, 2010 2:07 PM
    To: OB@AOMLISTS.PACE.EDU
    Subject: [OB-LIST] distributing questionnaires online using social networks

    Dear colleagues

    A student of mine intends to use a web based survey and an idea was
    mentioned to distribute the survey on one of the social networks (such as
    Facebook etc) to increase the number of responses. He targets the general
    population and the questionnaire is anonymous. Has anyone used this method
    before? what are the expected problems?
    any feedback or advice is appreciated.

    E.

    --
    ________________________
    Prof. Eran Vigoda-Gadot, Head
    School of Political Sciences,
    Center for Public Management & Policy (CPMP), The University of Haifa Haifa
    31905 Israel

    __________________________
    Voice: 972-4-8240709
    Fax: 972-4-8257785
    e-mail: eranv@poli.haifa.ac.il
    eranv@research.haifa.ac.il
    Home Page: http://poli.haifa.ac.il/~eranv/ CPMP-Center for Public Management
    & Policy: http://cpmp.hevra.haifa.ac.il/ MLGA Program:
    http://poli.haifa.ac.il/~eranv/material_vigoda/2ND.html


  • 5.  distributing questionnaires online using social networks

    Posted 02-08-2010 01:51

    I love those two forum links!  Great examples of how negative online responses are inversely correlated with trust!

     

    Social networking sites can be useful collection tools if used cautiously to target specific groups, but they probably will not help your student if the goal is to increase responses from the general population.  Most likely, non-parametric analyses will need to be run and (depending on the topic) Facebook responses may throw off the results if combined with previously collected data.  I've conducted a number of studies using targeted online populations and there can be fundamental differences (e.g., age, gender, Big 5 personality) between not just the populations of forums, Facebook, Craigslist, and so on, but the types of individuals who respond to questionnaires based on who is asking.  The people in our social network are more likely to be similar to us.  Since you need to control for the different populations anyway, their inclusion may or may not add to the study.

     

    If the student is dead set on using Facebook, one approach I have not yet tried myself is to purchase ad space that links to the questionnaire.  For example, one might buy 1000 ad clicks through Facebook.  I do not know the specifics, but I would imagine you could do this to target the either the general Facebook population or specific subgroups (by linking the ad to keywords).  Hope this helps!

     

    Cheers,

    Tim

     

    -----------------------------------------

    Timothy C. Lisk

    PhD student, Industrial/Organizational Psychology

    Claremont Graduate University

    http://claremont.academia.edu/TimLisk

     



  • 6.  distributing questionnaires online using social networks

    Posted 02-08-2010 08:55
    Eran,
    Facebook won't allow you to distribute multiple emails to non-friend recipients. If you attempt it, Facebook warns you, then it blocks you from accessing your account. As an alternative, I would suggest posting a link they can follow. However, the major pitfall is that the link will be limited only to your student's circle of friends, not the general population. Another method is to create a "group" that highlights the purpose of the research, and solicits individual participation. This approach is allowed by facebook, but response rates will be low. I wish I could be more optimistic, but I have not found Facebook to be a useful tool in data collection, largely because of the methods they use to mitigate spam.

    Sincerely,

    Tim Munyon



    -----Original Message-----
    From: Organizational Behavior Division Listserv [mailto:OB@AOMLISTS.PACE.EDU] On Behalf Of Eran Vigoda-Gadot
    Sent: Sunday, February 07, 2010 2:07 PM
    To: OB@AOMLISTS.PACE.EDU
    Subject: [OB-LIST] distributing questionnaires online using social networks

    Dear colleagues

    A student of mine intends to use a web based survey and an idea was mentioned to distribute the survey on one of the social networks (such as Facebook etc) to increase the number of responses. He targets the general population and the questionnaire is anonymous. Has anyone used this method before? what are the expected problems?
    any feedback or advice is appreciated.

    E.

    --
    ________________________
    Prof. Eran Vigoda-Gadot, Head
    School of Political Sciences,
    Center for Public Management & Policy (CPMP), The University of Haifa Haifa 31905 Israel

    __________________________
    Voice: 972-4-8240709
    Fax: 972-4-8257785
    e-mail: eranv@poli.haifa.ac.il
    eranv@research.haifa.ac.il
    Home Page: http://poli.haifa.ac.il/~eranv/ CPMP-Center for Public Management & Policy: http://cpmp.hevra.haifa.ac.il/ MLGA Program: http://poli.haifa.ac.il/~eranv/material_vigoda/2ND.html


  • 7.  distributing questionnaires online using social networks

    Posted 02-08-2010 10:44

    I believe that if what you want is to get opinions about social trends or social issues of universal interest, then the online networks may do the job.  And even if they cases I am not so sure.  But if what you want is to study a specific phenomenon with interest on a particular segment of the population, then nothing else can beat to good old targeted survey! 

     

    Thanks,

     

    Ivan

     

     

    From: Organizational Behavior Division Listserv [mailto:OB@AOMLISTS.PACE.EDU] On Behalf Of Linda J. Skitka
    Sent: Sunday, February 07, 2010 5:40 PM
    To: OB@AOMLISTS.PACE.EDU
    Subject: Re: [OB-LIST] distributing questionnaires online using social networks

     

    All-

     

    My students and I explored this possibility. When you get into the fine print of the user agreements on Facebook it becomes very clear that it is against their rules to use the site for this purpose. We have sought volunteers through Craigslist, however, with considerable success.

     

    Linda Skitka

     

     

    On Feb 7, 2010, at 1:06 PM, Eran Vigoda-Gadot wrote:



    Dear colleagues

    A student of mine intends to use a web based survey and an idea was mentioned to distribute the survey on one of the social networks (such as Facebook etc) to increase the number of responses. He targets the general population and the questionnaire is anonymous. Has anyone used this method before? what are the expected problems?
    any feedback or advice is appreciated.

    E.

    --
    ________________________
    Prof. Eran Vigoda-Gadot, Head
    School of Political Sciences,
    Center for Public Management & Policy (CPMP),
    The University of Haifa
    Haifa 31905 Israel

    __________________________
    Voice: 972-4-8240709
    Fax: 972-4-8257785
    e-mail: eranv@poli.haifa.ac.il
             eranv@research.haifa.ac.il
    Home Page: http://poli.haifa.ac.il/~eranv/
    CPMP-Center for Public Management & Policy: http://cpmp.hevra.haifa.ac.il/ MLGA Program: http://poli.haifa.ac.il/~eranv/material_vigoda/2ND.html

    <eranv.vcf>

     

     

     

    ________________________________________

    Linda J. Skitka, Ph.D., Professor of Psychology

    Social and Personality Division Chair

    University of Illinois at Chicago

    Department of Psychology (m/c 285)

    1007 W. Harrison St.

    Chicago, IL   60607-7137

     

    Phone: (312) 996-4464

    Fax: (312) 413-4122

    ISJR Homepage:  Http://www.isjr.org

     



     



  • 8.  distributing questionnaires online using social networks

    Posted 02-08-2010 17:56

    Eran,

    The issue you bring up is a central one that often gets ignored compared to measures and newest analytic technique of the day.  I think you had some indirect and direct responses to the potential problems of sampling from facebook.  Our interest might be in problems of execution.  Others have mentioned those problems.  I have never sampled that way and can't speak to them.  But, to me, the more important issues are the methodological problems that we always need to considered, some mentioned in that set of threads, though perhaps not in detail:

    1) The proposed strategy leads to a convenience sample, not a probability (random) sample.  I suppose one can rank convenience sampling strategies from poor to good.  If one did such a ranking,  the proposed strategy would be at the poor end.

    2) One would not be sampling from a "general population."  One would be sampling from the population of facebook users. This might not be a bad thing per se, but there are two other issues.  First, this is unlikely to be the population to which your students plans to generalize.  Second, even if this is the population your student is interested in, the way the sampling will take place, he will end up with a nonprobability (convenience) sample of facebook members.  So he can't generalize to even that population.  The utility of the data would be largely confined to the sample obtained (sample as population).  

    3) This sampling strategy will not likely provide useful information even if you ask about opinions regarding social trends or social issues of universal interest.  The universality of interest in the topic is not the critical issue, though it might impact response rates to some extent (but see below).  The critical issue is that the proposed sampling scheme is not likely to lead to unbiased estimates of opinions on these universally interesting issues in the population of interest (needs to be defined).  And, as mentioned above, the actual sampling does not lead to a probability sample from even the population of facebook members.

    4) There is no statistical basis on which to compute standard errors or estimates sampling error in a convenience sample.  But ignoring this broader issue, the strategy of sampling social networks will likely build in some indeterminate amount of nonindependence in the data that would bias standard errors downward.  This nonindependence needs to be addressed.  Nonindependence is not a flaw per se.  Most probability sampling builds in nonindependence.  But because most researchers are not used to thinking about it or dealing with it, it needs to be mentioned. Of course, the lack of a statistical basis to compute standard errors is a flaw of convenience sampling.

    5) There was some mention of response rates possibly being low.  But there is no justification for reporting response rates for convenience samples, despite the common practice to the contrary.  Outside of probability sampling from a defined target population, a response rate has no value.  But if one works from the "sample as population" approach, then the response rate is 100% and there is no need to estimate standard errors.  The downside is trying to make a case for the utility of the "population."

    Sampling affects some important broader issues.  First, regarding the "emerging" interest in evidence-based management or practice, its utility depends on the quality of the evidence.  GIGO.  Regarding this issue, sampling is as important as using appropriate measures and statistical procedures. Second, meta-analysis is often viewed as a higher quality form of evidence in formulating evidence-based practice.  It cumulates studies to provide adequate power given that most of our primary samples are small.  However, in addition to other assumptions, the utility of meta-analysis is based on the use of probability sampling.  A meta-analysis of several biased convenience samples results in a big biased convenience sample.  Not a useful as an evidence base.

    I understand that this research might be training exercise. After all, good sampling is expensive and requires a lot of effort.  But students need to understand the limitations of the all of the methods they use so they don't become too complacent on important issues.    

    Mike Frone      

    ****************************************************************
    Michael R. Frone, Ph.D.
    Senior Research Scientist
    Research Institute on Addictions
    State University of New York at Buffalo
    1021 Main Street
    Buffalo, New York 14203

    Office:    716-887-2519
    Fax:        716-887-2477
    E-mail:     frone@ria.buffalo.edu
    Internet:     http://www.ria.buffalo.edu/profiles/frone.html
    ***************************************************************

    > -----Original Message-----
    > From: Organizational Behavior Division Listserv [mailto:OB@AOMLISTS.
    > PACE.EDU] On Behalf Of Eran Vigoda-Gadot
    > Sent: Sunday, February 07, 2010 2:07 PM
    > To: OB@AOMLISTS.PACE.EDU
    > Subject: [OB-LIST] distributing questionnaires online using social networks
    >
    > Dear colleagues
    >
    > A student of mine intends to use a web based survey and an idea was
    > mentioned to distribute the survey on one of the social networks
    > (such as Facebook etc) to increase the number of responses. He
    > targets the general population and the questionnaire is anonymous.
    > Has anyone used this method before? what are the expected problems?
    > any feedback or advice is appreciated.
    >
    > E.
    >
    > --
    > ________________________
    > Prof. Eran Vigoda-Gadot, Head
    > School of Political Sciences,
    > Center for Public Management & Policy (CPMP), The University of
    > Haifa Haifa 31905 Israel
    >
    > __________________________
    > Voice: 972-4-8240709
    > Fax: 972-4-8257785
    > e-mail: eranv@poli.haifa.ac.il
    >            eranv@research.haifa.ac.il
    > Home Page:     http://poli.haifa.ac.il/~eranv/ CPMP-Center for Public
    > Management & Policy:     http://cpmp.hevra.haifa.ac.il/ MLGA Program:
    >     http://poli.haifa.ac.il/~eranv/material_vigoda/2ND.html