Discussion: View Thread

  • 1.  Student Peer Evaluation Systems

    Posted 10-13-2016 15:29
    We have typically handled student peer evaluations (for group work) in-house using our own homegrown systems. At the request of a colleague, I'm trying to benchmark what publicly available peer-evaluation systems are out there (free or paid). I've come across a few such as TEAMMATES and CATME. What experiences have you had with these or others?

    Main issues of interest include: 

    (1) flexibility of design (do you control what questions can be included in the peer evaluation or do you merely accept the questions provided) 
    (2) ease of reporting (how are the reports for each peer generated and delivered; are reports limited to numeric data or can they incorporate textual data as well)
    (3) accessibility of raw data (can the service securely provide raw data to instructor or researcher)

    No need for the systems to create the teams or provide team tasks. These would be used for existing teams already working on course-defined tasks. 

    Feel free to respond directly to me (eean-crawford@uiowa.edu). I'm happy to compile the responses and share them with the list. 

    _______________________________________
    Eean Crawford, Ph.D.
    Assistant Professor, Dept. of Management & Organizations
    Tippie College of Business
    University of Iowa
    W376 John Pappajohn Business Bldg. 
    Iowa City, IA 52242-1994
    Ph: (319) 335-2884
    Fx: (319) 335-1956







  • 2.  Student Peer Evaluation Systems

    Posted 11-30-2016 18:10
    I'm following up with compiled responses to my OB-LIST question about what publicly-available peer-evaluations systems are out there. 

    My original request is shown below followed by the email replies sent to me in the order I received them. I tried to synthesize the results the best I could in a public google spreadsheet. Anyone can click on this link and edit/improve/correct the information I've entered. The name of each peer-evaluation system in the spreadsheet is hyperlinked to its webpage so you can access it directly. 

    https://docs.google.com/spreadsheets/d/1O6enzKD2Gs1LPeu3IqxpMcFiIyi2I32OPFEd3P6q0XA/edit?usp=sharing

    Thanks for your input!
    Eean

    ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
    MY ORIGINAL REQUEST: 

    On Oct 13, 2016, at 2:28 PM, Crawford, Eean R <eean-crawford@uiowa.edu> wrote:

    We have typically handled student peer evaluations (for group work) in-house using our own homegrown systems. At the request of a colleague, I'm trying to benchmark what publicly available peer-evaluation systems are out there (free or paid). I've come across a few such as TEAMMATES and CATME. What experiences have you had with these or others?

    Main issues of interest include: 

    (1) flexibility of design (do you control what questions can be included in the peer evaluation or do you merely accept the questions provided) 
    (2) ease of reporting (how are the reports for each peer generated and delivered; are reports limited to numeric data or can they incorporate textual data as well)
    (3) accessibility of raw data (can the service securely provide raw data to instructor or researcher)

    No need for the systems to create the teams or provide team tasks. These would be used for existing teams already working on course-defined tasks. 

    Feel free to respond directly to me (eean-crawford@uiowa.edu). I'm happy to compile the responses and share them with the list. 

    _______________________________________
    Eean Crawford, Ph.D.
    Assistant Professor, Dept. of Management & Organizations
    Tippie College of Business
    University of Iowa
    W376 John Pappajohn Business Bldg. 
    Iowa City, IA 52242-1994
    Ph: (319) 335-2884
    Fx: (319) 335-1956


    ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
    From: David Hekman <dhekman@gmail.com>
    Subject: Re: [OB-LIST] Student Peer Evaluation Systems
    Date: October 13, 2016 at 5:35:43 PM CDT


    Catme is great for team generation but not so great for peer evals. 

    Not sure about the other one. 


    Best,
    Dave 

    Sent from my phone.

    _______________________________
    David R. Hekman, Ph.D.
    Associate Professor of Management
    Leeds School of Business
    University of Colorado at Boulder
    303.492.1076
    Koelbel 461
     
    ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

    From: "Tom O'Neill" <toneill7@gmail.com>
    Subject: Fwd: [OB-LIST] Student Peer Evaluation Systems
    Date: October 13, 2016 at 8:43:11 PM CDT

    I created one and we have had some success with it. It's at itpmetrics.com. Free for anyone, grant funded.


    Currently we are providing a Dropbox folder containing the below materials, let me know if you can't access (https://www.dropbox.com/sh/jfkwwa03ivxyqmy/AADOy1OgI5nChC16vck9v9JXa?dl=0):

     

    1.     Facilitation slides for preparing and debriefing students on the following assessments:

    a.     Team Dynamics

    b.     Peer Feedback

    c.      Conflict Management Styles

    d.     Personality (as it relates to relationships with others)

    2.     Sample reports from itpmetrics.com

    3.     SUIT Training Materials

    a.     Share, Understand, Integrate, and Team decision is a framework we recently published in the Academy of Management Learning and Education journal. The in press version is in the folder as well as the training materials

    4.     Goal setting forms and a team contract.

    While these materials have been used many times with success, we always strive to continuously improve what we offer, meaning these documents may be updated from time to time to better meet your needs. 

     

    If you would like to receive more regular updates regarding the Individual and Team Performance Lab, please follow us on twitter: @itpmetrics.com


    I also have a listserv, would you like to be added?

     

    Sincerely,



    Thomas O'Neill, Ph.D.

    Associate Professor

    Founder, Individual and Team Performance Lab


    Inline image 1


    Department of Psychology | University of Calgary

    2500 University Drive, N.W. | Calgary, AB T2N 1N4


    ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
    From: Amy Tian <amy.tian@curtin.edu.au>
    Subject: RE: Student Peer Evaluation Systems
    Date: October 13, 2016 at 11:17:00 PM CDT
     
    I have used SPARK before which is pretty user friendly.
     
     
    Cheers,
    Amy

    ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

    From: Paul Donovan <Paul.Donovan@nuim.ie>
    Subject: Peer Evaluation
    Date: October 14, 2016 at 5:25:53 AM CDT

    Dear Eean

    Go a call from my colleague,included here, about your request for information on peer evaluation.

    I've used CATME and SPARKPLUS

    1) flexibility of design - Sparkplus is more flexible (your set of questions or theirs). With CATME, it's theirs. 

    (2) ease of reporting - Both systems are equal with numbers and text.

    (3) accessibility of raw data - raw data is available.

    Eean, I would include a fourth criterion and that would be ease of use. I found with CATME that I was up and running in minutes. Sparkplus is much more 'designed by engineers for engineers' and I also found that it presented a much less friendly user experience overall. 

    Cost. Sparkplus is not free and CATME is.

    Come back to me if you need more.

    Regards

    Paul


    Dr Paul Donovan
    Principal Researcher

    School of Business
    Maynooth University, Maynooth, Co. Kildare, Ireland.

    T: +353 1 708 6627 M: +353 87 246 6134
    E: paul.donovan@nuim.ie W: www.maynoothuniversity.ie
    ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

    From: "Jensen, Jaclyn" <JJENSE10@depaul.edu>
    Subject: student peer evaluation system suggestion
    Date: October 14, 2016 at 10:09:35 AM CDT

    I would recommend you look into TeamMATE (http://www.capsim.com/teammate/), a product of Capsim (www.capsim.com). TeamMATE is a research-based peer evaluation system that focuses on both individual and team factors related to team success. It was created in collaboration with several management faculty at DePaul, along with members of the Capsim team.
     
    The system allows for both developmental assessments as well as evaluative assessments, and the questions can also be customized by the instructor.
     
    The feedback provided to users gives developmental suggestions on both the individual and team factors, which I've found to be quite helpful (and lacking in many other peer assessment tools).
     
    I have never asked for the raw data on student evaluations, but believe you could certainly ask for it and the team would be able to send it.
     
    If you'd like to learn more, here is the name of my primary contact there. I'm sure he'd be happy to provide a demo and answer any questions in more detail.
     
    Eric Smith | Manager, Client Relationship Services | Capsim Management Simulations Inc. | 55 E. Monroe Street, Suite 3210 | Chicago, IL 60603 | P: +1.312.477.7200 | D: +1.312.477.7248| eric@capsim.com | www.capsim.com
     
    Regards,
    Jaclyn
    ..............................................................................
    Jaclyn M. Jensen, Ph.D.
    Department of Management
    Richard H. Driehaus College of Business
    DePaul University
    1 E. Jackson Blvd., Suite 7000
    Chicago, IL 60604
    312.362.6852 | @jaclynmjensen

    ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
    From: Roxanne Zolin <r.zolin@qut.edu.au>
    Subject: Re: Student Peer Evaluation Systems
    Date: October 14, 2016 at 5:16:08 PM CDT

    I have used CATME and I love it. After getting used to it , it is not that hard to us and it has made a big difference to my team satisfaction. I do two surveys per semester. The first one is to make sure the teams got started OK and the second for 6the grade. That way the team can't come to me at the end and say a person has not contributed if they did not report a problem before.

    (1) flexibility of design (do you control what questions can be included in the peer evaluation or do you merely accept the questions provided) 

    YOu  can use standard qustions or wcreate your won.

    (2) ease of reporting (how are the reports for each peer generated and delivered; are reports limited to numeric data or can they incorporate textual data as well)

    You go in and look at the results online. Students are sent their scores.

    (3) accessibility of raw data (can the service securely provide raw data to instructor or researcher)
    Very accessable.


    Hope this helps!

    Cheers,
    Roxanne

    Dr. Roxanne Zolin | Associate Professor | School of Management

    QUT Business School | Queensland University of Technology | www.qut.edu.au/business

    Phone: + 61 7 3138 5095 | Mobile:  0433 400 113 | Email: r.zolin@qut.edu.au | CRICOS No. 00213J

    ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
    From: Jonathan Kush <jkush@andrew.cmu.edu>
    Subject: Response to Student Peer Evaluation Systems
    Date: October 17, 2016 at 10:17:57 AM CDT

    Hello Eean,

    I have used CATME a number of times and I think it works pretty well. There are a few 'hidden' functionalities that can make it have a bit of a learning curve. For example, it has a nice team builder survey but you can also upload a csv with a "team number" column and it will use that column in any peer evaluations.

    1. I thought that you could add questions to the peer evaluation in addition to the built in questions and I have done so in the past, but I couldn't find that functionality when looking again.

    2. CATME generates reports for the students that give them a summary of how their peers rated them, so there is no additional work for you. You can add free-response questions as part of the peer evaluation and I think you have the option of revealing that information to the students who completed the evaluation.

    3. The raw data can easily be downloaded as a csv.

    Good luck!
    Jonathan Kush

    ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

    From: Amanda Hinojosa <amanda.s.hinojosa@gmail.com>
    Subject: Team peer evaluations
    Date: October 18, 2016 at 12:16:13 PM CDT

    Re: your email to the OB listserv.

    I have used the TeamPlus platform and really liked it for the classes I used it for (both undergraduate and graduate classes). I have to teach online, so it really has helped in that setting in particular (when they don't necessarily get face to face interaction).


    The students pay a fee to use it for the semester. I thought my students might complain about this, but I actually did not receive any complaints about the price of the platform or the use of it in general. (Which is surprising, because they definitely had other complaints about the course). I think the students liked it because it makes it difficult to get away with free-riding, which can be particularly problematic in the online setting.

    Regarding your questions:
    1) Flexibility: They have a set use of questions. However, because it is a small company, if you have something specific in mind, I think they might be willing to work with you to add your own questions in. They seemed pretty open to helping me modify things the last time I used them. 
    2) Ease of reporting... this is the main reason I liked using it. You get automated spreadsheets that calculates their score for you (no need to compile and create the results yourself. Additionally, you get individualized reports as well as reports by team. I'm not actually sure how the output looks for the students, but I think you could request a sample from the company (again they were very helpful with providing what I requested when I was reviewing them). 
    3) Accessibility of raw data.. Related to the point above, the excel files they generate for the professor make the raw data really accessible. I would highly recommend asking for a sample of this from the company. This was what sold me on using the product for more than one semester, as it saves so much time on my end and provides a systematic way of recording the raw data that makes it easy for me to go back and verify if I ever need to for any reason.

    Last I checked, they have the full team platform (which does assign some basic team tasks throughout the semester...creating a team identity, creating checkpoints and goals throughout the semester etc.) OR you can choose to just use the peer evaluation components (a developmental checkpoint and a final evaluation at the end of the semester). I have used the full platform because of my need for creating cohesion in teams despite being in the online settting, but I would also use it for just the evaluations if I taught face to face classes that require teamwork. 

    I hope that helps.

    Amanda Hinojosa
    Assistant Professor of Management
    College of Business
    University of Houston-Clear Lake
    2700 Bay Area Blvd.
    Houston, TX 77062




  • 3.  Student Peer Evaluation Systems

    Posted 12-15-2017 22:51
    Hi all,

    Last year I queried the OB list about what publicly-available peer-evaluation systems are out there. You supplied many useful resources which I compiled in a public google spreadsheet (link here and below along with the previously compiled responses). 

    My follow-up query is this: If you use student peer evaluations for group project work, how do you incorporate those evaluations into each students' grade? For example, you might translate peer evaluations directly into a certain amount of points, or use them more indirectly as input for your judgment in adjusting individual student grades, or perhaps some other way. 

    If you'll reply privately to me with the program/type of class you teach in (e.g., undergrad, MBA, execs..) and a brief description of how you use peer evaluations to record grades, I will compile the responses for the list. My goal is to both provide some "benchmark" practices, as well as to evaluate whether the way I've been doing peer evaluations for the last 6.5 years needs improvement. I'll of course describe my own peer evaluation process along with the compiled responses. 

    Thank you all for your valuable insight!
    Eean

    P.S. I'm aware of research regarding the quality/accuracy of peer ratings (e.g., Loignin, A., Thomas, J., Woehr, D., Loughry, M., Ohland, M., & Ferguson, D. 2016. Facilitating peer evaluation in team contexts: The impact of frame-of-reference rater training. Academy of Management Learning & Education.). I'm more interested in practically how people are taking given ratings and translating them into student group project grades. 
     

    _______________________________________
    Eean Crawford, Ph.D.
    Assistant Professor, Dept. of Management & Organizations
    Tippie College of Business
    University of Iowa
    W376 John Pappajohn Business Bldg. 
    Iowa City, IA 52242-1994
    Ph: (319) 335-2884
    Fx: (319) 335-1956

    On Nov 30, 2016, at 5:09 PM, Crawford, Eean R <eean-crawford@uiowa.edu> wrote:

    I'm following up with compiled responses to my OB-LIST question about what publicly-available peer-evaluations systems are out there. 

    My original request is shown below followed by the email replies sent to me in the order I received them. I tried to synthesize the results the best I could in a public google spreadsheet. Anyone can click on this link and edit/improve/correct the information I've entered. The name of each peer-evaluation system in the spreadsheet is hyperlinked to its webpage so you can access it directly. 

    https://docs.google.com/spreadsheets/d/1O6enzKD2Gs1LPeu3IqxpMcFiIyi2I32OPFEd3P6q0XA/edit?usp=sharing

    Thanks for your input!
    Eean

    ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
    MY ORIGINAL REQUEST: 

    On Oct 13, 2016, at 2:28 PM, Crawford, Eean R <eean-crawford@uiowa.edu> wrote:

    We have typically handled student peer evaluations (for group work) in-house using our own homegrown systems. At the request of a colleague, I'm trying to benchmark what publicly available peer-evaluation systems are out there (free or paid). I've come across a few such as TEAMMATES and CATME. What experiences have you had with these or others?

    Main issues of interest include: 

    (1) flexibility of design (do you control what questions can be included in the peer evaluation or do you merely accept the questions provided) 
    (2) ease of reporting (how are the reports for each peer generated and delivered; are reports limited to numeric data or can they incorporate textual data as well)
    (3) accessibility of raw data (can the service securely provide raw data to instructor or researcher)

    No need for the systems to create the teams or provide team tasks. These would be used for existing teams already working on course-defined tasks. 

    Feel free to respond directly to me (eean-crawford@uiowa.edu). I'm happy to compile the responses and share them with the list. 

    _______________________________________
    Eean Crawford, Ph.D.
    Assistant Professor, Dept. of Management & Organizations
    Tippie College of Business
    University of Iowa
    W376 John Pappajohn Business Bldg. 
    Iowa City, IA 52242-1994
    Ph: (319) 335-2884
    Fx: (319) 335-1956


    ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
    From: David Hekman <dhekman@gmail.com>
    Subject: Re: [OB-LIST] Student Peer Evaluation Systems
    Date: October 13, 2016 at 5:35:43 PM CDT


    Catme is great for team generation but not so great for peer evals. 

    Not sure about the other one. 


    Best,
    Dave 

    Sent from my phone.

    _______________________________
    David R. Hekman, Ph.D.
    Associate Professor of Management
    Leeds School of Business
    University of Colorado at Boulder
    303.492.1076
    Koelbel 461
     
    ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

    From: "Tom O'Neill" <toneill7@gmail.com>
    Subject: Fwd: [OB-LIST] Student Peer Evaluation Systems
    Date: October 13, 2016 at 8:43:11 PM CDT

    I created one and we have had some success with it. It's at itpmetrics.com. Free for anyone, grant funded.


    Currently we are providing a Dropbox folder containing the below materials, let me know if you can't access (https://www.dropbox.com/sh/jfkwwa03ivxyqmy/AADOy1OgI5nChC16vck9v9JXa?dl=0):

     

    1.     Facilitation slides for preparing and debriefing students on the following assessments:

    a.     Team Dynamics

    b.     Peer Feedback

    c.      Conflict Management Styles

    d.     Personality (as it relates to relationships with others)

    2.     Sample reports from itpmetrics.com

    3.     SUIT Training Materials

    a.     Share, Understand, Integrate, and Team decision is a framework we recently published in the Academy of Management Learning and Education journal. The in press version is in the folder as well as the training materials

    4.     Goal setting forms and a team contract.

    While these materials have been used many times with success, we always strive to continuously improve what we offer, meaning these documents may be updated from time to time to better meet your needs. 

     

    If you would like to receive more regular updates regarding the Individual and Team Performance Lab, please follow us on twitter: @itpmetrics.com


    I also have a listserv, would you like to be added?

     

    Sincerely,



    Thomas O'Neill, Ph.D.

    Associate Professor

    Founder, Individual and Team Performance Lab


    <image.png>


    Department of Psychology | University of Calgary

    2500 University Drive, N.W. | Calgary, AB T2N 1N4


    ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
    From: Amy Tian <amy.tian@curtin.edu.au>
    Subject: RE: Student Peer Evaluation Systems
    Date: October 13, 2016 at 11:17:00 PM CDT
     
    I have used SPARK before which is pretty user friendly.
     
     
    Cheers,
    Amy

    ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

    From: Paul Donovan <Paul.Donovan@nuim.ie>
    Subject: Peer Evaluation
    Date: October 14, 2016 at 5:25:53 AM CDT

    Dear Eean

    Go a call from my colleague,included here, about your request for information on peer evaluation.

    I've used CATME and SPARKPLUS

    1) flexibility of design - Sparkplus is more flexible (your set of questions or theirs). With CATME, it's theirs. 

    (2) ease of reporting - Both systems are equal with numbers and text.

    (3) accessibility of raw data - raw data is available.

    Eean, I would include a fourth criterion and that would be ease of use. I found with CATME that I was up and running in minutes. Sparkplus is much more 'designed by engineers for engineers' and I also found that it presented a much less friendly user experience overall. 

    Cost. Sparkplus is not free and CATME is.

    Come back to me if you need more.

    Regards

    Paul


    Dr Paul Donovan
    Principal Researcher

    School of Business
    Maynooth University, Maynooth, Co. Kildare, Ireland.

    T: +353 1 708 6627 M: +353 87 246 6134
    E: paul.donovan@nuim.ie W: www.maynoothuniversity.ie
    ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

    From: "Jensen, Jaclyn" <JJENSE10@depaul.edu>
    Subject: student peer evaluation system suggestion
    Date: October 14, 2016 at 10:09:35 AM CDT

    I would recommend you look into TeamMATE (http://www.capsim.com/teammate/), a product of Capsim (www.capsim.com). TeamMATE is a research-based peer evaluation system that focuses on both individual and team factors related to team success. It was created in collaboration with several management faculty at DePaul, along with members of the Capsim team.
     
    The system allows for both developmental assessments as well as evaluative assessments, and the questions can also be customized by the instructor.
     
    The feedback provided to users gives developmental suggestions on both the individual and team factors, which I've found to be quite helpful (and lacking in many other peer assessment tools).
     
    I have never asked for the raw data on student evaluations, but believe you could certainly ask for it and the team would be able to send it.
     
    If you'd like to learn more, here is the name of my primary contact there. I'm sure he'd be happy to provide a demo and answer any questions in more detail.
     
    Eric Smith | Manager, Client Relationship Services | Capsim Management Simulations Inc. | 55 E. Monroe Street, Suite 3210 | Chicago, IL 60603 | P: +1.312.477.7200 | D: +1.312.477.7248| eric@capsim.com | www.capsim.com
     
    Regards,
    Jaclyn
    ..............................................................................
    Jaclyn M. Jensen, Ph.D.
    Department of Management
    Richard H. Driehaus College of Business
    DePaul University
    1 E. Jackson Blvd., Suite 7000
    Chicago, IL 60604
    312.362.6852 | @jaclynmjensen

    ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
    From: Roxanne Zolin <r.zolin@qut.edu.au>
    Subject: Re: Student Peer Evaluation Systems
    Date: October 14, 2016 at 5:16:08 PM CDT

    I have used CATME and I love it. After getting used to it , it is not that hard to us and it has made a big difference to my team satisfaction. I do two surveys per semester. The first one is to make sure the teams got started OK and the second for 6the grade. That way the team can't come to me at the end and say a person has not contributed if they did not report a problem before.

    (1) flexibility of design (do you control what questions can be included in the peer evaluation or do you merely accept the questions provided) 

    YOu  can use standard qustions or wcreate your won.

    (2) ease of reporting (how are the reports for each peer generated and delivered; are reports limited to numeric data or can they incorporate textual data as well)

    You go in and look at the results online. Students are sent their scores.

    (3) accessibility of raw data (can the service securely provide raw data to instructor or researcher)
    Very accessable.


    Hope this helps!

    Cheers,
    Roxanne

    Dr. Roxanne Zolin | Associate Professor | School of Management

    QUT Business School | Queensland University of Technology | www.qut.edu.au/business

    Phone: + 61 7 3138 5095 | Mobile:  0433 400 113 | Email: r.zolin@qut.edu.au | CRICOS No. 00213J



    ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
    From: Jonathan Kush <jkush@andrew.cmu.edu>
    Subject: Response to Student Peer Evaluation Systems
    Date: October 17, 2016 at 10:17:57 AM CDT

    Hello Eean,

    I have used CATME a number of times and I think it works pretty well. There are a few 'hidden' functionalities that can make it have a bit of a learning curve. For example, it has a nice team builder survey but you can also upload a csv with a "team number" column and it will use that column in any peer evaluations.

    1. I thought that you could add questions to the peer evaluation in addition to the built in questions and I have done so in the past, but I couldn't find that functionality when looking again.

    2. CATME generates reports for the students that give them a summary of how their peers rated them, so there is no additional work for you. You can add free-response questions as part of the peer evaluation and I think you have the option of revealing that information to the students who completed the evaluation.

    3. The raw data can easily be downloaded as a csv.

    Good luck!
    Jonathan Kush

    ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

    From: Amanda Hinojosa <amanda.s.hinojosa@gmail.com>
    Subject: Team peer evaluations
    Date: October 18, 2016 at 12:16:13 PM CDT

    Re: your email to the OB listserv.

    I have used the TeamPlus platform and really liked it for the classes I used it for (both undergraduate and graduate classes). I have to teach online, so it really has helped in that setting in particular (when they don't necessarily get face to face interaction).


    The students pay a fee to use it for the semester. I thought my students might complain about this, but I actually did not receive any complaints about the price of the platform or the use of it in general. (Which is surprising, because they definitely had other complaints about the course). I think the students liked it because it makes it difficult to get away with free-riding, which can be particularly problematic in the online setting.

    Regarding your questions:
    1) Flexibility: They have a set use of questions. However, because it is a small company, if you have something specific in mind, I think they might be willing to work with you to add your own questions in. They seemed pretty open to helping me modify things the last time I used them. 
    2) Ease of reporting... this is the main reason I liked using it. You get automated spreadsheets that calculates their score for you (no need to compile and create the results yourself. Additionally, you get individualized reports as well as reports by team. I'm not actually sure how the output looks for the students, but I think you could request a sample from the company (again they were very helpful with providing what I requested when I was reviewing them). 
    3) Accessibility of raw data.. Related to the point above, the excel files they generate for the professor make the raw data really accessible. I would highly recommend asking for a sample of this from the company. This was what sold me on using the product for more than one semester, as it saves so much time on my end and provides a systematic way of recording the raw data that makes it easy for me to go back and verify if I ever need to for any reason.

    Last I checked, they have the full team platform (which does assign some basic team tasks throughout the semester...creating a team identity, creating checkpoints and goals throughout the semester etc.) OR you can choose to just use the peer evaluation components (a developmental checkpoint and a final evaluation at the end of the semester). I have used the full platform because of my need for creating cohesion in teams despite being in the online settting, but I would also use it for just the evaluations if I taught face to face classes that require teamwork. 

    I hope that helps.

    Amanda Hinojosa
    Assistant Professor of Management
    College of Business
    University of Houston-Clear Lake
    2700 Bay Area Blvd.
    Houston, TX 77062