Review Process

The review process involves AI, Self, Peer, TA, and Instructor reviews, each giving you an opportunity to improve your work.

Peer Reviews
  • Review Assignments: You will receive up to 3 review requests per Work project once you have submitted your own draft.
    If by the end of the course you have received fewer than 3 review requests, no requests, or too many requests please contact the teaching assistant.
  • Peer Review Tool: You must complete 3 peer reviews using the CGMap tool. Sometimes this may be only 2 reviews and we will compensate for the system points.
    • For instructions on how to use the CGMap tool, you can watch this video.
    • If needed for better viewing, you may view the work in CG Scholar creator while reviewing the work in CG Map.
  • Annotations: For an explanation of how to use coded annotations, please refer to the CG Map instructional video.
  • Submission: In order to submit, you must create a node from each Rubric item (15). In addition, at least 15 annotations must be included. Annotations grading requirement is 20.
  • Review Length and Ratings: Providing constructive feedback is going to be the most helpful to the authors. Please write strong explanations in your comments. If you deduct a score explain why and how the author can improve. Even if you like a section and give full credit, explain what you liked and why.
  • Review content: Be kind to peers in your reviews! Be as helpful as you can, offering them constructive suggestions. Feedback should be unique (not copy/pasted) and be directly relevant to the work under the review section. Comments must be tailored to a specific work.
  • Review Submission: Wait until you have completed all pending reviews before submitting them. More ideas might come to you while you review another work.
  • Red Flags: Do not self-plagiarize, or copy work from updates or other courses. This must be new work. If you notice any problems with a work or the reviews you receive(e.g. plagiarism, offensive reviews), please inform the teaching assistant or instructor.
CGScholar Artificial Intelligence (AI) Reviews

It is crucial that all students generate an AI review for their work using this feature before submitting the final version, as this can significantly impact their grades. Therefore, we strongly encourage all students to take advantage of this opportunity and produce an AI review to enhance the quality of their work and maximize their chances of achieving a higher grade.

Video Tutorial

To generate an AI review, please follow the tutorial below and ensure that you save the review in the appropriate format.

Please allow time for the video to load. For a full-screen experience, choose the option to watch this on YouTube.

Step-by-Step Instructions

Here is the step-by-step instruction (with additional steps) as well:

  1. Open CGMap Login from the Shares tab
  2. Go to the Works tab, click on ‘Your Works’
  3. Select the desired work you want to review with AI
  4. Choose the latest version of the selected work
  5. Proceed to the Map Actions tab and click on New
  6. Create a new map by filling in the required details. It is important to select “OpenAI Peer Review” under Map Type, make it shareable and then click “Create.”
  7. Proceed to the Analyze tab and click on “OK.” Follow the instructions provided on the Analyze tab to generate your review.
  8. After generating the review, ensure that you SAVE IT from the Map Action tab.
  9. Once you have generated the review, it is essential to carefully review the feedback provided and make any necessary changes to your work based on the insights provided.
Review Process Order

Follow this order when doing reviews to refine your work:
1. Peer Review: Submit the first version (Draft) of your work for peer review to receive feedback from your peers.
2. AI Review: Use the CGMap AI tool to receive AI feedback and refine your work further.
3. Revision: Adjust your work based on peer and AI feedback suggestions and submit it.

Rubric Philosophy

The peer-review rubrics that we use in these courses are based on our Learning by Design research, and before that, the Multiliteracies project. This is grounded in an epistemologically based theory of learning, conceived as different kinds of knowledge processes, or “things you do to know.” In the first instance, this is an action-oriented view of knowledge and learning, and only secondarily a cognitively oriented theory.

The purpose of the rubric is to support creators and reviewers to think systematically about the fundamental knowledge orientations of their work. The coded annotations are in order to highlight specific examples of each generalization in the rubric. We are also analyzing these annotations as part of a research project exploring the potentials of artificial intelligence in education, in which users train the machine to detect higher-order thinking and epistemological moves.

Here is our “knowledge process” theory of learning in a visualization. The rubric is color-coded according to this visualization

See the Rubrics section below for work type-specific rubrics.

Self Review – Update 5

The Self-Review should be completed prior to your final submission. This is an opportunity to reflect on the improvements you have made to your work in response to the AI and Peer Reviews. This is completed as Update 5 and addresses both the peer and AI review processes.

CGMap

CGMap is a linked platform to CGSCholar. It enables you to annotate and review peers and your work in a visual way. You can find tutorials in the Course Community under Shares.

Evaluating your Experience with the Peer and AI Review Process

Review Reflection

At the end of the course, we would like you to reflect on your experience receiving reviews throughout the course.

Completing these tasks is an important part of CGScholar learning experience. We understand that some students may encounter difficulties during the process and if you have any questions or concerns, please do not hesitate to reach out to us for assistance. Our team is always available to help you navigate any challenges you may encounter while using this new feature.

Step 1

Please respond to the following prompt guidelines in your update in the Community to provide a comprehensive overview of your experience:

  • Peer/Human Review – Express your views of peer/human reviews by sharing an image and/or screenshot of your experience with them that reflects how you see this type of review. Accompany your image(s) with 3 words that summarize your perceptions of this type of review.
  • AI Review – Express your views of AI reviews by sharing an image and/or screenshot of your experience with them that reflects how you see this type of review. Accompany your image(s) with 3 words that summarize your perceptions of this type of review. Additionally, indicate the degree to which you agree or disagree with the provided feedback and ratings.
  • AI vs Peer/Human Review – Reflect on the main differences between AI and peer/human reviews based on your experience. Explain in 150 words or more, provide examples, and illustrate with screenshots.

Step 2

Once you have completed the tasks, please update the spreadsheet by putting “Yes” along with the link to the update in the designated column. If you had any difficulties or were unable to complete the task, please indicate “No” or “Having Difficulty” in the spreadsheet.

Rubrics

Knowledge Process Rubric – Foundation Rubric for all Work Types

PDF version of the Knowledge Processes Rubric.  

Learning Module Supplementary Rubric

PDF version of the Learning Module Rubric.

Meaning Patterns Supplementary Rubric

Here are some of the ways in which interpretive methods map against this rubric: