New at Marist College

Fall 2010 means a new start for me:
Marist College, I am the Director of Graduate Education programs: M. Ed., and M.A. in Educational Psychology (on campus format and hybrid format), and a five-year B.A./M.A. program.

I am learning and developing the programs and enjoy teaching again! :-)

Rubric for online discussions

Example of a Rubric for online discussions; the assignment values 10 points

NOTE: If there is no participation for the first two weeks then will fail class

Criteria
/
Points

Demonstrates an excellent understanding of key concepts; presents proof of critical thinking, ability of application, and evaluation of the theory; has personal reflection and comments to other postings; contributes in a timely and relevant manner; exceeds minimum number of required postings; writes clearly and logically
8-10


Demonstrates an average understanding of most key concepts, rarely has proof of critical thinking and application of the theory to practice; mostly shows agreement with other postings; does not bring consistent personal performance; generally contributes in a timely and relevant manner; meets minimum number of postings; generally writes clearly and logically
5-7


Demonstrates limited understanding of key concepts; contributes in a sporadic manner; short and rushed postings; only agrees or disagrees, does not contribute with personal reflection and comments; does not show critical thinking, and has no proof of examples for application of theory to practice. Has only the number of minimum required postings.
2-4


Demonstrates weak or no understanding of key concepts; Rarely participates freely; mostly “just to do the job” postings; short and irrelevant remarks. There is no proof of understanding the application of theory to practice.
Less than 2

Rubric for article critique

Writing a very simple and easy to use Rubric for Article Critique

Criteria

Each of the below listed six criteria will be evaluated on a three-point Likert scale, see below the Note on how to consider the points resulted:

1= Poor
2= Good
3= Excellent


List of criteria for evaluation:

1. Pertinent presentation of article.

2. Presentation is grounded in the academic literature (use of extra reference).

3. Comprehensively addresses the topic, reflecting on the theory and results presented by authors.

4. Shows personal reflection on the theory presented in the article

4. Includes descriptions, details, and examples of alternative ideas; and reason behind them

5. Shows the application to real life setting; brings in new ideas for future research

6. Written well (scholar expressions, grammar, spelling, and punctuation), APA format.


Note:
20 points = 5 excellent
18 points = 4 excellent + 1 good
16 points = 3 excellent + 2 good
14 points = 2 excellent + 3 good
12 points = 1 excellent + 4 good
10 points = 5 good
failed if less than 4 good

How to build a rubric

DEVELOPING A RUBRIC

Questions to ask when developing a rubric:
1. What criteria must be present in student work to be able to qualify it as high quality?
2. How many levels of achievement should be present?
3. What is a clear description for each level of quality?
4. What are the consequences of performing at each level?
5. What grading scheme will be used in the rubric?
6. What aspects of the rubric are good and bad; and how to improve the rubric?
7. Is the product more important than the process; or are they valued as equal?



Different ways of starting:

- collect student work and try to score it using your existent rubric; pay attention to and details and questions might arise and use them to perfect your rubric
- collect student work and sort them in different groups based by quality, then describe the features of each sorted level. Write specific details for each performance level. Think about different grading levels when placing the student work into different groups, this will make a link between your rubric and grading.
- list all particular details of a performance that you are interested to know about. The most important dimensions of performance will form your rubric’s traits (thus analytical rubric will be developed)
- read what other experts know about the skills and the different levels of performance you want to consider. Search for other rubrics that describe the same skills or performance. Compare that with your rubric traits. Write a clear and neutral definition for each trait. Neutral definitions describe the trait and do not label good or poor performance.
- Find multiple examples for each of the different levels of your rubric
METARUBRIC


To evaluate your rubric you have to consider each of the following features of your rubric:

1. Content (coverage) – tells students what they need in order to succeed for the respective assignment. The rubric content should cover ALL that is essential and leave out details that will not be graded. Leave space to your students to present the content in different modalities if the modality of presentation does not have to follow a very specific pattern, otherwise everyone will try to fit a certain pattern that you cover in the rubric.
2. Clarity (detail) – a rubric must describe in enough detail the dimensions of performance that two raters understand the rubric in the same way and would evaluate the same way a certain performance/product. It is always good to have samples of student work for each level. Avoid just listing categories of evaluation without giving a definition of the category. Explain what each level means so that if two raters use your rubric to evaluate the same product/performance they should rate the same way.
3. Usability – a rubric must be practical, so that a scored/graded performance will be reflection of the rubric traits (so, that the student will not question “why?” the performance/product resulted in a certain score). The rubric needs to help students be successful, and understand what is necessary to correct if they do not do well.
4. Technical quality – your rubric must be fair to all students (language level used in the rubric, and ESL students). The rubric must have a good reliability - If two raters use the rubric to rate the same work they should assign pretty close scores; raters should be in agreement.



GRADING WITH RUBRICS

1. Rubrics are used in order to show students what are the criteria, expected standards that we use for grading and assessment.

2. Rubrics help also to inform students about different degrees (levels) of performance (e.g., poor, good, excellent); in consequence students will be able to self-evaluate themselves and prepare for the assessment.

3. Rubrics are models of standard for students. In developing a rubric students can be asked to participate set the criteria and describe different levels, then the teacher can use the suggestions gathered from students.

4. Rubrics are useful in the process of assessment when using different graders, or when it is needed a panel of assessors, in order to control for grader differences.


Use of rubrics in grading

- Decide ahead of time how many scale points will meet your needs for best performance
- Decide how many traits (specific dimensions) you want to measure, what are the important dimensions of the performance/product that the rubric will measure
- It is not advised to use strict percentages to convert scores from each particular assessment into letter grades; a better choice is to tell the students that all assignments will be scored on one or more traits and points will be cumulated (or if you count only the final performance then only the last performance in the semester should be transformed into letter grades).

o Example:
- on a rubric that uses scores on a scale 1-5, a “3” is described as average (B). However, when the obtained score is divided by the total possible score (3/5), the percentage obtained (60%), often would represent a failing score (F)

- Try to come up with a logic rule to convert scores/points to letter grade instead of using percentages.

Example1:
- taking into consideration obtained scores on a series of assignments;
- At least 40% scores of 5, and no more than 10% scores lower than 4 = A
- At least 10% scores of 5, and no more than 30% scores lower than 4 = B
- At least 20% scores higher than 4, and no more than 10% lower than 3 = C
- At least 10% scores higher than 4, and no more than 30% lower than 3 = D
- Anything lower than the above = F



Example 2:
- add all points from a series of assignments then use a logical formula to transform the total points into letter grades:
91-100 =A; 81-90 = B; 71-80 = C; 61-70= D; below 69 =F.

Rubrics

Rubrics are explicit grids, schemas, lists, to evaluate and clasify products of learning outcomes into different categories that vary across a given continuum.

Rubrics are very handy to use in measuring student learning given some specific learning outcomes.

There are different ways to build a rubric, often the building process takes some thought and time, but the rubric gives a clear picture to the student and instructor about the level of performance expected for a certain task.

The title above has a link or you can find the information below and in the other link on Rubrics.


Rubrics defined

Rubrics = criteria that cover the essence of a performance that is judged with them.

The use of rubrics help define the expected performance, standards of quality, levels of accomplishment, also they are helpful in diagnostic assessment and providing feedback to students.


Holistic vs. Analytical

Holistic rubric = gives a single score or rating for an entire product or performance based on overall impression of a student’s work.
The rater considers all quality judgment in a big component and overall judgment and comes up with one single score.

Example:

Excellent level:
- Student shows complete understanding of the tasks and concepts
- Clear identification of key concepts and important elements
- Excellent writing style
- Pertinent insight and demonstration of appropriate application of main ideas

Good level:
- understanding of most critical concepts
- shows identification of some key concepts but most of the parts are missing
- adequate writing style with minor errors, some limited clarity in expressions
- scarce demonstration of application of main ideas

Poor level:
- misunderstanding of majority of concepts or no understanding of concepts and processes
- irrelevant or illegible response that has no relation to the key concepts
- unsuccessful attempt to communicate
- lack of demonstration in application of main ideas


Holistic rubrics are suited for the following:
- judging simple products or performances
- getting a quick snapshot of overall quality or achievement; often used when a large number of students are graded
- judging the impact of a product or performance more than the specific detailed parts of the performance.

Disadvantages:
There is no detailed analysis of the strengths and weaknesses of the performance/product; so it is not useful as diagnostic or to give students detailed feedback on their performance and help them improve their performance.


Analytical rubric = divides a product into essential dimensions (traits) and each dimension is judged separately. A separate score is given for each dimension/trait considered important for the assessed performance.
Scoring of each trait can be done by using a Likert scale (e.g., 1 to 5 where 1 is poor quality, 3 is average, and 5 is excellent quality).


Example (Six-trait writing analytical rubric):

Each of the following traits is scored separately on a scale of 1 to 5:
Ideas: the main idea of the assignment, content ideas, main theme, details in the theme

Organization: the internal structure of the individual, the core and central meaning, logical and creative pattern of ideas

Voice: the feeling and beliefs of the individual come through the words

Words choice: rich, precise, scholar, and eloquent language that enlightens the reader

Sentence fluency: the rhythm and flow of the language, the words patterns, the way the writing plays to the ear

Conventions: the mechanical correctness of the work as an entire piece, along with grammar and usage of words (spelling, capitals, punctuation).


Analytical rubric are suited for the following:

- judging complex performances that involve multiple dimensions (skills that must be assessed). Each step in the rubric can be designed to measure one specific trait.
- Provide more specific information and feedback to students about their strengths and weaknesses.
- Can be used to target instruction to specific areas in need for improvement.
- Analytical rubrics help students come to a better understanding about the nature and quality of work they must perform.

Disadvantage:
- more time consuming to craft and use in grading
- lower inter-rater agreement because of the detailed and many traits
- less desirable in large scale assessment context when many students must be graded and when speed in grading is essential



General vs. Specific rubrics

General (generic) rubrics = can be used across similar performances (e.g., all Blackboard postings across a semester; all group interactions in the semester).

- you construct one rubric and use it for similar tasks
- sometimes the tasks are so similar that there is no reason to construct a detailed rubric for each
- when grading portfolios or any collection of different assignments that must be graded overall; where developing a rubric for each piece would be time consuming and not important
- help students understand the big picture and when students need to be able to apply what they learned in one task to the next task
- when the product is more important for assessment purposes than the process/steps of that performance
- when the students must come up with general ideas and when they must think in order to develop particularized performance
- when there are many different possibilities to solve a problem or there are many ways for a successful performance



Task-specific rubrics = each rubric can be used only for a particular task/assignment (e.g., rubric for the Final exam,; rubric for a project proposal).

- it is easier and more consistent to get fast scoring. Easier to train the scorers
- if you want to know if students are able to perform specific methods and procedures

There are times when the best rubric is a combination of generic and task-specific rubric traits.
For example when you are interested if the students know how to develop specific parts/steps in a project proposal but in the same time you are interested in the overall quality of writing.


Number of score points for rubrics

- A good range to evaluate the performance
- Enough points to evaluate different levels of quality
- Does your rubric need to fit a particular standard? Use the same amount of points.
- Tracking changes over a longer period of time requires more points than limited/shorter time
- Have a good reason to pick odd or even number of points: odd number gives the tendency to gravitate to the mean to a more neutral assessment; but if your goal is to delimitate the average then odd number of points is the best choice.
- 1-5 point scale is similar to A-F grading, at times students confuse points with grades







Reference:

Arter, J. & McTighe, J. (2001). Scoring rubrics in the classroom. Thousand Oaks, CA: Corwin.

Brennan, R. L. (Ed.) (2006). Educational measurement (4th Ed.). NCME. Westport, CT: Praeger.

Earl, L. M. (2003). Assessment as learning. Thousand Oaks, CA: Corwin.

Huba, M. E. (2000). Learner-centered assessment on college campuses. Boston, MA: Allyn & Bacon.