Using Standards and Assessments to Support Student Learning
 
Prepared for the SSTA by Loraine Thompson Information Services Ltd.
SSTA Research Centre Report #99-11: 28 pages, $11
 
Table of Contents
  
Introduction 
Part I:  How Are Our Students Doing? 
Evaluation – An Overview  Findings Regarding Student Achievement  Findings Regarding Curriculum and Instruction  
Part II: Developing a Policy on Student Evaluation 
Why a Policy on Student Evaluation? 
The Policy Development Process 
The Components of a Student Evaluation Policy  References 
Appendix A:  Principles for Fair Student Assessment Practices for Education in Canada 
Appendix B:  A Guide to Establishing School Division Benchmarks and Standards
Overview
 
This report has two parts.  Part I:  How Are Our Students Doing? Reviews the results of provincial and national student evaluations in specific subject areas.  Part II:  Developing a Policy on Student Evaluation offers suggestions to boards of education who may be considering developing a policy on student evaluation.  Part II describes why a student evaluation policy may be useful, briefly outlines the policy development process, and describes specific components of a typical student evaluation policy.  Questions for discuss and thought are included throughout.  This document is intended to stimulate further discussion about student evaluation as well as provide practical information for boards of education.
 
Introduction
 
At their 1997 convention, the members of the Saskatchewan School Trustees Association passed the following resolutions:
 
  • That the SSTA urge Saskatchewan Education to lead and support the development of well-defined evaluation criteria and carefully crafted assessments which will serve as the foundation for defining performance standards for each curriculum area.
  • That the SSTA establish processes to develop a provincial consensus among educational partners regarding the nature, appropriateness and utility of educational standards in improving teaching and learning. 
  •  
    These resolutions focus on province-wide actions – actions that can also include initiatives by local boards of education.  This booklet was developed by the Saskatchewan School Trustees Association to support individual boards of education as they develop their own processes to respond to these resolutions.

    This booklet has two parts.  Part I:  How Are Our Students Doing? reviews the results of provincial and national student evaluations in specific subject areas.  Part II:  Developing a Policy on Student Evaluation offers suggestions to boards of education who may be considering developing a policy on student evaluation.

    Back to Table of Contents 


     
    Part I:  How Are Our Students Doing?
     
    Evaluation – An Overview
     
    “Are our students achieving at a level that is appropriate for their age and grade level?  How do Saskatchewan students compare to those in other provinces or countries?  How do the students in our division compare to other students in Saskatchewan or in Canada?”  These questions are often asked by boards of education.  The information that follows is an initial step toward answering questions such as these.

    Back to Table of Contents 



    Provincial and National Evaluation Programs

    There are two provincial and one national evaluation programs.

    The two provincial evaluation programs are:
     

  • The Provincial Learning Assessment Program (PLAP) - This program assesses the mathematics and language arts achievement of Saskatchewan Grade 5, 8 and 11 students.  To date, two mathematics assessments (1995(1) and 1997(2)) and three language arts assessments (1994,(3) 1996(4) and 1998(5)) have been published.  The 1994 and 1996 language arts assessments focused on reading and writing only.  The 1998 language arts assessment focused on listening and speaking only.
  • The Provincial Curriculum Evaluation Program - This program assesses the effectiveness of specific Saskatchewan curricula.  To date, the areas that have been assessed are Grades 1-5 Science (1993)(6), Grades 7, 8 and 9 Health (1994)(7), Grades 5, 8 and 11 Mathematics (1995),(8) Grades 7, 8 and 9 Social Studies (1995)(9) and Grade 1-9 Arts Education (1998)(10).  Although the primary focus of this program is curriculum evaluation, it also provides a limited amount of information about student achievement.
  • The national evaluation program is:
     

  • The School Achievement Indicators Program (SAIP), Council of Ministers of Education, Canada - This national program examines the achievement of Canadian 13- and 16-year-olds in selected subject areas.  The evaluations published to date have been in science (1996),(11) mathematics (1993(12) and 1997(13)) and language arts (1994(14) and 1998(15)).
  • The information that follows is drawn from the provincial and national evaluation programs and is presented in two categories:
     

  • findings regarding student achievement; and,
  • findings regarding curriculum and instruction.

  •  
    Back to Table of Contents 

    Findings Regarding Student Achievement
     
    The results of these provincial and national assessments are just one indicator of students’ achievement.  It is necessary to consider the results of other assessment tools, as well, in order to get a comprehensive, well-rounded picture.  All of the provincial and national assessments are low-stakes assessments.  That is, students are told to do their best, but they know that the test doesn’t count toward their report card or year-end mark.  In addition, the national SAIP program is designed to apply all across Canada.  Thus, the knowledge and skills tested are not closely linked with the Saskatchewan curriculum or with expected learning outcomes for Saskatchewan students.

    Student achievement in specific subject areas is summarized in the sections that follow.
     
    Back to Table of Contents 



    How Are Students Doing in Science?

    The science performance of Saskatchewan 13- and 16-year-olds in 1996 was comparable to that of all Canadian students.  However, for Canada as a whole, and for Saskatchewan, the percentage of students who achieved mid-range performance met standards, but fewer than expected Canadian and Saskatchewan students achieved top levels of excellence.

    An area of strength among Saskatchewan elementary students in 1993 was their understanding of the nature of science.  Areas of weakness included observing, describing, hypothesizing, questioning, inferring, written communication, and the effect of science on one personally.
     
    Back to Table of Contents 



    How Are Students Doing in Mathematics?

    The 1997 provincial assessment of Grade 5, 8 and 11 students found that Grade 5 student outcomes for good math proficiency met provincial expectations; otherwise students’ overall achievement was below expectations for good and top-level proficiency.

    The 1995 provincial assessment found that fewer Grade 5 students than expected achieved excellent and acceptable performance in most areas.  At Grade 8 and 11, more students than expected achieved excellent performance and fewer than expected achieved adequate performance in most areas.

    In general, in the 1995 assessment, Saskatchewan students were stronger on math concepts, procedures and relationships and weaker on practical applications and problem solving.

    In the 1997 national assessment, both Canadian students as a whole and Saskatchewan students achieved below expectations in math content and problem solving.  However, the percentage of Saskatchewan 13- and 16-year-olds who achieved adequately was lower than for Canada as a whole.
     
    Back to Table of Contents 



    How Are Students Doing in Language Arts?

    The 1998 national assessment of reading and writing showed that:
     

  • Performance of 13-year-olds in reading and writing was similar to performance in the rest of Canada except that fewer Saskatchewan students performed at the very highest level in reading.
  • When Saskatchewan students’ reading performance was compared to the reading performance of the rest of Canada, fewer Saskatchewan 16-year-olds performed at the middle and two top achievement levels.  In writing, fewer Saskatchewan 16-year-olds performed at the two top achievement levels.
  • The 1998 provincial assessment of listening and speaking skills showed that Grade 5 students met provincial expectations, but both Grade 8 and 11 students fell short of expectations in some dimensions of listening and speaking.

    The 1996 provincial assessment of Grade 5, 8 and 11 students  showed that most Saskatchewan students had adequate or better reading skills and that students met or exceeded expectations in most reading areas.  In writing, more students performed at the middle achievement levels than expected, and fewer performed at both very high and very low achievement levels.

    There was some evidence that Grade 8 and 11 students’ writing skills had improved between 1994 and 1996.

    The 1994 provincial assessment showed that, in general, Grade 5, 8 and 11 students exceeded expectations in reading, and were below expectations for writing for all three grades.

    Back to Table of Contents 



    How Are Students Doing in Health?

    The teaching-learning process is central to the Grade 7, 8 and 9 Health Education curriculum.  This process can be summarized as follows:
     

    Health Education:
    Teaching-Learning Process
    • Level A: Get information about health-related issues.
    • Level B: Make informed decisions about actions that will enhance one's health
    • Level C: Develop and carry out a health action plan for specific activities to enhance one's health
     
    During the 1994 evaluation of the Grade 7, 8 and 9 Health Education curriculum, it was found that:
     
  • Most students had not learned the decision-making skills necessary for success with the teaching-learning process.
  • Students did not know as much as expected about health-related issues, but student learning was slightly higher for topics where knowledge can be obtained from the media and community programs as well as from school.  Examples of these types of topics are the harmful effects of smoking and drug use.

  •  
    Back to Table of Contents 

    How Are Students Doing in Social Studies?

    As part of the assessment of the 1995 Grade 7, 8 and 9 Social Studies curriculum, students, teachers and administrators were asked to report on achievement.  In addition, students wrote tests to measure their actual achievement.
     

  • Students tended to rate their own achievements as acceptable or strong, to express positive attitudes towards social studies and to say that what they learn in social studies class is important.
  • Teachers believed that the curriculum had developed student learning in the following ways:
  • In the tests that students completed, students tended to do better on earlier units of work than on later units, because teachers devoted more time to earlier units than to later.

  •  
    Back to Table of Contents 

    How Are Students Doing in Arts Education?

    The 1998 evaluation of the Grade 1-9 Arts Education program included a review of student outcomes and experiences.  The review found that:
     

  • Students like the creative/productive component of arts education more than they do the critical/responsive or cultural/historical components.  For example, they would rather paint a picture than critically analyze the work of famous painters or study specific genres of painting such as Impressionism or Cubism.
  • Arts education, generally, has positive benefits for students.  It expands their horizons, gives them new experiences and helps them develop talents that have previously been unrecognized.
  • Arts education is organized into four strands:  dance, drama, music and visual arts.  Students have differing opportunities in these four strands with visual art implemented most extensively followed by music.  Drama and dance have not been implemented to the same extent as the other two strands; thus students have fewer opportunities in these areas.

  •  
    Back to Table of Contents 

    Findings Regarding Curriculum and Instruction
     
    Overall, curriculum implementation and use of specific instructional strategies recommended in curriculum guides was spotty.  Although there was variation among subject areas, the following generalizations apply:
     
  • For individual curriculum guides, some units of study were emphasized more than others.
  • For specific subject areas, teachers at some grade levels used the curriculum guide more than teachers at other grade levels.  In at least one subject area, teachers relied on the textbook more than the curriculum guide.
  • Some important instructional strategies have not been implemented to the extent the curriculum guides suggest.  These include use of manipulatives (hands-on materials), group work and computers in math; portfolios in language arts and mathematics; Indian and Métis perspectives in social studies and language arts.
  • Teachers’ reports regarding instructional strategies varied.  Teachers in Grade 1-5 Science and Grade 7, 8 and 9 Social Studies reported using a wide range of instructional strategies, but teachers in some other subject areas reported using a limited number of traditional instructional strategies such as lectures, worksheets and taking notes.

  • Back to Table of Contents 
    Part II: Developing a Policy on Student Evaluation
     
    Why a Policy on Student Evaluation?
     
    A policy on student evaluation is one of the most common types of policies developed by boards of education.

    A policy on student evaluation is important for five reasons:
     

  • It can increase student learning.  An important purpose of student evaluation is to provide information about student performance so that teachers can modify their instructional programs appropriately and to provide students with information about their strengths and weaknesses so they can better focus their energy.
  • It can give the school division a mandate to set local-level standards for student achievement.  Typically a school board policy outlines processes that are used to set standards and responsibilities of the various groups and individuals involved, rather than listing specific standards since these can change regularly.
  • It helps ensure that student evaluation within the school division is conducted in accordance with recognized national and international principles for fair student evaluation.
  • It promotes fairness and consistency for all students.  A student evaluation policy will help ensure that students in similar situations are treated the same in every classroom and every school across the division.  It will also help ensure that similar situations are handled in the same way from one year to the next.
  • It may reduce the number of problems relating to student evaluation.  If students, teachers and parents know the guidelines beforehand for issues such as missed exams and appeals of exams, fewer problems may arise.

  •  
    Back to Table of Contents 

    The Policy Development Process
     
    A policy on student evaluation, like most other types of policies, is best developed with the active participation of stakeholder groups.

    Stakeholder involvement has several advantages.  It:
     

  • ensures the policy development process is open and transparent;
  • allows everyone with an interest to express their opinions;
  • builds a sense of commitment and shared ownership;
  • makes people affected by the policy aware of its existence; and,
  • increases the knowledge base upon which the policy is built.
  • Involving students, parents, teachers and community members in policy development increases the time needed to develop the policy and may also increase the amount of work involved.  However, involvement is worth the extra time and effort, because it results in a policy that has a much higher level of acceptance and is more likely to be observed.

    The process of policy development is described in detail in the SSTA publication Policy Leadership.  This publication on student evaluation policies focuses on the components that could appropriately be included in a student evaluation policy.

    Back to Table of Contents 



    The Components of a Student Evaluation Policy
     
    The components that typically appear in a school division student evaluation policy are:
     
  • a statement of philosophy;
  • a policy statement;
  • references to the legislation;
  • principles for fair student assessment practices;
  • guidelines for specific student evaluation issues; and,
  • guidelines for making the policy known.
  • Each of these components of a student evaluation policy is discussed in more detail in the sections that follow.
     
    Back to Table of Contents 



    A Statement of Philosophy

    The statement of philosophy or rationale that begins a student evaluation policy gives the reasons for the policy.  For example, it might state that the board of education:
     

  • is committed to continuous improvement of learning and that the information obtained through student evaluation will help the board modify and change programs to enhance student achievement;
  • is committed to providing students and their parents with feedback on students’ progress ? feedback that comes through student evaluation;
  • is committed to public accountability; and,
  • is committed to fair and equitable testing practices for all students.

  •  
    Back to Table of Contents 

    A Policy Statement

    The policy statement is a short, one- to three-sentence statement of what the board intends to do.  For example:

    Back to Table of Contents 

    References to the Legislation

    Most school boards contain reference to the legislation under which the policy was established.  Section 17.8(1) of The Education Act, 1995 says:

    All educational experts will agree that a good program of instruction includes evaluation of students’ progress.  Thus, board policies regarding student evaluation fall under the mandate of Section 17.8(1) of The Education Act, 1995.

    Back to Table of Contents 



    Principles for Fair Student Assessment

    Most school division policies on student evaluation list a few fundamental principles of fair student evaluation.  The policies make it clear that these principles underlie all student evaluation activities that take place in the division.

    The best source for fundamental principles of student evaluation is The Principles for Fair Student Assessment Practices for Education in Canada which appear in Appendix A.  These principles were developed by a group comprised of several major Canadian educational organizations and are widely accepted across Canada. However, the full set of principles is lengthy and complex.  Most school divisions use The Principles for Fair Student Assessment Practices for Education in Canada as a guide in the development of a few key principles.

    Key principles identified in Saskatoon (West) School Division’s policy on student evaluation appear below.
     

    Saskatoon (West) School Division Policy on Student Evaluation – Key Principles(16)
     
    Goals Objectives 
    1. A wide range of evaluation methods will be used. 1. Formal methods will be used to assess student progress.  Examples of formal methods are teacher-developed tests and examinations, commercially-developed evaluation instruments and standardized tests. 
    2. Informal methods will be used to assess student progress.  Examples of informal methods are teacher observation, anecdotal records, performance appraisals, peer appraisal, and self-appraisal. 
    3. Professional development opportunities and support will be provided to teachers to assist in expanding and enhancing evaluation methods. 
    4. Standardized tests will be used in accordance with divisional policy.  (Policy to be developed.)
    2. The purpose of evaluation is to promote student growth and development in the cognitive, affective, and psychomotor domains. 1. Teachers will incorporate evaluative activities, which will assess affective and psychomotor objectives as well as cognitive objectives. 
    2. Student growth in the cognitive, affective, and psychomotor domains should be reported to students and parents. 
    3. Evaluation is primarily formative in elementary and middle years. 
    4. A list of the major competencies will be developed at each grade level in each area of learning, as resources are available. 
    5. Retention at the K-8 level will occur only as part of a carefully designed development plan for an individual student.  (Promotion and retention policy to be developed.) 
    3. Evaluation practices will be fair and equitable. 1. Teachers will provide evaluation plans to the students/parents as part of the course outline. 
    2. Students must know on what basis and when they will be evaluated. 
    3. Inservice will be provided.  Teachers will be familiarized with the principles and practices of fair student assessment. 
    4. Parents and students will be made aware of the divisional appeals policy (to be developed). 
    5. Each school will develop a program to annually monitor their evaluation plan and principles. 
    4. The results of evaluation will be regularly communicated to students and parents. 1. Mechanisms for reporting student programs will be developed which are appropriate for the needs of the various school populations. 
    2. Student progress will be formally reported to parents at least twice per semester at the high school level and at least four times per year at other levels.  The report can be a documented conference, a report card, or both. 
    3. School staffs will work towards increasing the informal reporting process. 
    4. Students should be active participants in the reporting process. 
    5. A committee will be formed to study alternatives to the existing grading structure and current report cards. 
    6. Any changes in the grading structure or report cards will be clearly communicated to parents and students. 
    7. School staffs will develop a plan to communicate the results of all major tests, assignments, etc. to parents.
    5. Evaluation will be continuous and carefully planned to reflect instructional techniques and curricular objectives. 1. Teachers and administrators will seek to match evaluation devices to instructional techniques. 
    2. Final comprehensive examinations based on at least 50 percent of the course content will be utilized in Grades 9 to 12, unless precluded by subject contents.  The mark for the comprehensive examination should not represent more than 40 percent of the final mark. 
    3. Evaluation devices should be weighted to reflect the relative importance of the curricular objectives. 
    6. Evaluation will be a major focus of professional development activities at the divisional level and school levels for 3-5 years, beginning in the 1993-94 school year. 1. Inservice sessions will be provided to ensure all teachers know and understand the divisional evaluation policy. 
    2. Additional inservices will be provided to build on and enhance the strengths of teachers in the area of evaluation. 
    3. Opportunities will be provided for teachers to collaboratively plan and implement new evaluation devices and/or strategies.
     
    Back to Table of Contents 

    Guidelines for Specific Issues

    The specific issues addressed in the student evaluation policy will vary from one school division to another depending on local circumstances.  Many student evaluation policies include guidelines for:
     

  • establishing school division benchmarks and standards;
  • use of standardized tests;
  • student retention in grade;
  • return of completed tests;
  • recommendations;
  • bonus marks;
  • appeals of final examinations;
  • reporting to parents;
  • missed tests and exams;
  • ownership and retention of student records;
  • the basis on which marks are assigned;
  • making the policy known.
  • Each of these topics is discussed in more detail in the sections that follow.
     
    Back to Table of Contents 



    Establishing school division benchmarks and standards

    Some Saskatchewan school divisions are beginning to establish benchmarks and standards to supplement those set through Saskatchewan’s Provincial Learning Assessment Program (PLAP) and the national School Achievement Indicators Program (SAIP).
     

  • A benchmark is a “snapshot” of student achievement at a particular level at a particular moment in time.  Benchmarks allow comparison of student achievement from one year to the next within the school division.  They also allow school divisions to compare their students’ achievement with provincial and national achievement.
  • Standards describe what student achievement should be in a particular skill area at a particular grade.  Comparing students’ achievement to the standard tells us whether students are achieving above or below expectations.
  • Board policy usually describes the processes that will be used to establish benchmarks and standards, rather than specifying the benchmarks and standards themselves, because these can change from year to year.

    The process that is frequently used to establish benchmarks involves:
     

  • collecting samples of students’ work.
  • scoring that work using written descriptions of student work at various levels (rubrics) and comparisons with students work at various levels (exemplars).  Teachers are trained to do the scoring.
  • returning students’ work along with the written descriptions and examples used for scoring to classroom teachers so that students and teachers can compare individual pieces of work to sample work at various levels.
  • A complete description of this process appears in Appendix B.

    In addition to being an evaluation tool, benchmarking can play an important role in improving student learning.

    Benchmarking involves developing written descriptions of student work at various levels (rubrics) and collecting samples of student work at each of the levels (exemplars).  The rubrics and exemplars provide very clear examples of the criteria that students must meet in order to get a particular mark.  When these rubrics and exemplars are distributed to students and teachers, students can compare their work and identify changes and improvements they need to make in order to reach a certain level.

    The benchmarking process involves training teachers to score student work based on rubrics and exemplars rather than on criteria developed by individual teachers.  The objective is for all teachers to score the same piece of work in the same way.  This process promotes greater fairness and consistency, because it is based on accepted external standards rather than on teachers’ individualistic criteria.

    Back to Table of Contents 



    Use of standardized tests

    In recent years, use of standardized tests has become a somewhat controversial topic.  Some groups are strong advocates of their use and others are strongly opposed.  Establishing a policy on the use of standardized tests will help ensure they are not misused and will provide useful information to the public.  Such a policy often includes:
     

  • a statement about the purposes for which standardized tests are used.  They can be used for diagnostic purposes, evaluative purposes or both.
  • a commitment that when standardized tests are used for evaluative purposes, they will never be the only assessment tool used to assign a mark.  Marks should incorporate the results of several assessment tools such as portfolios of students’ work, students’ self-assessments, teacher-made tests, homework assignments, students’ practical work during science experiments and math performance stations, and teacher observation.
  • a statement about when standardized tests are administered, if they are used for evaluative purposes.  For example, some school divisions administer standardized reading and math tests at specific grade levels and specific times of the year.
  • a commitment to use bias-free tests.  Gender and racial/cultural bias are the most obvious types of bias, but there are other types, too.  An American-made reading test may include references to President’s Day and Martin Luther King Day.  Because Canadian children are not familiar with these holidays, they may not understand the question and thus will get it wrong.  Similarly, a math test from eastern Canada or the U.S. may ask students to calculate distances traveled on the subway, the commuter train or the LRT, all of which are unfamiliar to Saskatchewan children.
  • guidelines for who gets information about the results of the test.  Are students and parents informed of the results or is this information held confidential at the school?  If students and parents get the results, how is this information provided - in a special letter to the parent, in a parent/teacher interview, only if the parent asks?
  • Back to Table of Contents 



    Student retention in grade

    It is sometimes believed that requiring an elementary student who is not doing well to repeat the grade will produce achievement in later grades.  In fact, there is a huge body of research that shows that the opposite is usually true.(17)  Children who are retained in grade, usually don’t do as well later on as children with comparable achievement who are promoted.  The sense of being a loser and of being incapable of learning that comes with school failure often stays with a child for the rest of his or her school career and leads to continuing low performance - an example of the self-fulfilling prophecy in action.

    School board policies on retention in grade often contain the following guidelines:
     

  • A statement about whether retention in grade is an option in the school division.  Some school divisions do not permit it at all, others at Kindergarten and Grade 1, and still others throughout the elementary and middle years.
  • A description of the circumstances under which retention in grade is permitted.  Usually students may be retained only on the basis of achievement in the courses in which learning is sequential and cumulative, such as math and language arts.
  • A description of decision-making processes to be followed when retention in grade is under consideration.  For example, most policies require that the teacher discuss the retention with the director of education and the principal and with the child’s parents.  Some require the written permission of the child’s parents before the child can be retained.
  • A requirement for a written remediation plan.  Some school divisions require that teachers who are proposing to retain a student prepare a written plan outlining how the student’s needs will be addressed during the year of retention.  This plan may be discussed with parents, principal and director of education and it is usually kept on file in the school or school division office.
  • Guidelines for repeated retentions in grade.  Usually, students may be retained only once during their school career.  Students who continue to experience difficulties after retention then become candidates for special education classes, highly individualized programs and/or programs established under the Adaptive Dimension of Core Curriculum.
  • Student retention in grade is less of an issue at the high school level than the elementary and middle years level.  In high school, promotion is on a subject by subject basis and is often determined strictly by marks.

    Back to Table of Contents 



    Return of completed tests

    The two sides to this issue are:
     

  • Some teachers prefer to keep students’ completed tests and simply give students their mark.  This practice reduces teacher workload, because teachers can use the same test over and over.
  • Most students prefer to get their completed tests back, so they can see where they went wrong and use this information to improve in the future.
  • Generally, it is preferable to return students’ tests because this promotes greater learning.

    Many student evaluation policies contain a statement about the return of completed tests and specify the circumstances under which tests must be returned to the students and the circumstances under which they may be kept by the teacher.

    Back to Table of Contents 



    Recommendations

    Sometimes students are “recommended”, which means they pass a course without having to take the final exam.  It may be appropriate to include a statement about recommendations in a student evaluation policy.  This statement might respond to questions such as:
     

  • Are recommendations permitted in our school division?  If so, in what grades and what subject areas?
  • On what basis are recommendations given – student marks throughout the year, student effort, student attitude and behaviour?
  • Back to Table of Contents 



    Bonus marks

    Sometimes teachers give bonus marks on exams and routine assignments.  Bonus marks are typically given for factors such as student attitude, student effort, completing work on time and/or neatness of finished work.  Bonus marks mean that some students can get a mark in excess of 100 percent.  It may be appropriate to include a statement about bonus marks in a student evaluation policy.  This statement might respond to questions such as:
     

  • Are bonus marks permitted in our school division?  If so, in what grades and what subject areas?
  • For what type of work can bonus marks be given – routine class assignments, final exams, weekly or monthly exams?
  • On what basis are bonus marks given – student attitude, student effort, neatness of finished work, completing work on time, exceptional quality of work?
  • Can students achieve a mark greater than 100 percent?
  • Back to Table of Contents 



    Appeals of final examinations

    Most school division student evaluation policies give students the right to appeal the grade received on a final exam.  The policy may contain provisions such as the following:
     

  • description of the process for filing the appeal.  For example, appeals must be made in writing to the school principal within 14 days of the date on which the student receives the grade.
  • description of the steps in the review process.  For example:
  • guidelines for who gets information about the outcome of the appeal.  For example, the students, her/his parents, the subject teacher and the school principal.
  • Back to Table of Contents 



    Reporting to parents

    School division student evaluation policies often address both informal and formal reporting to parents.

    Informal Reporting

    Informal reporting can include telephone calls to parents and notes sent via e-mail or regular mail.  It is extremely important that informal reporting focus on students’ strengths and achievements as well as on their problems.  Otherwise, parents will come to dread every phone call from school.

    Formal Reporting

    The policy on formal reporting can include items such as:
     

  • the frequency and dates on which report cards are sent home.  For example, a policy might specify that report cards are completed and sent home three times a year:  in late November, late February and the end of June.
  • the way that achievement is reported on the report card at the elementary and middle years.  This is typically with letters such as N = not yet, S = satisfactory progress, A = above average progress.  At the high school level, progress may be reported with percentages or via a letter grade (A, B, C, etc.).
  • provision for parent/teacher/student conferences in conjunction with the report cards.  Some school divisions involve students in these conferences as well as teachers and parents.
  • The way that school divisions and individual schools handle parent/teacher/student conferences varies greatly.  It may be appropriate to include a policy statement that responds to questions such as:
     

  • Who attends the conference – teacher, parents, students, other school staff?
  • How much time is scheduled for the conference?  Some schools schedule an hour per student, others as little as five minutes.
  • When are conferences held?  Usually, they are held during the school day, but it may be advisable to have a few late afternoon or evening time slots for parents whose schedules don’t allow them to attend during the day.
  • What is the format of the conference?  For example, in some schools, the student leads the conference by talking about his/her own accomplishments and areas for improvement; in some, the conference is one-way with the teacher providing information to the parents; in still others, the parents are given an opportunity to talk about their perceptions of their child’s school experience.
  • Back to Table of Contents 



    Missed tests and exams

    Most school division student evaluation policies contain guidelines concerning missed tests and exams at the high school level.  These guidelines typically address:
     

  • the number of missed exams allowed.  For example, students may be permitted to miss up to three exams per year, no questions asked.  After three missed exams, an explanation may be required.
  • the types of explanations accepted for missed tests or exams.  For example, acceptable explanations may include a death in the family or other family crisis (in some cases, it may be appropriate to ask for a note from a minister, social worker or police officer); illness (with a doctor’s certificate); school bus breakdown, bad roads or bad weather.
  • how the missed exam will be handled.  A few options include:
  • Back to Table of Contents 

    Ownership and retention of student records

    Most school divisions will need to address the issue of access to and ownership of student records.

    At one time students’ records were held confidential by the school, and students and parents were not permitted to see them.  This is no longer the case.  With increasing openness in the education system, students and parents now usually have access to all school records relating to the specific student.  In their policy on student evaluation, school divisions may wish to respond to questions such as the following:
     

  • What are students/parents required to do to get access to a student’s records ? ask verbally, submit a formal written request?  How much notice do they have to give (one day, two weeks) before they want to access the records?
  • How is access provided?  Are photocopies of student’s records mailed to the student/parents?  Must parents come to school to see actual records?
  • When a student leaves the school, what happens to the student’s records?  Options include:
  • Back to Table of Contents 

    The basis on which marks are assigned

    This section of the student evaluation policy can address topics such as:
     

  • using a variety of sources for evaluation data;
  • the role of achievement and effort;
  • marks for attitude and behaviour;
  • attendance; and,
  • the role of achievement and growth.
  • Using a Variety of Sources for Evaluation Data

    A single data source gives a limited view of student achievement.  Evaluating a student and/or assigning a mark on the basis of a single test, exam or other data source is bad evaluation practice and very unfair to the student.  A comprehensive picture of student achievement in relation to course objectives is best obtained by using several different data sources.  These can include:
     

  • routine classroom assignments;
  • portfolios of student work;
  • teacher-made tests;
  • standardized tests;
  • homework assignments;
  • observational checklists, rating scales, and anecdotal records, completed by the teacher;
  • student self-evaluation;
  • student peer-evaluation; and,
  • student performance - This applies particularly to areas such as physical education, drama and music, and also to performance during science experiments and at math performance stations.
  • The Role of Achievement and Effort

    When expectations and objectives are the same for all students, an average student may need to work very hard to achieve top-level performance, while a highly gifted student may put in little effort and still achieve the same level of performance.  Some student evaluation policies contain statements in response to questions such as:
     

  • Is achievement the only criterion considered when assigning marks, or is the amount of effort the student puts in to accomplish that level of achievement also considered?
  • Are programs and expectations individualized for highly gifted and slower students?  This isn’t always the best solution for slow students. Generally, if you expect less of a student, the student will achieve in accordance with expectations.  Thus, it is important not to underestimate what a student can do.  However, it is also important not to have such high expectations that slower students have no chance of success and experience only failure and frustration.
  • Marks for Attitude and Behaviour

    Sometimes, teachers also consider attitude and behaviour when assigning marks.  A student who demonstrates good achievement, but is troublesome in the classroom will get a lower mark.  A student whose achievement is low will still get a good mark because s/he is obedient and well behaved.  This is not a good idea.  It is best to consider achievement and behaviour separately and to report on them separately.  The student evaluation policy might clearly state that achievement and attitude/behaviour are evaluated separately.

    Attendance

    Another issue to consider in the policy on assigning marks is attendance.  Some teachers lower a student’s mark if the student has many absences regardless of the student’s level of achievement.  Generally, it is best to consider attendance and achievement separately and to report upon them separately.

    The Role of Achievement and Growth

    Some teachers give marks for achievement only, others focus primarily on student growth.  For example, a student who performs very well but shows little growth over the year might get a lower final mark than a student whose performance is low but shows significant growth over the year.  A student evaluation policy might contain a statement about the relative importance of achievement and growth when assigning marks.  Some school divisions prefer to evaluate these two elements separately.
     
    Back to Table of Contents 



    Making the policy known

    A policy of student evaluation will only fulfill its objectives (promoting student learning, ensuring fairness and consistency for all students, and preventing problems and issues from arising) if the guidelines it contains are well known to all students, teachers and parents.

    Thus, many student evaluation policies contain guidelines for making the existence of the entire policy known and for familiarizing students, teachers and parents with guidelines for specific issues such as missed exams and appeals of exams.  Some ideas for doing this include:
     

  • Make the principles for fair student assessment practices known by making these principles the focus of:
  • Make guidelines for specific issues known by:
  • Back to Table of Contents 

    References
     
    1. Saskatchewan Education.  (1996).  1995 provincial learning assessment in mathematics.  Regina, SK:  Author.
    (www.Sasked.gov.sk.ca/k/p_e/eval/plap/math/1998pdf)
    2. Saskatchewan Education.  (1998).  1997 provincial learning assessment in mathematics.  Regina, SK:  Author.
    (www.Sasked.gov.sk.ca/k/p_e/eval/plap/math/1997pdf)
    3. Saskatchewan Education.  (1996).  1994 provincial learning assessment in language arts (reading and writing).  Regina, SK:  Author.  (www.Sasked.gov.sk.ca/k/p_e/eval/plap/language/1994pdf)
    4. Saskatchewan Education.  (1997).  1996 provincial learning assessment in language arts (reading and writing).  Regina, SK:  Author. (www.Sasked.gov.sk.ca/k/p_e/eval/plap/language/1996pdf)
    5. Saskatchewan Education.  (1999).  1998 provincial learning assessment in English  language arts (listening and speaking).  Regina, SK:  Author. (www.Sasked.gov.sk.ca/k/p_e/eval/plap/listening_speaking/1998pdf)
    6. Saskatchewan Education.  (1994).  Science 1-5 curriculum evaluation project report [1993].  Regina, SK:  Author.  (www.Sasked.gov.sk.ca/k/p_e/eval/curriculum/science-5/1994pdf)
    7. Saskatchewan Education.  (1996).  Curriculum evaluation report:  Health education 7, 8 and 9 [1994].  Regina, SK:  Author.  (www.Sasked.gov.sk.ca/k/p_e/eval/curriculum/health7,8,9/1996pdf)
    8. Saskatchewan Education.  (1995).  Mathematics 5, 8 and 11 curriculum project [1995].  Regina, SK:  Author. (www.Sasked.gov.sk.ca/k/p_e/eval/assess/math5_8_11/language/index.htm/)
    9. Saskatchewan Education.  (1996).  Curriculum evaluation report:  Grades 7, 8 and 9 Social Studies [1995].  Regina, SK:  Author. (www.Sasked.gov.sk.ca/k/p_e/eval/curriculum/social-studies/1996pdf)
    10. Saskatchewan Education.  (1998).  Arts education:  Grades 1-9 curriculum evaluation report.  Regina, SK:  Author.
    11. Council of Ministers of Education, Canada.  (1996).  SAIP:  School achievement indicators program:  1996 report on science assessment.  Toronto, ON:  Author.  (www.cmec.ca/saip/sci96/)
    12. Council of Ministers of Education, Canada.  (1995).  SAIP:  School achievement indicators program:  1993 report on mathematics assessment.  Toronto, ON:  Author.  (www.cmec.ca/saip/math93en.htm)
    13. Council of Ministers of Education, Canada.  (1997).  SAIP:  School achievement indicators program:  1997 report on mathematics assessment.  Toronto, ON:  Author.  (www.cmec.ca/saip/math97/index.stm)
    14. Council of Ministers of Education, Canada.  (1994).  SAIP:  School achievement indicators program:  1994 report on reading and writing assessment.  Toronto, ON:  Author.  (www.cmec.ca/saip/writ94en.htm)
    15. Council of Ministers of Education, Canada.  (1999).  SAIP:  School achievement indicators program:  1998 report on reading and writing assessment.  Toronto, ON:  Author.  (URL unavailable at date of writing.)
    16. Saskatoon (West) School Division #42 of Saskatchewan.  (1993).  Student evaluation policy.
    17. For information on the generally harmful effects of student retention in grade refer to: Back to Table of Contents 
    Appendix A
    Principles for Fair Student Assessment Practices for Education in Canada
     
    The Principles for Fair Student Assessment Practices for Education in Canada contains a set of principles and related guidelines accepted by professional organizations as indicative of fair assessment practice within the Canadian educational context.  Assessments depend on professional judgement; the principles and related guidelines presented in this document identify the issues to consider in exercising this professional judgement and in striving for the fair and equitable assessment of all students.

    Assessment practice is broadly defined in the Principles as the process of collecting and interpreting information that can be used:  (i) to provide feedback to students, and to their parents/guardians where applicable, about the progress they are making toward attaining the knowledge, skills, attitudes, and behaviours to be learned or acquired, and (ii) to inform the various educational decisions (instructional, diagnostic, placement, promotion, graduation, curriculum planning, program development, policy) that are made with reference to students.  Principles and related guidelines are set out for both developers and users of assessments.  Developers include people who construct assessment methods and people who set policies for particular assessment programs.  Users include people who select and administer assessment methods, commission assessment development services, or make decision on the basis of assessment results and findings.  The roles may overlap, as when a teacher or instructor develops and administers an assessment instrument and then scores and interprets the students’ responses, or when a ministry or department of education or local school system commissions the development and implementation of an assessment program and scoring services and makes decisions on the basis of the assessment results.



    The Principles of Fair Student Assessment Practices for Education in Canada was developed by a Working Group guided by a Joint Advisory Committee.  The Joint Advisory Committee included two representatives appointed by each of the following professional organizations:  Canadian Education Association, Canadian School Boards Association, Canadian Association for School Administrators, Canadian Teachers’ Federation, Canadian Guidance Counselling Association, Canadian Association of School Psychologists, Canadian Council for Exceptional Children, Canadian Psychological Association, and Canadian Society for the Study of Education.  In addition, the Joint Advisory Committee included a representative of the Provincial and Territorial Ministries and Departments of Education.

    Financial support for the development and dissemination of the Principles was provided principally by the Walter and Duncan Gordon Charitable Foundation, with additional support provided by various Faculties, Institutes, and Colleges of Education and Provincial and Territorial Ministries and Departments of Education in Canada.  This support is gratefully acknowledged.

    The Joint Advisory Committee invites users to share their experiences in working with the Principles and to submit any suggestions that could be used to revise and improve the Principles.  Comments and suggestions should be sent to the Joint Advisory Committee at the address shown below.

    The Principles for Fair Student Assessment Practices for Education in Canada is not copyrighted.  Reproduction and dissemination are encouraged.  Principles reproduced in this document with permission.  Please cite the Principles as follows:

    Principles for Fair Student Assessment Practices for Education in Canada.  (1993).  Edmonton, Alberta:  Joint Advisory Committee (Mailing address:  Joint Advisory Committee, Centre for Research in Applied Measurement and Evaluation, 3-104 Education Building North, University of Alberta, Edmonton, Alberta, T6G 2G5).


    The Principles for Fair Student Assessment Practices for Education in Canada is the product of a comprehensive effort to reach consensus on what constitutes sound principles to guide the fair assessment of students.  The principles and their related guidelines should be considered neither exhaustive nor mandatory; however, organizations, institutions, and individual professionals who endorse them are committing themselves to endeavour to follow their intent and spirit so as to achieve fair and equitable assessments of students.

    Organization and Use of the Principles

    The Principles and their related guidelines are organized in two parts.  Part A is directed at assessments carried out by teachers at the elementary and secondary school levels.  Part A is also applicable at the post-secondary level with some modifications, particularly with respect to whom assessment results are reported.  Part B is directed at standardized assessments developed external to the classroom by commercial test publishers, provincial and territorial ministries and departments of education, and local school jurisdictions.1
     

    1  Boards, boroughs, counties, and school districts.
     
    Five general principles of fair assessment practices are provided in each Part.  Each principle is followed by a series of guidelines for practice.  In the case of Part A where no prior sets of standards for fair practice exist, a brief comment accompanies each guideline to help clarify and illuminate the guideline and its application.

    The Joint Advisory Committee recognizes that in the field of assessment some terms are defined or used differently by different groups of people.  To maintain as much consistency in terminology as possible, an attempt has been made to employ generic terms in the Principles.
     
    Part A:  Classroom Assessments

    Part A is directed toward the development and selection of assessment methods and their use in the classroom by teachers.  It is organized around five interrelated themes:

    I.  Developing and Choosing Methods for Assessment
    II. Collecting Assessment Information
    III.  Judging and Scoring Student Performance
    IV. Summarizing and Interpreting Results
    V.  Reporting Assessment Findings
     

    I.  Developing and Choosing Methods for Assessment
     

    Assessment methods should be appropriate for and compatible with the purpose and context of the assessment.


    Assessment method is used here to refer to the various strategies and techniques that teachers might use to acquire assessment information.  These strategies and techniques include, but are not limited to, observations, text- and curriculum- embedded questions and tests, paper-and-pencil tests, oral questioning, benchmarks or reference sets, interviews, peer-and self-assessments, standardized criterion-referenced and norm-referenced tests, performance assessments, writing samples, exhibitions, portfolio assessment, and project and product assessments.  Several labels have been used to describe subsets of these alternatives, with the most common being "direct assessment," "authentic assessment," "performance assessment," and "alternative assessment".  However, for the purpose of the Principles, the term assessment method has been used to encompass all the strategies and techniques that might be used to collect information from students about their progress toward attaining the knowledge, skills, attitudes, or behaviours to be learned.
     
    Principle  Commentary
    1. Assessment methods should be developed or chosen so that inferences drawn about the knowledge, skills, attitudes, and behaviors possessed by each student are valid and not open to misinterpretation. Validity refers to the degree to which inferences drawn from assessments results are meaningful.  Therefore, development or selection of assessment methods for collecting information should be clearly linked to the purposes for which inferences and decisions are to be made.  For example, to monitor the progress of students as proofreaders and editors of their own work, it is better to assign an actual writing task, to allow time and resources for editing (dictionaries, handbooks, etc.) and to observe students for evidence of proofreading and editing skill as they work than to use a test containing discrete items on usage and grammar that are relatively devoid of context. 
    2. Assessment methods should be clearly related to the goals and objectives of instruction, and be compatible with the instructional approaches used. To enhance validity, assessment methods should be in harmony with the instructional objectives to which they are referenced.  Planning an assessment design at the same time as planning instruction will help integrate the two in meaningful ways.  Such joint planning provides an overall perspective on the knowledge, skills, attitudes, and behaviors to be learned and assessed, and the contexts in which they will be learned and assessed. 
    3. When developing or choosing assessment methods, consideration should be given to the consequences of the decisions to be made in light of the obtained information. The outcomes of some assessments may be more critical than others. For example, misinterpretation of the level of performance on an end-of-unit test may result in incorrectly holding a student form proceeding to the next to the next instructional unit in a continuous progress situation. In such "high-stake" situations, every effort should be made to ensure the assessment method will yield consistent and valid results.  "Low stake" situations, such as determining if a student has correctly completed an in-class assignment, can be less stringent.  Low stake assessments are often repeated during the course of a reporting period using a variety of methods. If the results are aggregated to form a summary comment or grade, the summary will have greater consistency and validity than its component elements. 
    4. More than one assessment method should be used to ensure comprehensive and consistent indications of student performance. To obtain a more complete picture or profile of a student's knowledge, skills, attitudes, or behaviors, and to discern consistent patterns and trends, more than one assessment method should be used.  Student knowledge might be assessed using completion items; process or reasoning skills might be assessed by observing performance on a relevant task; evaluation skills might be assessed by reflecting upon the discussion with a student about what materials to include in a portfolio. Self-assessment may help to clarify and add meaning to the assessment of a written communication, science project, piece of artwork, or an attitude.  Use of more than one method will also help minimize inconsistency brought about by different sources of measurement error (for example, poor performance because of an "off-day"; lack of agreement among items included in a test, rating scale, or questionnaire; lack of agreement among observers; instability across time). 
    5. Assessment methods should be suited to the backgrounds and prior experiences of students. Assessment methods should be free from bias brought about by student factors extraneous to the purpose of the assessment.  Possible factors to consider include culture, developmental stage, ethnicity, gender, socio-economic background, language, special interests, and special needs.  Students' success in answering questions on a test or in an oral quiz, for example, should not be dependent upon prior cultural knowledge, such as understanding an allusion to a cultural tradition or value, unless such knowledge falls within the content domain being assessed.  All students should be given the same opportunity to display their strengths. 
    6. Content and language that would generally be viewed as sensitive, sexist, or offensive should be avoided. The vocabulary and problem situation in each test item or performance task should not favour or discriminate against any group of students. Steps should be taken to ensure that stereotyping is not condoned. Language that might be offensive to particular groups of students should be avoided.  A judicious use of different roles for males and females and for minorities and the careful use of language should contribute to more effective and, therefore, fairer assessments. 
    7. Assessment instruments translated into a second language or transferred from another context or location should be accompanied by evidence that inferences based on these instruments are valid for the intended purpose.  Translation of an assessment instrument from one language to another is a complex and demanding task. Similarly, the adoption or modification of an instrument developed in another country is often not simple and straightforward.  Care must be taken to ensure that the results from translated and imported instruments are not misinterpreted or misleading. 
     
    II.  Collecting Assessment Information
     

    Students should be provided with a fair opportunity to demonstrate the knowledge, skills, attitudes, or behaviors being assessed.


     
    Assessment information can be collected in a variety of ways (observations, oral questioning, interviews, oral and written reports, paper-and-pencil tests).  The guidelines that follow are not all equally applicable to each of these procedures.
     
    Principle  Commentary 
    1. Students should be told why assessment information is being collected and how this information will be used.  Students who know the purpose of an assessment are in a position to respond in a manner that will provide information relevant to that purpose.  For example, if students know that their participation in a group activity is to be used to assess cooperative skills, they can be encouraged to contribute to the activity. If students know that the purpose of an assessment is to diagnose strengths and weaknesses rather than to assign a grade, they can be encouraged to reveal weaknesses as well as strengths. If the students know that the purpose is to assign a grade, they are well advised to respond in a way that will maximize strengths. This is especially true for assessment methods that allow students to make choices, such as with optional writing assignments or research projects. 
    2. An assessment procedure should be used under conditions suitable to its purpose and form. Optimum conditions should be provided for obtaining data from and information about students so as to maximize the validity and consistency of the data and information collected.  Common conditions include such things as proper light and ventilation, comfortable room temperature, and freedom from distraction (e.g. movement in and out of the room, noise).  Adequate workspace, sufficient materials, and adequate time limits appropriate to the purpose and form of the assessment are also necessary.  For example, if the intent is to assess student participation in a small group, adequate work space should be provided for each student group, with sufficient space between subgroups so that the groups do not interfere with or otherwise influence one another and so that the teacher has the same opportunity to observe and assess each student within each group.
    3. In assessments involving observations, checklists, or rating scales, the number of characteristics to be assessed at one time should be small enough and concretely described so that the observations can be made accurately. Student behaviors often change so rapidly that it may not be possible simultaneously to observe and record all the behavior components.  In such instances, the number of components to be observed should be reduced and the components should be described as concretely as possible.  One way to manage an observation is to divide the behavior into a series of components and assess each component in sequence. By limiting the number of components assessed at one time, the data and information become more focused, and time is not spent observing later behaviour until prerequisite behaviours are achieved. 
    4. The directions provided to students should be clear, complete, and appropriate for the ability, age and grade level of the students. Lack of understanding of the assessment task may prevent maximum performance or display of the behavior called for.  In the case of timed assessments, for example, teachers should describe the time limits, explain how students might distribute their time among parts of those assessment instruments with parts, and describe how students should record their responses.  For a portfolio assessment, teachers should describe the criteria to be used to select the materials to be included in a portfolio, who will select these materials, and it more than one person will be involved in the selection process, how the judgments from the different people will be combined.  Where appropriate, sample material and practice should be provided to further increase the likelihood that instructions will be understood. 
    5. In assessments involving selection items (i.e., true/false, multiple-choice), the directions should encourage students to answer all items without threat of penalty. A correction formula is sometimes used to discourage "guessing" on selection items.  The formula is intended to encourage students to omit items for which they do not know the answer rather than to "guess" the 
    Answer.  Because research evidence indicates that the benefits expected from the correction are not realized, the use of the formula is discouraged. Students should be encouraged to use whatever partial knowledge they have when choosing their answers, and to answer all items. 
    6. When collecting assessment information, interactions with students should be appropriate and consistent.  Care must be taken when collecting assessment information to treat all students fairly. For example, when oral presentations by students are assessed, questioning and probes should be distributed among the students so that all students have the same opportunity to demonstrate their knowledge. While writing a paper-and-pencil test, a student may ask to have an ambiguous item clarified, and, if warranted, the item should be explained to the entire class. 
    7. Unanticipated circumstances that interfere with the collection of assessment information should be noted and recorded.  Events such as a fire drill, an unscheduled assembly, or insufficient materials may interfere in the way in which assessment information is collected.  Such events should be recorded and subsequently considered when interpreting the information obtained. 
    8. A written policy should guide decisions about the use of alternate procedures for collecting assessment information from students with special needs and students whose proficiency in the language of instruction is inadequate for them to respond in the anticipated manner.  It may be necessary to develop alternative assessment procedures to ensure a consistent and valid assessment of those students who, because of special needs or inadequate language, are not able to respond to an assessment method (for example, oral instead of written format, individual instead of group administered, translation into first language, providing additional time).  The use of alternate procedures should be guided by a written policy developed by teachers, administrators, and other jurisdictional personnel. 
     
    III.  Judging and Scoring Student Performance
     

    Procedures for judging or scoring student performance should be appropriate for the assessment method used and be consistently applied and monitored. 

     
    Judging and scoring refers to the process of determining the quality of a student's performance, the appropriateness of an attitude or behavior, or the correctness of an answer. Results derived from judging and scoring may be expressed as written or oral comments, ratings, categorizations, letters, numbers, or as some combination of these forms.
     
    Principle  Commentary 
    1. Before an assessment method is used a procedure for scoring should be prepared to guide the process of judging the quality of a performance or product, the appropriateness of an attitude or behaviour, or the correctness of an answer. To increase consistency and validity, properly developed scoring procedures should be used.  Different assessment methods require different forms of scoring.  Scoring selections items (true/false, multiple-choice, matching) requires the identification of the correct or, in some instances, best answer.  Guides for scoring essays might include factors such as the major points to be included in the "best answer" or models or exemplars corresponding to different levels of performance at different age levels and against which comparisons can be made.  Procedures for judging other performances or products might include specification of the characteristics to be rated in performance terms and, to the extent possible, clear descriptions of the different levels of performance or quality of a product. 
    2. Before an assessment method is used, students should be told how their responses or the information they provide will be judged or scored. Informing students prior to the use of an assessment method about the scoring procedures to be followed should help ensure that similar expectations are held by both students and their teachers. 
    3. Care should be taken to ensure that results are not influenced by factors that are not relevant to the purpose of the assessment. Various types of errors occur in scoring, particularly when a degree of subjectivity is involved (e.g., marking essays, rating a performance, judging a debate).  For example, if the intent of a written communication is to assess content alone, the scoring should not be influenced by stylistic factors such as vocabulary and sentence structure.  Personal bias errors are indicated by a general tendency to rate all students in approximately the same way (e.g., too generously or too severely). Halo effects can occur when a rater's general impression of a student influences the rating of individual characteristics or when a previous rating influences a subsequent rating. Pooled results from two or more independent raters (teachers, other students) will generally produce a more consistent description of student performance than a result obtained from a single rater. In combining results, the personal biases of individual raters tend to cancel one another. 
    4. Comments formed as part of scoring should be based on the responses made by the students and presented in a way that students can understand and use them. Comments, in oral and written form, are provided to encourage learning and to point out correctable errors or inconsistencies in performance.  In addition, comments can be used to clarify a result. Such feedback should be based on evidence pertinent to the learning outcomes being assessed. 
    5. Any changes made during scoring should be based upon a demonstrated problem with the initial scoring procedure.  The modified procedure should then be used to restore all previously scored responses. Anticipating the full range of student responses is a difficult task for several forms of assessment.  There is always the danger that unanticipated responses or incidents that are relevant to the purposes of the assessment may be overlooked.  Consequently, scoring should be continuously monitored for unanticipated responses and these responses should be taken into proper account. 
    6. An appeal process should be described to students at the beginning of each school year or course of instruction that they may use to appeal a result. Situations may arise where a student believes a result incorrectly reflects his/her level of performance. A procedure by which students can appeal such a situation should be developed and made known to them.  This procedure might include, for example, checking for addition or other recording errors or, perhaps, judging or scoring by a second qualified person. 
     
    IV.  Summarizing and Interpreting Results


    Procedures for summarizing and interpreting assessment results should yield accurate and informative representations of a student's performance in relation to the goals and objectives of instruction for the reporting period. 

    Summarizing and interpreting results refers to the procedures used to combine assessment results in the form of summary comments and grades that indicate both a student's level of performance and the valuing of that performance.
     
    Principle  Commentary 
    1. Procedures for summarizing and interpreting results for a reporting period should be guided by a written policy. Summary comments and grades, when interpreted, serve a variety of functions.  They inform students of their progress.  Parents, teachers, counsellors, and administrators use them to guide learning, determine promotion, identify students for special attention (e.g., honours, remediation), and to help students develop future plans.  Comments and grades also provide a basis for reporting to other schools in the case of school transfer and, in the case of senior high school students, post-secondary institutions and prospective employers.  They are more likely to serve their many functions and those functions are less likely to be confused if they are guided by a written rationale or policy sensitive to these different needs. This policy should be developed by teachers, school administrators, and other jurisdictional personnel in consultation with representatives of the audiences entitled to receive a report of summary comments and grades. 
    2. The way in which summary comments and grades are formulated and interpreted should be explained to students and their parents/guardians. Students and their parents/guardians have the "right-to-know" how student performance is summarized and interpreted.  With this information, they can make constructive use of the findings and fully review the assessment procedures followed.  It should be noted that some aspects of summarizing and interpreting are based upon a teacher's best judgment of what is good or appropriate.  This judgment is derived from training and experience and may be difficult to describe specifically in advance. In such circumstances, examples might be used to show how summary comments and grades were formulated and interpreted. 
    3. The individual results used and the process followed in deriving summary comments and grades should be described in sufficient detail so that the meaning of a summary comment or grade is clear. Summary comments and grades are best interrupted in the light of an adequate description of the results upon which they are based, the relative emphasis given to each result, and the process followed to combine the results. Many assessments conducted during a reporting period are of a formative nature.  The intent of these assessments (e.g., informal observations, quizzes, text-and-curriculum embedded questions, oral questioning) is to inform decisions regarding daily learning, and to inform or otherwise refine the instructional sequence. Other assessments are of a summative nature.  It is the summative assessments that should be considered when formulating and interpreting summary comments and grades for the reporting period. 
    4. Combine disparate kinds of results into a single summary should be done cautiously. To the extent possible, achievement, effort, participation, and other behaviors should be graded separately. A single comment or grade cannot adequately serve all functions.  For example, letter grades used to summarize achievement are most meaningful when they represent only achievement.  When they include other aspects of student performance such as effort, amount (as opposed to quality) of work completed, neatness, class participation, personal conduct, or punctuality, not only do they lose their meaningfulness as a measure of achievement, but they also suppress information concerning other important aspects of learning and invite inequities.  Thus, to more adequately and fairly summarize the different aspects of student performance, letter grades for achievement might be complemented with alternate summary forms (e.g. checklists, written comments) suitable for summarizing results related to these other behaviours. 
    5. Summary comments and grades should be based on more than one assessment result so as to ensure adequate sampling of broadly defined learning outcomes.  More than one or two assessments are needed to adequately assess performance in multi-facet areas such as Reading. Under-representation of such broadly defined constructs can be avoided by ensuring that the comments and grades used to summarize performance are based on multiple assessments, each referenced to a particular facet of the construct. 
    6. The results used to produce summary comments and grades should be combined in a way that ensures that each result receives its intended emphasis or weight.  When the results of a series of assessments are combined into a summary comment, care should be taken to ensure that the actual emphasis placed on the various results matches the intended emphasis for each student. 

    When numerical results are combined, attention should be paid to differences in the variability, or spread, of the different sets of results and appropriate account taken where such differences exist.  If, for example, a grade is to be formed from a series of paper-and-pencil tests, and if each test is to count equally in the grade, then the variability of each set of scores must be the same. 

    7. The basis for interpretation should be carefully described and justified. Interpretation of the information gathered for a reporting period for a student is a complex and, at times, controversial issue.  Such information, whether written or numerical, will be of little interest or use if it is not interpreted against some pertinent and defensible idea of what is good and what is poor.  The frame of reference used for interpretation should be in accord with the type of decision to be made.  Typical frames of reference are performance in relation to pre-specified standards, performance in relation to peers, performance in relation to aptitude or expected growth, and performance in terms of the amount of improvement or amount learned. If, for example, decisions are to be made as to whether or not a student is ready to move to the next unit in an instructional sequence, interpretations based on pre-specified standards would be most relevant. 
    8. Interpretations of assessment results should take account of the backgrounds and learning experiences of the students.  Assessment results should be interpreted in relation to a student's personal and social context.  Among the factors to consider are age, ability, gender, language, motivation, opportunity to learn, self-esteem, socio-economic backgrounds, special interests, special needs, and "test-taking" skills. Motivation to do school tasks, language capability, or home environment can influence learning of the concepts assessed, for example.  Poor reading ability, poorly developed psychomotor or manipulative skills, lack of test-taking skills, anxiety, and low self-esteem can lead to lower scores.  Poor performance in an assessment may be attributable to a lack of opportunity to learn because required learning materials and supplies were not available, learning activities were not provided, or inadequate time was allowed for learning.  When a student performs poorly, the possibility that one or more factors such as these might have interfered with a student's response or performance should be considered. 
    9. Assessment results that will be combined into summary comments and grades should be stored in a way that ensures their accuracy at the time they are summarized and interpreted. Comments and grades and their interpretations, formulated form a series of related assessments, can be no better than the data and information upon which they are based.  Systematic data control minimizes errors, which would otherwise be introduced into a student's record or information base, and provides protection of confidentiality. 
    10. Interpretations of assessment results should be made with due regard for limitations in the assessment methods used, problems encountered in collecting the information and judging or scoring it, and limitations in the basis used for interpretation.  To be valid, interpretations must be based on results determined from assessment methods that are relevant and representative of the performance assessed.  Administrative constraints, the presence of measurement error, and the limitations of the frames of reference used for interpretation also need to be accounted for. 
     
    V.  Reporting Assessment Findings
     

    Assessment reports should be clear, accurate, and of practical value to the audiences
    for whom they are intended. 

     
    Principle  Commentary 
    1. The reporting system for a school or jurisdiction should be guided by a written policy. Elements to consider include such aspects as audiences, medium, format, content, level of detail, frequency, timing, and confidentiality. The policy to guide the preparation of school reports (e.g., reports of separate assessments; reports for a reporting period) should be developed by teachers, school administrators, and other jurisdictional personnel in consultation with representatives of the audiences entitled to receive a report.  Cooperative participation not only leads to more adequate and helpful reporting, but also increases the likelihood that the reports will be understood and used by reporting, but also increases the likelihood that the reports will be understood and used by those for whom they are intended. 
    2. Written and oral reports should contain a description of the goals and objectives of instruction to which the assessments are referenced.  The goals and objectives that guided instruction should serve as the basis for reporting.  A report will be limited by a number of practical considerations, but the central focus should be on the instructional objectives and the types of performance that represent achievement of these objectives. 
    3. Reports should be complete in their descriptions of strengths and weaknesses of students, so that strengths can be built upon and problem areas addressed. Reports can be incorrectly slanted towards "faults" in a student or toward giving unqualified praise.  Both biases reduce the validity and utility of assessment.  Accuracy in reporting strengths and weaknesses helps to reduce systematic error and is essential for stimulating and reinforcing improved performance.  Reports should contain the information that will assist and guide students, their parents/guardians, and teachers to take relevant follow-up actions. 
    4. The reporting system should provide for conferences between teachers and parents/guardians. Whenever it is appropriate, students should participate in these conferences.  Conferences scheduled at regular intervals and, if necessary, upon request provide parents/guardians and, when appropriate, students with an opportunity to discuss assessment procedures, clarify and elaborate their understanding of the assessment results, summary comments and grades, and reports, and, where warranted, to work with teachers to develop relevant follow-up activities or action plans.
    5. An appeal process should be described to students and their parents/guardians at the beginning of each school year or course of instruction that they may use to appeal a report.  Situations may arise where a student and his/her parents/guardian believe the summary comments and grades inaccurately reflect the level of performance of the student.  A procedure by which they can appeal such a situation should be developed and made known to them (for example, in a school handbook or newsletter provided to students and their parents/guardians at the beginning of the school year). 
    6. Access to assessment information should be governed by a written policy that is consistent with applicable laws and with basic principles of fairness and human rights. A written policy, developed by teachers, administrators, and other jurisdictional personnel, should be used to guide decisions regarding the release of student assessment information.  Assessment information should be available to those people to whom it applies ? students and their parents/guardians, and to teachers and other educational personnel obligated by profession to use the information constructively on behalf of students. In addition, assessment information might be made available to others who justify their need for the information (e.g., post-secondary institutions, potential employers, researchers).  Issues of informed consent should also be addressed in this policy. 
    7. Transfer of assessment information from one school to another should be guided by a written policy with stringent provisions to ensure the maintenance of confidentiality.  To make a student's transition from one school to another as smooth as possible, a clear policy should be prepared indicating the type of information to go with the student and the form in which it will be reported.  Such a policy, developed by jurisdictional and ministry personnel, should ensure that the information transferred will be sent by and received by the appropriate person within the "sending" and "receiving" schools respectively. 
     
    Part B:  Assessments Produced External to the Classroom

    Part B applies to the development and use of standardized assessment methods used in student admissions, placement, certification, and educational diagnosis, and in curriculum and program evaluation. These methods are primarily developed by commercial test publishers, ministries and departments of education, and local school systems.

    The principles and accompanying guidelines are organized in terms of four areas:

    I. Developing and Selecting Methods for Assessment
    II. Collecting and Interpreting Assessment Information
    III. Informing Students Being Assessed
    IV. Implementing Mandated Assessment Programs

    The first three areas of Part B are adapted from the Code of Fair Testing Practices for Education (1988) developed in the United States.  The principles and guidelines as modified in these three sections are intended to be consistent with the Guidelines for Educational and Psychological Testing (1986) developed in Canada.  The fourth area has been added to contain guidelines particularly pertinent for mandated educational assessment and testing programs developed and conducted at the national, provincial, and local levels.
     

    I.  Developing and Selecting Methods for Assessment
     


    Developers of assessment methods should strive to make them as fair as possible for use with students who have different backgrounds or special needs. Developers should provide the information users need to select methods appropriate to their assessment needs. Users should select assessment methods that have been developed to be as fair as possible for students who have different backgrounds or special needs.  Users should select methods that are appropriate for the intended purposes and suitable for the students to be assessed.


    Developers should: Users should: 
    1. Define what the Assessment method is intended to measure and how it is to be used.  Describe the characteristics of the students with which the method may be used. 1. Determine the purpose(s) for assessment and the characteristics of the students to be assessed. Then select an assessment method suited to that purpose and type of student.
    2. Warn users against common misuses of the assessment method. 2. Avoid using assessment methods for purposes not specifically recommended by the developer unless evidence is obtained to support the intended use. 
    3. Describe the process by which the method was developed. Include a description of the theoretical basis, rationale for selection of content and procedures, and derivation of scores.  3. Review available assessment methods for relevance of content and appropriateness of scores with reference to the intended purpose(s) and characteristics of the students to be assessed. 
    4. Provide evidence that the assessment method yields results that satisfy its intended purpose(s).  4. Read independent evaluations of the methods being considered.  Look for evidence supporting the claims of developers with reference to the intended application of each method. 
    5. Investigate the performance of students with special needs and students from different backgrounds. Report evidence of the consistency and validity of the results produced by the assessment method for these groups. 5. Ascertain whether the content of the assessment method and the norm group(s) or comparison group(s) or comparison group(s) are appropriate for the students to be assessed. For assessment methods developed in other regions or countries, look for evidence that the characteristics of the norm group(s) or comparison group(s) are comparable to the characteristics of the students to be assessed.
    6. Provide potential users with representative samples or complete copies of questions or tasks, directions, answer sheets, score reports, guidelines, guidelines for interpretation, and manuals.  6. Examine specimen sets, samples or complete copies of assessment instruments, directions, answer sheets, score reports, guidelines for interpretation, and manuals and judge their appropriateness for the intended application. 
    7. Review printed assessment methods and related materials for content or language generally perceived to be sensitive, offensive, or misleading. 7. Review printed assessment methods and related materials for content or language that would offend or mislead the students to be assessed. 
    8. Describe the specialized skills and training needed to administer an assessment method correctly, and the specialized knowledge to make valid interpretations of scores. 8. Ensure that all individuals who administer the assessment method, score the responses, and interpret the results have the necessary knowledge and skills to perform these tasks (e.g., learning assistance teachers, speech and language pathologists, counsellors, school psychologists, psychologists). 
    9. Limit sales of restricted assessment materials to persons who possess the necessary qualifications.  9. Ensure access to restricted assessment materials is limited to persons with the necessary qualifications. 
    10. Provide for periodic review and revision of content and norms, and, if applicable, passing or cut-off scores, and inform users. 10. Obtain information about the appropriateness of content, the regency of norms, and, if applicable, the appropriateness of the cut-off scores for use with the students to be assessed. 
    11. Provide evidence of the comparability of different forms of an instrument where the forms are intended to be interchangeable, such as parallel forms or the adaptation of an instrument for computer administration.  11. Obtain information about the comparability of interchangeable forms, including computer adaptations. 
    12. Provide evidence that an assessment method translated into a second language is valid for use with the second language.  This information should be provided in the second language.  12. Obtain evidence about the validity of the use of an assessment method translated into a second language.
    13. Advertise an assessment method in a way that states it can be used only for the purposes for which it was intended. 13. Verify advertising claims made for an assessment method. 


     
    II.  Collecting and Interpreting Assessment Information
     


    Developers should provide information to help users administer an assessment method correctly and interpret assessment results accurately.  Users should follow directions for proper administration of an assessment method and interpretation of assessment results. 


    Developers should: Users should: 
    1. Provide clear instructions for administering the assessment method and identify the qualifications that should be held by the people who should administer the method.  1. Ensure that the assessment method is administered by qualified personnel or under the supervision of qualified personnel. 
    2. When feasible, make available appropriately modified forms of assessment methods for students with special needs or whose proficiency in the original language of administration is inadequate to respond in the anticipated manner.  2. When necessary and feasible, use appropriately modified forms of assessment methods with students who have special needs or whose proficiency in the original language of administration is inadequate to respond in the anticipated manner. 

    Ensure that instruments translated from one language to another are administered by persons who are proficient in the translated language. 

    3. Provide answer keys and describe procedures for scoring when scoring is to be done by the user.  3. Follow procedures for scoring as set out for the assessment method. 
    4. Provide score reports or procedures for generating score reports that describe assessment results clearly and accurately. Identify and explain possible misinterpretations of the scores yielded by the scoring system (grade equivalents, percentile ranks, standard scores) used.  4. Interpret scores taking into account the limitations of the scoring system used.  Avoid misinterpreting scores on the basis of unjustified assumptions about the scoring system (grade-equivalents, percentile ranks, standard scores) used. 
    5. Provide evidence of the effects on assessment results of such factors as speed, test-taking strategies, and attempts by students to present themselves favorably in their responses.  5. Interpret scores taking into account the effects of such factors as speed, test-taking strategies, and attempts by students to present themselves favorably in their responses. 
    6. Warn against using published norms when the prescribed assessment method has been modified in any way.  6. Interpret scores taking account of major differences between the norm group(s) or comparison group(s) and the students being assessed. Also take account of discrepancies between recommended and actual procedures and differences in familiarity with the assessment method between the norm group(s) and the students being assessed. 

    Examine the need for local norms, and, if called for, develop these norms.

    7. Describe how passing and cut-off scores, where used, were set and provide evidence regarding rates of misclassification.  7. Explain how passing or cut-off scores were set and discuss the appropriateness of these scores in terms of rates of misclassification. 
     
    Examine the need for local passing or cut-off scores and, if called for, reset these scores.
    8. Provide evidence to support the use of any computer scoring or computer generated interpretations.  The documentation should include the rationale for such scoring and interpretations and their comparability with the results of scoring and interpretations made by qualified judges. 8. Ensure that any computer administration and computer interpretations of assessment results are accurate and appropriate for the intended use.  If necessary, ensure that relevant information not included in computer reports is also considered. 
    9. Observe jurisdictional policies regarding storage of and subsequent access to the results. Ensure that computer files are not accessible to unauthorized users. 
    10. Ensure that all copyright and user agreements are observed. 


     
    III.  Informing Students Being Assessed
     

    Direct communication with those being assessed may come from either the developer or the user of the assessment method. In either case, the students being assessed and, where applicable, their parents/guardians should be provided with complete information presented in an understandable way. 

    Developers or Users should:

    1. Develop materials and procedures for informing the students being assessed about the content of the assessment, types of question formats used, and appropriate strategies, if any, for responding.

    2. Obtain informed consent from students or, where applicable, their parents/guardians in the case of individual assessments to be used for identification or placement purposes.

    3. Provide students or, where applicable, their parents/guardians with information to help them decide whether to participate in the assessment when participation is optional.

    4. Provide information to students or, where applicable, their parents/guardians of alternate assessment methods where available and applicable.
     


    Control of results may rest with either the developer or user of the assessment method.  In either case, the following steps should be followed. 

    Developers and Users should:

    1. Provide students or, where applicable, their parents/guardians with information as to their rights to copies of instruments and completed answer forms, to reassessment, to rescoring, or to cancellation of scores and other records.

    2. Inform students or, where applicable, their parents/guardians of the length of time assessment results will be kept on file and of the circumstances under which the assessment results will be released and to whom.

    3. Describe the procedures that students or, where applicable, their parents/guardians may follow to register concerns about the assessment and endeavor to have problems resolved.
     

    IV.  Implementing Mandated Assessment Programs1
     
    1 The Joint Advisory Committee wishes to point out it has not taken a position on the value of mandated assessment and testing programs.  Rather, given the presence of these programs, the intent of the guidelines presented in Section Iv, when combined with applicable guidelines in the first three sections of Part B, is to help ensure fairness and equity for the students being assessed.
     

    Under some circumstances, the administration of an assessment method is required by law. In such cases, the following guidelines should be added to the applicable guidelines outlined in Sections I, II, and III of Part B. 

    Developers and Users should:

    1. Inform all persons with a stake in the assessment (administrators, teachers, students, parents/guardians) of the purpose(s) of the assessment, the uses to be made of the results, and who has access to the results.

    2. Design and describe procedures for developing or choosing the methods of assessment, selecting students where sampling is used, administering the assessment materials, and scoring and summarizing student responses.

    3. Interpret results in light of factors that might influence them. Important factors to consider include characteristics of the students, opportunity to learn, and comprehensiveness and representatives of the assessment method in terms of the learning outcomes to be reported on.

    4. Specify procedures for reporting, storing, controlling access to, and destroying results.

    5. Provide reports and explanations of results that can be readily understood by the intended audience(s). If necessary, employ multiple reports designed for different audiences.

    References

    Code of Fair Testing Practices for Education.  (1988).  Washington, DC:  Joint Committee on Testing Practices.

    Guidelines for Educational and Psychological Testing.  (1986).  Ottawa, ON:  Canadian Psychological Association.

    Standards for Teacher Competence in Educational Assessment of Students.  (1990).  Washington, DC:  American Federation of Teachers, National Council on Measurement in Education, and National Educational Association.
     



    The membership of the Working Group (WG) that developed the Principles for Fair Student Assessment Practices for Education in Canada and of the Joint Advisory Committee that oversaw the development was as follows:
     
    Marvin Betts 
    Gary Broker 
    Clement Dassa (WG) 
    Dick Dodds 
    Tom Dunn 
    Bob Gilchrist 
    Nicholas Head 
    Douglas Hodgkinson 
    Barbara Holmes (WG)
    Michael Jackson 
    Michel Laurier (WG) 
    Tom Maguire (WG) 
    Romulo Magsino 
    Linda McAlpine 
    Allan McDonald 
    Stirling McDowell 
    Craig Melvin 
    Kathy Oberle (WG) 
    Frank Oliva 
    Jean Pettifor 
    Sharon Robertson 
    Don Saklofske 
    Marvin Simner 
    Marielle Simon (WG) 
    Ross Traub (WG) 
    Sue Wagner 
    Kim Wolff 
    Todd Rogers (Chair, Working Group and Joint Advisory Committee)
     
    Back to Table of Contents 
    Appendix B
    A Guide to Establishing School Division
    Benchmarks and Standards
     
    “What is the role of the school system in establishing educational standards?”

    There are large, well organized programs to set standards at the provincial and national levels.  Provincial-level benchmarks and standards are set through Saskatchewan Education’s Provincial Learning Assessment Program (PLAP).  National-level benchmarks and standards are set through the Council of Ministers of Education, Canada, School Achievement Indicators Program (SAIP).

    School divisions may also wish to establish benchmarks and set standards.  A benchmark is a “snapshot” of student achievement at a particular level at a particular moment in time.  Benchmarks allow comparison of student achievement from one year to the next within the school division.  They also allow school divisions to compare their students’ achievement with provincial and national achievement.  Standards describe what student achievement should be in a particular skill area at a particular grade.  Comparing students’ achievement to the standard tells us whether students are achieving above or below expectations.
     
    Part I - Establishing Benchmarks

    Several Saskatchewan school divisions are establishing benchmarks at a variety of grade levels and subject areas.  The description of the benchmarking process that follows is adapted from one developed by Regina Public School Division for its Writing Benchmarks project.  However, the benchmarking process can be used for any subject area at any grade level(s).

    The benchmarking process produces detailed descriptions of student performance at several levels of achievement and allows school divisions to determine the percentage of students who achieve at various levels.

    The benchmarking process has five steps:

    1. Collect samples of students’ work;
    2. Determine the range of students’ work;
    3. Train teacher-scorers;
    4. Score students’ work; and,
    5. Report the results.

    Each of these steps in the benchmarking process is described in more detail below.

    Benchmarking - Step 1:  Collect samples of students’ work

    The nature of the work collected will vary with the subject area and the grade level.  It is important that the same math question, science experiment or writing prompt be used with all students participating in the benchmarking activity, so that student performance can be fairly assessed.

    Below is a description of the process that was used by Regina Public Schools to collect samples of students’ writing.
     

  • Give students a prompt that will enable them to write a response in one main mode or format (descriptive, narrative, expository, persuasive).  The purpose of the prompt is to ensure that all students are writing on the same theme, which makes for greater consistency, and also to ensure that work is not done ahead of time.  An example of a prompt is:

  •  
    Grade Four Prompt 
     
    This is the beginning of a story.  Read it.  Then continue writing the story and end it:  

    Alfred was looking out the window while his parents were preparing supper. A gust of wind started to shake the house, making the bedroom curtains flap.  Alfred began to shiver.  Suddenly, all the lights in the house went out… 

    Source: Nicholson, L. L.  (1989).  Test de rendement.  Immersion Journal, 13(1), 7-13. 

     
  • Give students an extended period of writing time, so that classes have the opportunity to approximate as closely as possible the usual procedures in writing class.
  • Collect other data if desired.  For example, a student survey can be used to collect information about the process that students used while writing.

  •  
    Sample Student Survey 

    Before you hand in your story, please answer the following questions: 
     
    Yes No
    Did you make a web or do any other kinds of planning before writing your story?
    Did you make any changes to your first draft other than for spelling and punctuation?
    Did you work with a partner to make changes or corrections to your story?
    Do you enjoy writing? 
    Source: Regina School Division #4 of Saskatchewan.  (1998).  Writing Benchmarks (English and French Immersion) in Regina Public Schools.  Regina, SK. 

     

    Benchmarking - Step 2:  Determine the range of students’ work

    Range finding is the process of developing descriptions of the characteristics of several levels of student work (rubrics) and then locating examples that illustrate student work at each level (exemplars).  The specific steps are:
     

  • Develop a rubric that describes the characteristics of student achievement at several levels; and,
  • Select exemplars – representative samples of student work (exemplars) at the various achievement levels.
  • Each of these steps is described in the sections that follow:

    Develop a Rubric

    The rubric describes the characteristics of student achievement at several (usually 4 or 5) achievement levels.  The rubric makes it possible for scorers to assess student work against objective descriptions of work at various levels, rather than comparing students to each other.  Rubrics for informational or persuasive writing, and for informational or narrative writing appear on the next two pages.
     
    Criteria (Rubric) for Informational or Persuasive Writing 
     
    Regina Board of Education Scoring Guide:  Informational Writing (adapted from the 1987 GED Testing Service Guide) 
    Papers will show some or all of the following characteristics: 
    Upper-half papers make clear a definite purpose, pursued with varying degrees of effectiveness.  They also have a structure that shows evidence of some deliberate planning.  The writer’s control of the conventions of standard written English (spelling, punctuation, grammar, word choice and sentence structure) ranges from fairly reliable at 4 to confident and accomplished at 6. 
    The 6 paper
    offers sophisticated ideas within an organizational framework that is clear and appropriate for the topic.  The supporting statements are particularly effective because of their sense of voice, substance, specificity, or illustrative quality.  The writing is vivid and precise, although it may contain an occasional flaw in the conventions of standard English. 
    The 5 paper is clearly organized with effective support for each of the writer’s major points.  While the writing offers substantive ideas, it lacks the fluency and clarity of voice found in the 6 paper.  Although there are some errors, the conventions of standard English are consistently under control. 
    The 4 paper shows evidence of the writer’s organizational plan, even when listing is evident.  Support, though adequate, tends to be less extensive or effective than that found in the 5 paper.  The writer generally observes the conventions of standard English.  The errors that are present are not severe enough to interfere significantly with the writer’s main purpose. 
    Lower-half papers either fail to convey a purpose sufficiently or lack one entirely.  Consequently, their structure ranges from rudimentary at 3, to random at 2, to absent at 1.  Control of the conventions of standard written English tends to follow this same gradient. 
    The 3 paper
    usually shows some evidence of planning, although the development may be insufficient  The supporting statements may be limited to a listing or a repetition of ideas, often without a consistent plan or with poor transitions.  The 3 paper often demonstrates repeated weaknesses in the conventions of standard English. 
    The 2 paper is characterized by a marked lack of organization or inadequate support for ideas.  The development may be superficial or unfocused.  Voice is confused or unclear.  Errors in the conventions of standard English may seriously interfere with the overall effectiveness of this paper. 
    The 1 paper lacks purpose or development.  The dominant feature is the absence of control of structure or the conventions of standard English.  Little effort has been invested in addressing the topic.  The deficiencies are so severe that the writer’s ideas are difficult or impossible to understand. 
    An asterisk code is reserved for papers that are blank, illegible, or written on a topic other than the one assigned.  Because these papers cannot be scored, a Writing Skills Test composite score cannot be reported. 
    Source: Pace, Sandra Falconer.  (1996).  Holistic marking and the development of system-wide standards for writing.  Query, 24(2).
     
     
    Criteria (Rubric) for Informational or Narrative Writing
     
    Regina Board of Education Scoring Guide:  Informational Writing (adapted from the 1987 GED Testing Service Guide) 

    Papers will show some or all of the following characteristics: 
    Upper-half papers make clear a definite purpose, pursued with varying degrees of effectiveness.  They also have a structure that shows evidence of some deliberate planning.  The writer’s control of the conventions of standard written English (spelling, punctuation, grammar, word choice and sentence structure) ranges from fairly reliable at 4 to confident and accomplished at 6. 
    The 6 paper
    offers sophisticated ideas within an organizational framework that is clear and appropriate for the genre of story.  Interesting and effective use is made of point of view, dialogue, or other narrative techniques.  Description is vivid and precise, creating a sense of voice, a mood or atmosphere.  Both language and storyline are fluent.  Particular conventions of standard English may not yet be mastered, but in general the use of conventions is correct.  In no case, however, would the few errors impede understanding. 
    The 5 paper is clearly organized with effective support for each of the writer’s major points.  The storyline is developed with substantive ideas, though it lacks the fluency and clarity of voice found in the 5 paper.  It uses dialogue and descriptive language much more extensively and effectively than a 4 paper does.  Although there are some errors, the conventions of standard English are consistently under control. 
    The 4 paper shows evidence of the writer’s organizational plan, even when listing of events is evident.  The plot line is adequate but mundane and lacks the extensive or effective support that is found in the 5 paper.  Dialogue, while present, is not incorporated into the descriptive language.  Simple modifiers such as adjectives and adverbs are used, and there may be simple embedding of subordinate clauses.  The writer generally observes the conventions of standard English.  The errors that are present are not severe enough to interfere significantly with the writer’s main purpose. 
    Lower-half papers either fail to convey a purpose sufficiently or lack one entirely.  Consequently, their structure ranges from rudimentary at 3, to random at 2, to absent at 1.  Control of the conventions of standard written English tends to follow this same gradient. 
    The 3 paper
    usually shows some evidence of planning, although the development may be insufficient  The storyline exists, but may be vague, and consists of a listing of events with little logical connections.  The connections between ideas are basic and strung together with then and so.  The 3 paper often demonstrates repeated weaknesses in the conventions of standard English. 
    The 2 paper is characterized by a marked lack of organization or inadequate support for ideas.  There may be a thread of a storyline, but not much more.  The development may be superficial or unfocused.  Voice is confused or unclear.  Errors in the conventions of standard English may seriously interfere with the overall effectiveness of this paper. 
    The 1 paper lacks purpose or development.  The storyline is absent, negligible or cannot be discerned.  The dominant feature is the absence of control of structure or the conventions of standard English.  The deficiencies are so severe that the writer’s ideas are difficult or impossible to understand. 
    An asterisk code is reserved for papers that are blank, illegible, or written on a topic other than the one assigned.  Because these papers cannot be scored, a Writing Skills Test composite score cannot be reported. 
    Source: Pace, Sandra Falconer.  (1996).  Holistic marking and the development of system-wide standards for writing.  Query, 24(2).

    Select Exemplars

    Using the rubric, expert scorers read a cross-section of the student’s writing samples and select representative writing (exemplars) at the various achievement levels.  Examples of exemplars are:
     

    Grade Four Prompt 

    This is the beginning of a story.  Read it.  Then continue writing the story and end it:  

    Alfred was looking out the window while his parents were preparing supper. A gust of wind started to shake the house, making the bedroom curtains flap.  Alfred began to shiver.  Suddenly, all the lights in the house went out… 

     
     
    Grade 4 Exemplar - Level 1  

    … The house was empty and everything was on the floor the t.v. was broken everything was was broken it was scary to nite the curtains flap aroand and aroand the trees brok and fall. 

    Comments  
    The mechanics of this piece are so weak that they interfere with the reader’s understanding of the piece.  No storyline is evident. 

     
     
    Grade 4 Exemplar - Level 2 

    … Alfred always opens the window in his room But he doesn’t know ghosts come in from The barn.  The floors in the barn were about to cavein.  About a month later hestarted to shud the windows. 
    So the ghost’s don’t come in anymore.  The ghosts’ that were in still the got vacumed up. 

    Comments  
    There is minimal evidence of a storyline, but mechanics continue to interfere seriously with the piece.  Vocabulary is simple and verb tenses are confused. 

     
     
    Grade 4 Exemplar - Level 3  

    … Because a gust of wind nocked over a power post.  I looked all over the house, but mom and dad were not home.  So Alfred want over to his frnds but nobody was home.  So Alfred went back hom.  And looked in the garage to see if the car was home.  It was.  Alfred sed to himself Mom and dad never leve without the car. The light came back on and my mom and dad came back to.  She was over at her frends house.  I sed to my mom.  “I was scared.” 

    Comments  
    This is a marginally unacceptable piece. There is some evidence of a storyline, but it is strung together as a sequence of events.  Mechanics are weak, but do not interfere seriously with the story.  Sentences tend to be similar in length and structure. 

     
     
    Grade 4 Exemplar - Level 4  

    … Alfred was frightened.  Alfred’s dad went to check the small power box outside.  Mean while Alfred’s mom tried to find our tiny brown lanturn.  Alfred’s dad was about to step in when he saw a gigantic twister!  They all worked quick to get everything ready.  Alfred’s mom said, “Hurry to the cellar.”  Alfred brought lots of fruit with him to eat.  When they were down there they ate and talked to each other.  Alfred was terribly horrified! 

    Soon all the food was gone.  Alfred’s dad and mom polked their heads through the large door.  “The twister was still going.” They said to Alfred.  After awhile Alfred got really bored, so he fell fast asleep!  Zzz.  In afew hours Alfred was woken up by his parents.  They whispered gently in his ears.  “The twister is finally over.”  All of them went up stairs.  Nothing was broken at all, exept the outside!  They decided to fix it up, when they when they got enough money.  Alfred and his folks were happy.  Then they all went out to find a job! 

    Comments  
    This is a marginally acceptable piece.  There is evidence of a story plan, which deteriorates toward the end of the piece.  Dialogue has been introduced, but without paragraphing and consistently correct punctuation.  Most spelling errors are developmentally appropriate.  Some description is evident in phrases like “ terribly horrified” and “Zzz”. 

     
     
    Grade 4 Exemplar - Level 5  

      … Alfred waited, then ran downstairs.  He called “Mom Dad!” But no reply.  Alfred thought they might be downstairs.  He walked down the stairs ever so quietly.  Creak Creak was the only noise.  Alfred tried to turn on the lights.  “SHOOT” He Whispered.  The lights were burnt out.  Alfred picked up a flashlight and clicked it on.  They were not down there. 

      Alfred decided to check the attic.  So Alfred ever so quietly walked up the stairs, through the kitchen, and up the attic ladder.  It was going nuts in the attic.  Music was playing, lights were flashing, and all the boxes were opened.  Suddenly a ghost bride appeared in front of Alfred.  He ran down the ladder, through the kitchen, out the back door and down the hill.  AAAAAAHHHH!  There was a ghost picking potatoes. 

      “Alfred!” 

      “Dad!” 

      “Here let me explain.  When the lights went out we went up to the attic to fix the lights.  But we found all this neat stuff.  Your mother has her old wedding gown and I am wereing this old coat.”  Alfred and his dad went into the house, had supper, and went to bed with a stomach full of potatoes. 

    Comments  
    This piece has well developed storyline that is nicely tied together at the conclusion. Conventions are generally observed.  Dialogue is used effectively and the writer’s voice clearly emerges through the use of descriptive language. 

     
     
    Grade 4 Exemplar - Level 6  
    The Quest 

      Fresh air flooded in through the open windows. 
      “Gee it’s cold,” said Alfred’s dad.  He went to close the windows. 
      “Where are the matches,” Alfred asked.  They looked and looked.  Alfred finally found a match. 
      “Children shouldn’t play with matches,” said Alfred. 
      “Just light the match,” said his mother.  Alfred lit the match on his jeans.  A small light glowed in the darkness. 
      Suddenly the lights came on. 
      “What a relief,” they all sighed. 

      Alfred’s parents were going out that night and they told him to take care of himself. 
      Alfred’s parents said he could have a sleepover if they obeyed one rule.  Don’t go in the attic.  The person Alfred wanted to have over was his best friend Tom. 
      Tom and Alfred had once ventured into the attic.  They were each four-year-olds and held teddy bears.  They had climbed the creaky old stairs to the attic.  At first they weren’t scared, but then they saw the thing they dreaded most in life.  Baby food.  Not just any baby food though, it was made with green beans and it had arms and legs and….and….and it was then they were so scared that they screamed, dropped their teddy bears and shot down the stairs. 

      Little did their parents know that they were planning to go to the attic that night.  They said their goodbys and left. 
      As soon as Alfred’s parents left that night Alfred and Tom ran up the attic stairs. 
      Alfred saw something round jutting out of the stair ahead of them. 
      “Oooh, look at that boobie trap like button,” said Alfred. 
      “I think I’ll activate it.” Alfred pushed the button.  5 knives shot out of the wall and pinned him on the oppisite wall. 
     “Help!!” said Alfred through tears. 
      “Your problem,” said Tom in a very cool voice. 
      Tom continued the rest of the way up the stairs while Alfred continued whining at the other end of the stairs.  Tom opened the attic door and there, in the doorway, was the babyfood. 
     The baby food jumped, exposing all its beanlike arms.  Tom ducked and the baby food fell past him and down the stairs.  There was a scream from Alfred, then all was silent.  Then Tom had an idea.  He ran down the stairs past the baby food and into Albert’s baby sister’s room.  She was fast asleep in her crib. Tom woke her up and then ran back to the attic stairs and showed her the baby food. 

      “Mmmmm!” she shouted.  The baby food tried to run but couldn’t.  There was a great slurp and the baby food was gone. 
      Tom set the baby down and went back to the attic, and there, in front of him was the reason he had come.  His Teddy Bear!!!!!! 

    Comments  
    This piece has a very well organized and well developed storyline.  An example of its effectiveness is found in the mention of the teddy bear early in the story, then the return to it at the end.  There is subtle humour in the reference to the boys’ fear of the baby food, yet the baby eats it.  There is effective use of vocabulary, sentence structure and characterizations.  Conventions are generally correct, but perfection is not expected at this level. 

    Source: Regina School Division #4 of Saskatchewan.  (1998).  Writing Benchmarks (English and French Immersion) in Regina Public Schools.  Regina, SK.

    Benchmarking - Step 3:  Train teacher-scorers

    Scorers are selected from those teachers who work at the grade level from which the writing sample was taken. They participate in a two-day training session.  At the training session, the rubrics are introduced and the exemplars are read together, so teacher-scorers can see where the exemplars fall on the rubric.  The teachers practice scoring on training pieces.  They compare the training pieces to the rubrics and the exemplars and do not compare one paper to another.

    The focus during training is on consistency, so that teachers’ scoring is consistent with the rubrics and exemplars every time, and consistent with each other’s scoring every time.

    Benchmarking - Step 4:  Score students’ work

    Teacher-scorers score the writing samples submitted by students.  They work in pairs and change partners every half-day.  Thus, two people score each paper.  When there is a difference between the two scorers’ marks for a paper, the head scorer helps them negotiate the score.  The head scorer also reviews a sample of the scored papers to make sure all scored papers are consistent with the rubrics and exemplars.

    Benchmarking - Step 5:  Report the Results

    The reporting process used by Regina Public Schools is as follows:
     

  • Assemble baseline data for student achievement the first time a writing sample is collected for a particular subject area or skill at a particular grade level.
  • Organize results according to school and classroom.  Provide this information, along with the rubrics and exemplars, to classroom teachers, who use it to improve instruction in their classroom.
  • Classroom teachers may share information about each child’s work with the child and his/her parents.  Displaying the child’s work next to the rubrics and exemplars helps children and their parents know what is required to reach a high level of achievement.
  • Data from a benchmarking activity can also be reported to the public.  For example, it may be appropriate to report to the public the percentage of students achieving at each level and to provide the public with examples of students’ work at each level.

    Advantages and disadvantages of benchmarking

    The benchmarking process has many advantages.  Benchmarking:
     

  • involves teachers in all stages of design and implementation.  Generally, when people have input into a process they have more commitment to it and more of a sense of ownership.
  • provides for consistency in scoring.  Past experience has shown that different teachers may score the same paper very differently.  Because the benchmarking process emphasizes scoring that is consistent with rubrics and exemplars and scoring that is consistent with other teachers, there is less variation.  This results in greater fairness to students and a more accurate picture of students’ performance.
  • provides useful information and materials for teaching and learning.  When teachers compare their students’ work to the rubrics and exemplars, they can see where they need to focus their instruction.  For example, if a teacher sees that her students’ work is strong on mechanics (spelling, punctuation) but contains limited, unoriginal ideas (or vice-versa), the teacher can adapt her lesson plans appropriately.  The rubrics and exemplars also can be shared with students and parents, so students know exactly what they have to do to achieve top levels of performance.
  • makes expectations explicit.  Traditionally, the teacher has mental criteria and expectations that s/he uses to mark students’ work, but the teacher often doesn’t explain these criteria to students.  Benchmarking is a more transparent process, because the criteria that are used to mark students’ work are available for all to see.
  • provides inservice for teachers.  Because benchmarking requires teachers to use new methods to mark students’ work and to discuss with colleagues their own beliefs relating to student performance, it is a valuable learning experience for teachers.
  • The chief disadvantage of benchmarking is cost.  Because it requires that teachers be released from the classroom to participate in developing rubrics and identifying exemplars, in training, and in scoring, there will be costs for substitute teachers.
     
    Part II - Setting Standards

    Establishing benchmarks for achievement for any specific subject or skill area will provide baseline data, so that future years’ achievement can be compared.  It will allow schools and school divisions to assess whether they are making progress in specific areas.

    While it isn’t necessary to set standards for achievement, some school divisions may choose to build upon the benchmarking process by doing so.  Setting standards is the process of specifying the percentage of students who are expected to reach each level of achievement.  For example:
     

    As part of the 1997 Provincial Learning Assessment in Mathematics, standards were set for achievement in performance-based mathematics tasks. 

    These included items such as calculating volume and surface area of wood blocks and calculating the ratio of the length, surface, area and volumes of these small blocks. 

    A five-level scale was used to describe student performance.  Level 5 represented the highest level of performance; Level 1 the lowest.  The percentage of students who were expected to achieve each level was: 

    Level 5 -   7%
    Level 4 - 18%
    Level 3 -  44%
    Level 2 -  23%
    Level 1 -    8%

    Source: Saskatchewan Education.  (1998).  1997 Provincial Learning Assessment in Mathematics.  Regina, SK. 

     
    The standard-setting process has three steps:

    1. Establish a standard-setting panel;
    2. Set the standard; and,
    3. Report the results.

    Standard-setting - Step 1:  Establish a standard-setting panel

    The ideal standard-setting panel includes about 50 percent teachers who are presently teaching at the grade levels and subject area for which standards are being set.  The other 50 percent of the panel can be made up of students, parents, educational administrators and community representatives.

    If you are setting standards for one subject area at one grade level, a panel of 10-12 people is appropriate.  If you are setting standards for several grade levels, establish a panel that is large enough so that it can split up into grade level panels of 9-12 people for each grade.
     
    Standard-setting - Step 2:  Set the standards

    The standard-setting process is as follows:
     

  • Panelists become familiar with the rubrics and exemplars used to benchmark students’ work.  These can be provided to panelists beforehand and briefly reviewed during standard-setting.
  • Panelists vote by secret ballot to specify the percentage of students who are expected to achieve each level of performance.
  • Votes are tabulated and results presented to the panel.  Panelists discuss the reasons why they voted as they did.
  • Panelists view comparison of the standards they set during their first vote with students’ actual performance and discuss differences.
  • Panelists vote again a second time by secret ballot, specifying the percentage of students who are expected to reach each level of achievement.
  • Second round votes are tabulated, results are presented to the panel, panelists discuss their votes and compare their expectations to students’ actual performance.
  • Panelists vote a third time, by secret ballot, specifying the percentage of students who are expected to reach each level of achievement.
  • The third vote becomes the standard for that subject or skill area at that grade level.
  • Standard-setting - Step 3:  Report the results

    Results can be reported in a variety of ways ranging from a formal indicators report, to a page in the school division’s annual report, to a page in the school newsletter.
     
    Advantages and disadvantages of setting standards

    It may be useful to build on benchmarking by setting standards in school divisions where there is public pressure and concern about standards.

    In other school divisions, setting standards may have disadvantages.  Standards replace the expectation for slow and steady progress with pressure to achieve to a specific level.  The cost of standard setting is also a factor, since substitute teachers will be required and travel and honorariums are usually paid to non-teachers.
     



    Note: The standard-setting procedures described above are adapted from the procedures used by Saskatchewan Education in the Provincial Learning Assessment Program (PLAP) and the Council of Ministers of Education, Canada in the School Achievement Indicators Program (SAIP).  For detailed technical information about standard setting refer to:

    Council of Ministers of Education, Canada.  (1999).  SAIP:  School achievement indicators program:  1998 report on reading and writing assessment.  Toronto, ON:  Author.  (URL unavailable at date of writing.)

    Council of Ministers of Education, Canada.  (1996).  SAIP:  School achievement indicators program:  1996 report on science assessment.  Toronto, ON:  Author.  (www.cmec.ca/saip/sci96/)

    Saskatchewan Education.  (1999).  1998 provincial learning assessment in English  language arts (listening and speaking).  Regina, SK:  Author.  (www.Sasked.gov.sk.ca/k/p_e/eval/plap/listening_speaking/1998pdf)

    Saskatchewan Education.  (1998).  1997 provincial learning assessment in mathematics.  Regina, SK:  Author.  (www.Sasked.gov.sk.ca/k/p_e/eval/plap/math/1997pdf)

    Saskatchewan Education.  (1997).  1996 provincial learning assessment in language arts (reading and writing).  Regina, SK:  Author.  (www.Sasked.gov.sk.ca/k/p_e/eval/plap/language/1996pdf)

    Saskatchewan Education.  (1996).  1995 provincial learning assessment in mathematics.  Regina, SK:  Author.  (www.Sasked.gov.sk.ca/k/p_e/eval/plap/math/1998pdf)

    Back to Table of Contents