|
General Policy
Faculty will be paired up
at the beginning of the semester. They will score each other's
class's portfolios at mid-term and end-of-term so there won't be the chaos
of figuring out who scores whose portfolios. Because the instructors
will know each other's classes better due to working together the entire
semester, the challenge of using an appropriate tone for the comments will
likely be ameliorated. The choice of pairs will reduce the problem
that may come due to difference in rank between instructors. To maintain
consistent standards the Director of Basic Skills (Ed) will collect from
each pair two portfolios, one of which the instructors will agree upon as
a Low Pass and the other a borderline High Pass. Ed (with the assistance
of the Director of First-Year Writing, Nancy Enright when needed) will
review these scores and discuss with the scorers if appropriate.
Pairing should be a mixture of pure choice and
director guidelines and input.
·
Two inexperienced instructors should not work together.
·
The optimal situation is where there is some difference of
experience but not novice with very experienced.
·
All portfolio assessment participants should decide on
partners by end of week two.
Mid-Term Norming:
There will be a one-hour norming for everyone on
Friday, Oct. 24, at noon. Afterwards, pairs Ed will collect 2 portfolios from
each portfolio pair. Criteria for these 2 portfolios are that one lie
between a High Pass and Pass and another be a Low Pass. Use scoring as
described below for norming. (This scoring doesn’t have to be followed
literally when all the portfolios are scored.) The pairs should write a
one-sentence comment on each of the 2 portfolios, directed to the
student. Finally, the instructors should write a very brief comment on
each to explain why they scored each portfolio as they did. Ed, with some
assistance from Nancy, scores the portfolios. Ed (and/or Nancy?)
discusses these scores with the pairs and reach consensus. After the
pairs have scored their respective portfolios, they submit a log of the
scores that they give their students and a copy of the one-sentence
comments given on the three lowest scoring portfolios.
Scoring:
At the norming session, we will assign points based
on each of the six criteria for portfolio assessment on the 1201 website,
in large part to internalize a common set of standards. 0-3 points will
be awarded each criterion but, at midterm, no points will be awarded to
the range criterion and 0-1 points to the improvement criterion. Total
possible will be 14 at midterm and 18 at the end. If we think of 0 as
Fail, 1 as Low Pass, 2 as Pass, and 3 as High Pass, then for the midterm
portfolio, 2 and below will be Fail, 3-7 will be Low Pass, 8-12 will be
Pass, and 13-15 will be High Pass. For the final portfolio, 3 and below
will be a Fail, 4-9 will be Low Pass, 10-15 will be Pass, and 16-18 will
be High Pass.
Scorers write brief, one-sentence comments on each
portfolio that capture the essence of their judgment. Comments, however
brief, must be constructive and must be addressed to the student.
The final score is determined by the instructor and
one other reader. In cases of disagreement, one or both of the directors
will decide.
Final norming:
Ed picks four portfolios from mid-term, 2 low and 2
high, and the instructors of the selected portfolios add the second essay,
but not the reflective piece, of the final portfolio. We discuss what
effect the essay has on the final portfolio grade. (The aim here is to
reduce the time that norming takes but to focus on the key new component:
the second essay.)
Faculty development component
Instructors write a brief observation of what they
notice in the portfolio. This may mean noticing one thing most of the
students seem to be understanding and one thing they seem to be missing.
The larger picture is that the pairing should encourage dialogue over the
course of the entire semester.
Impact of portfolio assessment on grades
Student must pass portfolio to pass 1201/0160, but as
long as the student passes, the score only affects a portion of the
grade. Usually instructors award it something like 10% of the final
course grade. In regular 1201 courses, the portfolio counts as the final
exam. Rationale behind the mandatory passing of the portfolio in the
linked course:
1. If
a student can’t produce 2 essays that are of passing quality, along with a
reflective piece that is of passing quality, s/he really isn’t ready for
1202.
2.
Students should not be able to pass a class with failing essay scores and
still pass the course. Participation grades and effort alone should not
result in passing grades.
3.
Although it seems unlikely, there may be exceptions to this rule that
would be considered by the directors on a case-by-case basis.
Time considerations:
What portfolio assessment replaces (per section of
1201-0160):
Final exam: 2.0 hours during finals and 2.0 to grade = 4.0 hours
Post-testing: 1.25 hours during last week of class
Total hours: 5.25
How much time portfolio assessment takes (per section of 1201-0160):
Midterm: 1.7 hours of norming and 2.5 hours of scoring = 4.2 hours
Final portfolio assessment: 1 hour of norming and 3.0 hours of scoring
= 4.0 hours
Total hours: 8.2
Thus portfolio assessment appears to take about 3 hours more per semester
than the regular system for 1201-0160 instructors. About 4 hours more for
1201 instructors.
What should be turned in to Ed:
At mid-term:
1. One
portfolio judged LP and one judged a borderline P/HP.
2. A
class list with portfolio assessment scores for each student
3. A
copy of comments on the three lowest scoring portfolios
At end of term:
1. A
class list with portfolio assessment scores for each student
2. A
copy of comments on the three lowest scoring portfolios (Ed and Nancy may
determine this isn’t necessary after the midterm.) |
