| Volume 8 |
Winter 2000 |
Issue No. 1 |
NCAAPMT ANNUAL
BOARD MEETING
The Board of Directors of The North
Carolina Association of Advanced Placement Mathematics Teachers held its annual meeting on
August 7, 1999, at Lexington Senior High School in Lexington, North Carolina.
The following items were covered in the
meeting.
Treasurer Jeff Lucia reported that
NCAAPMT membership has increased to 369 including 136 from North Carolina, 227 from 38
other states, and 6 from outside of the United States.
Sue W. Sams from Providence High School
in Charlotte, North Carolina, was appointed to a second two-year term on the Board as a
regional representative Judy Busick from New Hanover High School in Wilmington, North
Carolina, completed a two-year term as a regional representative.
There is currently a vacancy on the
Board for a representative from the eastern region. Anyone interested in filling this
vacancy should contact
NCAAPMT President
Carolyn Walmsley
2030 Huntington Woods Drive
Lexington, NC, 27295
(336) 243-7003
swalms4236@aol.com
NCAAPMT had a 90-minute session at the
NCCTM conference in Greensboro, North Carolina, on October 6, 1999.
President-Elect Deborah Britt will make
a presentation at the western region meeting of NCCTM at the University of North Carolina
at Asheville on March 10, 2000.
The next annual meeting of the Board
will be held at Lexington Senior High School in Lexington, North Carolina, on June 24,
2000.
CHANGES IN THE EXAMINATION FORMAT IN 2000
Educational Testing Service
Beginning with the May 2000 Calculus AB
and Calculus BC Examinations, Section II will consist of two parts. There will be no
change to the format for Section I.
Part A of Section II will consist of
three (3) free-response questions and will have a time limit of forty-five minutes. It
will contain some questions or parts of questions for which a graphing calculator is
required.
Part B of Section II will consist of
three (3) free-response questions and will have a time limit of forty-five minutes. The
use of a calculator will not be permitted to solve these problems. During the timed
portion for Part B, students will be permitted to continue to work on questions in Part A,
but they will not be permitted to use a calculator during this time.
The change in the format is an effort to
respond to heightened concerns with equity as more students may use graphing calculators
with computer algebra system (CAS) features. The AP Calculus Examinations are designed to
accurately assess student mastery of both concepts and techniques of calculus. The
two-part format for the free-response section provides greater flexibility in the types of
questions that can be asked while ensuring fairness to all students taking the
examination, regardless of the graphing calculator used.
Specific details of the new format can be
found in the Acorn Book “May 2000, May 2001 Course Description. ”Details can
also be found in the “What’s New”? section at the following Internet
address.
http://www.collegeboard.org/ap/calculus/
The distribution of the grades for the
1998 examination candidates and the members of the current Test Development Committee are
also listed at this address.
GENERAL INSTRUCTIONS FOR SECTION II FOR 2000
EXAMINATIONS
These are the instructions for the May
2000 AP Calculus Examinations.
Note: The problems will NOT appear in the
Section II test booklet. Part A questions will be printed in the green insert only; Part B
questions will be printed in a separate sealed blue insert. Each part of every problem
will have a designated workspace in the test booklet. ALL WORK MUST BE SHOWN IN THE TEST
BOOKLET. (For students taking the examination using an alternate form of the exam, the
Part A questions will be printed in the test booklet only; the Part B questions will
appear in a separate sealed insert.)
Part A: 45 minutes, 3 problems
A graphing calculator is required for
some problems or parts of problems
During the timed portion for Part A, you
may work only on the problems in Part A. The Part A problems are printed in the green
insert only. When you are told to begin, open your booklet, carefully tear out the green
insert, and write your solution to each problem in the space provided for that problem in
the pink test booklet.
On Part A, you are permitted to use your
calculator to solve an equation, find the derivative of a function at a point, or
calculate the value of a definite integral. However, you must clearly indicate the setup
of your problem, namely the equation, function, or integral you are using. If you use
other built-in features or programs, you must show the mathematical steps necessary to
produce your results.
Part B: 45 minutes, 3 problems
No calculator is allowed for these
problems.
The problems for Part B are printed in
the blue insert only. When you are told to begin, open the blue insert, and write your
solution to each problem in the space provided for that problem in the pink test booklet.
During the timed portion for Part B, you may continue to work on the problems in Part A
without the use of any calculator.
GENERAL INSTRUCTIONS FOR SECTION II PART A AND SECTION II
PART B
For each part of Section II, you may wish
to look over the problems before starting to work on them, since it is not expected that
everyone will be able to complete all parts of all problems. All problems are given equal
weight, but the parts of a particular problem are not necessarily given equal weight.
Show all your work. You will be graded
on the correctness and completeness of your methods as well as the accuracy of your final
answers. Correct answers without supporting work may not receive credit.
Justifications require that you give
mathematical (non-calculator) reasons and that you clearly identify functions, graphs,
tables, or other objects you use.
Unless otherwise specified, answers
(numeric or algebraic) need not be simplified. if your answer is given as a decimal
approximation, it should be correct to three places after the decimal point.
Your work must be expressed in standard
mathematical notation rather than calculator syntax. For example,  may not be written as  .
Unless otherwise specified, the domain
of a function  is assumed to be the set of
all real numbers  for which  is a real number.
OFFICIAL RESULTS FOR THE 1999 AP CALCULUS EXAMINATIONS
Calculus AB
Mean, standard deviation, and percent of responses with a
score of 9:
Question 1: 4.58 (2.60); 7.2% 9's
Question 2: 4.37 (2.69); 7.4% 9's
Question 3: 3.52 (2.87); 3.2% 9's
Question 4: 2.92 (2.72); 2.5% 9's
Question 5: 1.54 (2.14); 1.2% 9's
Question 6: 1.99 (2.57); 2.6% 9's
Note: Free-response questions 2, 3, and 5 were on both
the AB & BC examinations
Distribution of grades:
Number of candidates: 125,705
Percent at 5: 16.1
Percent at 4: 22.7
Percent at 3: 24.9
Percent at 2: 19.1
Percent at 1: 17.2
Mean Grade: 3.02
OFFICIAL RESULTS FOR THE 1999 AP CALCULUS EXAMINATIONS
Calculus BC
Mean, standard deviation, and percent of responses with a
score of 9:
Question 1: 4.88 (2.53); 3.3% 9's
Question 2: 5.65 (2.41); 15.6% 9's
Question 3: 4.99 (2.80); 8.2% 9's
Question 4: 3.25 (2.12); 0.1% 9's
Question 5: 3.26 (2.77); 5% 9's
Question 6-AB part-parts a/c (5 points): 3.17 (1.69); 34.6% 5's
Question 6-BC only part (4 points)-part b: 1.19 (1.24); 1.9% 4's
Distribution of grades:
Number of candidates: 30,451
Percent at 5: 38.5
Percent at 4: 18.4
Percent at 3: 22.5
Percent at 2: 9.2
Percent at 1: 11.5
Mean Grade: 3.63
OFFICIAL RESULTS FOR THE 1999 AP CALCULUS EXAMINATIONS
Calculus AB Subscore Grade On Calculus BC Exam
The sub score grade is based on performance on 26
multiple-choice questions on AB topics
and free-response questions 2, 3, 5, and 6 (parts a and c)
Distribution of grades:
Percent at 5: 43.1
Percent at 4: 27.3
Percent at 3: 17.7
Percent at 2: 8.7
Percent at 1: 3.3
Mean Grade: 3.98
DECIMAL APPROXIMATIONS ON THE AP EXAMINATIONS
Earl Mitchelle - The Asheville School - Asheville, North Carolina
Grading the advanced placement
examinations for the past six years has impressed upon me the extreme importance of
determining decimal approximations correctly. The instructions for the examinations state
that “if your answer is given as a decimal approximation, then the answer should be
correct to three places after the decimal.” This continues to be a problem for many
students. Many times a student will do correct mathematics for some very
challenging problem only to lose an answer point because he makes an error determining a
decimal answer correct to three (3) decimal places.
As an example, in the past, if the
answer to a question was p, and the student left the answer as the symbol p, the student
got full credit for the answer. If the student ex-pressed the answer as 3.141 (truncated)
or 3.142 (rounded off), the student got full credit for the answer. The student also got
full credit for the answer 3.14159... because the answer was correct to three decimal
places even though there are more than three decimal places given in the answer. The first
three decimal places are correct and what comes after these three places is not relevant.
Unless there is a compelling reason to do
so such as in the multiple choice section, students do not need to give numerical
approximations to exact symbols such as  ,  ,  ,  ,  . . These exact forms are
perfectly acceptable.
A final point that should be mentioned is
that of premature rounding. As an example, suppose you need to compute the volume of a
sphere of radius  . This
volume is 11.848 rounded to three (3) decimal places. However, if you use the intermediate
value  , you will compute the
volume to be 11.842 which is not correct to three (3) decimal places.
WHO GRADES THE AP EXAMINATIONS?
The grading of the advanced placement
examinations’ free-response questions is a very structured and exact process to
insure that every problem on every student’s paper is graded with the same
consistency and fairness as every other student’s paper. The grading personnel
hierarchy is in five levels. From top to bottom, these levels are chief reader,
examination leader, question leader, table leader, and reader. In this issue of the
NCAAPMT newsletter, the roles of these positions are explained by people who have held
these positions. These include Bernie Madison, retired chief reader; Steve Davis, 1999 AB
6 question leader; Jeff Lucia, table leader; and Earl Mitchelle, reader.
THE CHIEF READER’S ROLE
Bernard L. Madison - University of Arkansas - Fayetteville, Arkansas
AP Calculus Chief Reader, 1995-99
I have been a college professor,
department chair, dean, husband and father, but the four-year term as AP Calculus Chief
Reader easily charged me with more absolute authority than ever before. With that heavy
responsibility came some of the most frightful possibilities for failure. Nonetheless, my
work as Chief Reader was one of the most rewarding and pleasant experiences of my
professional life.
Basically, the Chief Reader (CR) is
charged with overseeing the grading process for AB and BC calculus: making sure the
examinations (exams) are valid and effective evaluation instruments; organizing and
directing the scoring of the free-response examination (exam) section; and setting the cut
scores for the five AP grades. During my four years as CR, these duties required 60-65
days per year, mostly on weekends.
A few years ago, Educational Testing
Service (ETS), which administers the Advanced Placement (AP) Examinations, changed the
title of Chief Readers to Chief Faculty Consultants. Jokingly, but with a germ of truth, I
said that the title was changed because Chief Reader was far too optimistic, meaning that
the Chief Reader actually read very few examinations. Of course, at the same time, ETS
changed the title of Readers to Faculty Consultants, but the same joke did not hold.
My principal mentor for the CR position
was Ray Cannon (Baylor University) who preceded me. Ray was excellent as CR and understood
more about calculus than anyone I had ever worked with. I remember remarking to Ray in his
last year that he had learned an incredible amount about being CR and now that information
was going to be nearly useless since almost none of it applied to his normal duties back
at Baylor.
The structure of the Reading and the
duties of the CR are essentially passed on from one CR to the next, and almost none of the
procedures were recorded until the increasing number of exams and Readers forced us to
codify some of the organization and structure. One modification that proved very
successful was formalizing the position of Question Leader (QL) to provide a person with
principal responsibility for the standard and scoring of each of the 9 or 10 free response
questions (6 each on AB and BC with 2 or 3 common to both). Since the Reading had grown so
large, it was no longer possible for the CR to manage the logistics of the Reading and
answer all the questions that arose in the scoring of questions. Responsibility had to be
distributed and is now divided among three Examination Leaders (one each for the AB exam,
the BC exam, and the alternate exams), and the nine QL's.
As best I could, I passed on what I knew
to Larry Riddle (Agnes Scott College) who succeeded me. Larry will direct his first
Reading in June 2000. One nightmare of the Chief Reader is that he or she will not
identify someone to be the next CR, and I was much relieved when Larry and ETS agreed that
he would succeed me. Like Ray, Larry will be excellent.
I served as a Reader, Table Leader (TL),
and Examination Leader (EL) under four CR's: Ray Cannon, George Rosenstein (Franklin and
Marshall College), Dick Prosl (College of William and Mary), and Pat Henry (Weber State
University). All of them were inspirations to me and influenced my performance as CR. The
constant awareness of how able they had been spurred me to work harder to uphold the
tradition. The legacy of past CR's, the assistance of many very talented EL's, QL's, and
TL's, plus great support from ETS staff make the work of the CR pleasant and rewarding.
Beginning in 1983, I participated in
seventeen consecutive AP Calculus Readings, that is, the annual sessions in June when
faculty from colleges and schools gather together to score the free-response section of
the AP calculus exams. In 1983, I knew very little about the AP Calculus program and
approached the Reading with reservations. Scoring calculus exams for six days had
little intrinsic appeal. What I learned was that the collegiality of the Reading had
special appeal, so I continued.
Two nightmares stand out during my tenure
as a Reader. One was being publicly derided by my TL's for misscoring a paper. Luckily,
this never happened, and the more I learned about the Reading, the more I understood why
this would never happen unless someone went over the edge. The second requires that I
confess, as a younger college research mathematics professor, I had those mixed feelings
about scoring calculus problems. Calculus was supposed to be easy for me, completely and
instantly understood. Yet, questions for which I had no answers arose at the Reading,
shattering my cocky self-confidence about this elementary subject.
Being a TL was my favorite position at
the Reading. The term table is used by ETS to designate a group of 6-8 Readers. One TL is
assigned supervisory responsibility for a table. The AP Calculus Reading is organized into
rooms of 12-16 Readers in a room with two TL's assigned to each room. Therefore, a room
equals two tables. TL's get to score questions, back read Readers, walk around, chat with
Readers about their scoring, and pass the difficult questions on up to QL's and EL's and,
eventually, the ultimate authority, the CR.
In 1983 at my first Reading, the total of
AB and BC exams was 35,489. Seventeen years later when I retired, there were 155,932. I
recall that there were about 120 Readers, including TL's, in 1983. In 1999, there
were 530. Having the responsibility for scoring the AP Calculus exams, the CR must choose
the Readers to be invited (ETS staff do all the clerical work and record-keeping), make
assignments as TL's and EL's, and assign Readers to rooms. In addition, the CR drafts
scoring standards for each of the free-response questions and directs the meetings of EL's
and QL's (one and one-half days in 1999) and the meeting of TL's (three days in 1999) to
discuss and refine the scoring standards in preparation for presentation to the Readers
and use in scoring. The CR has final approval for all scoring standards. When the Readers
arrive on the eve of the Reading, I, as CR, had been on site for 5-6 days and had spent
most of the previous two weeks drafting scoring standards and organizing 500 or so Readers
into 30 - 36 Reading rooms.
There were several recurring nightmares
for me around Reading time. One was losing or damaging the computer disk with all the
assignments, organizational documents, and opening session notes. One we nervously joked
about was that Readers wouldn’t show up. Actually, I always kept a wary eye on the
weather on the day the Readers were to travel to the Reading. Storms at major airport hubs
could, and did, delay arrivals, requiring that plans be adjusted.
Another nightmare was that some scoring
standard would be flawed and would have to be withdrawn and rewritten. To a limited
extent, this would occur with some standards at most Readings. I will forever remember the
pure fear of standing in front of 500 school and college mathematics teachers on that
morning of the first day of the Reading preparing to present a scoring standard so that
each of the teachers would go and score student responses in exactly the same way.
The possibilities for screwing up are endless, and the subsequent confusion and
embarrassment enormous. The audience to be instructed includes ace AP calculus teachers,
ever ready to protect and defend their students work, and hotshot young college
professors, eager to catch you in some grievous mathematical misstep. Some 150,000
students await their AP calculus grades, and you must make this work, and work perfectly,
finishing reading all 150,000 examinations in late morning of the seventh Reading day so
the 500 plus Readers can board the line of busses to go to the airport for the trip home.
The opportunities for delays in the
Reading are seemingly infinite, and many occurred over my seventeen years. There is
the question nearly impossible to read, like the trough problem on the 1987 AB exam. On
such a question, rooms of Readers struggle and struggle, moving at a snail’s pace.
And the CR worries and worries, looking at options. In 1998, we had such a problem, a BC
question about motion of a particle along a curve. Eventually we had to put together a
special SWAT Team room of experienced and fast Readers to bail out this question and get
everyone to finish at the same time, on time.
We had a fire alarm in 1999 and evacuated
the entire building of the Reading. I vividly remember standing outside the Colorado State
University Union in the warm June sun wondering how long before the 530 Readers could
return to work and how we might read books wet by sprinklers or charred by fire. The alarm
was triggered by a (thankfully) small kitchen fire, and we were able to resume scoring
within an hour.
At that same Fort Collins Reading, a
Reader fell and was injured on the steps entering an auditorium for a briefing on a new
question requiring an anxious wait for medical and police personnel to arrive and tend to
the Reader. Nearly three hundred Readers waited and waited, and I worried and worried both
for the health of the fallen Reader and the beginning of the briefing. In the end, the
Reader received medical attention, but we postponed the briefing until the next morning
since the day was nearly over.
A nightmare I never had even dreamed
about threatened to upset many of my plans for my last Reading in 1999. Since the number
of alternate exams has increased to a few thousand, we have had to devise a way to get
them read. In earlier years with a different policy about who took the alternate exams,
there were only a few alternates and the CR would read those. Now, mind you, the alternate
exams, one for AB and one for BC, have six questions each including the three common
questions, so that is nine more questions to write standards for and to score. When Ray
Cannon was CR and I was an EL, Ray and I came up with the plan to have TL's read the
alternate exams on the afternoon before the Reading began, requiring that the alternate
exams be at the Reading site at least one day before the actual Reading started. This
process had worked very well until 1999 when someone at ETS decided that the alternate
exam booklets need not be at the Reading site that early. So, in 1999, we were all set to
read the more than 2000 alternate exams using TL's, but the alternate exam books were on a
truck somewhere between Princeton, NJ, and Fort Collins, CO. So we had to devise alternate
plans for reading the alternate exams. It all worked out but not without many anxious
hours for me, and, more to the point, a 7:00 a. m. start by a great group of TL's on the
first day of the Reading.
In addition to directing the Reading, the
CR meets with the Calculus Development Committee, usually for four four-day working
meetings per year. This committee writes the AP course description, exams, and recommends
program policy to ETS and the College Board. AP is a program of the College Board. ETS
contracts with the College Board to write and score AP exams. While the Calculus
Development Committee is formulating the free-response exam questions, the Committee and
the CR essentially agree how the questions will be scored. Therefore when the CR drafts
the scoring standards, much work and understanding has been invested into these drafts.
Prior to becoming CR Designate in 1994, I had served for five years (1989-94) as a member
of the Development Committee. Consequently, I had considerable experience with the
process, and, by the time I retired as CR in 1999, I had met continuously with the
Committee for a decade. The meetings themselves, over that ten years, constituted 160 days
of writing and discussing calculus questions. Fortunately, each of those days was capped
by a usually wonderful evening meal and great conversation.
During that decade, 1989-99, AP Calculus
changed dramatically. Calculators were incorporated into both the course description and
the examinations; a new reform-influenced course description was written; the Reading was
extended from six to seven days; the scoring standards were changed from being handwritten
to being set in text; the AB sub score was developed and implemented; the new Reading
Management System using scanning sheets for Readers to record scores was implemented; a
college comparability study was completed; and the Reading was moved from Clemson, where
it had been for ten years,1989-98, to Colorado State University where no AP Reading had
ever been held. All these changes created extra work but also enormous satisfaction
when they were successful.
As I look back and think more carefully,
the absolute power of the CR makes a good story, but, in reality, the choices of options
for the CR are usually well restricted by program specifications, by history, by best
educational practices, and by the enormous wisdom and good will of the thousands of
teachers and other professionals who personify AP Calculus. The CR has the wonderful
opportunity to direct the culminating process where this good work and wisdom are
transformed into grades for students.
The Reading itself is very special to the
AP Calculus teachers who participate, and directing that Reading is automatically a
cherished task.
While I served in the AP Calculus
Reading, both my son and daughter took the AB examination. Many times, when faced with a
decision as CR, I would remind myself that the exams were written by real students like my
son and daughter. Fairness and consistency in upholding the high standards of AP were
enhanced by this imagery.
THE QUESTION LEADER’S ROLE
Steve Davis - Davidson College - Davidson, North Carolina
It has been my pleasure to serve as a
Question Leader (QL) at the AP Calculus reading for the last three years, most recently as
QL for question AB6 at the 1999 reading at Colorado State University. At the 1999 reading
we had about 550 readers, including 72 Table Leaders, 9 who double as QL’s, 3 Exam
Leaders (AB, BC, and Alternate), the Chief Reader Designate, and the Chief Reader.
The process starts with a request from
the Chief Reader in November or December to participate in the reading as a QL. In
May, after the test date, the QL receives a draft standard from the Chief Reader. The last
two years this has been followed soon afterward by a collection of 30 or so sample student
responses collected by our friends at ETS. Throughout the standard development process,
the ability to survey actual student efforts on a question is a big help in focusing
what-if speculation. For example, on 1999's AB6, I don't think we appreciated the extent
to which students would refrain from simplifying the expression for "k in terms of
w" (part b; answer: k = (3w/2) when faced with finding the rate of change of k with
respect to time (given the rate of change of w with respect to time; see part c).
Inspection of samples gave us a gauge for this reluctance and had some effect on the
standard. The goal is to produce a standard that is fair in allocating partial credit to
the students, is clear to the readers, and succinctly covers 99% of the ways that students
will go wrong on a question. The standard is the product of a great deal of debate, but
the debate should be resolved before its presentation to the readers.
The Chief Reader, Exam Leaders, and
QL’s arrive at the reading a couple of days in advance of the Table Leaders to
discuss the QL revision of the standard. A QL meets individually with the Exam Leader and
the Chief Reader to discuss possible changes in the standard and presents the standard to
the TL’s. The QL also selects a "question team" from the list of Table
Leaders, two people designated as the "writer" and "collector" for the
problem. More sample books are inspected, and the QL again reviews the preliminary
standard with the Exam Leader and the Chief Reader in preparation for its presentation to
the Table Leaders.
As the Table Leaders arrive, the QL finds
the members of the question team and provides a short briefing on the
standard-in-progress. The team provides important support and alternate perspectives for
the QL. At the Table Leader meeting, the QL presents the standard, now in its adolescent
phase, and walks through some sample student responses. Members of the question team
record issues raised by the Table Leaders and document proposed revisions to the standard.
Each Table Leader participates in the setting of standards for at least the four questions
they will read, and per-haps the other two questions on the test they are reading. Table
Leaders contribute an even greater variety of speculations on possible student responses,
and each possible response must be considered as to how well the current version or a
proposed revision of the standard handles it. At the end of the Table Leader briefing, the
Table Leaders assist in a broader check through boxes of student books, looking for
responses that may not "road map" by the standard, and generally getting a feel
for how well the standard will weather its application by the readers. Some fine tuning of
the standard is yet possible, sometimes to the surprise of the Table Leaders at the
official briefing.
The afternoon that the readers arrive is
usually a time that the question team can confer with the QL to polish the final standard
and prepare for the briefing of the readers on the problem. Adjustments are made to the
format of the printed standard to follow convention of other problems, and the final,
ready-to-photocopy standard is carefully scrutinized and approved by the Exam Leader and
Chief Reader prior to copying. The question team helps select and order the training
samples; only those sample books on which students have checked the "Y"
permission box are available for briefing purposes. The question team patiently helps the
QL stumble through several rehearsals for the briefing. Their input is critical to a
briefing that will sell the philosophy behind the standard as well as the details of the
standard. The team also helps with the details of insuring that the standard and samples
are copied and delivered to the briefing room on time.
The Chief Reader and Exam Leader keep the
QL apprised of the progress of the reading, queuing the QL when the time for the briefing
is nigh. Ideally, the briefing goes smoothly and all issues are resolved, but with 160,000
student books, something is bound to come up. The QL is usually busy answering questions
from other Table Leaders in his or her own reading room, or checking on the progress of
the question by surveying other reading rooms. The QL's Table Leader partner takes on an
additional load in the reading room as the QL puts out the brush fires. Fortunately,
I've had fantastic partners! Sometimes a special circumstance occurs with such
frequency that an addendum to the standard called a “board note” is
required. As with the standard, a board note must be approved by the Exam Leader and
Chief Reader before it is promulgated.
The question team monitors the progress
of the reading of the question. The "collector" is responsible for selecting the
official "typical" fair/good/excellent samples for ETS, and the
"writer" surveys Table Leaders and develops a report on the question---common
student errors, the impact of the calculator, and issues concerning the wording of the
question and of the standard. In the end, the QL returns to the role of table leading and
tries to make up for the grief to which his or her partner was subjected. The
establishment of a scoring standard is truly a team effort involving many people: the
Chief Reader, Exam Leader, QL, writer, and collector, and the QL's table partner.
During my years as a QL I have been blessed with superb teammates and I am grateful for
their keen insights and good humor.
THE TABLE LEADER’S ROLE
Jeff Lucia - Providence Day School - Charlotte, North Carolina
I was first selected to be an AP Calculus
reader in 1989, and though I knew some of the readers already, I knew little about the
process. I arrived at Clemson University the night before the actual reading began, full
of my own ideas about how certain problems would be a nightmare to grade, e.g.,1989 AB 6 -
the “Oil Well Problem.” As I introduced myself to some of the hundreds of
readers at the “get reacquainted” evening social gathering, I remarked that I
thought that problem would be particularly tough to grade because of multiple teaching
approaches to exponential growth problems. Some people agreed; others hadn’t
really thought about it. One man replied, “No, actually it’s reading very well
right now. ”What I didn’t know was that he was a Table Leader and had already
been at Clemson for a couple of days with Chief Reader George Rosenstein and the rest of
the “TL’s,” as they are called. They had been busy discussing each of
the free-response problems, setting preliminary standards, grading sample stacks of books,
and refining the standards they initially set.A good, clear standard can make an
incredible difference in how smoothly a question reads, and though I would only find out
as the week went on how wrong I was, this was my first exposure to what a Table Leader is
responsible for.
Needless to say, I have learned a great
deal more about the reading process in the ten years I have attended the reading.
Generally, a person may serve as a reader for six years, and at the closing ceremony at
the end of the week, he or she will be “retired.” At that point, most readers
will not return until after taking a hiatus for several years and reapplying. But a few,
perhaps 10% or so, will be invited to return in the capacity of Table Leader. This
invitation can extend to another six years, after which the real retirement occurs. Only
the Chief Reader and the Exam Leaders continue after serving twelve years. Some of the
veteran Table Leaders are asked to be Question Leaders, the ones responsible for guiding
the evolution of each standard from its initial stages to the final version.
The Table Leaders arrive at the reading
three days before the rest of the readers. The Chief Reader, Exam Leaders and Question
Leaders leaders have already been there and set some proposed standards to grade each
question. For those three days the Table Leaders discuss and debate each standard,
collectively coming up with more things like “what-if” and “the student who
does it this way will be unfairly penalized” than we thought possible. We do this
together by reading several hundred student solutions to see what the typical and
non-typical solutions are. At the close of the discussion of each standard the
Question Leader approves a final version -- he or she will then in turn “teach”
the standard to the readers in a briefing prior to the grading of that question during the
week.
Once the three days of Table Leader
meetings are over, the debate about each standard is over as well. Whether they
agree with it or not, it is the reader’s job to learn each standard and to apply it
fairly and consistently to every paper. Pairs of Table Leaders are assigned a room of
twelve to fourteen readers. The standards are “road maps” of how to assign
partial credit for incorrect or incomplete solutions, and the Table Leaders serve as local
authorities for interpretation of how to apply the standard to a particular student’s
paper. We answer questions, emphasize specific points discussed in the briefings and work
towards assuring that a particular student's solution would be given the same score no
matter which reader reads it. One of the essential elements of this process is
called “back reading,” which is an extension of the training of readers.
Table leaders read and score some of the same papers already read by one of the
readers. If the scores agree, it is a good indication that the reader understands
and is consistently applying the standard. If there are disagreements, the Table
Leader “gently and privately” discusses the interpretation with the reader and
they try to find the reason for the discrepancy. Soon the standard becomes
“locked in” for each reader and enough papers have been read so that most of the
typical solutions have appeared many times. At this point, the readers usually gain
momentum and the question reads at a much faster rate. The Table Leaders then just
try to keep the room focused and on task, while still being available to discuss any
unusual situation which may arise.
Most rooms of readers read four different
questions out of the nine in the AB and BC free-response sections during the week of the
reading. It gets easier for everyone to shift gears to a new problem as the week goes
along. By the last couple of days readers may be getting tired, but everyone senses
that the end is near. Table Leaders continue back reading in order to ensure that
reading consistency is maintained. Scheduling is such that all rooms finish their last
question at the same time, and everyone then gathers together for a final
“debriefing.” After suppressing their strong opinions for a week in favor of
correctly applying the official standards, the readers get their opportunity to speak back
to the Question Leaders, Exam Leaders and the Chief Reader. They offer comments,
suggestions, and/or criticism about each question which the leaders take under advisement
as they participate in the process of producing future exams.
I have talked to many people who have
gone on from being a Table Leader to the higher leadership levels, and most of them have
said that the role they enjoyed the most at the reading was that of Table Leader. I
can certainly attest that it has been a rewarding experience for me, because I get to be
part of the decision-making before the reading and because it is where I get to be a
teacher. And I've been getting satisfaction from that my whole career.
THE
READER'S ROLE
Earl Mitchelle - The Asheville School - Asheville, North Carolina
The roles of chief reader, question
leader, and table leader for the grading of the free- response questions on the advanced
placement calculus examinations have been described and explained elsewhere in this
newsletter. The other major position in the reading (grading) process is that of reader.
Although this position is at the bottom of the hierarchy of the grading process, the
readers do the vast majority of the work.
I completed a six-year term as a reader
of the advanced placement examinations in the summer of 1999. After being selected to be a
reader, a person is permitted to serve in this position for a maximum of six years before
having to rotate out and wait a period of four years before being allowed to apply again
to be a reader. However, after serving a six-year term as a reader, an individual
may serve up to six years as a table leader without having to wait for four years.
The process for selecting readers uses a
number of demographic criteria to insure that the group of readers is a representative
sample of the general population. I waited three or four years after applying to be a
reader to get an appointment. I was appointed two days before the reading began in
1994 to fill a position that had become vacant just before the reading was to start.
I suspect that I received this late invitation to read at Clemson University because I
live in Asheville, NC, which is less than 100 miles from Clemson. But regardless of why I
got the call, I had made it to the "big dance," the Final Four of calculus.
The first day of reading begins with a
presentation of the grading standard for the first problem that will be graded The chief
reader, examination leaders, questions leaders, and table leaders have already determined
how each problem will be graded and how the nine points for each free-response question
will be allotted. The question leader for the particular question presents the grading
standards and explains how to apply these standards. Although there was some hot debate
about these standards during my first couple of years as a reader, it became apparent that
the gods of calculus had determined what the standards were going to be and that only
intercession by either Isaac Newton or Gottfried Leibniz would bring about any changes in
the standards.
It is the job of the reader to develop a
clear understanding of the standard to be used in grading a problem although the standard
may not be the one that he would use to grade the problem in his course. During the six
years I served as a reader, I gradually changed the grading rubrics for problems on my
quizzes, tests, and examinations to reflect what I had learned about the grading process
and standards as a reader. After the presentation and explanation of the standard for a
particular problem, selected student responses for that question are graded for practice
and clarification purposes.
At this point, the group of approximately
six hundred readers is broken down into groups with each including ten to fifteen people.
A group will be a cross-section of college and secondary teachers, first-year (Acorns) to
sixth-year readers, males and females, etc. Now the grading process begins under the
direction of two table leaders. A reader grades the same problem, in sets of twenty-five
papers, for a period of time. The first set of papers graded by a particular grader is
then back read, i. e., a table leader grades the same problem on the same set of papers.
If there is any difference between the grade determined by the reader and the grade
determined by the table leader for any student's particular response, the two discuss the
difference and arrive at a mutually acceptable grade for that response.
A reader will probably be back read
several more times during the grading of a particular problem. The ultimate goal of the
grading process is to insure that the standard is accurately and consistently applied to
every student response for the particular question. As might be expected, students do come
up with solutions that cannot be graded using the standard that has been developed. In
these cases, a parallel standard is developed which insures that consistent grading is
maintained.
After grading a single problem for as
long as a day and a half, readers are briefed on the standard for another problem, and the
process repeats itself. An individual reader grades four problems during the week of
grading. Both AB and BC readers grade the problems common to the two examinations.
The remaining four problems on each examination are graded by AB and BC readers,
respectively. Half of the AB graders grade two of the remaining AB problems while
the other half of the AB graders grade the other two remaining AB problems. The four
remaining BC problems are graded in a similar fashion by the BC readers.
I can say, without reservation, that
being a reader for six years was an extremely valuable professional experience. I
have learned about the very fine points of calculus in new ways. My contacts with other
calculus teachers, both secondary and college level instructors, have been extremely
valuable to me as a classroom teacher. The grades my students receive on the examinations
have improved since I have been a reader. It should be pointed out that at no time do the
college level teachers "look down their noses" at the secondary level
teachers. The readers are a homogeneous group and show respect for each other.
During the week of grading, there are
other opportunities for professional growth. There is an open forum with members of the
Test Development Committee when members of the Committee outline possible future changes
in the examinations' questions. There is also a question-and-answer session when
teachers/graders can ask questions about the future direction of the testing philosophy
that will be followed by the Committee. There is a presentation by the College Board
followed by a question-and-answer session. At the conclusion of the grading process, there
is a debriefing when graders can ask questions about the free response questions that have
just been graded. Graders criticize and commend the current year's questions, and make
suggestions about how to improve the testing and grading process.
I have been asked what I do as a
grader. I tell the questioner that I grade the same problem over and over again
for a day and half. Then she looks at me and asks me why I would do such a thing for
seven straight days. After I list the professional advantages of participating in
the grading process, she usually wants to know if the process is boring. My reply is that
grading the examinations is tiring but never boring.
|
