NCAAPMT ANNUAL BOARD MEETING

The Board of Directors of The North Carolina Association of Advanced Placement Mathematics Teachers held its annual meeting on August 7, 1999, at Lexington Senior High School in Lexington, North Carolina.

The following items were covered in the meeting. 

• Treasurer Jeff Lucia reported that NCAAPMT membership has increased to 369 including 136 from North Carolina, 227 from 38 other states, and 6 from outside of the United States.

• Sue W. Sams from Providence High School in Charlotte, North Carolina, was appointed to a second two-year term on the Board as a regional representative Judy Busick from New Hanover High School in Wilmington, North Carolina, completed a two-year term as a regional representative.

• There is currently a vacancy on the Board for a representative from the eastern region. Anyone interested in filling this vacancy should contact

NCAAPMT President
Carolyn Walmsley
2030 Huntington Woods Drive
Lexington, NC, 27295
(336) 243-7003
swalms4236@aol.com

• NCAAPMT had a 90-minute session at the NCCTM conference in Greensboro, North Carolina, on October 6, 1999.

• President-Elect Deborah Britt will make a presentation at the western region meeting of NCCTM at the University of North Carolina at Asheville on March 10, 2000.

• The next annual meeting of the Board will be held at Lexington Senior High School in Lexington, North Carolina, on June 24, 2000.

CHANGES IN THE EXAMINATION FORMAT IN 2000
Educational Testing Service

Beginning with the May 2000 Calculus AB and Calculus BC Examinations, Section II will consist of two parts. There will be no change to the format for Section I.

Part A of Section II will consist of three (3) free-response questions and will have a time limit of forty-five minutes. It will contain some questions or parts of questions for which a graphing calculator is required.

Part B of Section II will consist of three (3) free-response questions and will have a time limit of forty-five minutes. The use of a calculator will not be permitted to solve these problems. During the timed portion for Part B, students will be permitted to continue to work on questions in Part A, but they will not be permitted to use a calculator during this time.

The change in the format is an effort to respond to heightened concerns with equity as more students may use graphing calculators with computer algebra system (CAS) features. The AP Calculus Examinations are designed to accurately assess student mastery of both concepts and techniques of calculus. The two-part format for the free-response section provides greater flexibility in the types of questions that can be asked while ensuring fairness to all students taking the examination, regardless of the graphing calculator used.

Specific details of the new format can be found in the Acorn Book “May 2000, May 2001 Course Description. ”Details can also be found in the “What’s New”? section at the following Internet address.

http://www.collegeboard.org/ap/calculus/

The distribution of the grades for the 1998 examination candidates and the members of the current Test Development Committee are also listed at this address.

GENERAL INSTRUCTIONS FOR SECTION II FOR 2000 EXAMINATIONS

These are the instructions for the May 2000 AP Calculus Examinations.

Note: The problems will NOT appear in the Section II test booklet. Part A questions will be printed in the green insert only; Part B questions will be printed in a separate sealed blue insert. Each part of every problem will have a designated workspace in the test booklet. ALL WORK MUST BE SHOWN IN THE TEST BOOKLET. (For students taking the examination using an alternate form of the exam, the Part A questions will be printed in the test booklet only; the Part B questions will appear in a separate sealed insert.)

Part A: 45 minutes, 3 problems

A graphing calculator is required for some problems or parts of problems

During the timed portion for Part A, you may work only on the problems in Part A. The Part A problems are printed in the green insert only. When you are told to begin, open your booklet, carefully tear out the green insert, and write your solution to each problem in the space provided for that problem in the pink test booklet.

On Part A, you are permitted to use your calculator to solve an equation, find the derivative of a function at a point, or calculate the value of a definite integral. However, you must clearly indicate the setup of your problem, namely the equation, function, or integral you are using. If you use other built-in features or programs, you must show the mathematical steps necessary to produce your results.

Part B: 45 minutes, 3 problems

No calculator is allowed for these problems.

The problems for Part B are printed in the blue insert only. When you are told to begin, open the blue insert, and write your solution to each problem in the space provided for that problem in the pink test booklet. During the timed portion for Part B, you may continue to work on the problems in Part A without the use of any calculator.

GENERAL INSTRUCTIONS FOR SECTION II PART A AND SECTION II PART B

For each part of Section II, you may wish to look over the problems before starting to work on them, since it is not expected that everyone will be able to complete all parts of all problems. All problems are given equal weight, but the parts of a particular problem are not necessarily given equal weight.

• YOU SHOULD WRITE ALL WORK FOR EACH PART OF EACH PROBLEM IN THE SPACE PROVIDED FOR THAT PART IN THE PINK TEST BOOKLET. Be sure to write clearly and legibly. If you make an error, you may save time by crossing it out rather than trying to erase it. Erased or crossed-out work will not be graded.

• Show all your work. You will be graded on the correctness and completeness of your methods as well as the accuracy of your final answers. Correct answers without supporting work may not receive credit.

• Justifications require that you give mathematical (non-calculator) reasons and that you clearly identify functions, graphs, tables, or other objects you use.

• Unless otherwise specified, answers (numeric or algebraic) need not be simplified. if your answer is given as a decimal approximation, it should be correct to three places after the decimal point.

• Your work must be expressed in standard mathematical notation rather than calculator syntax. For example, may not be written as .

• Unless otherwise specified, the domain of a function is assumed to be the set of all real numbers for which is a real number.

OFFICIAL RESULTS FOR THE 1999 AP CALCULUS EXAMINATIONS
Calculus AB

Mean, standard deviation, and percent of responses with a score of 9:

Question 1: 4.58 (2.60); 7.2% 9's
Question 2: 4.37 (2.69); 7.4% 9's
Question 3: 3.52 (2.87); 3.2% 9's
Question 4: 2.92 (2.72); 2.5% 9's
Question 5: 1.54 (2.14); 1.2% 9's
Question 6: 1.99 (2.57); 2.6% 9's

Note: Free-response questions 2, 3, and 5 were on both the AB & BC examinations

Distribution of grades:

Number of candidates: 125,705
Percent at 5: 16.1
Percent at 4: 22.7
Percent at 3: 24.9
Percent at 2: 19.1
Percent at 1: 17.2
Mean Grade: 3.02

OFFICIAL RESULTS FOR THE 1999 AP CALCULUS EXAMINATIONS
Calculus BC

Mean, standard deviation, and percent of responses with a score of 9:

Question 1: 4.88 (2.53); 3.3% 9's
Question 2: 5.65 (2.41); 15.6% 9's
Question 3: 4.99 (2.80); 8.2% 9's
Question 4: 3.25 (2.12); 0.1% 9's
Question 5: 3.26 (2.77); 5% 9's
Question 6-AB part-parts a/c (5 points): 3.17 (1.69); 34.6% 5's
Question 6-BC only part (4 points)-part b: 1.19 (1.24); 1.9% 4's

Distribution of grades:

Number of candidates: 30,451
Percent at 5: 38.5
Percent at 4: 18.4
Percent at 3: 22.5
Percent at 2: 9.2
Percent at 1: 11.5

Mean Grade: 3.63

OFFICIAL RESULTS FOR THE 1999 AP CALCULUS EXAMINATIONS
Calculus AB Subscore Grade On Calculus BC Exam

The sub score grade is based on performance on 26 multiple-choice questions on AB topics
and free-response questions 2, 3, 5, and 6 (parts a and c)

Distribution of grades:

Percent at 5: 43.1
Percent at 4: 27.3
Percent at 3: 17.7
Percent at 2: 8.7
Percent at 1: 3.3
Mean Grade: 3.98

DECIMAL APPROXIMATIONS ON THE AP EXAMINATIONS 
Earl Mitchelle - The Asheville School - Asheville, North Carolina

Grading the advanced placement examinations for the past six years has impressed upon me the extreme importance of determining decimal approximations correctly. The instructions for the examinations state that “if your answer is given as a decimal approximation, then the answer should be correct to three places after the decimal.” This continues to be a problem for many students. Many times a student will do correct mathematics for some very challenging problem only to lose an answer point because he makes an error determining a decimal answer correct to three (3) decimal places.

As an example, in the past, if the answer to a question was p, and the student left the answer as the symbol p, the student got full credit for the answer. If the student ex-pressed the answer as 3.141 (truncated) or 3.142 (rounded off), the student got full credit for the answer. The student also got full credit for the answer 3.14159... because the answer was correct to three decimal places even though there are more than three decimal places given in the answer. The first three decimal places are correct and what comes after these three places is not relevant.

Unless there is a compelling reason to do so such as in the multiple choice section, students do not need to give numerical approximations to exact symbols such as , , , , . . These exact forms are perfectly acceptable.

A final point that should be mentioned is that of premature rounding. As an example, suppose you need to compute the volume of a sphere of radius . This volume is 11.848 rounded to three (3) decimal places. However, if you use the intermediate value , you will compute the volume to be 11.842 which is not correct to three (3) decimal places.

WHO GRADES THE AP EXAMINATIONS?

The grading of the advanced placement examinations’ free-response questions is a very structured and exact process to insure that every problem on every student’s paper is graded with the same consistency and fairness as every other student’s paper. The grading personnel hierarchy is in five levels. From top to bottom, these levels are chief reader, examination leader, question leader, table leader, and reader. In this issue of the NCAAPMT newsletter, the roles of these positions are explained by people who have held these positions. These include Bernie Madison, retired chief reader; Steve Davis, 1999 AB 6 question leader; Jeff Lucia, table leader; and Earl Mitchelle, reader.

THE CHIEF READER’S ROLE
Bernard L. Madison - University of Arkansas - Fayetteville, Arkansas
AP Calculus Chief Reader, 1995-99

I have been a college professor, department chair, dean, husband and father, but the four-year term as AP Calculus Chief Reader easily charged me with more absolute authority than ever before. With that heavy responsibility came some of the most frightful possibilities for failure. Nonetheless, my work as Chief Reader was one of the most rewarding and pleasant experiences of my professional life.

Basically, the Chief Reader (CR) is charged with overseeing the grading process for AB and BC calculus: making sure the examinations (exams) are valid and effective evaluation instruments; organizing and directing the scoring of the free-response examination (exam) section; and setting the cut scores for the five AP grades. During my four years as CR, these duties required 60-65 days per year, mostly on weekends.

A few years ago, Educational Testing Service (ETS), which administers the Advanced Placement (AP) Examinations, changed the title of Chief Readers to Chief Faculty Consultants. Jokingly, but with a germ of truth, I said that the title was changed because Chief Reader was far too optimistic, meaning that the Chief Reader actually read very few examinations. Of course, at the same time, ETS changed the title of Readers to Faculty Consultants, but the same joke did not hold.

My principal mentor for the CR position was Ray Cannon (Baylor University) who preceded me. Ray was excellent as CR and understood more about calculus than anyone I had ever worked with. I remember remarking to Ray in his last year that he had learned an incredible amount about being CR and now that information was going to be nearly useless since almost none of it applied to his normal duties back at Baylor.

The structure of the Reading and the duties of the CR are essentially passed on from one CR to the next, and almost none of the procedures were recorded until the increasing number of exams and Readers forced us to codify some of the organization and structure. One modification that proved very successful was formalizing the position of Question Leader (QL) to provide a person with principal responsibility for the standard and scoring of each of the 9 or 10 free response questions (6 each on AB and BC with 2 or 3 common to both). Since the Reading had grown so large, it was no longer possible for the CR to manage the logistics of the Reading and answer all the questions that arose in the scoring of questions. Responsibility had to be distributed and is now divided among three Examination Leaders (one each for the AB exam, the BC exam, and the alternate exams), and the nine QL's.

As best I could, I passed on what I knew to Larry Riddle (Agnes Scott College) who succeeded me. Larry will direct his first Reading in June 2000. One nightmare of the Chief Reader is that he or she will not identify someone to be the next CR, and I was much relieved when Larry and ETS agreed that he would succeed me. Like Ray, Larry will be excellent.

I served as a Reader, Table Leader (TL), and Examination Leader (EL) under four CR's: Ray Cannon, George Rosenstein (Franklin and Marshall College), Dick Prosl (College of William and Mary), and Pat Henry (Weber State University). All of them were inspirations to me and influenced my performance as CR. The constant awareness of how able they had been spurred me to work harder to uphold the tradition. The legacy of past CR's, the assistance of many very talented EL's, QL's, and TL's, plus great support from ETS staff make the work of the CR pleasant and rewarding.

Beginning in 1983, I participated in seventeen consecutive AP Calculus Readings, that is, the annual sessions in June when faculty from colleges and schools gather together to score the free-response section of the AP calculus exams. In 1983, I knew very little about the AP Calculus program and approached the Reading with reservations.   Scoring calculus exams for six days had little intrinsic appeal. What I learned was that the collegiality of the Reading had special appeal, so I continued.

Two nightmares stand out during my tenure as a Reader. One was being publicly derided by my TL's for misscoring a paper. Luckily, this never happened, and the more I learned about the Reading, the more I understood why this would never happen unless someone went over the edge. The second requires that I confess, as a younger college research mathematics professor, I had those mixed feelings about scoring calculus problems. Calculus was supposed to be easy for me, completely and instantly understood. Yet, questions for which I had no answers arose at the Reading, shattering my cocky self-confidence about this elementary subject.

Being a TL was my favorite position at the Reading. The term table is used by ETS to designate a group of 6-8 Readers. One TL is assigned supervisory responsibility for a table. The AP Calculus Reading is organized into rooms of 12-16 Readers in a room with two TL's assigned to each room. Therefore, a room equals two tables. TL's get to score questions, back read Readers, walk around, chat with Readers about their scoring, and pass the difficult questions on up to QL's and EL's and, eventually, the ultimate authority, the CR.

In 1983 at my first Reading, the total of AB and BC exams was 35,489. Seventeen years later when I retired, there were 155,932. I recall that there were about 120 Readers, including TL's, in 1983.   In 1999, there were 530. Having the responsibility for scoring the AP Calculus exams, the CR must choose the Readers to be invited (ETS staff do all the clerical work and record-keeping), make assignments as TL's and EL's, and assign Readers to rooms. In addition, the CR drafts scoring standards for each of the free-response questions and directs the meetings of EL's and QL's (one and one-half days in 1999) and the meeting of TL's (three days in 1999) to discuss and refine the scoring standards in preparation for presentation to the Readers and use in scoring. The CR has final approval for all scoring standards. When the Readers arrive on the eve of the Reading, I, as CR, had been on site for 5-6 days and had spent most of the previous two weeks drafting scoring standards and organizing 500 or so Readers into 30 - 36 Reading rooms.

There were several recurring nightmares for me around Reading time. One was losing or damaging the computer disk with all the assignments, organizational documents, and opening session notes. One we nervously joked about was that Readers wouldn’t show up. Actually, I always kept a wary eye on the weather on the day the Readers were to travel to the Reading. Storms at major airport hubs could, and did, delay arrivals, requiring that plans be adjusted.

Another nightmare was that some scoring standard would be flawed and would have to be withdrawn and rewritten.   To a limited extent, this would occur with some standards at most Readings. I will forever remember the pure fear of standing in front of 500 school and college mathematics teachers on that morning of the first day of the Reading preparing to present a scoring standard so that each of the teachers would go and score student responses in exactly the same way.   The possibilities for screwing up are endless, and the subsequent confusion and embarrassment enormous. The audience to be instructed includes ace AP calculus teachers, ever ready to protect and defend their students work, and hotshot young college professors, eager to catch you in some grievous mathematical misstep. Some 150,000 students await their AP calculus grades, and you must make this work, and work perfectly, finishing reading all 150,000 examinations in late morning of the seventh Reading day so the 500 plus Readers can board the line of busses to go to the airport for the trip home.

The opportunities for delays in the Reading are seemingly infinite, and many occurred over my seventeen years.   There is the question nearly impossible to read, like the trough problem on the 1987 AB exam. On such a question, rooms of Readers struggle and struggle, moving at a snail’s pace. And the CR worries and worries, looking at options. In 1998, we had such a problem, a BC question about motion of a particle along a curve. Eventually we had to put together a special SWAT Team room of experienced and fast Readers to bail out this question and get everyone to finish at the same time, on time.

We had a fire alarm in 1999 and evacuated the entire building of the Reading. I vividly remember standing outside the Colorado State University Union in the warm June sun wondering how long before the 530 Readers could return to work and how we might read books wet by sprinklers or charred by fire. The alarm was triggered by a (thankfully) small kitchen fire, and we were able to resume scoring within an hour.

At that same Fort Collins Reading, a Reader fell and was injured on the steps entering an auditorium for a briefing on a new question requiring an anxious wait for medical and police personnel to arrive and tend to the Reader. Nearly three hundred Readers waited and waited, and I worried and worried both for the health of the fallen Reader and the beginning of the briefing. In the end, the Reader received medical attention, but we postponed the briefing until the next morning since the day was nearly over.

A nightmare I never had even dreamed about threatened to upset many of my plans for my last Reading in 1999. Since the number of alternate exams has increased to a few thousand, we have had to devise a way to get them read. In earlier years with a different policy about who took the alternate exams, there were only a few alternates and the CR would read those. Now, mind you, the alternate exams, one for AB and one for BC, have six questions each including the three common questions, so that is nine more questions to write standards for and to score. When Ray Cannon was CR and I was an EL, Ray and I came up with the plan to have TL's read the alternate exams on the afternoon before the Reading began, requiring that the alternate exams be at the Reading site at least one day before the actual Reading started. This process had worked very well until 1999 when someone at ETS decided that the alternate exam booklets need not be at the Reading site that early. So, in 1999, we were all set to read the more than 2000 alternate exams using TL's, but the alternate exam books were on a truck somewhere between Princeton, NJ, and Fort Collins, CO. So we had to devise alternate plans for reading the alternate exams. It all worked out but not without many anxious hours for me, and, more to the point, a 7:00 a. m. start by a great group of TL's on the first day of the Reading.

In addition to directing the Reading, the CR meets with the Calculus Development Committee, usually for four four-day working meetings per year. This committee writes the AP course description, exams, and recommends program policy to ETS and the College Board. AP is a program of the College Board. ETS contracts with the College Board to write and score AP exams. While the Calculus Development Committee is formulating the free-response exam questions, the Committee and the CR essentially agree how the questions will be scored. Therefore when the CR drafts the scoring standards, much work and understanding has been invested into these drafts. Prior to becoming CR Designate in 1994, I had served for five years (1989-94) as a member of the Development Committee. Consequently, I had considerable experience with the process, and, by the time I retired as CR in 1999, I had met continuously with the Committee for a decade. The meetings themselves, over that ten years, constituted 160 days of writing and discussing calculus questions. Fortunately, each of those days was capped by a usually wonderful evening meal and great conversation.

During that decade, 1989-99, AP Calculus changed dramatically. Calculators were incorporated into both the course description and the examinations; a new reform-influenced course description was written; the Reading was extended from six to seven days; the scoring standards were changed from being handwritten to being set in text; the AB sub score was developed and implemented; the new Reading Management System using scanning sheets for Readers to record scores was implemented; a college comparability study was completed; and the Reading was moved from Clemson, where it had been for ten years,1989-98, to Colorado State University where no AP Reading had ever been held.   All these changes created extra work but also enormous satisfaction when they were successful.

As I look back and think more carefully, the absolute power of the CR makes a good story, but, in reality, the choices of options for the CR are usually well restricted by program specifications, by history, by best educational practices, and by the enormous wisdom and good will of the thousands of teachers and other professionals who personify AP Calculus. The CR has the wonderful opportunity to direct the culminating process where this good work and wisdom are transformed into grades for students.

The Reading itself is very special to the AP Calculus teachers who participate, and directing that Reading is automatically a cherished task.

While I served in the AP Calculus Reading, both my son and daughter took the AB examination. Many times, when faced with a decision as CR, I would remind myself that the exams were written by real students like my son and daughter. Fairness and consistency in upholding the high standards of AP were enhanced by this imagery.

THE QUESTION LEADER’S ROLE
Steve Davis - Davidson College - Davidson, North Carolina

It has been my pleasure to serve as a Question Leader (QL) at the AP Calculus reading for the last three years, most recently as QL for question AB6 at the 1999 reading at Colorado State University. At the 1999 reading we had about 550 readers, including 72 Table Leaders, 9 who double as QL’s, 3 Exam Leaders (AB, BC, and Alternate), the Chief Reader Designate, and the Chief Reader.

The process starts with a request from the Chief Reader in November or December to participate in the reading as a QL.   In May, after the test date, the QL receives a draft standard from the Chief Reader. The last two years this has been followed soon afterward by a collection of 30 or so sample student responses collected by our friends at ETS. Throughout the standard development process, the ability to survey actual student efforts on a question is a big help in focusing what-if speculation. For example, on 1999's AB6, I don't think we appreciated the extent to which students would refrain from simplifying the expression for "k in terms of w" (part b; answer: k = (3w/2) when faced with finding the rate of change of k with respect to time (given the rate of change of w with respect to time; see part c). Inspection of samples gave us a gauge for this reluctance and had some effect on the standard. The goal is to produce a standard that is fair in allocating partial credit to the students, is clear to the readers, and succinctly covers 99% of the ways that students will go wrong on a question. The standard is the product of a great deal of debate, but the debate should be resolved before its presentation to the readers.

The Chief Reader, Exam Leaders, and QL’s arrive at the reading a couple of days in advance of the Table Leaders to discuss the QL revision of the standard. A QL meets individually with the Exam Leader and the Chief Reader to discuss possible changes in the standard and presents the standard to the TL’s. The QL also selects a "question team" from the list of Table Leaders, two people designated as the "writer" and "collector" for the problem. More sample books are inspected, and the QL again reviews the preliminary standard with the Exam Leader and the Chief Reader in preparation for its presentation to the Table Leaders.

As the Table Leaders arrive, the QL finds the members of the question team and provides a short briefing on the standard-in-progress. The team provides important support and alternate perspectives for the QL. At the Table Leader meeting, the QL presents the standard, now in its adolescent phase, and walks through some sample student responses. Members of the question team record issues raised by the Table Leaders and document proposed revisions to the standard. Each Table Leader participates in the setting of standards for at least the four questions they will read, and per-haps the other two questions on the test they are reading. Table Leaders contribute an even greater variety of speculations on possible student responses, and each possible response must be considered as to how well the current version or a proposed revision of the standard handles it. At the end of the Table Leader briefing, the Table Leaders assist in a broader check through boxes of student books, looking for responses that may not "road map" by the standard, and generally getting a feel for how well the standard will weather its application by the readers. Some fine tuning of the standard is yet possible, sometimes to the surprise of the Table Leaders at the official briefing.

The afternoon that the readers arrive is usually a time that the question team can confer with the QL to polish the final standard and prepare for the briefing of the readers on the problem. Adjustments are made to the format of the printed standard to follow convention of other problems, and the final, ready-to-photocopy standard is carefully scrutinized and approved by the Exam Leader and Chief Reader prior to copying. The question team helps select and order the training samples; only those sample books on which students have checked the "Y" permission box are available for briefing purposes. The question team patiently helps the QL stumble through several rehearsals for the briefing. Their input is critical to a briefing that will sell the philosophy behind the standard as well as the details of the standard. The team also helps with the details of insuring that the standard and samples are copied and delivered to the briefing room on time.

The Chief Reader and Exam Leader keep the QL apprised of the progress of the reading, queuing the QL when the time for the briefing is nigh. Ideally, the briefing goes smoothly and all issues are resolved, but with 160,000 student books, something is bound to come up. The QL is usually busy answering questions from other Table Leaders in his or her own reading room, or checking on the progress of the question by surveying other reading rooms. The QL's Table Leader partner takes on an additional load in the reading room as the QL puts out the brush fires.  Fortunately, I've had fantastic partners!  Sometimes a special circumstance occurs with such frequency that an addendum to the standard called a “board note” is required.  As with the standard, a board note must be approved by the Exam Leader and Chief Reader before it is promulgated.

The question team monitors the progress of the reading of the question. The "collector" is responsible for selecting the official "typical" fair/good/excellent samples for ETS, and the "writer" surveys Table Leaders and develops a report on the question---common student errors, the impact of the calculator, and issues concerning the wording of the question and of the standard. In the end, the QL returns to the role of table leading and tries to make up for the grief to which his or her partner was subjected. The establishment of a scoring standard is truly a team effort involving many people: the Chief Reader, Exam Leader, QL, writer, and collector, and the QL's table partner.  During my years as a QL I have been blessed with superb teammates and I am grateful for their keen insights and good humor.

THE TABLE LEADER’S ROLE
Jeff Lucia - Providence Day School - Charlotte, North Carolina

I was first selected to be an AP Calculus reader in 1989, and though I knew some of the readers already, I knew little about the process. I arrived at Clemson University the night before the actual reading began, full of my own ideas about how certain problems would be a nightmare to grade, e.g.,1989 AB 6 - the “Oil Well Problem.” As I introduced myself to some of the hundreds of readers at the “get reacquainted” evening social gathering, I remarked that I thought that problem would be particularly tough to grade because of multiple teaching approaches to exponential growth problems.  Some people agreed; others hadn’t really thought about it. One man replied, “No, actually it’s reading very well right now. ”What I didn’t know was that he was a Table Leader and had already been at Clemson for a couple of days with Chief Reader George Rosenstein and the rest of the “TL’s,” as they are called.  They had been busy discussing each of the free-response problems, setting preliminary standards, grading sample stacks of books, and refining the standards they initially set.A good, clear standard can make an incredible difference in how smoothly a question reads, and though I would only find out as the week went on how wrong I was, this was my first exposure to what a Table Leader is responsible for.

Needless to say, I have learned a great deal more about the reading process in the ten years I have attended the reading.  Generally, a person may serve as a reader for six years, and at the closing ceremony at the end of the week, he or she will be “retired.” At that point, most readers will not return until after taking a hiatus for several years and reapplying. But a few, perhaps 10% or so, will be invited to return in the capacity of Table Leader. This invitation can extend to another six years, after which the real retirement occurs. Only the Chief Reader and the Exam Leaders continue after serving twelve years. Some of the veteran Table Leaders are asked to be Question Leaders, the ones responsible for guiding the evolution of each standard from its initial stages to the final version.

The Table Leaders arrive at the reading three days before the rest of the readers. The Chief Reader, Exam Leaders and Question Leaders leaders have already been there and set some proposed standards to grade each question. For those three days the Table Leaders discuss and debate each standard, collectively coming up with more things like “what-if” and “the student who does it this way will be unfairly penalized” than we thought possible. We do this together by reading several hundred student solutions to see what the typical and non-typical solutions are.  At the close of the discussion of each standard the Question Leader approves a final version -- he or she will then in turn “teach” the standard to the readers in a briefing prior to the grading of that question during the week.

Once the three days of Table Leader meetings are over, the debate about each standard is over as well.  Whether they agree with it or not, it is the reader’s job to learn each standard and to apply it fairly and consistently to every paper. Pairs of Table Leaders are assigned a room of twelve to fourteen readers. The standards are “road maps” of how to assign partial credit for incorrect or incomplete solutions, and the Table Leaders serve as local authorities for interpretation of how to apply the standard to a particular student’s paper. We answer questions, emphasize specific points discussed in the briefings and work towards assuring that a particular student's solution would be given the same score no matter which reader reads it.  One of the essential elements of this process is called “back reading,” which is an extension of the training of readers.  Table leaders read and score some of the same papers already read by one of the readers.  If the scores agree, it is a good indication that the reader understands and is consistently applying the standard.  If there are disagreements, the Table Leader “gently and privately” discusses the interpretation with the reader and they try to find the reason for the discrepancy.  Soon the standard becomes “locked in” for each reader and enough papers have been read so that most of the typical solutions have appeared many times.  At this point, the readers usually gain momentum and the question reads at a much faster rate.  The Table Leaders then just try to keep the room focused and on task, while still being available to discuss any unusual situation which may arise.

Most rooms of readers read four different questions out of the nine in the AB and BC free-response sections during the week of the reading. It gets easier for everyone to shift gears to a new problem as the week goes along.  By the last couple of days readers may be getting tired, but everyone senses that the end is near.  Table Leaders continue back reading in order to ensure that reading consistency is maintained. Scheduling is such that all rooms finish their last question at the same time, and everyone then gathers together for a final “debriefing.” After suppressing their strong opinions for a week in favor of correctly applying the official standards, the readers get their opportunity to speak back to the Question Leaders, Exam Leaders and the Chief Reader. They offer comments, suggestions, and/or criticism about each question which the leaders take under advisement as they participate in the process of producing future exams.

I have talked to many people who have gone on from being a Table Leader to the higher leadership levels, and most of them have said that the role they enjoyed the most at the reading was that of Table Leader.  I can certainly attest that it has been a rewarding experience for me, because I get to be part of the decision-making before the reading and because it is where I get to be a teacher. And I've been getting satisfaction from that my whole career.

THE READER'S ROLE
Earl Mitchelle - The Asheville School - Asheville, North Carolina

The roles of chief reader, question leader, and table leader for the grading of the free- response questions on the advanced placement calculus examinations have been described and explained elsewhere in this newsletter. The other major position in the reading (grading) process is that of reader. Although this position is at the bottom of the hierarchy of the grading process, the readers do the vast majority of the work.

I completed a six-year term as a reader of the advanced placement examinations in the summer of 1999. After being selected to be a reader, a person is permitted to serve in this position for a maximum of six years before having to rotate out and wait a period of four years before being allowed to apply again to be a reader.  However, after serving a six-year term as a reader, an individual may serve up to six years as a table leader without having to wait for four years.

The process for selecting readers uses a number of demographic criteria to insure that the group of readers is a representative sample of the general population. I waited three or four years after applying to be a reader to get an appointment.  I was appointed two days before the reading began in 1994 to fill a position that had become vacant just before the reading was to start.  I suspect that I received this late invitation to read at Clemson University because I live in Asheville, NC, which is less than 100 miles from Clemson. But regardless of why I got the call, I had made it to the "big dance," the Final Four of calculus.

The first day of reading begins with a presentation of the grading standard for the first problem that will be graded The chief reader, examination leaders, questions leaders, and table leaders have already determined how each problem will be graded and how the nine points for each free-response question will be allotted. The question leader for the particular question presents the grading standards and explains how to apply these standards. Although there was some hot debate about these standards during my first couple of years as a reader, it became apparent that the gods of calculus had determined what the standards were going to be and that only intercession by either Isaac Newton or Gottfried Leibniz would bring about any changes in the standards.

It is the job of the reader to develop a clear understanding of the standard to be used in grading a problem although the standard may not be the one that he would use to grade the problem in his course. During the six years I served as a reader, I gradually changed the grading rubrics for problems on my quizzes, tests, and examinations to reflect what I had learned about the grading process and standards as a reader. After the presentation and explanation of the standard for a particular problem, selected student responses for that question are graded for practice and clarification purposes.

At this point, the group of approximately six hundred readers is broken down into groups with each including ten to fifteen people. A group will be a cross-section of college and secondary teachers, first-year (Acorns) to sixth-year readers, males and females, etc. Now the grading process begins under the direction of two table leaders. A reader grades the same problem, in sets of twenty-five papers, for a period of time. The first set of papers graded by a particular grader is then back read, i. e., a table leader grades the same problem on the same set of papers. If there is any difference between the grade determined by the reader and the grade determined by the table leader for any student's particular response, the two discuss the difference and arrive at a mutually acceptable grade for that response.

A reader will probably be back read several more times during the grading of a particular problem. The ultimate goal of the grading process is to insure that the standard is accurately and consistently applied to every student response for the particular question. As might be expected, students do come up with solutions that cannot be graded using the standard that has been developed. In these cases, a parallel standard is developed which insures that consistent grading is maintained.

After grading a single problem for as long as a day and a half, readers are briefed on the standard for another problem, and the process repeats itself.  An individual reader grades four problems during the week of grading.  Both AB and BC readers grade the problems common to the two examinations. The remaining four problems on each examination are graded by AB and BC readers, respectively.  Half of the AB graders grade two of the remaining AB problems while the other half of the AB graders grade the other two remaining AB problems. The four remaining BC problems are graded in a similar fashion by the BC readers.

I can say, without reservation, that being a reader for six years was an extremely valuable professional experience.  I have learned about the very fine points of calculus in new ways. My contacts with other calculus teachers, both secondary and college level instructors, have been extremely valuable to me as a classroom teacher. The grades my students receive on the examinations have improved since I have been a reader. It should be pointed out that at no time do the college level teachers  "look down their noses" at the secondary level teachers. The readers are a homogeneous group and show respect for each other.

During the week of grading, there are other opportunities for professional growth. There is an open forum with members of the Test Development Committee when members of the Committee outline possible future changes in the examinations' questions. There is also a question-and-answer session when teachers/graders can ask questions about the future direction of the testing philosophy that will be followed by the Committee. There is a presentation by the College Board followed by a question-and-answer session. At the conclusion of the grading process, there is a debriefing when graders can ask questions about the free response questions that have just been graded. Graders criticize and commend the current year's questions, and make suggestions about how to improve the testing and grading process.

I have been asked what I do as a grader.  I tell the questioner that I grade the same problem over and over again for a day and half. Then she looks at me and asks me why I would do such a thing for seven straight days.  After I list the professional advantages of participating in the grading process, she usually wants to know if the process is boring. My reply is that grading the examinations is tiring but never boring.