Assessment's Next Wave: The Computerized Placement Tests

The movement of millions of students to online learning because of the pandemic upended how nearly everything at school was done: class learning, sure, but also test taking. Last spring, the College Board adapted the AP program to this new reality, creating the AP Live series of YouTube videos to help kids (and teachers) prepare for the AP Exams—which would be administered digitally. Naturally that got people wondering if the SAT would go digital, too.

Spoiler alert: there is no news being made in this introduction. Rather, seeing the question raised reminded us of an article we saw in an old issue of the College Board Review. In the mid-1980s, the personal computer was still primarily a creature of the university and the (expensive) domain of hobbyists. It would be another decade before it became a fixture in the home and classroom. But that didn’t stop some educators and researchers from thinking outside the bubble sheet and imagining a world of computerized placement tests.

In partnership and cooperation with other community colleges, IBM, and the College Board, Sante Fe Community College in Gainesville, Florida, “initiated a project to investigate the feasibility of using the Computerized Placement Tests (CPTs) for entry-level assessment and placement,” Pat Smittle writes in the Summer 1990 issue of the Review. In the article, Smittle, then the chairman of learning labs and developmental studies at Santa Fe Community College, presented an overview of the work, from how it was conceived to the challenges CPTs posed, to the opportunities they created. “We are in the midst of a technological revolution that is moving testing into a new era,” Smittle writes. “The computerized adaptive mode provides, for test administrators and test takers, an efficiency and effectiveness previously unavailable.”

It’s a fascinating read from a tech history point of view—not only from the standpoint of the research, but also the alternate reality it points toward: What if these had caught on? How different would generations of students’ experiences with assessment tests be? How different would such tests be? Our current moment of remote schooling gives it an additional resonance, adding layers of what-ifs and what’s-nexts, and demonstrating that our contemporary wrestling with the value and feasibility of digital assessment tests is not only not new, it’s as old as the PC revolution.

For the purposes of efficient advisement and appropriate placement in coursework, accurate assessment of basic skills is a major concern for most postsecondary institutions, especially those that have open admission policies that provide access for all students, without regard to their academic preparedness. At some point, this concern forces each of these institutions to examine the relationship between the various tests it uses and the institution's specific mission and student population.

In 1986, Santa Fe Community College in Gainesville, Florida, in cooperation with the College Board, IBM, and Miami-Dade Community College, initiated a project to investigate the feasibility of using the Computerized Placement Tests (CPTs) for entry-level assessment and placement. The project was sponsored by the League for Innovation in the Community College. Developed by the College Board, the CPTs constitute a new testing program that is based on the theory of computerized adaptive testing. Computerized adaptive testing is a method of testing that utilizes the capabilities of microcomputers to adapt examinations, by level of difficulty, to the abilities of individual students. Instead of facing a battery of prearranged test questions, many of which are either too hard or too easy, the computer quickly presents students with questions that are appropriate to their ability and knowledge.

The CPTs consist of four untimed tests that cover reading comprehension, sentence skills, arithmetic, and elementary algebra. A fifth test—college-level mathematics—is undergoing field trials and will be introduced this September. The issue of entry-level testing is a critical one for those colleges involved in the project because the Florida Community College system adheres to an open-admission policy and promotes access and excellence for all students. To assure academic excellence for the underprepared students who are accepted, Florida legislation mandates that every first-time-in-college student who is seeking a degree shall take an assessment test, register for basic skills courses if deficiencies are indicated, and demonstrate designated competencies before enrolling in most college-level courses.

Since more than 50 percent of the incoming freshmen at Santa Fe Community College are enrolled in at least one developmental course, our initial exploration of the CPTs focused on this area. To collect data and gain experience with testing procedures that were quite different from the traditional paper-and-pencil tests, the CPTs were administered to those students who had been placed in developmental reading, writing, and mathematics courses primarily on the basis of their ACT or SAT scores.

Soon after the project began, it became obvious that students liked the "new" test. Student evaluations of the CPTs indicated an overwhelming preference for the CPTs over paper-and-pencil tests; 63 percent of the students reported that they preferred the computer test. Students reported that they thought the computer test was very easy, even those whose skill deficiencies were still manifest.

It was this initial, enthusiastic reaction that piqued our interest and encouraged us to continue our exploration. The initial assumption was that this positive reaction was due to the novelty of using the computer. However, after approximately three years of exploration with the CPTs and observation of students who use computers on a regular basis, it has become apparent that satisfaction with the test is based on factors other than novelty. Among students taking basic skills courses, we wanted to explore what it was about them that might explain their positive reaction to the CPTs, as contrasted with their usual fearful reaction to tests.

Underprepared Students and CPTs
 

In our work, we found that several characteristics of underprepared students make them likely candidates for administration of a CPT. For one thing, the short attention span of many developmental students creates an automatic barrier to the traditional paper-and-pencil tests that are usually designed to assess a wide range of abilities. According to William C. Ward, senior development leader at Educational Testing Service, this "traditional" type of test usually consists of some easy and difficult items, with most of the items being at a moderate level of difficulty. It is a rigid structure that requires students to respond to many items at all degrees of difficulty to determine their level of achievement. The computerized adaptive test, however, is individualized, having a fluid structure that depends on a student's response to each question. This allows the level of achievement to be determined in a short time, using only a few items in the general ability range of each student.

Since the CPTs are adjusted for each student and since that student works only with questions in his ability range, the element of discouragement is diminished, allowing underprepared students to demonstrate what they know instead of what they do not know. A statistical procedure called Item Response Theory, which is one of the critical components of computerized adaptive testing, estimates the individual's ability based on the difficulty of the questions he can answer correctly. The procedure allows us to determine achievement levels, with items presented at whatever level of difficulty is appropriate for the student.

Developmental students perform optimally when given simple and specific instructions. The introduction and instructions for the CPTs are easy to read and very clear. After students enter personal information such as name and social security number, they are instructed to use only three different keystrokes throughout the entire test. Instructions are so clear that little additional information has been required by our test administrators, regardless of a particular student's level of familiarity with the computer.
 
Instructions further inform the student that the tests they are about to take are untimed, eliminating the pressure factor.

Although it is not always evident, many developmental students at the postsecondary level have low self-esteem, especially in relation to academics. But when students are seated before computers that are used to administer the CPTs, they infer that they are capable of working with the same high technology equipment that "successful" people use. And while feelings of distraction often characterize many developmental students, the computer demands a focused interaction from them that is generally forthcoming, perhaps because students know they have to focus only for a short period and they feel challenged by the presentation of questions at appropriate ability levels.

At the beginning of the test, students are informed that they will be given their scores immediately upon completion. As soon as the tests are completed, the computer calculates the scores and displays them on the screen. This immediate feedback is another important element in raising the self-esteem of developmental students.

The computer eliminates another problem particular to underprepared adult students: embarrassment. Though many students are self-conscious about their deficiencies in basic skills, the computer's immediate scoring capability removes the "human" element that students often view as judgmental.

Developmental students thrive on success. Regardless of the level of achievement, every student who completes the CPTs has been successful with high technology equipment. This successful experience, although small, provides an incentive to move to the next step in one's college career. All students need reinforcement, but it is critical for underprepared students who have experienced repeated failure in their academic work. Reinforcement is the "green light" that tells the student to keep going. The test-retest capabilities of the CPTs allow for frequent testing to reinforce the efforts of the students themselves. These capabilities also make the tests valuable measures of progress. The minimal cost of 25 cents for each retest makes the CPT an inexpensive motivator for students and a counseling and advisement tool for the instructor.

Considering the similarities between characteristics of developmental students and the attributes of the CPTs, it is not surprising that developmental students respond well to the tests. It should be noted that included in these common characteristics are the basic principles of individualized instruction that K. Patricia Cross identifies as being essential ingredients for effective learning. These individualized tests have all the necessary components to become a vital part of the entire instructional program for underprepared students.

Conclusion
 

The large numbers of underprepared students admitted to open-admission institutions demand that we actively pursue nontraditional methods of meeting the needs of our at-risk population. Given our limited resources in terms of time and money, it behooves us to strive for the most efficient and effective assessment and instructional materials available. Our objective is to admit these students, regardless of their level of academic preparedness, and remediate their deficiencies, thus preparing them for college-level courses that aspire to high academic standards. Obviously, the first step for success is an accurate assessment of their skills, so they can begin their work at the appropriate level.

At Santa Fe Community College, we have recently made another advancement in meeting the needs of underprepared students who are admitted to our college. We have adopted the CPT as our on-site assessment instrument. Although our students were major factors in our adoption of this particular test, it would be wrong to conclude that the CPTs are not appropriate for higher level students also. Generally, students with good skills perform well on most tests. (Our extended project included students at all educational levels, and we continue to see positive results from all of the students at all different levels.)

Our many years of experience with underprepared students demonstrate that those who are placed at the appropriate level of academic work upon their entrance to college can develop the skills necessary to earn a degree. Longitudinal studies reveal that approximately 30 percent of the graduates at Santa Fe Community College were placed in at least one developmental course at the beginning of their college career. This is significant when we acknowledge that, due to basic skill deficiencies, these students would not even have been admitted to some institutions that have selective admission policies.

In 1983, Howard Wainer, senior research scientist at Educational Testing Service, wrote, "We are on the verge of a technological revolution in testing as a result of the availability of extensive, inexpensive computing and some recent developments in statistical test theory." In 1990, I would like to expand on that thought: We are in the midst of a technological revolution that is moving testing into a new era. The computerized adaptive mode provides, for test administrators and test takers, an efficiency and effectiveness previously unavailable. The computerized placement tests are, indeed, assessment's "next wave."

Bibliography

Cross, K. Patricia. Accent on Learning. San Francisco: Jossey-Bass, 1976.

College Board Computerized Placement Tests:  Validation of an Adaptive Test of Basic Skills. Princeton, New Jersey: Educational Testing Service and the College Entrance Examination Board, 1986.

Computerized Adaptive Testing: The state of the art in assessment at three community colleges. Laguna Hills, California: League for Innovation in the Community College, 1988.

Wainer, Howard. "On Item Response Theory and Computerized Adaptive Tests: The Coming Technological Revolution in Testing." In The Journal of College Admissions. Vol. 28, No. 4., April 1983.

Ward, William C. "Using Microcomputers to Administer Tests." In Educational Measurement: Issues and Practices. Summer 1984, 16-20.