Assessing Career Skills—A New Approach

Nearly 50 years ago, five state education agencies and the College Board got together to form the Career Education Consortium. Its remit was to determine how educators can ensure “that students have a chance to develop the skills they need/or career planning.” The result of their work over more than two years was the Career Skills Assessment Program. Arthur M. Kroll, then Director of Guidance Programs at Educational Testing Service, and Linda A. Pfister, then Program Planning Officer at the College Board, introduced the program and shared its goals in the Fall 1977 issue of the College Board Review.

Why we’re surfacing this piece now, in 2023, will be evident to anyone involved in career and technical education (CTE) efforts or career skills work generally, be it in high schools, community colleges or elsewhere. How to assess such skills—and how to translate results into meaningful, actionable recommendations for students and families—is central to the renewed focus on work, trades, and vocations. Looking back at how leaders and researchers approached this decades ago reveals that, well, everything old is new again.

It's impossible to ignore echoes of long-ago work in today’s contemporary conversations while reading through Kroll and Pfister’s article. How do states handle this work? How do they collaborate? What’s the role of a third-party institution like the College Board? What kinds of things should an assessment assess? What results will they bear? How will those results be given to students and educators, and what do they do with them? The Consortium had some novel ideas, from an exercise booklet, a machine/self-scorable response sheet, and a self-instructional guide created for six different target areas to “sound filmstrips” that “provide background and rationale for both staff and students as well as techniques for follow-up.”

But perhaps its most relevant learning is the role of standardization. “The Career Skills Assessment Program measures will not be standardized on a national population, because a national reference group is of limited value as a basis for interpreting performance in the area of career development,” Kroll and Pfister wrote. “Schools nationally vary considerably in the degree to which they emphasize mastery of career skills and knowledge. The most appropriate reference group for understanding an individual's career development is the group he or she most resembles educationally and demographically.”

Where today’s career skills work leads is still to be determined. But what’s not in doubt is that people still need jobs, still need guidance, still need resources to ensure opportunity and, more importantly, success. Like it was a half century ago, the College Board is in this conversation through BigFuture, Career Kickstart, and other initiatives. Revisiting its prior efforts—indeed, the prior efforts of anyone involved in this space—can be a useful guide in navigating the present and building the tools and systems of tomorrow.

It probably comes as no surprise to most educators that the words "career education" have diverse interpretations—depending on the educational setting. But a primary element that seems to be present in virtually all career education programs is a deep concern for the individual student—a concern for providing the learner with opportunities to acquire basic skills and knowledge essential for functioning in the world of work.

Students traditionally have received educational and vocational guidance through separate programs in the secondary schools; however, recent trends have been to incorporate career development activities into many other aspects of instructional programs. This approach to the curriculum has created a need for new tools to foster and assess student career competencies or "skills." Responding to these new needs, the College Board, in cooperation with five state education agencies, has developed a program to provide students and educators the materials and tools necessary to assess an individual's career preparedness.

The Career Skills Assessment Program
 

The new program, the Career Skills Assessment Program, is a product of the combined efforts of a Career Education Consortium composed of the state education agencies of Georgia, Maryland, Minnesota, New Jersey, Ohio, and the College Board. Initiated in February 1975, the consortium members agreed to begin development of new tools that would aid in providing individuals more information about their strengths and weaknesses in specific career development areas, while at the same time providing institutions with information regarding group performance in these same areas.

Prior to the agreement, meetings had been held with state and local educators in each of the consortium states in an effort to discover what kinds of instructional and guidance tools were lacking—both in terms of format and content. In addition, a national survey of secondary schools was completed to gain a broad picture of the needs for career education tools and programs.

During early meetings, it was agreed that the area of career development was too broad to measure in one instrument. In fact, consortium members strongly believed that career development consisted of specific skills that could be learned—and taught—so that individuals could be tested for competence. One of the first steps to be taken was to determine what skills were necessary for individuals to make and carry out their career plans effectively. Six areas were identified as essential: Self-Evaluation and Development Skills, Career Awareness Skills, Career Decision-Making Skills, Employment-Seeking Skills, Work Effectiveness Skills, and Personal Economics Skills.*

The Career Skills Assessment Program has been designed to serve both the guidance needs of students and the assessment needs of administrators and program planners. Through the program, students receive immediate and personal feedback about their career skills, and professional staff receive group information regarding student achievement in specific career development content areas.

The Career Skills Assessment Program can be used with learners in various settings. In the secondary school, the materials can be used with students in academic, vocational, work-study, or career education programs. Teachers can use it in the classroom; guidance counselors will find it useful in individual counseling and group guidance settings. The materials are equally appropriate for community college and adult continuing education students.

Included in each of the six content areas are an exercise booklet, a machine/self-scorable response sheet, and a self-instructional guide. Questions in the exercise booklets use diverse situations and problems, cartoons, facsimiles of forms, charts, and other graphics.

Responses to the multiple-choice items in the exercise booklets are made on a special three-ply response sheet. This combination machine-scorable/self-scorable form permits collection of responses for both central machine scoring and analysis—and immediate scoring and tabulation by the individual. This technique provides the flexibility needed for discussion, guidance, and self-assessment purposes—yet it also allows central scoring, analysis, and summary report generation for program evaluation purposes.

After scoring their own response sheets, each individual is provided with an accompanying self-instructional guide for each content area. Students will find a brief discussion of the skills involved in the area being addressed, sometimes including an interwoven scenario of a person engaged in activities that utilize these skills. Each guide also suggests activities, materials, and resources that the student might use to strengthen those skills, and concludes with brief statements of the rationale for the preferred response to each question in the exercise booklet.

As the program is designed to assess functional career skills, a detailed set of objectives and representative tasks has been developed for each of the six content areas. As a result, the Career Skills Assessment Program is curriculum- or instructionally-based, and each question in the various exercise booklets is directly related to a specific objective in a given content domain. This objectives-based feature of the program will aid in planning career guidance programs and career education instructional activities. The objectives and item linkages for each content area are presented in a handbook associated with the program.

Each of the content modules may be used separately, or any number of them may be used in an integrated program. Offered as a part of a comprehensive developmental career education program, the components together provide a motivational impetus for learning in several major career skill areas. Although the measures and related materials in the program were developed simultaneously and designed as an integrated series, the separate sets of materials for each of the six skill areas are not dependent upon each other. The materials for each skill area may be used independently on an as-needed basis.

The Career Skills Assessment Program materials are designed to be easy to use. Three accompanying sound filmstrips provide background and rationale for both staff and students as well as techniques for follow-up. Administration procedures require no special training or orientation and can be readily mastered by counselors, teachers, and other professional educational personnel. Paraprofessionals working as information specialists in career resource centers, with appropriate supervision, may also be involved in administration and use of the materials. In some settings, such as community colleges, students working as peer counselors, in conjunction with professional supervision and appropriate training, should be capable of supervising use of the Career Skills materials.

Several group summary reports will be available to users who request central computer scoring. These reports permit a combination of objectives-referenced and local norm-referenced interpretation. They provide an overall view of a designated group's performance on a given measure, as well as information about the performance of each individual in the group. No report has been designed for distribution to individual students, since the self-scoring feature of the response sheet, together with the item explanations in the guide, permits immediate feedback.

The Career Skills Assessment Program measures will not be standardized on a national population, because a national reference group is of limited value as a basis for interpreting performance in the area of career development. Schools nationally vary considerably in the degree to which they emphasize mastery of career skills and knowledge. The most appropriate reference group for understanding an individual's career development is the group he or she most resembles educationally and demographically. Thus, the reporting procedure makes provision for the production of local norm data.

Possible Uses of the Program
 

The Career Skills Assessment Program measures resemble achievement tests in that they measure skills and knowledge that are teachable, as opposed to assessing a hypothetical developmental construct like career maturity. The various uses of the materials include:

• Assessing a student's educational needs in the area of career skills and tracking his/her progress. Data collected provide a useful "benchmark" or beginning point against which future changes in skills and knowledge may be assessed.

• Assessing overall career program needs. The measures help reveal knowledge and skill areas where groups of students are proficient and deficient, and where program emphases might be strengthened to provide additional knowledge and skill-building experiences.

• Enhancing career counseling. Counselors may use any of the program materials for instructional or motivational purposes in individual or group counseling situations. Performance on the measures helps to provide a basis for discussion of specific career skills that may have been assessed only informally or subjectively. Counselors using the materials will have a stronger basis for suggesting activities and resources for skill building.

• Identifying bases for curriculum development and/or reemphasis. Strengths and deficiencies identified from student performance on the measures might be considered in the design of a school's curriculum materials and instructional emphases.

• Identifying bases for staff development. In-service training of guidance and instructional staff in career development concepts and methodology might be planned using information about student performance on one or more of the measures.

• Evaluating program effectiveness. The measures may be used as beginning- or end-of-program evaluation tools to help assess the effectiveness of specific curricula, instructional methodologies, or programs.

The Career Skills Assessment Program is a unique development in that it is designed to meet the needs of both individuals and institutions. Its structure is based on the belief that career development skills are not merely a result of maturation but are learned and are, therefore, teachable.
 

* These skill areas were determined after a thorough review of such existing documents as Objectives for Career and Occupational Development (National Assessment of Educational Progress, 1971), An Introduction to Career Education: A Policy Paper of the U.S. Office of Education (Hoyt, 1975), as well as the career development goals and objectives which were present in each of the consortium states. It was the intention of the consortium that any materials developed be supportive, but not redundant, of prior efforts in the field of career development. (See references below.)

References:

Bailey, Larry, and Stadt, Ronald. Career Education: New Approaches to Human Development. Bloomington, Illinois: McKnight, 1973.

Career Education: An Annotated Bibliography for Teachers and Curriculum Developers. Palo Alto: American Institutes for Research, 1973.

Developmental Program Goals for the Comprehensive Career Education Model (Preliminary Edition). Columbus, Ohio: The Center for Vocational and Technical Education, The Ohio State University, August 1972.

Dunn, James A., et al. Career Education: A Curriculum Design and Instructional Objectives Catalog. Palo Alto: American Institutes for Research, April 1973.

Hoyt, Kenneth B. An Introduction to Career Education: A Policy Paper of the U.S. Office of Education. Washington, D.C.: U.S. Government Printing Office, 1975.

Jesser, David L. Career Education: A Priority of the Chief State School Officers. Salt Lake City: Olympus, 1976.

Marland, Sidney P., Jr. Career Education, A Proposal for Reform. New York: McGraw-Hill, 1974.

Mitchell, Anita M., Jones, G. Brian, and Krumboltz, John D. A Social Learning Theory of Career Decision Making, Final Report. (Contract #NIE C-74-0134) Palo Alto: American Institutes for Research.

Objectives for Career and Occupational Development. Denver: National Assessment of Educational Progress, 1971.

Position Paper on Career Development. Washington, D.C.: National Vocational Guidance Association-American Vocational Association Commission on Career Guidance and Vocational Education, 1973.

Westbrook, Bert W. "Content Analysis of Six Career Development Tests." Measurement and Evaluation in Guidance, Vol. 7, No. 3, October 1974.