Planning a new test to complement the LSAT

(the Law School Admissions Test)

Status: Summer 2001

"Criteria that will predict successful lawyering as opposed to successful law studenting" get the attention of Professor Marjorie Shultz at the UC Berkeley Law School. Along with a psychology professor, she has designed a project to determine how to test such criteria in law school applicants. The project begins in late summer 2001 by surveying students, faculty, and alumni of Berkeley's Law school on the skills and traits an attorney needs.

Law educators nationwide are interested in the work of Shultz's researchers. Schools of law have been facing declining admissions of minority applicants, as affirmative action programs are being phased out due to court decisions. Minority applicants generally get lower scores on the IQ-oriented Law School Admissions Test (LSAT). But to date, systems for evaluating non-LSAT skills have been both expensive and not as reliable as had been hoped.

Skills other than purely intellectual ones are a focus of additional questions that have recently been asked of some LSAT test takers. Applicants are invited to discuss their qualities of integrity, initiative, perseverance, compassion, generosity, and civility, among other questions. The Law School Admission Council hopes to determine if answers to such questions might result in more minorities being admitted. Under current admissions criteria, based largely on LSAT scores, minority law school applicants are at a disadvantage.

Problems with self-evaluation

However flawed the LSAT may be, it does have some major benefits. It generates a numerical score that may be reasonably be compared with the scores of other applicants. And it avoids the bias that comes with most psychological tests and questionnaires where an applicant is asked to do a self-evaluation. If all applicants exaggerated their capabilities to the same extent, it would be easy to compensate for self-bias. But they do not, and there are similar ranges of truthfulness in letters of recommendation or personal references.

Even in tests/ questionnaires of minor importance on oneself, psychologists know that self-bias is present in varying degrees. So in matters of great concern to an individual, as in gaining admission to the better law schools, self-bias tends to be substantial. It's no wonder that law schools stick to reliance on the LSAT, rather than on untested approaches so hard to evaluate and likely to be full of self-bias.

A proven way to eliminate self-bias

Over the past five years, a system that avoids self-bias has been developed and tested with 2500+ students. It measures social skills, but could easily be modified to measure integrity, compassion, and other non-academic traits considered useful in a good attorney. The system avoids self-bias because each individual's results are based solely on the private opinions of others. It is simple to execute and could readily be administered along with the LSAT in many settings.

In addition to the avoidance of self-bias, the new system can generate a numerical score on each trait for each individual. Thus results might be as readily compared as LSAT scores, even among widely dispersed applicants. Trait scores show a bell-shaped pattern similar to IQ scores. Usually results show little correlation between traits. For example, individuals scoring high in social skills (useful to a lawyer) do not necessarily score highly in intelligence as well. Among 2500 individuals surveyed, minorities tend to score at least as well as Whites in social skills - even in groups over 80% white! Minorities may score similarly well in the other traits that the Shultz team finds are desirable for lawyers. This result may enable law schools to increase the percentage of minority students in a fair, rational way, without having to use affirmative action measures.

The process of measuring non-academic skills

Professor Shultz has suggested multi-day projects in which a group of applicants would be assigned a specific task. They could be graded on how much they contributed to the group's success, as is often done for associates in a law firm. This approach may yield useful results, but it would be both time-consuming and expensive. And minor incidents, an illness, etc. may skew results in a way that would not give an accurate picture of an individual over the long term.

The approach described above for social skills avoids both these problems. Within just 20 minutes, a measure is typically recorded for social skills (or another trait) for EACH individual in a group of 30. And the grading is based on group members' longer-term knowledge of each other, not on just performance in a one-time, competitive, and likely stressful event.

One drawback to the above approach is that it does not work with relative strangers. But many applicants taking the LSAT at a college would normally know each other to some degree, often over several years. The system compensates for the natural tendency of individuals to give mainly good grades to others, by computing average scores on any trait of 100. Thus admissions officials could have the confidence that for each high score, there exists a comparable lower than average score for any trait. This rigorous score centering is often difficult to achieve in non-academic measurements, and may be one reason why LSAT scores remain prominent in law school admissions.

For more info:

Go to: Measuring Integrity and Other Traits like trustworthiness and reliability

Go to: New Exam for med students: Peer review

Go to: home page

Comments to:



Web (this website, 170+ pages on IQ and Success skills)

Web (this website, 170+ pages on IQ and Success skills)

The ads below are placed by - they are not necessarily endorsed by this site