St. George’s Hospital Case
A racist and sexist computer system used for medical school admissions
Country: United Kingdom
Forum: Commission for Racial Equality
This case concerned a computer program that was used to score and rank applications to a medical school. This ranking was used to determine who would be invited to interview for a place at the school.
After an investigation into the program, the Commission for Racial Equality found its use to be discriminatory as it gave negative weightings to candidates based on race and gender. The program was later described by the British Medical Journal as a “blot on the profession.”
It is also worth noting that, at the time, there was significant evidence to suggest that there was racial and sexual discrimination in medical school admissions. The computer program itself was designed to reflect selections that had been made previously by the school’s selection panel.
The Commission found there to be discrimination under the law and made a number of recommendations to the medical school. This included recommendations to review admissions procedures to ensure selection procedures are not biased.
This was a report published by the Commission for Racial Equality, a public body in the UK aimed at addressing racial discrimination and promoting racial equality. The Commission for Racial Equality has since been replaced by the Equality and Human Rights Commission.
The report itself came about after the Commission became aware of research suggesting that racial and sexual discrimination was taking place when students were selected for medical education at London colleges. One of these colleges was St. George’s Hospital Medical School.
While conducting research into these allegations at St. George’s Hospital Medical School, one of the researchers discovered, by chance, that a computer program was being used by the school to make an initial selection of applicants for placement with the college.
The asmissions procedure
It is useful to know a bit more about how school admissions were generally handled in the UK at this time to understand the context in which this computer program was deployed. Medical school admissions followed the procedure outlined above.
Those wishing to attend medical school, or any university, had to submit an application form (called a UCCA form) that included academic and personal information about a candidate and their choices for where and what they wanted to study. These forms were completed and submitted before final school exams (called A-Levels) were completed.
At medical schools, like St. George’s, these application forms would be reviewed and, following this review, a number of applicants would be invited to interview. Following an interview, successful applicants would be offered a place at the school. However, these offers were contingent on the candidate obtaining certain results in their A-levels.
After candidates received their exam results, some vacancies would open up at the schools. Those candidates who had not been offered places by any school and who had obtained the requisite exam results were placed in a “clearing” system. The school can then select candidates from this pool and offer them a place.
At St. George’s, a computer program was used for the first step of this admissions procedure, i.e. the examination of application forms and the selection of candidates for interview.
The computer system
The computer program that was being used at St. George’s Hospital Medical School generated a ranking for each applicant, which was then used to determine whether that applicant would be invited for an interview. This program had been designed by a member of academic staff, who aimed to remove any “inconsistencies” in the selection of candidates.
In devising the program, he took account of “academic factors,” as well as a number of other factors that he deemed relevant in the selection of students. These factors were to be derived from information provided by candidates in their application forms.
He determined the factors to be used by the program, as well as the weightings to be given to these factors, based on his observations of selections made by the selection panel over a number of years. He, therefore, sought to mimic the decisions of the panel who had previously selected applicants for interview. These selections themselves had been suspected of being discriminatory.
While developing the program, he found a strong correlation (90-95%) between the outputs of the program and the choices made by the selection panel.
The academic factors that were considered included exam grades (actual and, for A-levels, expected) and subjects. These factors accounted for 75% of a final scoring given to applicants. Personal factors that were considered included age, hobbies, social and community work, handwriting, type of school, and parental occupation. They also included race and gender, despite the fact that college applications did not have entries for race on their form.
Race was defined as either “Caucasian” or “Non-Caucasian” and was inferred from the candidate’s name, place of birth, or “other indications.” The factors “Non-Caucasian” and female carried negative weightings in the program, lowering these applicants’ position in the ranking order for selection for interviews. This meant they were less likely to be called to interview. As a result, some candidates who would otherwise have been selected for interview were rejected.
Section 1(1)(a) of the Race Relations Act 1976
A person discriminates against another [according to this law] if on racial grounds he treats that other less favourably than he treats or would treat other persons.
Section 17(b) of the Race Relations Act 1976
It is unlawful, in relation to an educational establishment [as defined by this law] for a [person deemed a responsible body for that established by this law] to discriminate against a person […] by refusing or deliberately omitting to accept an application for his admission to the establishment as a pupil.
NOTE: this Act has been repealed and replaced with the Equality Act 2010.
The Commission found that St. George’s Hospital Medical School had “directly discriminated on racial grounds” in breach of the Race Relations Act 1976 by refusing or deliberately omitting to accept applications, through the operation of a computer program, for the admission of persons to the school as students.
The Commission reasoned that not all candidates whose scores were altered by the weightings based on race or gender suffered unlawful discrimination “since a proportion of them would not have been selected for interview anyway because of their already low scores on other factors.”
However, those candidates who would have been invited to interview but for the adverse racial and gender weightings did suffer discrimination. The best estimate for this was about 60 applications a year. It was irrelevant that some of these students were accepted by other medical schools, or eventually got into St. George’s through the “clearing” system, or eventually failed to get exam results that would have permitted them to attend the school had they been admitted, they still suffered discrimination.
The Commission went on to observe that the percentage of “Non-Caucasian” total intake that were admitted through the “clearing” system (rather than through interviews) was significant (37% in 1985/86). They reasoned that this was probably because of the discrimination occurring in earlier stages of the selection process, as this had excluded a disproportionately high number of able and well-qualified “Non-Caucasian” students. They noted that the “clearing” system, to some extent, statistically off-set the results that would otherwise have flowed from discrimination occurring at other stages of the selection process. Nonetheless, “[f]rom the point of view of the individual objects of discrimination, that does not undo the harm that they suffer; and while in the aggregate it may be viewed as mitigating the mischief, it does not excuse it.”
The Commission noted that a couple of employees at the school were aware of the racial bias in the computer program, and that “it was clearly the responsibility of those in authority at the School to find out what the computer program contained and to stop any discrimination that might be occurring.” Although those in authority at the school had no reason to believe the computer program was racially discriminatory, they “manifestly failed to take reasonably practicable steps” to make sure that this was not the case.
The Commission concluded that St. George’s Hospital Medical School had directly discriminated on racial grounds contrary to the Race Relations Act 1976 where applicants were not called for interview and would have been so called if it were not for the computer program weighting operating against them on racial grounds.
The Commission also recognised that, since the program was mimicking the approach of the selection panel, further acts of discrimination were likely to be committed unless broader changes were made to the school’s admission practices and arrangements.
The Commission made a number of recommendations to St. George’s Hospital Medical School. They stated that computerised sorting of application forms was not, in itself, objectionable. Nonetheless, any such program should be open and its use should be controlled by a senior member of staff. They also warned how strict application of academic and personal criteria may have the effect of excluding disproportionately high numbers of candidates from particular backgrounds. Such criteria should be reviewed accordingly.