Plaintiffs,
v.
CITY OF CHICAGO,
Defendant.
Plaintiffs, a group of African-American and Hispanic Chicago police officers, sued their employer, alleging that the City of Chicago deprived them of equal employment opportunities by basing promotions upon the 1993-94 Sergeant Promotional Test ("Test"), which allegedly has a significantly disparate impact on African-American and Hispanic police officers when compared to their white counterparts. Plaintiffs have withdrawn their intentional discrimination claim. The Court has jurisdiction under Title VII of the Civil Rights Act of 1964, as amended, 42 U.S.C. S 2000e et seg.
The case is currently before the Court on Plaintiffs, Motion for Preliminary Injunction, having been tried upon the facts without a jury pursuant to Fed. R. Civ. P. 65. At the hearing, the Court heard testimony from Plaintiffs Kenneth M. Webb, Andrew Jones, Jr., Jacqueline Kimber, and Thomas A. Anderson, Jr., Dr. Gerald Barrett, Lieutenant John Klein, Professor Christopher Jencks, Sergeants Brenda J. Shead, Doris Byrd, Reginald Carpenter, and Vance T. Kimber, Commissioner of Personnel of the City of Chicago Glenn Carr, Deputy Commissioner of Personnel of the City of Chicago Robert Joyce, Commander Joseph A. DeLopez, Dr. Myra Lawrence, Chief-John Cadogan, and Lieutenant Daniel A. Gibson. In addition, the parties submitted portions of the deposition testimony of Matt L. Rodriguez, the Superintendent of the Chicago Police Department. The Court has reviewed the memoranda and arguments of counsel, proposed findings of fact and conclusions of law, the testimony of the witnesses, the deposition of the absent witness, the exhibits and stipulations received into evidence, the transcript, and detailed notes taken by the Court. In listening to the testimony at the hearing, the Court took care to appraise each witness, credibility and to determine the weight to accord the witness' testimony. In so doing, the Court considered the witnesses' opportunity to observe; their manner while testifying; any interest, bias, or prejudice they may have had; and the reasonableness of their testimony in light of all the evidence presented in the case. The Court recorded its impressions of the witnesses in its notes.
In reaching its conclusion in this case, the Court has sought to draw reasonable inferences from the evidence and has considered the parties' legal arguments. In the opinion of the Court, the evidence submitted at the hearing requires denial of Plaintiffs, Motion for Preliminary Injunction. Based upon the evidence presented, the Court now makes its findings of fact and conclusions of law pursuant to Fed. R. Civ. P. 52(a).
The sergeant shortage is not static; it varies based upon deaths, resignations, retirement, and promotions. Tr. at 1341. The average monthly attrition rate is five to seven sergeants. Tr. at 1342. The CPD is creating a new appointed position entitled District Watch Commander. Tr. at 1343. There will be 75 District Watch Commanders, one for each of the three eight-hour shifts in the 25 police districts. The 33 sworn officers holding the rank of police captain, which is being phased out, will fill some of the positions. The remaining 42 District Watch Commander positions will be filled by police lieutenants. Tr. at 1343. After 42 lieutenants are appointed to the position of District Watch Commander, there will be 42 openings in the rank of lieutenant. These openings will be filled by sergeants, creating an additional 42 vacancies in the rank of sergeant. Tr. at 1344. If the promotion of the 228 new sergeants is not enjoined and Plaintiffs ultimately prevail on the merits in this litigation and the Court orderB retroactive promotions, none of the 228 promoted sergeants would have to be demoted to make room for plaintiffs. Tr. at 1351-52; Def.'s Ex. N, pp. 45-46; Defense Counsel's representation to the Court.
Defendant concedes that the Test had a disparate impact on African-American and Hispanic candidates. The Test was taken by 4700 police officers: 1494 African-American, 506 Hispanic, 2653 White, 46 other, and 1 with no race identification. Stip., P.'s Ex. 89. Thus, 43% of those who took the test were African-American and Hispanic. In 1994, Defendant made 114 promotions in rank order from the test: 109 white, 3 African-American, and 2 Hispanic. Id. Defendant,now seeks to fill 228 sergeant vacancies by making promotions in rank order from the results of the test. Id. Of these 228, 195 are white, 19 are African-American, 10 are Hispanic, and 4 are Asian- American. Of the additional 70 promotions scheduled, 59 will be white, 9 will be African-American, and 2 will be Hispanic. Id. Defendant will use the test to make a total of 412 promotions: 363 (or 88%) to white officers, 31 (or 7.5%) to African-American officers, and 14 (or 3.3%) to Hispanic officers. Thus, the [4] "success rate" for white officers (the ratio of those who will be promoted to those who applied for promotion) is .13; the "success rate" for African-American officers is .021; and the "success rate" for Hispanic officers is .027. The "success rate" for African-American and Hispanic officers is far less than the 80% standard of the EEOC guidelines. 29 C.F.R. S 1607.4(D)(1981).
The Test sought the most qualified candidates and consisted of three parts. Tr. at 105. The first part -- the written job knowledge test -- was administered to 4700 police officers. Def.'s Ex. D, App. Qr p. 5. The average number of correct answers was 96.575; the average score for white officers was 100.9, the average score for African-American officers was 89.7, and the average score was 93.9 for Hispanic officers. Id. There was no "passing score" on the "written knowledge" test: scores on this test were combined with scores on the 'in basket simulation" and the persons with the highest 1937 scores were permitted to take the "oral simulation." Tr. 209. As a practical matter, only the persons with the highest 1200 combined scores on the "written job knowledge" test and the "in basket simulation" had a chance of being promoted. Id. Scores on the "written job knowledge" test had a significant disparate impact on African-American and Hispanic police officers:
(a) White officers made up 56.4% of those who took the test and 83.5% of those who scored in the top 515. Def.'s Ex. D at Q-5. African-American officers made up 31.8% of those who took the test and 8.9% who scored in the top 515. Hispanic officers [5] made up 10.8% of those who took the test and 6.4% of those who scored in the top 515. Id.
(b) 34 of the 36 police officers who received a score of 130 or higher on the written job knowledge test were white; 106 of the 122 police officers who received a score of 124 and higher on the written job knowledge test werewhite; 198 of the 231 police officers who received a score of 121 and higher on the written job knowledge test were white; 289 of the 339 police officers who received a score of 118 and higher on the written job knowledge test were white. Stip., Pl.'s Ex. 89, 1 7.
The second component -- the "in basket simulation" was administered to 4643 police officers. Def.'s Ex. D, App. p. 7. The mean score was 49.2; the mean score for white officers was 51.3, the mean score for African-American officers was 45.7, and the mean score for Hispanic officers was 48.1. Id. White officers made up 56.4% of those who took the test and 83% of those who scored in the top 553. Id. African-American officers made up 31.8% of those who took the test and 9.4% of those who scored in the top 553. Id. Hispanic officers made up 10.8% of those who took the test and 5.8% of those who scored in the top 553. Id.
35 of the 40 police officers who received a perfect score of 60 on the in-basket test were white; 121 of the 136 police officers who received a score of 59 and higher on the in-basket test were white; 262 of the 306 police officers who received a score of 58 and higher on the in-basket test were white; 179 of the 222 sergeants who received a score of 57 on the in-basket test were white. Pl.'s Ex. 89, p. 2.
The third component -- the oral briefing exercise -- was completed by 1937 police officers: 300-African-American, 150 Hispanic, 1454 White, 22 other, and one candidate with no race indication. 24 of the 26 police officers who scored perfectly were white; 138 of the 162 who scored 14 and higher were white; 403 of the 483 who scored 13 and higher were white; 750 of the 952 who scored 12 and higher were white. Id.
Of the total Test score, white officers scored 97 of the top 101, 189 of the top 203, 262 of the top 288, 347 of the top 391, 426 of the top 491, and 508 of the next 591 scores that comprise the final promotional roster derived from the Test. Id.
Chief John Cadogan, one of Defendant's subject matter experts (IISME's"), is the Chief of the CPDIs Patrol Division, a position he has held since July, 1992. Tr. at 1298-99, 1407. Chief Cadogan became a sworn police officer with the CPD in 1961. Tr. at 1299. Based on his 35 years' experience in various CPD ranks and positions, Chief Cadogan is familiar with the duties and responsibilities of a Chicago police sergeant. Chief Cadogan participated in the development of the Test as a SME for Barrett & Associates, the test developer. Tr. at 1305-08, 1315-20. He also reviewed challenges candidates made to the examination items and process. Tr. at 1322. The Court finds Chief Cadogan to be a credible witness. .P Defendant's SME Lieutenant John Klein has been a lieutenant with the CPD since 1988. Tr. at 142-43. He has been a Chicago police officer since 1973. Tr. at 143. Lt. Klein is currently assigned to the training academy.' Tr. at 1056. Lt. Klein is familiar with the job duties and responsibilities of a Chicago police sergeant. Tr. at 143. Lt. Klein participated in the development of the Test as a SME for Barrett & Associates. Tr. at 142. He also reviewed challenges candidates made to the examination items and process. Tr. at 154-55. The Court finds Lt. Klein to be a credible witness.
Defendant's SME Commander Joseph DeLopez is a career service lieutenant with the CPD. Tr. at 689. Before becoming a District Commander, his current position, Commander DeLopez was the commander of the CPDIs Training Division, a position he held from 1992 to 1995. Tr. at 690. While commander of that Division, Commander DeLopez also responsible for training sergeants. Tr. at 693. Commander DeLopez is familiar with the job duties and responsibilities of a Chicago police sergeant. The Court finds Commander DeLopez to be a credible witness.
The CPD is a paramilitary organization that operates pursuant to a chain of command beginning with entry level police officers who are supervised by sergeants. Thus, sergeants are selected from the rank of police officer. Upon selection for promotion, prospective sergeants receive three weeks of training ("sergeant school"). During that three-week training period, they are taught how to supervise. Tr. at 693-94. Sergeants are not taught the elements of crimes or the duties of police officers at sergeant school; rather, prospective sergeants are expected to know the elements of crimes, and, because they supervise police officers, sergeants are expected to know all of the duties and responsibilities of police officers when they start sergeant school. Tr. at 694, 1104. Sergeants do not receive a copy of the Illinois Compiled Statutes when they complete their training. Tr. at 1067.
Chicago police sergeants perform the following general duties: (1) monitor and supervise the activities of police officers; (2) review and forward forms and reports submitted by police officers; (3) investigate actions of police officers; (4) assist the Watch Commander in conducting roll calls and inspections; (5) conduct watch check-offs; (6) perform desk duties; (7) conduct medical absence follow-ups; (8) conduct weekly vehicle inspections; and (9) testify in court. Tr. at 312-13, 326-27, 365-66. These are the sergeant work behaviors identified by Barrett & Associates, the developer of the Test based upon the results of the job analysis. Report on the Development and Administration of a Promotion Process for the Job of Chicago Police Sergeant, Def.'s Ex. D, p. 82.
Sergeants may perform different functions, including beat sergeant, rapid response sergeant, neighborhood relations sergeant, tactical sergeant, gang sergeant, and desk sergeant. Tr. 159, 695, 1300-04. The CPD does not provide additional training when a sergeant is transferred from one of these functions to another. Tr. at 695-96.
Sergeants must know the General Orders and Special Orders of the CPD the first day on the job because those Orders establish the basic policies and guidelines that govern the conduct of all police officers. Tr. at 691. Sergeants are responsible for ensuring that police officers whom they supervise conduct themselves in accordance with those directives. Tr. at 691. Sworn members of the CPD receive copies of the General Orders and Special Orders when they graduate from the police academy and become probationary police officers. Sergeants are not taught the information contained in the General Orders and Special Orders at sergeant school, and they do not receive a copy of those directives after they complete sergeant school. Tr. at 1060. The General Orders and Special Orders are amended and updated on a regular basis. Id. New General Orders and Special Orders and addenda to those Orders are distributed to all sworn personnel by police mail delivery. Tr. at 1060-61. The General Orders and Special Orders of the CPD are extensive. Tr. at 1060.
Sergeants must know the CPDIs new community-based policing philosophy embodied in the Chicago Alternative Policing Strategy ("CAPS"). Tr. at 958-61; Def.'s Ex. N, p. 18. Sergeants have the critical role in the CAPS program of designing the beat plan and ensuring that plan's implementation. Tr. at 1346, 1350. After sergeants complete sergeant school, they are [10] usually assigned out in the field supervising beat officers who work in squad cars and on foot. Tr. at 1057. Sergeants generally have no office or desk at the station. Tr. at 1058. Once a beat sergeant starts his tour of duty for the day, he is required to remain in the field as much as possible to respond to calls from citizens for service and to provide guidance and direction to subordinates. Tr. at 1067. To do this, sergeants carry a radio to monitor calls for assistance. Tr. at 1067-68. Sergeants should respond to as many calls concerning crimes-in- progress and domestic violence as possible during their tours of duty. Tr. at 1071-72.
Sergeants are responsible for ensuring that arrests are made properly, CPD procedures are followed, and the necessary forms and reports are filled out correctly. Tr. at 699-700, 710- 11, 715. Prior to arresting a person and depriving him of his liberty, a police officer must have probable cause to believe that the person has committed a crime. Tr. at 702. Probable cause is determined by the elements of the crime. Tr. at 702. Thus, police officers must know the elements of crimes to make proper probable cause determinations. Tr. at 702. It is more important for a sergeant to know the elements of crimes and offenses as stated in the Illinois Criminal Code and municipal ordinances than it is for a police officer, because sergeants must ensure that the probable cause requirement has been met and because police officers turn to their sergeants if they are uncertain about the elements of a crime. Tr. at 710-11.
[11] Further, sergeants supervise the charge approval process. Tr. at 706-07, 743. A sergeant must know the elements of misdemeanors and felonies, because the charge approval process differs for misdemeanors and felonies. Tr. at 1105-06.- Misdemeanor charges are approved by CPD watch commanders, whereas felony charges generally are approved by the State's Attorney. Tr. at 1085-86, 1099-1104. Thus, it is important for a sergeant to know the difference between whether a person is being arrested for a felony or a misdemeanor, because it is part of the responsibility and role of a sergeant to advise, direct, and guide police officers. Sergeants must supervise and explain when asked: the proper procedures to follow in investigating alleged criminal activity; the proper way to begin conducting that preliminary investigation to establish the elements of a crime; the probable cause to effect an arrest; and the information/evidence gathering process. Ultimately the case needs to be presented to someone for approval of charges. Tr. at 1037.
Sergeants must know the elements of crimes because they often must react quickly in the field where they do not have the time or resources to research crimes in the statute books and the Codes. Tr. at 700-01, 703, 720, 1079-80, 1113. The possible consequences of not knowing the law and improperly charging a person with a crime or offense include the denial of that person's constitutional rights, a wrongful arrest, injustice to the victim, an inability to prosecute a crime, and a resultant civil lawsuit. Tr. at 704, 708, 722, 941.
[12]
Sergeants review CPD arrest reports prepared by police officers. On the arrest report, the arresting officer must provide the facts that establish probable cause. Tr. at 1089. It is the sergeant's responsibility to make'sure that the facts contained in these arrest reports satisfy the requirements of probable cause. Tr. at 146, 1038, 1087. To accomplish this review, sergeants must know the elements of crimes. Tr. at 699- 700. Sergeants also review the accuracy and completeness of police officers' case reports that will be submitted to the state and the FBI. Tr. at 147-48, 728. Those case reports must contain the elements necessary to establish probable cause for the arrest. Tr. at 702. Sergeants generally review arrest reports and case reports in the field. Tr. at 728-29, 1038-39, 1094. A sergeant may have 20 to 30 case reports to review at one time. Tr. at 729. Sergeants usually do not have an opportunity to research crimes during their review of case reports. Tr. at 729, 1113. If sergeants had to seek reference documents to review arrest reports and case reports, they would be less available to respond to calls for assistance from citizens or to supervise their subordinates. Tr. at 729.
Sergeants also are responsible for ensuring that incidents are handled in accordance with CPD policy and procedure. Tr. at 1109-10. They often must make quick judgments regarding whether an action conforms with that policy and procedure. Tr. at 1113. To do this, sergeants must know the General Orders and Special Orders. Tr. at 710-11. Sergeants [13] must know those Orders more thoroughly than their subordinates who turn to their sergeants for guidance on those directives. id.
Field personnel must be able to adjudicate matters at the scene in most circumstances. There will be times where there will be complicating factors or other reasons why things have to be handled in a different fashion. In the normal course of business, however, the CPD needs its officers and sergeants to have the requisite knowledge and skills to be able to deal with situations at the scene. Tr. at 1226-27. It is necessary to make determinations at the scene because the Department needs to make sure that it can dispense with the incident and solve the problem as quickly and efficiently as possible so that police officers remain available for police service. Id.
It is not practical for sergeants, who often are travelling in squad cars or working in busy, noisy stations, to look up General and Special Orders. Tr. at 1079-80. Thus, it is imperative that sergeants have a good working knowledge of the General Orders and Special Orders. Id.
Dr. Gerald Barrett is an industrial psychologist. Def.'s Ex. I. In March of 1993 his consulting company, Barrett & Associates, Inc., submitted a proposal to develop a police- sergeant promotional test for Defendant. P.'s Ex. 12. A report had been submitted to Defendant by the "Mayor's Blue Ribbon Panel on Police Testing, Hiring, and Promotion," which recommended the [14] hiring of an outside consultant to develop a promotional procedure. Pl.'s Ex. 60, p. 12. Dr. Barrett and Defendant signed an agreement for development of the Test on November 24, 1993, which requires Defendant to pay Dr. Barrett his customary hourly rate of $250 per hour for services rendered in any litigation arising from the Test. P1.1s Ex. 12. Barrett & Associates has been in the business of developing employment related examinations for 23 years. Tr. at 372. The majority of those examinations were developed for municipal police and fire departments. Id. The Court finds Dr. Barrett to be a very credible witness.
The proposal recommended a three-component test for promotion to sergeant: a job knowledge component, an in-basket component, and an oral component. This proposal was based upon Dr. Barrett's years of experience developing promotional tests for municipal police and fire departments. Tr. at 93, 171. Specifically, the proposal was based upon past job analyses that Dr. Barrett conducted for the job of police sergeant in other jurisdiction. It was Dr. Barrett's experience that this type of test battery tended to cover the major work behaviors of a police sergeant. Tr. at 171. Further, the proposal included only the three-part framework rather than specific items or details of the materials that would be used in the actual Test, because Dr. Barrett had not yet conducted a job analysis for the job of Chicago police sergeant. Tr. at 171. [15]
Prior to developing the exam, Dr. Barrett conducted a scientific literature review relevant to a test development process. The purpose of this literature review was to explore alternative selection procedures, identify state-of-the-art selection procedures, and provide support for the procedures used in the Chicago police sergeant promotional process. Def.'s Ex. D, pp. 4-77. This literature review also included a review of the current literature on the job analysis process, test construction, and validation in general. Def.'s Ex. D, pp. 6-10.
The first step in the development of the sergeant examination was a job analysis. The job analysis included interviews, ride-a-longs, observations and a review of all departmental and organizational documents and materials. Def.'s Ex. D, p. 78; Tr. at 390. One hundred twenty four (124) semi- structured interviews were conducted with 120 incumbent sergeants and other supervisors. Interviews were conducted with one commander, two captains, twenty-eight lieutenants, and eighty- nine sergeants, including Sgt. Doris Byrd, who testified for Plaintiffs. Interviews averaged one hour in duration. During the interviews, relevant materials, forms, and reports were collected. Tours of districts, work areas, and beats were conducted to develop a thorough and complete understanding of the job of Chicago police sergeant. Def.'s Ex. D, pp. 78-79; Tr. at 384, 385, 390. Dr. Barrett's team interviewed 89 sergeants in order to identify the major work behaviors and important job knowledges. Tr. at 189 1n. 22-25, 190. They interviewed [16] lieutenants and other supervisory personnel because it is important to ask supervisors of sergeants about the job of sergeant in order to identify those critical incidents that always confront sergeants on the job. By identifying critical incidents, a test developer can identify the important job knowledges, the areas of administrative decision making that are part of a sergeant's job, and the type of oral communication that is important. Tr. at 190.
As part of his job analysis, the team interviewed a broad and representative sample of Chicago police sergeants. Tr. at 386; Def.'s Ex. D at 79-80. The incumbent sergeants and supervisors who were interviewed were diverse in terms of race and gender. Tr. at 386, 389; Def.'s Ex. D, pp. 79-80. Dr. Barrett also took steps to ensure that his team interviewed sergeants from as many of the 25 police districts as possible and to ensure that the sergeants interviewed held different assignments, such as field sergeant and desk sergeant, so that his team could understand all of the functions of a sergeant and obtain all viewpoints of the job. Tr. at 386, 389. Dr. Barrett began his job analysis on May 11, 1993 and completed the job analysis on September 14, 1993. Def.'s Ex. D, App- C, p- 2; Tr. at 386, 387. The majority of the interviews were conducted between May 11, 1993 and June 3, 1993. Def.'s Ex. D, App. C, pp. 2-5.
The job analysis interviews focused extensively on the identification of the major work behaviors and responsibilities [17] required of incumbents in the position of police sergeant, their relative importance, and the frequency of performance of these behaviors. Def.'s Ex. D, p. 81, App. C; Tr. at 385. The job analysis interviews focused specifically on-the identification of the knowledge, skills, and abilities ("KSA's") associated with the major work behaviors, as well as how the KSA's are acquired. The job analysis interviews also focused on measuring the importance and frequency of each major work behavior or responsibility identified during that process. Tr. at 399; Def.'s Ex. D, p. 81. The team met periodically during the job analysis process to discuss their findings and identify areas of the job of sergeant that they did not fully understand. They also identified what information about the job of sergeant they were still missing. Tr. at 388. The purpose of the meetings was to ensure that Barrett & Associates had a complete and accurate picture of the job of sergeant at the end of the job analysis process. Tr. at 388. The job analysis included ride-a-longs and observations in order to get a fuller understanding of the job of sergeant and to verify the information the job analysts had received during the interview process. Tr. at 391.
Through the job analysis process, Dr. Barrett identified the source material that was important for a sergeant to know. This material included General and Special Orders, department directives, portions of the Illinois Criminal Code and Municipal Code, and a pamphlet on community policing. Def.'s Ex. D, App. F; Tr. at 392, 395, 399. The source list prepared by Dr. [18] Barrett was reviewed by a panel of four SME's from the CPD to ensure that the material was relevant and important for a sergeant to know. Tr. at 396-97, 398. As a part of the job analysis process, confirmatory job analysis-interviews were conducted where some incumbents and supervisors were asked to review earlier drafts of the job description for completeness, comprehensiveness, and accuracy. Specifically, they were asked to mark or strike out any work behaviors, duties, or responsibilities that they thought a Chicago police sergeant did not perform, and to add any that had been omitted. Def.'s Ex. D, p. 78; Tr. at 388.
At the end of the job analysis process, Dr. Barrett completed a Master Job Description for the job of Chicago police sergeant. Def.'s Ex. D, App. C; Tr. at 191-92. The Master Job Description sets forth the major work behaviors and responsibilities of a sergeant that were identified during the job analysis process, their relative importance and frequencyl and the associated KSA's. Def.'s Ex. D, pp. 81-88 & App. C; Tr. at 385, 192, 392-93, 399, 391.
The Master Job Description was reviewed by subject matter experts from the CPD to ensure that it was complete and that no major tasks or responsibilities were omitted. Tr. at 393, 394, 388, 389.
The job analysis revealed that the job of Chicago police sergeant consists of an oral component, knowledge component, and an administrative component. The written job [19] knowledge examination was utilized to measure the knowledge component, the in-basket examination was utilized to measure the administrative component, and the oral briefing exercise was utilized to measure the oral component. Tr; at 199; Def.'s Ex. D, pp. 88-89, 99-105, 106-121.
The written job knowledge test developed by Dr. Barrett for the sergeant examination is a specific-job-knowledge test tailored specifically to the knowledge required of a Chicago police sergeant. Tr. at 168. Thus, the written job knowledge test is not a general cognitive ability test. Tr. at 167-68. The written job knowledge test measured the knowledge identified during the job analysis process as necessary for successful performance as a Chicago police sergeant. Def.'s Ex. D at 88. Evidence from the job analysis demonstrated that the more knowledge a candidate has of the source materials identified as important for a sergeant to know the better that individual will perform on the job. Tr. at 401. Empirical evidence from past studies and research in the field of industrial psychology demonstrates that there is a positive correlation between performance on job knowledge tests and performance on the job. Tr. at 211, 401, 402. Numerous studies indicate that job knowledge is related to job performance. Tr. at 102-104; Def.'s Ex. D, p. 70. The job of Chicago police sergeant is a complex job and complex jobs require more knowledge. Tr. at 402. Further, there is a higher correlation between performance on job [20] knowledge tests and job performance when a job is complex. Tr. at 403.
The items were designed to measure only the KSA's required for successful performance on the job as determined by the job analysis. Def.'s Ex. D, p. 91. The items on the written job knowledge examination were written to sample the source materials identified during the job analysis as important for a sergeant to know. Def.'s Ex. D, p. 89.
The item writers relied on job analysis information to determine the number of questions to be included from various sources. Def.'s Ex. D, p. 89. The item writers developed a pool of items that tapped the job domain of knowledge needed to perform the major work behaviors of a sergeant. Dr. Barrett then selected a representative sample of items from this pool. Tr. at 456, 459, 460. In order to develop a good test, the developer should choose a large number of items that sample the job domain, because it is the aggregation of the correct responses on that large number of items that allows an industrial psychologist to draw the inference that the candidates who score highest on the examination have a mastery of the job knowledge that is required for performing well as a sergeant. Tr. at 458-59. No one item on the written job knowledge examination is important by itself; rather the aggregation of the items is important. Tr. at 459. Individual items are only exemplars of the broader domain of knowledge. Test developers rely upon the responses to the aggregation of items, not a response to a single item, to
[21] determine that a candidate who gets a higher score will perform better as a sergeant. Tr. at 891. Dr. Barrett relied on his many years of test development experience and literature in the field in making the determination of which items to include on the written job knowledge examination he developed for the Test. Tr. at 465.
Questions were written in a multiple-choice format. Each item included a stem and five alternative answers. The stem of the item establishes the elements of the situation from which the examinee should base his or her response. There are four alternatives known as distractors, which are incorrect given the information in the stem. There is only one correct answer. Def.'s Ex. D, p. 93; Tr. at 461, 462, 463. The instructions of the examination required the candidate to choose the "most correct" alternative. The answer to each question was to be based solely upon the source material indicated on the examination. Pl.'s Ex. 63. Distractors were designed to be plausible in order to prevent someone with lack of knowledge of the job of police sergeant from answering correctly. Tr. at 462,, 463. Items were designed to tap a domain of knowledge. If the item distinguished between plausible and implausible answers, the item would not be testing that domain of knowledge. Tr. at 782. Each item on the written job knowledge test was linked to the major work behaviors and tasks identified in the Master Job Description and the knowledge identified as important to successful performance on the job. Tr. at 193; Def.'s Ex. D at [22] 89-94. Appendix I of the Technical Report (Def.'s Ex. D) contains a chart that sets forth the linkages between each item of the 150 item written job knowledge test and the major work behaviors set forth in the Master Job Description in Appendix C to the Technical Report. This chart establishes a link between the knowledge being tapped by that test item and the work behavior and knowledge required for the job. Tr. at 194, 195, 396. Many of the items on the written job knowledge test are linked to more than one major work behavior because the same knowledge underlies different work behaviors. Tr. at 197. The linkage chart set forth in Appendix I is one piece of evidence of content validity. Tr. at 121.
The test development process for the written job knowledge examination incorporated extensive review procedures. Each item written for this examination underwent extensive review and revision over a four month period to ensure that the item was job relevant, that it could be easily understood and that it conformed with the policies and practices of the CPD. Def.'s Ex. D, p. 93; Tr. at 463, 464. The written job knowledge examination was pilot tested on subjects who had no connection with the CPD. Tr. at 464, 465; Def.'s Ex. D, p. 97-98. In order to ensure the relevance and criticality of test materials, test items were reviewed by SME's from the CPD. Def.'s Ex. D, p. 92. All items developed by Dr. Barrett for the written job knowledge examination were reviewed by Chief Cadogan and Lieutenant Klein. Id. One member of Barrett & Associates worked with each SME. [23] For each item, Barrett asked the SME whether the item was relevant and important for a sergeant to know. He also asked Chief Cadogan and Lt. Klein whether there were any problems with the item. Tr. at 403 ln. 12-18. Barrett &-Associates incorporated all recommendations and modifications made by the SME's. If the SME's determined that an item was inappropriate, that item was deleted. Tr. at 403. The 150 items on the job knowledge component of the Test had been reviewed by Chief Cadogan and Lieutenant Klein. Tr. at 404.
The 150 items on the written job knowledge examination tested knowledge that a sergeant must know in order to be effective on the job. Tr. at 152, 1309, 404, 1114, 1122, 1314. The source list material provided to candidates taking the Test contained all of the information an applicant would need to properly answer the questions on the written job knowledge examination. Tr. at 1226.
Defendant has not conducted a criterion-related study to obtain a correlation coefficient between performance on the written job knowledge component and a measure of job performance.
The in-basket component was designed to (1) measure administrative decision making, including a candidate's analytical, organizational, and administrative abilities, and (2) reflect the type of work a sergeant would have to perform on the job. Tr. at 407. It consisted of several tasks that are actually performed by Chicago police sergeants, such as reviewing [24] reports, investigating and disciplining subordinates, and determining crime patterns. Def.'s Ex. 99; Tr. at 407.
Each candidate was given 60 minutes to review a set of materials and to take notes on those materials. The candidate was then given 75 minutes to complete 60 multiple choice questions. Candidates were allowed to refer to their notes and the materials while they answered the questions. Def.'s Ex. D, pp. 92-93.
Past research and studies indicate that in-basket exercises like the in-basket simulation used by Dr. Barrett in the Test are very effective tools for selecting supervisors or managers. These studies indicate a positive correlation between performance on in-basket and oral briefing exercises and performance on the job. Def.'s Ex. D, pp. 72-77; Tr. at 22.
All items on the in-basket simulation were linked to the Master Job Description. Tr. at 202; Def.'s Ex. D, App. L. It is important for a sergeant to have some skills as an administrator because they are required to perform a variety of administrative tasks. They must be analytical, logical, and able to articulate their reasons and processes. Tr. at 152.
The in-basket simulation development process incorporated extensive review procedures. Def.'s Ex. D, p. 101. The in-basket simulation was pilot tested on graduate students, some of whom had studied the material and some had not, in order to determine the time limit. Def.'s Ex. D, pp. 104-05. Once this process was completed, the exercise materials and items were [25] reviewed by the SME's Chief Cadogan and Lt. Klein to ensure that the materials and items were relevant and consistent with the rules, policies, and procedures of the CPD. Def.'s Ex. D, p. 101; Tr. at 406, 150, 1317-18. Barrett & Associates incorporated all modifications and recommendations made by Chief Cadogan and Lieutenant Klein. Tr. at 403, 1308, 1314-15, 1318-19. The in- basket tested for skills and abilities that are important for a sergeant to have. Tr. at 159, 1317-18, 408. The in-basket tested for a candidate's ability to synthesize and understand information given to him or her. Tr. at 409 & 410.
Defendant has not conducted a criterion-related study
to obtain a correlation coefficient between performance on the
in-basket and a measure of job performance.
The oral briefing exercise was linked to the major work behaviors and responsibilities of a sergeant set forth in the [26] Master Job Description. Appendix 0 of the Technical Report (Def.'s Ex. D) contains the linkage chart for the oral briefing exercise. Tr. at 203-204. Thheoral briefing exercise tested for a candidate's ability to communicate clearly, concisely, and effectively. Tr. at 410. It did not directly test listening comprehension. Tr. at 410.
There is evidence in the professional literature relied upon by Dr. Barrett that oral communication skills are directly linked to success on the job. Tr. at 898-903, 921-22. Oral communication is an important skill for the job of sergeant, as the CPD relies upon communication in many instances. Tr. at 152.
In reviewing the oral component, Lt. Klein determined that the type of materials the sergeant reviewed were representative of a very wide range of materials that a sergeant comes across. Further, Lt. Klein and Chief Cadogan determined that the skill required to respond to the oral component is the skill required for the job of sergeant. Tr. at 160, 1320.
Defendants have not conducted a criterion-related study to obtain a correlation coefficient between performance on the oral component of the exam and a measure of job performance.
Dr. Barrett took the following steps during the test development process to try to minimize adverse impact:
a. written short answer and essay type examinations may have low reliability because they require subjective grading by raters;
b. written short answer and essay examinations are very costly and time-consuming to administer and score;
c. Dr. Barrett anticipated that over 8,000 applicants would be taking the examination; because of the time and cost constraints, it was impractical to include an essay examination;
d. scores on written short answer and essay examinations tend to be bunched together at one end of the spectrum making it impossible to differentiate between candidates;
e. constructed response tests like written short answer and essay examinations tend to have as much, if not more, adverse impact on minority candidates.
f. structured response tests like multiple choice are more reliable and economical to score. Tr. at 445-50. [28]
Listening comprehension is an important ability for a sergeant to have. Tr. at 1229; Pl.'s Ex. 53, C7-10. The 1993 police sergeant examination did not include a direct listening comprehension examination for the following-reasons:
a. research in the field of industrial psychology, including research conducted by Dr. Barrett, indicates that reading comprehension and listening comprehension are very similar. Tr. at 97, 410, 411, 437, 439, 443;
b. there are reliability problems with listening comprehension tests because it is difficult to accurately rate a candidate's ability to listen in a way that would be perceived as fair and objective. Tr. at 96-97.
c. there are practical concerns about how to actually test for an applicant's ability to listen -- particularly when attempting to test interactive listening skills. Id; Def.'s Ex. D, p. 32.
However, although listening comprehension is typically tested in an interview, standardized listening comprehension tests exist. Tr. at 1054.
Sergeants Vance Kimber, Doris Byrd, Brenda Shead, and Reginald Carpenter -- Plaintiffs' SME's -- are plaintiffs in Brown v. City of Chicago, No. 95 C 1890 (N.D. Ill.). In that suit, they are challenging the 1994 Chicago police lieutenant examination, which was developed by Barrett & Associates, on the grounds that it has a disparate impact on minorities. The SME's are represented by Kenneth N. Flaxman, who also represents Plaintiffs in this case. Tr. at 310-11, 323-24, 330, 353. Plaintiffs' SME's reviewed the 150 question multiple choice job knowledge examination, pursuant to a February 18, 1996 letter from Mr. Flaxman. Tr. at 353-54; D.'s Ex. J. In that letter, [29] Mr. Flaxman asked them to "help to block the City from making promotions to sergeant." D.'s Ex. J. Mr. Flaxman also asked them "to prove that a reasonable passing score on the [multiple choice job knowledge component] would have been about 88 [out of 150]." Id. They had not reviewed the examination prior to the meeting called by Mr. Flaxman. Tr. at 316, 355. All of Plaintiffs' SME's agreed that Chicago police sergeants perform the duties identified in Barrett & Associate's Technical Report as the major work behaviors of a Chicago police sergeant. Tr. at 312-13, 326-27, 337-38, 366; D.'s Ex. D, p. 82.
Plaintiffs' evidence that certain examination items were not job-related was based on false premises and does not controvert the evidence of content validity. The job analysis is the end result of a process that involved interviews with 124 CPD personnel, including incumbent sergeants and personnel who supervise sergeants. The Court finds that the opinions of the individual witnesses for Plaintiffs have less validity and weight than the aggregation of views and opinions of the Chicago police personnel and review of source material, as represented in the job analysis and the testimony of the witnesses for Defendant.
Seventy-two of the one hundred fifty items on the written job knowledge test are not challenged. All of the items are based on the statutes, ordinances, General Orders, Special Orders, and other source materials used in the CPD and provided to candidates prior to the examination as the source material that would be tested on the examination. Def.'s Ex. D, App. F; [30] Def.'s Ex. K. The correct answer to these items is found in the source materials included in Def.'s Ex. K. The language "seeking approval to charge" is contained in many of the 150 items of the job knowledge test. Pl.'s Ex. 14. That language was utilized by Barrett & Associates based upon the recommendations of the CPD's SME's, Klein and Cadogan. Tr. at 1180, 1181. Fifteen of the seventy-two items that Plaintiffs do not challenge contain the "seeking approval to charge" language. Pl.'s Ex. 14.
In his testimony concerning items challenged in Plaintiff's Exhibit 55, Commander DeLopez testified that the knowledge required to answer the items was important for a sergeant to have. Tr. at 714-43, 940-61. Further, Commander DeLopez testified that having reviewed all 150 items of the job knowledge test, he formed an opinion that the information contained in those questions was information that a sergeant needed to know and that the information was relevant to the responsibilities of a sergeant. Tr. at 961-62.
In his testimony concerning the items challenged in Plaintiffs' Exhibit No. 54, Lt. John Klein testified that the knowledge and information contained in the questions included is important for a sergeant to have. Tr. at 1130-1225. Lt. Klein testified that having reviewed all 150 items of the job knowledge test, he formed the opinion that the items contained knowledge that a sergeant needs to have in order to be able to perform that function. Tr. at 1122.
[31] Defendant selected Arthur Andersen and Co. to administer the test and provide for validity and test security. Tr. at 471. Every candidate was given the opportunity to challenge any item or material contained in-the three components of the Test. Tr. at 471, '1,50, 754-55. The challenge procedures were designed to ensure that the examination process was fair, that there were no mistakes, that items were not miskeyed, and to clarify ambiguities in the testing process. Tr. at 750. However, to avoid compromising the Test's confidentiality, Defendant did not provide access to a copy of the questions and answer key, thus, applicants could not check to see whether a correctly keyed answer had received credit. Arthur Andersen provided a hot line that candidates could call to lodge a complaint about any portion of the examination, including its administration. Tr. at 154-55, 754-55.
All challenges were reviewed by acommittee comprised of Barrett & Associates personnel and SME's from the CPD. Tr. at 751-52, 755. Dr. Barrett received a total of seven challenges 'to the Test. Tr. at 751. As a result of these challenges, item 19 on the written multiple choice was double-keyed; every candidate who chose alternative B or D on item 19 was given credit for the correct answer. Def.'s Ex. 0; Tr. at 757-61. As a result of these challenges, all individuals were given credit for item 11 on the in-basket and item 60 on the in-basket was double keyed.; every candidate who chose alternative D or E on item 60 was given credit for the correct answer. Def.'s Ex. 0. Dr. Barrett [32] received far fewer challenges than would be normal for this type of promotional examination. Tr. at 764.
The decision to weight the three examination components equally was made at the completion of the job analysis and before the examination announcement was posted in September, 1993. Tr. at 176. The decision to weight the three test components equally is supported by the scientific literature on equal weighting, the job analysis conducted by Barrett & Associates, and a judgmental rationale study that involved linking various work behaviors with the various test components and was conducted by two analysts who work for Barrett & Associates. Tr. at 177-179; Def.'s Ex. E. The decision to weight the three test components equally also was supported by a computer simulation. Tr. at 179; Def.'s Ex. E; Def.'s Ex. D, App. R.
There is evidence from the job analysis that a higher score relates to better job performance. Tr. at 766-67; Def.'s Ex. E. Scientific research shows that test scores, particularly on content-validated job knowledge tests, relate in a linear fashion to job performance, which means the higher the test score, the higher the probability of being a successful job performer. Tr. at 767-68; Def.'s Ex. E. This is the type of empirical evidence that industrial psychologists rely upon in reaching decisions on how to use the results of examinations. Tr. at 768. The psychometric properties of each component of the [33] test met accepted standards in the field of industrial psychology. First, the examination had a high reliability coefficient of .87. Low reliability precludes validity. Second, the scores on the examination were not bunched together. This meant that the test was differentiating among individuals based upon their KSA's. Third, there was generally a positive correlation between each item on the test and the total test scores. Tr. at 768-770, 777-782; Def.'s Ex. E.
In the Technical Report describing the development of this examination, Dr. Barrett referred to published articles on assessment centers. Def.'s Ex. E, p. 72. The two most common assessment center exercises are in-basket and oral exercises. Tr. at 921. There is evidence in the literature that these types of exercises are effective in selecting supervisors. Tr. at 922. Success on assessment center devices is related to success on the job. Tr. at 888-903.
In sum, the Court finds, for purposes of this Motion for Preliminary Injunction, that the test is contont valid. Plaintiffs, expert Professor Jencks testified that tests of a substantial body of knowledge which require protracted study tertd to have an adverse impact on some minorities. Tr. at 258-65. The fact that a test has an adverse impact on some minorities does not mean that the test is not content valid. Tr. at 264.
The Blue Ribbon Panel recommended that performance evaluations play a role in promotions under appropriate [34] standards. Both Dr. Barrett and Professor Jencks testified to the difficulty of implementing any promotional system based on supervisor ratings. Tr. at 218-19, 274-76, 377-78, 931-32. It is difficult to obtain reliable and accurate measures of performance from supervisors because there has been so much litigation over promotional examinations in the safety forces and supervisors realize that they may be accused of favoritism, bias, and discrimination based on their supervisory ratings. Thus, supervisors are reluctant to rate the performance of their subordinates. In addition, supervisory ratings used as part of a promotional process are not confidential. Consequently, supervisors are concerned about what they say or do concerning their subordinates. Tr. at 377-78; Def.'s Ex. D, p. 40. Plaintiffs have not proven that a job performance measure for promoting Chicago police sergeants is practical, reliable, valid, or available.
The type of criterion-related validation study that might be of value to an organization would be a concurrent validation study where the examination is administered to a large number of incumbents and, at the same time, a measure of job performance is obtained to determine whether there is a correlation between test scores and job performance. Tr. at 377. It was not feasible to conduct a concurrent criterion-related validation study in the CPD to obtain a correlation between performance on the Test and performance on the job for the following reasons: [35]
a. it is impossible to maintain the confidential nature of the test battery if the examination components are administered to incumbent police sergeants in order to conduct a criterion related study. Once the test items have been revealed the validity of the test is destroyed;
b. In safety forces like the CPD, supervisors are very reluctant to give performance appraisal ratings to their subordinates because they are worried about being labeled as biased and unfair. Tr. at 218-19, 377.
Mastery testing involves multiple test administrations. Those who score well on the first test are not tested further. Others are given time to study and additional opportunities to take new mastery tests. Tr. at 849. Mastery testing is not practical.
No evidence was presented regarding the validity and adverse impact of cut-off scores or mastery testing.
The Chicago Police Department ("CPD") has a 1996 budget of 1,230 police sergeant positions. Prior to March 11, 1996, only 1,017 of those budgeted positions were filled, creating a shortage of 213 police sergeants. Tr. at 1337. The Patrol Division's District Law Enforcement (the 25 CPD Districts responsible for day to day law enforcement) had a shortage of 158 sergeants. Tr. at 1340. On March 11, 1996, the CPD promoted 18 sergeants to the rank of lieutenant, creating 18 more sergeant vacancies. Tr. at 1324-25. Thus, the CPD has a current shortage of approximately 231 sergeants. Tr. at 1337. The Police Superintendent has determined that there is an operational need to make 228 promotions to the rank of sergeant now. Tr. at 1433.
[36] The CPD plans to make an additional 70 sergeant promotions later this year. Pl.'s Ex. 89. The average monthly attrition rate is five to seven sergeants. Tr. at 1342. The CPD is creating 75 new appointed positions called District Watch Commander to which 42 sergeants will be promoted, creating an additional 42 vacancies in the rank of sergeant. Tr. at 1344. The CPD is attempting to implement a new policing philosophy known as CAPS, which is a community-based crime prevention program. Tr. at 1345; Def.'s Ex. N, pp. 17-20. Sergeants play an important role in the new CAPS program. Under CAPS, sergeants must identify the crime in the beat, develop a beat plan to address the crime, and ensure that the police officers under their supervision implement the plan. Tr. at 1350. With the introduction of CAPS, the number of police beats has increased from 1,090 to 1,314. Tr. at 1345. The increase in the number of beats creates a need for additional sergeants. Tr. at 1347. The CAPS program has not been fully implemented in the CPD. Tr. at 1347. CAPS cannot be fully implemented without additional sergeants. Tr. at 1349-50.
During the last three weeks in December, 1995, and the first week in January, 1996, there should have been 3,444 tours of duty completed by sergeants. Tr. at 1353. During that period, the CPD did not have sergeants for 1,600 of those tours. Tr. at 1353. Specifically, approximately 42% of the time there was no sergeant on the third watch (4:00 p.m. to 12:00 a.m.); 42% of the time there was no sergeant on the second watch (8:00 a.m. to 4:00 p.m.); and 25% of the time there was no sergeant on the [37] first watch (12:00 a.m. to 8:00 a.m.). Tr. at 1353-54. When sergeants are not available for a tour of duty, delivery of police services to the public suffers, as there is no supervision to make sure police officers are performing-their jobs properly. Tr. at 1355.
Over the last two years, the CPD has hired approximately 1,850 new police officers and plans to hire 500-550 more by the end of 1996. Tr. at 1354. That accounts for about 30% of the total number of police officers. Tr. at 1354; Def.'s Ex. N, pp. 33-34, 48. New police officers need supervision to guide them in performing their jobs. If supervision is not available, potentially harmful errors in the provision of police services may result, including possible corruption of new officers and problems within the organization. Tr. at 1354-55; Def.'s Ex. N, pp. 47-48.
The following upcoming events in Chicago will place an additional burden on CPD resources and require additional police supervision from sergeants.
a. The Democratic National Convention in August, 1996. Tr. at 1357. This is a major event which requires heightened police security to ensure the safety of the citizens of Chicago, Convention delegates and other visitors. Tr. at 1359-60. The CPD must prepare for the possibility of demonstrations, riots and other disturbances that marred the 1968 Democratic National Convention. Tr. at 1360-62; Def.'s Ex. N, pp. 48-49.
b. The Bulls' participation in the NBA playoffs in June, 1996. Tr. at 1356. When the Bulls won the NBA championship in 1993, the CPD put approximately 3,000 police officers on the street to prevent and respond to the crime resulting [38] during the post-game celebrations. Tr. at 1358-59.
c. The Puerto Rican Carnival in June, which last year required over 1,750 police officers due to the crime and gang activity during the six-day event. Tr. at 1356.
d. The Fourth of July fireworks show on the lakefront, which draws almost one millioi,-i people and requires about 900 police officers. Tr. at 1357.
e. Fiesta del Sol in late July, which requires about 800 police officers during that four-day event. Tr. at 1357.
f. Taste of Chicago, the Blues Fest, The Bud Billiken Parade, Venetian Night, the Air and Water Show, as well as the 150 other special spring and summer events, which call for additional police services. Tr. at 1356-57.
If this Court enters a preliminary order enjoining the scheduled promotions, Plaintiffs will not be immediately promoted to sergeant and will not receive any additional compensation.
None of the Plaintiffs testified that they would have been promoted if the challenged questions from the job knowledge test were thrown out. At this stage of the proceedings, Plaintiffs, injury is speculative, as Plaintiffs have conceded that they do not know which, if any, of them are entitled to a promotion. (Pl.'s Trial Br. at 8). No examination for promotion to lieutenant, the next CPD rank after sergeant, is currently scheduled. Def.'s Ex. N, p. 22. Further, Plaintiffs have not offered an equally valid alternative selection device with less adverse impact that could be employed in the near future if promotions from the Test are enjoined. The Personnel Rules of the City of Chicago prohibit provisional or temporary promotions
[39] when there is a promotional eligibility list in place. Tr. at 1426. Because there is a sergeant promotional list in place, under its Personnel Rules, the City could not make any temporary or provisional promotions to the rank of sergeant. Tr. at 1429. The one time where the City did make temporary police promotions there was no promotional iist in place. Tr. at 1428. Those promotions created morale and supervision problems, as the temporary appointees were unsure of their authority. Tr. at 1410-11, 1428.
Plaintiffs argue that they will suffer irreparable harm absent the injunction in that retroactive promotions are an inadequate remedy. (Pl.'s Trial Br. at 3). However, some delay in promotion does not constitute irreparable harm. Cox v. City of Chicago, 868 F.2d 217, 223 (7th Cir. 1989); Ceichon v. City of Chicago, 634 F.2d 1055, 1057 (7th Cir. 1980). Although Plaintiffs argue in conclusory fashion that the demotion of the officers promoted to sergeant "is the only mechanism available to the Court that could undo the harm that Plaintiffs will suffer," (P1.1s Trial Br. at 3), the Court disagrees and concludes that, if Plaintiffs ultimately prevail, an adequate remedy exists: retroactive promotions with back pay, pension benefits, and seniority in title. See Sampson v. Murray, 415 U.S. 61, 91-92 (1974); Lasco v. Northern, 733 F.2d 477, 481 (7th Cir. 1984); Brown v. City of Chicago, No. 95 C 1890, slip. op. at 16-17 (N.D. Ill. Mar. 1, 1996). The case upon which Plaintiffs rely for the proposition that the Court would need to order demotions if the injunction is denied and Plaintiffs ultimately prevail, Doll v. Brown, 75 F.3d 1200, 1205 (7th Cir. 1996), while recognizing a district court's power to order demotions in a promotion discrimination case, also explains that this is not "an absolute duty"; rather, considerations of "concern with friction in the [41] workplace or changed circumstances of some kind, or solicitude for the current incumbent" require ordering "an award of front pay, an equitable substitute for reinstatement (or, here, linstatement') where reinstatement would be-inequitable." Id. (citations omitted). Moreover, the Court has made the finding of fact that no demotions would be needed if the Court ultimately orders retroactive promotions.
Plaintiffs also argue that retroactive promotions are not a plausible remedy for the injury of denial of an opportunity to compete for a promotion because "it is impossible to identify with certainty the persons who would have been promoted in a non- discriminatory" system. However, in dictum the Doll court discussed the propriety of a remedy that "split[s] the difference in cases where proof of injury is inescapably uncertain": the lost chance theory of recovery. Id. Indeed, the Seventh Circuit stated that "it strikes us as peculiarly appropriate in employment cases involving competitive promotion" and, while not holding that the lost chance theory is available in such cases because the issue was not before it, "commend[ed] it to the consideration of the bench and bar as a Possible method." id. at 1206-07.
Contrary to Plaintiffs' contention otherwise, the Court does not engage in a sliding scale analysis of the two-pronged threshold burden; Plaintiffs, failure to satisfy the irreparable harm prong of its threshold burden warrants denial of the Motion for Preliminary Injunction. Abbott, 971 F.2d at 11. [42]
Evidence of the validity of a test or other selection procedure by a content validity study should consist of data showing that the content of the selection procedure is representative of important aspects of performance on the job for which the candidates are to be evaluated.29 C.F.R. S 1607.5(B). The Seventh Circuit has interpreted this such that "Title VII does not require that an employer test all or nearly all of the skills required for the occupation." Instead, "an employment test must neither: (1) focus exclusively on a minor aspect of the position; nor (2) fail to test a significant skill required by the position." Gillespie, 771 F.2d at 1044. In so holding, the Seventh Circuit followed the Second Circuit's interpretation of "representative," as enunciated in Guardians Assoc. v. Civil Service Commission, 630 F.2d 79, 98-100 (2d Cir. 1980). Thus, this Court turns to Guardians for guidance. First, in concluding that the employer need not test for nearly all of the job content, the court explained that to hold otherwise would "foreclose any possibility of constructing a valid test" as "some of the required capacities cannot be tested for in any valid manner." Id. at 98. Second, the court [44] emphasized that the policy behind the "representative" requirement is to prevent "the needless elimination of some significant part of the job's requirements from the selection process entirely." Id. at 99 (emphasis added). The court summed up the analysis as follows: "it is reasonable to insist that the teat measure important aspects of the job, at least those for ,rAhich appropriate measurement is feasible, but not that it measure all aspects, regardless of significance, in their exact proportions." Id.
Applying that standard to the above findings of fact, the Court rejects Plaintiffs, argument that the test was not representative. First, Plaintiffs complain that the test did not directly measure the ability to write clearly and concisely. However, the evidence shows that such a test would be problematic because essay and short answer tests tend to have (1) low reliability, which precludes validity, and (2) constraints in terms of time and expense and difficulty of grading. Also, research indicates that whether an open-ended or multiple-choice test format is utilized has little impact upon total test scores. There is no evidence that minorities do better on constructive response tests. Def.'s Ex. D, pp. 36-39. Thus, the Court does not conclude that it was needlessly omitted.
Second, Plaintiffs point out that listening comprehension is not directly measured by the test and contest Defendant's response that the test indirectly measured it by testing reading comprehension. However, although Plaintiffs, [45] expert testified that different parts of the brain are involved in understanding reading comprehension than in understanding oral communication, the expert further explained that the brain works differently when it is required to "read a selection and then answer questions to see if they can identify material from it, . . a passive receptive language process" Compared to when "you want to see whether or not that person can read that selection and then articulate and describe adequately to someone else what it is they understand." (Tr. at 1053). The latter is an expressive language process. (Tr. at 1051). Interestingly, this explanation does not address listening comprehension and, thus, is somewhat contradictory of the testimony that it seeks to explain. Rather, this testimony is merely evidence that the in- basket component of the exam did not test the same brain process as the oral component. Moreover, on voir dire the expert testified that a test where one speaks to a candidate and then asks her to respond measures "whether or not people are able to express themselves clearly, if what they seem to understand in the abstract or in the passive sense they can also express in a more active sense." (Tr. at 1051). This seems to describe-the oral component of the test equally well. However, the expert explicitly opined that a test of reading comprehension does not measure the ability to understand oral communication. In sum, because of the inconsistencies in the expert's testimony and the extensive evidence presented that supports the content validity of the selection procedure, the Plaintiffs have demonstrated only [46] a mere scintilla of evidence that they will ultimately succeed on the merits based upon this argument.
Finally, Plaintiffs contend that the test is not representative because the job analysis excluded additional duties that were added to the job of sergeant pursuant to the CAPS program in April of 1993. However, the Court concludes that the duties imposed by CAPS involve the skills and abilities encompassed by the In-Basket and Oral components of the exam. Further, the evidence shows that the CAPS program heightened the criticality of the existing duties, rather than adding duties. Thus, the job analysis did not exclude these duties.
Selection procedures which purport to measure knowledges, skills, or abilities may in certain [47] circumstances be justified by content validity, although they may not be representative samples, if the knowledge, skill, or ability measured by the selection procedure can be operationally defined as provided in section 14C(4) below, and if that knowledge, skill, or ability is a necessary prerequisite to successful job performance.The Guidelines further explain that a knowledge "should be operationally defined as that body of learned information which is used in and is a necessary prerequisite for observable aspects of work behavior of the job." S 1607.14(C)(4). Curiously, that Section contradicts 14(C)(l)ls apparent modification of representativeness when it goes on to require that:
For any selection procedure measuring a knowledge, skill, or ability the user should show that (a) the selection procedure measures and is a representative sample of that knowledge, skill, or ability; and (b) that knowledge, skill, or ability is used in and is a necessary prerequisite to performance of critical or important work behaviors).Thus, an inconsistency appears because, whereas the presence of (b) obviates the need for (a) pursuant to 14(C)(1), 14(C)(4) requires both, unless the Guidelines distinguish between "successful job performance," 14(C)(1), and "performance of critical or important work behavior(s)," 14(C)(4)(b). Yet it is difficult to conceive of a meaningful distinction between critical work behavior and successful job performance. The Court turns to the "Adoption of Questions and Answers to Clarify and Provide a Common Interpretation of the Uniform Guidelines on Employee Selection Procedures." 44 Fed. Reg. 11996 et sect. (1978)("EEOC Questions and Answers"). The answer to Question 79 [48] -- "What is required to show the content validity of a test of a job knowledge?" -- states:
There must be a defined, well recognized body of information, and knowledge of the information must be prerequisite to performance of the required work behaviors. The work behaviors to which each knowledge is related should be identified on an item by item basis. The test should fairly sample the information that is actually used by the employee on the job, so that the level of difficulty of the test items should correspond to the level of difficulty of the knowledge as used in the work behavior. See Section 14C(1) and (4).44 Fed. Reg. 12007. Accordingly, the Court agrees with Plaintiffs that a job knowledge test should sample the knowledge necessary to performance of the job. Of course, this requirement is more generally discussed above in the context of Gillespie, 771 F.2d 1035 (7th Cir. 1985) and Guardians, 630 F.2d 79 (2d Cir. 1980).
Nevertheless, the Court concludes that the written job knowledge component of the test was representative, because despite the testimony of Plaintiffs' witnesses that the knowledges tested for are not important, the Court accords more weight to the testimony of Defendant's witnesses, a Commander and Lieutenant who both have sergeant's experience, that each of the questions under attack relates to important knowledge. Further, although Plaintiffs provided evidence that two of the questions might have more than one correct answer, there is no evidence that two problematic questions can invalidate a 150-item test. Rather, the evidence is that responses to an aggregation of items [49] on a test, as opposed to the response to a single item, are significant.
Defendant has introduced evidence of a job analysis conducted in accordance with the Guidelines; Plaintiffs argue that the chart linking each question to a ma.jor work behavior is insufficient because it links an 'overwhelming number of questions to three broad work behaviors. However, Defendant's "Master Job Description and KSA's for Chicago Police Sergeant," (Ex. D, App. C) sets out the frequency of the major work behaviors and their associated knowledges. The evidence of frequency and associations is undisputed.
Applying the above standard to the findings of facts, the Court concludes that both the in-basket and the oral components of the exam measured observable characteristics in a form that simulated actual work behavior, rendering the content valid strategy appropriate. Moreover, the Court concludes that Plaintiffs do not have a likelihood of success on this issue, because the evidence shows that these components are content valid.
If a user can show, by a job analysis or otherwise, that a higher score on a content valid selection procedure is likely to result in better job performance, the results may be used to rank persons who score above minimum levels. Where a selection procedure supported solely or primarily by content validity is used to rank job candidates, the selection procedure should measure those aspects of performance which differentiate among levels of job performance.29 C.F.R. S 1607.14(C)(9)(emphasis added).
The issue is further addressed in the EEOC Questions and Answers, which explains that [u]se of a selection procedure [52] on a ranking basis may be supported by content validity if there is evidence from job analysis or other empirical data that what is measured by the selection procedure is associated with differences in levels of job performance." -44 Fed. Reg. 11996, 12005 (1978) (Question 62)(emphasis added). Further, to "justify use of (a job knowledge] test for ranking, it would also have to be demonstrated from empirical evidence either that mastery of more difficult work behaviors, or that mastery of a greater scope of knowledge corresponds to a greater scope of important work behaviors." Id. However, this form of "empirical evidence" is distinguishable from "empirical data demonstrating that the selection procedure is predictive of or significantly correlated with important elements of job performance," which applies to criterion-related validity. S 1607.5(B).
The Guidelines, by their plain language and as interpreted, contemplate that a job analysis may provide the evidence to support rank ordering, as opposed to the "mathematical support for the proposition" which is required when ranking is supported by criterion-related and construct validity studies. Id. In other words, the Court declines Plaintiffs' invitation to engraft a requirement that a user provide mathematical support in the nature of criterion-related validity when rank ordering based upon a content validity strategy.
This conclusion is supported by the seminal case of Guardians Assoc. v. Civil Service Commission, 630 F.2d 79, 100 (2d Cir. 1980), where the Second Circuit analyzed the propriety [53] of rank ordering police officer candidates based upon a content validity strategy and concluded that the Guidelines, as interpreted, do not require rank ordering to be justified with a criterion-related study. Rather, the Court-extensively delineated the ways to justify rank ordering with a content validity strategy: (1) a job analysis so that the test is constructed with a high degree of adherence to the Guidelines (i.e., "there must be a substantial demonstration of job relatedness and representativeness to show a sound basis for making rank-ordering hiring decisions") and (2) achieving "an adequate degree of reliability" and "using questions that are shown to have significant differentiating power, so that scores are not bunched at the high end of the scale." Id. at 100-04. hts v. City of Columbus, See also Police Officers for Egual Rights v. City of Columbus, 916 F.2d 1092, 1101-03 (6th Cir. 1990)(upholding rank ordering upon such a showing); Nash v. Consolidated City of Jacksonville, 895 F. Supp. 1536, 1549-52 (M.D. Fla. 1995)(same); Cuesta v. State of New York, 708 F. Supp. 583, 586 (S.D.N.Y 1989)(same). Further, contrary to Plaintiffs, argument, Evans v. City of Evanston, 881 F.2d 382, 384 (7th Cir. 1989) does not hold otherwise.
Applying this standard to the above findings of fact, the Court concludes that Plaintiffs do not have a likelihood of success on the merits as to the propriety of using the test scores to rank order candidates. Specifically, Defendants have constructed a test that is content valid and highly reliable, [54] with a good range of scores, thus demonstrating that the test measures those aspects of performance which differentiate among levels of job performance. Furthermore, Defendant has presented convincing evidence, for purposes of a preliminary injunction hearing, that mastery of a greater scope of knowledge corresponds to a greater scope of work behaviors.
Further, assuming arguendo thaL delay in promotion does constitute irreparable harm, the harm to the public and Defendant so outweighs the harm to Plaintiffs that, pursuant to the sliding-scale nature of the balance, Plaintiff must demonstrate a proportionately stronger likelihood of success on the merits in order to obtain a preliminary injunction. The Court concludes that Plaintiffs have not made such a showing.