In Response to Police Misconduct, a Flourishing of Online Databases

On July 17, 2014, New York City police officer Daniel Pantaleo tried to arrest a man named Eric Garner on Staten Island for illegally selling cigarettes. Instead, Pantaleo held Garner in a chokehold and killed him. It took nearly three years for the public to learn that Pantaleo had a history of abuse complaints against him, thanks to a New York Police Department policy not to release officer’s past records. A staff member of New York City’s Civilian Complaint Review Board, or CCRB, ultimately leaked the information.

It wasn’t until last month that the city finally moved forward with a disciplinary hearing, after the board brought charges against the officer.

The Pantaleo case isn’t unusual: Across the country, nearly 1,000 people were shot and killed by police last year, as well as in 2016 and 2017 — a much higher rate than in other countries — yet only about 50 officers have seen criminal charges over a recent 10-year period. In response, transparency advocates from New York to Illinois to California are pushing to open up records about police misconduct. At the core of their movement, they say, is a conviction that the public has a right to this information, and that disclosure is an essential part of holding police accountable.

Public websites making police records accessible are central to the transparency movement, and over the past several years, nonprofit organizations, public defenders, academics, and journalists have mounted efforts like the Citizens Police Data Project, a large database of civilian complaints in Chicago, and CAPstat from the Cop Accountability Project, which contains data on federal civil rights lawsuits against police in New York City. Advocates in New York State, meanwhile, are pushing to repeal law 50-a, which allows police departments to shield misconduct records from disclosure. It is considered by many transparency advocates to be one of the most secretive such laws in the country.

“The records of any other public servant are available to the public” said Michael Sisitzky, the lead policy counsel at the New York Civil Liberties Union (NYCLU), in a promotional video produced by the NYCLU in 2018. This would include those of lawyers, doctors, teachers, and veterinarians, Sisitzky suggested. Sisitzky’s organization is among the groups pushing for 50-a’s repeal. “When police officers violate that public trust,” he said, “we need to know whether or not the police department leadership is taking measures to hold those officers accountable.”



Police departments and unions often resist sharing disciplinary records, arguing that doing so violates police privacy and could unfairly tarnish reputations. In response to a request for comment on CAPstat, the New York City Police Benevolent Association (PBA) directed Undark to a public statement from PBA president Patrick Lynch: “The intent of this database,” Lynch wrote, “is clearly to help guilty criminals beat the charges against them.”

Advocates, meanwhile, counter that the vast power police officers hold should be matched by a high level of transparency and accountability.

As it stands, officers like Pantaleo aren’t promptly charged or disciplined when they abuse their authority. While there are many reasons for failures of accountability — for example, review boards tasked with investigating complaints often lack independence from police departments and police officers often abide by a “code of silence” — Sisitzky and other transparency advocates say part of the problem is the lack of public information.

The new databases seek to fill that gap. Such transparency, says Yul-san Liem, co-director of Justice Committee, a New York nonprofit that works with families that have lost loved ones to police violence, is about wanting to understand “who is actually in our neighborhoods with guns.”


Some of the largest new police transparency efforts come from cities with the country’s biggest police forces. Increasingly, they are being used by the public, lawyers, and even the police themselves. In March, for example, the NYC-based nonprofit Legal Aid Society — a public defender organization — publicly released the CAPstat database, which compiles thousands of records about lawsuits alleging police misconduct from 2015 to 2018, and several years of New York City police disciplinary records that were leaked to BuzzFeed News. Cynthia Conti-Cook, a Legal Aid attorney, says the database started as an effort to provide the organization’s defense attorneys with information about officers who had previously lied as witnesses.

CAPstat’s lawsuit data reveal that some officers have been sued dozens of times. Strikingly, even in the three years following Garner’s death, dozens of officers were sued for using chokeholds — an act of force that the NYPD banned in 1993 — with most ending in a settlement, and little to no discipline for officers.

While civilian complaint boards, which investigate complaints against police, release some information about misconduct, the data are most often displayed as summaries and rarely include individual records. CAPstat goes a step further, allowing people who are affected by police violence to look up police officers by name. Having this information available is important, Conti-Cook says, in helping residents to know who is patrolling their communities.

CAPstat was inspired, Conti-Cook says, by another database: the Citizens Police Data Project (CPDP), launched by the Invisible Institute, a Chicago journalism nonprofit. According to Jamie Kalven, the institute’s executive director, the database grew out of work documenting civil rights abuse by police in Chicago. Kalven and his team sued the city for release of police misconduct data, and eventually got it in 2014.

The Invisible Institute launched the database in 2015, and later re-launched an expanded site in partnership with the investigative journalism website The Intercept in 2018. As of late May 2019, the database included nearly 250,000 complaints against more than 23,000 officers spanning three decades. Only 7 percent of the officers were disciplined; in cases involving the use of force, officers were less than half as likely to face any consequences. The data also help show that Chicago police have disproportionately used force on people of color.

So far, the databases have had a range of users. In some cases, investigators have used the information to conduct external reviews of police departments. For example, a 2017 Department of Justice investigation of the Chicago Police Department relied in part on the CPDP database, Kalven says. The DOJ report states that the department “tolerated racially discriminatory conduct that not only undermines police legitimacy, but also contributes to the pattern of unreasonable force.” Attorneys also use the information to learn of potentially abusive and unreliable officers, while academics who study police misconduct can use it for research, since the Invisible Institute makes all the relevant data and programming code available for download on Github.

Even individual police officers in Chicago are accessing the records. Rajiv Sinclair, the CPDP director, says police officers have anonymously told his team that they use the database “in order to avoid getting partnered up with someone who they don’t want to get,” out of concern that they will be caught up in misconduct.

But perhaps one of the most powerful uses comes from the public. The most high-profile example is the case of Jason Van Dyke, a Chicago officer who was captured on a police dashboard camera video in 2014, shooting 17-year-old Laquan McDonald 16 times as the young man walked away from the officer. Kalven wrote about the shooting and learned about the video, which contradicted the police department’s account of what happened. Along with University of Chicago law professor Craig Futterman, he urged the city to make the video available. When it was released in 2015, Kalven says, many people immediately looked up Van Dyke on the CPDP database, which had launched just weeks before. (Van Dyke had received nearly a dozen prior complaints relating to use of force. In 2018, he was convicted of second-degree murder in the McDonald case — the first Chicago police officer in nearly 50 years to be convicted of murder).

Kalven emphasizes that merely receiving complaints isn’t in itself a sign of a bad officer, and that it’s important to look at the details. In Van Dyke’s case, he sees evidence of warning signs that something was wrong. “Supervisors should have been intervening to see what was going on,” Kalven said. And when information exists that suggests bad behavior, he added, failure to act on it shows just “how dysfunctional the system is.”

Still, it’s not always that straightforward, and Sinclair noted that even some police supervisors don’t have access to this sort of personnel data on their underlings. “[They] told us that they couldn’t access this information about their own subordinates,” he said, “until they could access it through us.”

Transparency projects like these still face logistical barriers, of course. Lawsuit records of alleged police misconduct are public, but can be nonetheless difficult to access and compile. Legal Aid uses a federal court record site — the Public Access to Court Electronic Records, or PACER — to obtain information on such lawsuits. But PACER typically charges site users, which can be costly, and while Legal Aid enjoys a fee exemption, putting together CAPstat was still onerous and time-consuming: Legal Aid interns worked for several years to review the PACER site and manually extract information.

There are also technical hurdles. For instance, one way that the Invisible Institute finds data is by having its computer programmers write code to regularly search and pull information from relevant websites, a process called web scraping. But writing code for web scraping takes technical expertise and it is not a trivial task. Tracing out an officer’s disciplinary history can also be a challenge, Sinclair and Conti-Cook say, since it requires uniquely identifying officers over time (names are insufficient since some officers may share them). In Chicago, for example, the data processed by the Invisible Institute does not have one unique identification number for each officer — instead, the team has to combine different elements, such as names and a series of badge numbers, together.

Then there are the legal challenges. Police union contracts sometimes include language about shielding police misconduct from public disclosure, and unions have also successfully lobbied for state legislation to bar sharing disciplinary records. A 2015 WNYC investigation showed that in 23 states (plus Washington, D.C.), officer disciplinary histories are mostly confidential; in 12 they are generally made public; and the rest have limited public disclosure.

While the overall numbers remain more or less the same, there have been some changes since then. For example, in March, California passed a law under which some kinds of police misconduct records, including those that involve sexual assault and use of force, must be public.


For their part, police departments and their representatives often argue that releasing civilian complaints and police disciplinary records will unfairly tarnish officer reputations and potentially put them in danger.

“Anyone with a grudge against cops will be free to peruse the false and frivolous allegations against specific officers,” said Police Benevolent Association president Patrick Lynch in his statement on CAPstat, “and use them as inspiration for a campaign of harassment, intimidation or worse.”

The police aren’t alone in their objections. Some researchers have also argued against releasing individual police records, including legal scholar Kate Levine of Duke University. While Levine argues that police misconduct records can and should be shared with the defense in court cases, she sees issues with public sharing of individual records, which she says could violate officer privacy and cause unfair reputational damage. (Other academics disagree, including Rachel Moran of the University of St. Thomas School of Law, who points out that courts have generally found that the right to privacy involves “highly personal, intimate, or political information,” which she argues doesn’t cover allegations of police misconduct.)

Moran and Conti-Cook acknowledge there is room for debate. For instance, should all complaints be public? Or only those that led to discipline? Or only cases in which civilian complaint boards found an officer guilty? Moran writes that there a reason to prefer full transparency: Without seeing all allegations, the public “cannot fairly judge whether its accountability system is working.” Civilian complaint review boards — which don’t exist in every city — may have significant limitations: They are often ill-equipped for full investigations, and some boards have very high rates of complaints ending with no clear conclusion. In addition, nearly all of the boards lack independent authority. All they can do is recommend discipline, leaving it to police departments to enforce it.

In about half of the misconduct cases in New York over the past several years, the police department either imposed less discipline than the Civilian Complaint Review Board recommended or ignored the recommendations altogether.

Looking to the future, Conti-Cook and Kalven hope to expand their respective databases, and both have been in contact with organizations in other cities that may start police transparency initiatives. There are also similar projects underway, including a database from USA Today, to which the Invisible Institute contributed, which lists records of officers that have had their police certification removed; the Stanford Open Policing Project on traffic stops; and the Police Crime Database out of Bowling Green State University, which includes data on officers charged and convicted with crimes.

For families directly impacted by police violence, efforts to hold police accountable are personal — and a shift to greater transparency and accountability, they say, is taking far too long. The charges brought against Pantaleo in the Garner chokehold case may lead to a recommendation for discipline, but the police department will ultimately make the decision about Pantaleo’s fate. After the third day of the hearing, Gwen Carr, Garner’s mother, described how she had been pushing, along with many others, for the case against Pantaleo to move forward: “It is just heart-wrenching,” she said in a video posted to Facebook, “that five years have gone by and there’s nothing been done.”


Stephanie Wykstra is a freelance writer and researcher with a focus on transparency and criminal justice reform. Her work has recently appeared in Vox, Slate, and Stanford Social Innovation Review.

Stephanie Wykstra is a freelance writer and researcher with a focus on transparency and criminal justice reform.