With media outlets losing advertising revenues to digital platforms, some newsrooms no longer have the resources for investigative and public service reporting.

The inability of news outlets to perform this traditional “watchdog” role puts democracy at risk, said communication scholar James Hamilton , who is concerned about what could happen – or not happen – when there are no reporters covering substantive issues in their communities.

“Without investigative reporting, how will the public know what their elected officials are doing and which policies are working?” Hamilton asked.

Hamilton, who is also trained as an economist, estimated that it can cost newsrooms up to $300,000 and six months of a reporter’s time to do a deep dive into public interest issues like crime and corruption. In one case, it cost a newsroom $487,000 to produce an investigative series on local police shootings.

But many media outlets are losing money – and jobs. Newsroom employment dropped 45 percent between 2008 and 2017, according to a recent Pew survey .

And cutting jobs means cutting coverage of the stories that play a vital role for democracy, Hamilton said. “Important stories are going untold.”

The Stanford Journalism and Democracy Initiative (JDI), a cross-departmental collaboration, could change that.

The initiative’s goal is to develop technological and data-driven tools that will help journalists do high-quality public affairs reporting that could hold public officials and others in positions of power accountable.

“How do you lower the cost of discovering stories through better use of data? We want to give journalists the tools and the data to do it,” said Hamilton, who has joined together with computer science Professor Maneesh Agrawala ; Dawn Garcia , director of the John S. Knight Journalism Fellowships; and Krishna Bharat , a research scientist who created Google News.

“How can we take technology to understand what decisions are being made at an institution? At JDI, Stanford faculty and students are going to address these questions,” Hamilton said.

Societal impact of investigative news

The first project JDI is initiating is “Big Local News,” an effort to help local newsrooms with the data collection and analysis needed for investigative reporting.

Spearheading Big Local News is computational journalist and Stanford scholar Cheryl Phillips. It builds on her work with the Stanford Open Policing Project , which obtained more than 130 million state police traffic stops that were then made publicly available in a comprehensive database. The data-driven project led to dozens of news stories about racial disparities and bias in police stops.

“What data journalism allows us to do is go beyond the anecdote and provide evidence that something is happening, and we can show you what it is and why,” Phillips said.

But gathering and analyzing this type of data is a challenge, especially for local journalists who are already resource and time strapped, she said. That’s where Big Local News will step in.

Stanford students in Phillips’ fall course will work with local newsrooms to collect data needed for in-depth stories – such as gathering government records, in some cases by making Freedom of Information Act requests. The students will then transform that information into datasets that journalists can analyze. Students will also help journalists navigate the data.

Students will be working with a Bay Area newspaper on a story about housing issues in the region, as well as other news outlets on different topics.

“We are trying to take unstructured communication, turn it into structured data and tell you, this is how an institution is operating,” said Hamilton, adding that it will be an interdisciplinary effort. Students in the class have backgrounds in computer science, engineering and statistical analysis as well as media and communication.

The datasets Big Local News gathers will then be made available through the Stanford Libraries’ Digital Repository . Also included will be “story recipes,” guides that can show journalists how to analyze the information.

“We want to process data in a standardized way so a newsroom can just take the data, follow the story recipe and do the analysis,” Phillips said. “By helping do that, it makes it more likely that you are going to see that type of investigative, data-driven journalism done.”

Next steps

The Big Local News project is just the beginning for JDI.

In 2019, JDI plans to launch “Trust and Verify,” a project that will help news organizations and journalists defend against technology that can manipulate videos and photos.

For Hamilton, the unique collaboration across disciplines is the heart of JDI’s work. “It’s the combination of insights from social science, computer science and journalism that will ultimately provide reporters with new ways to discover and tell stories that hold institutions accountable,” he said.

Newsrooms interested in partnering should contact R.B. Brenner , managing director of the Journalism and Democracy Initiative, at rbbrenner@stanford.edu .

JDI is a joint effort between Stanford’s Journalism Program , John S. Knight Journalism Fellowships and the Brown Institute for Media Innovation . Funding for JDI was made possible from a gift from John and Laura Arnold. Big Local News also has received a Magic Grant from the Brown Institute, a collaboration between Stanford University’s School of Engineering and Columbia Journalism School.