Events set in motion by a recent string of high-profile cases regarding disclosure of foreign influence and lobbying efforts have heightened interest among government officials, the media and the public. With the once obscure disclosure rule known as the Foreign Agent Registration Act (FARA) implicating new faces in the Trump administration and lobbying stalwarts alike, activities of lobbyists and other operatives taking millions of dollars from foreign interests to conduct influence operations in the United States have catapulted into the national spotlight. The breadth and scale of foreign influence operations in the United States make it more vital than ever to expose the connections between money, governance and policy.
Foreign Lobby Watch is a searchable database of foreign lobbying and influence spending launched by Center for Responsive Politics (CRP) in August 2018, the first time foreign lobbying and influence spending totals have been released for 2017 through the most recent business day and the only publicly accessible full-text search tool containing decades of FARA filings.
Created through the analysis of data extracted from records on file with the U.S. Department of Justice (DOJ), the Foreign Lobby Watch tool lets users conduct keyword searches on standardized, value-added data by coding lobbyist, firm and country along with visualizations like mapping to enhance usability.
The tool allows users to quickly learn which foreign interests are seeking to influence U.S. policy, and how much they are spending to accomplish this goal.
What makes this project innovative?
CRP’s Foreign Lobby Watch database is the first online tool that makes foreign influence spending totals for 2017 through the most recent business day and the full-text of FARA filings dating back to the 1940s available to the public This project extracted financial data from more than a hundred thousand records that were nominally available to the public into an easy-to-use searchable database. The launch of Foreign Lobby Watch was the first time foreign influence spending totals for 2017 and 2018 were released. Since there is no other existing database of all recent foreign influence spending numbers and DOJ’s FARA data is in a non-searchable format that requires laborious review, CRP has collected FARA records and data directly from DOJ daily, manually standardizing and coding the raw information to make the data more meaningful. The breadth of Foreign Lobby Watch content developed by CRP includes traditional data journalism alongside interactives, search tools, mapping and other resources to keep the public better informed about foreign influence on U.S. policy, with access to the tools needed to hold government accountable.
What was the impact of your project? How did you measure it?
Since the project’s launch, CRP has extracted text and analyzed FARA data from more than 120,000 documents with spending totals tallying over $882 million now searchable using the Foreign Lobby Watch tool. More than 2,000 receipt records were analyzed and their spending totals were entered into the tool during the first three months of 2019. In addition to our own original reporting, Foreign Lobby Watch’s FARA data has been featured in major articles from The Washington Post to the Economist and in broadcasts from the BBC and NPR to C-SPAN. CRP routinely provides guidance on best practices in analyzing FARA data and using Foreign Lobby Watch with journalists, academics, and other users contacting CRP multiple times each week. From the Foreign Lobby Watch launch to the end of 2018, CRP assisted more than 50 journalists with inquiries about Foreign Lobby Watch. CRP held a launch program in September 2018, with a demonstration and an expert panel on the history, abuses and proposed reforms of FARA, now in its 80th year. The event was attended by journalists and live-streamed on YouTube for wider distribution. Following the event, CRP created a digital guide to using the FLW tool and an explainer of the demonstration for the website to better enable users to navigate the data on their own. Since that time, CRP has also been invited to participate in a variety of external events that featured the Foreign Lobby Watch tool.
Source and methodology
With no existing database of recent foreign influence and lobbying spending numbers, CRP collects Foreign Agents Registration Act records and data directly from U.S. Justice Department (DOJ) daily then standardizes and codes the raw information. DOJ data is in a non-searchable format and requires laborious review. CRP launched a beta version of Foreign Lobby Watch in April 2017 before launching a more dynamic public-facing tool with spending totals and keyword searches on standardized, value-added data in August 2018. Since then, CRP has extracted and converted the data into a useful format and built and improved a new data entry tool to digitize and make essential information more accessible for journalists, scholars, and the public to review. This allows users to conduct keyword searches on standardized, value-added data by lobbyist, industry, firm and country and provides visualizations like mapping to enhance usability. CRP collects all records made available from DOJ’s FARA unit to provide links to filings and metadata showing who spent money to influence U.S. policy or public opinion. New filings are collected daily, organization names are standardized and financial receipts reported in the Supplemental Statements are entered into a database that allows calculations of a foreign principal's spending based on their country of origin, the agents they hire in the U.S. to work on their behalf and their total spending. Receipts data is kept up-to-date for payments reported from January 1, 2017 to the most recent business day and CRP is currently working to expand the database to prior years. CRP developed a categorization system to classify spending by year based on the reported dates of monetary receipts in supplemental statements rather than the summary amounts included in the semi-annual reports to Congress since reported dates of receipt sometimes occur in a different year than the work happened. When no dates were provided, the end of the six month period covered by the filing was used. CRP uses information from the filings themselves which identifies the level of government control over a foreign principal. This allows us to fairly and consistently apply the distinction between money spent on behalf of government and non-government sources.
Data visualizations and analytics for the reporting portion of this project were primarily created using SQL, Microsoft Access, Microsoft Excel, and Flourish. CRP created a custom data entry tool to extract and digitize “amount” field to make data more accessible and accurate through a verification process. CRP has developed backend software that queries FARA data from DOJ every hour, looks for new reports, downloads those reports, parses the data, puts it in internal database, and sends alerts when there is a new filing to manually parse into the database.
Dan Auble Anna Massoglia The Center for Responsive Politics team