Sunlight by Request and by Default: How FOIA and Open Data Keep Government Accountable

Accountability rarely happens by accident. It usually depends on two mechanisms working in tandem: a legal right to ask for…
1 Min Read 0 1

Accountability rarely happens by accident. It usually depends on two mechanisms working in tandem: a legal right to ask for records that agencies would rather keep in a filing cabinet, and a growing expectation that ordinary government information should be published before anyone asks. The first is the Freedom of Information Act. The second is the open data movement. Together, they give journalists, researchers, auditors, advocates, and ordinary residents a practical way to check what government actually did with public money and public authority.

Neither mechanism is self-executing. A request can be delayed, narrowed, or partly denied. A dataset can be published and still mislead if the definitions behind it are ignored. The most useful accountability work happens when people understand how each channel operates, what it can and cannot reveal, and where to go when a record is missing.

FOIA: The Right to Ask, and the Duty to Answer

The Freedom of Information Act, enacted in 1966 and codified at 5 U.S.C. § 552, established a presumption that records held by federal executive branch agencies are available to the public unless a specific exemption applies. Before FOIA, access to agency files generally depended on the agency’s discretion or on being a party to a lawsuit. After FOIA, any person — citizen or not, journalist or not — could request records without explaining why they wanted them. The statute has been amended repeatedly since then, including the FOIA Improvement Act, which reinforced the idea that agencies should release information proactively and should not withhold material unless they can identify a foreseeable harm from disclosure.

In practice, FOIA is a procedural right, and procedure matters. A requester must describe records with enough specificity that an agency can locate them; agencies are not obligated to answer questions, write new analyses, or assemble data that does not already exist in some form. Requests are typically handled by a dedicated FOIA office, assigned a tracking number, and answered within a statutory response window measured in working days — though complex requests and backlogged offices often take considerably longer. The statute provides nine categories of exemption, covering areas such as classified national security information, personal privacy, law enforcement sensitivities, and internal deliberative discussions. Those exemptions are generally discretionary in structure but constrained by law and policy, and agencies are expected to release reasonably segregable non-exempt portions of a record rather than withholding a document wholesale.

Requesters can strengthen their position by narrowing scope, identifying the specific office or program involved, and asking for records in a preferred format. Fee categories — commercial, educational, news media, and “other” — determine what an agency may charge, and public interest fee waivers are available when disclosure would meaningfully inform the public about government operations. When a request is denied in whole or in part, the administrative appeal goes back to the agency, and litigation in federal court remains the ultimate backstop. The Office of Government Information Services at the National Archives serves as a mediator and ombudsman for disputes.

It is also worth remembering that FOIA applies only to federal agencies. States, counties, cities, school districts, and police departments operate under their own public records and sunshine laws, which vary widely in scope, deadlines, and enforcement. Anyone investigating local spending, local policing, or a school board decision should identify the correct state statute first.

Open Data: Publishing Records Before Anyone Asks

FOIA is reactive. Open data is proactive. Rather than waiting for individual requests, agencies publish structured datasets that anyone can download, query, and combine. This shift was reinforced by federal legislation and policy, including the DATA Act, which standardized how federal spending is reported, and the Foundations for Evidence-Based Policymaking Act and related open government data provisions, which pushed agencies to catalog their data assets and release them in usable formats. The practical result is that many of the records people used to request one at a time are now available in bulk.

Several sources anchor this ecosystem. USAspending.gov publishes federal contracts, grants, and loans, with recipient names, amounts, and awarding agencies. The Census Bureau releases decennial counts, the American Community Survey, and economic data, much of it through an API and bulk files. The Federal Register documents rules and notices as they happen. CMS publishes Medicare Advantage and Part D plan information, including enrollment and plan-level details that change by county and year. The FBI Crime Data Explorer distributes Uniform Crime Reporting and NIBRS data on offenses and arrests. The IRS makes nonprofit Form 990 filings available, and the SEC maintains EDGAR for corporate filings. Many states and cities now run their own open data portals as well.

Aggregators add a layer of convenience by reorganizing these official releases into browsable pages — sorted by state, county, or city, with methodology notes and pointers back to the original publication. For a resident trying to understand a local contract, a county’s crime trends, or a hospital’s Medicare plan options, that kind of navigation makes it far easier to explore government transparency data without losing the trail back to the primary source. The important habit is to follow that trail: any secondary compilation should be checked against the agency’s own documentation, release date, and definitions.

Open data also changes the economics of accountability. A single FOIA request produces a record for one requester, often after months. A published dataset informs everyone at once, supports replication, and lets independent analysts check each other’s work. The limitation is that agencies choose what to publish and how to structure it, so the availability of a dataset is not the same as the availability of the underlying record.

Reading the Numbers: Gaps, Definitions, and Responsible Interpretation

Transparency delivers accountability only when the numbers are read carefully. Every dataset carries assumptions. Spending records distinguish between obligations and outlays, between prime awards and subawards, and between initial amounts and later deobligations. A contract listed at a headline figure may represent a ceiling rather than money actually disbursed. Crime statistics present similar traps. The FBI’s Uniform Crime Reporting program and the newer National Incident-Based Reporting System depend on voluntary submissions from thousands of law enforcement agencies, and reporting coverage is uneven. The transition from UCR’s summary format to NIBRS changed what gets recorded and how categories align, which means year-over-year comparisons can reflect a change in reporting method rather than a change in underlying conditions.

Crime data also requires particular care. Arrest data reflects law enforcement activity, not the underlying rate of offending, and arrest patterns are shaped by policing priorities, deployment, and community reporting behavior. Offense data reflects incidents that were reported to and recorded by police, which means underreporting and recording practices matter. Race fields are frequently incomplete or marked unknown, and where race is recorded, the data documents who was arrested or described as an offender — not why an incident occurred. Socioeconomic conditions, geography, and enforcement patterns all influence what appears in the tables. Responsible use means describing what the records measure, noting their limits, and resisting the temptation to imply causation that the data cannot support.

Similar cautions apply across other sources. Medicare Advantage and Part D plan availability and premiums vary by state, county, and year, so a figure from one market does not generalize; readers should compare current CMS Plan Finder details during open enrollment. Unclaimed property is held by states under their own programs, and searching official state treasury or controller databases is free. Data on immigration courts, federal salaries, farm subsidies, and school finance each come with their own units, geographies, and suppression rules, including privacy protections that hide small counts.

A sound workflow is straightforward: identify the publishing agency, read the methodology and release notes, note whether figures are provisional or final, check the reporting period and geography, confirm whether counts and rates use appropriate population denominators, and compare against at least one independent source. Where published data falls short, FOIA and state public records requests fill the gap. Where requesters are stonewalled, published datasets often provide enough context to frame a sharper, more targeted follow-up. Transparency works best when the two channels are used together — and when readers treat every figure as a claim that deserves a citation.

WilmaVRanson