Missing spending for DoD and others; inaccurate percentages of total
Closed this issue · 3 comments
Congrats on the new version of the site! I know a lot of great and hard work has gone into it!
There are a few key issues with the summary data on the site:
- Inaccurate number given for DoD / most DoD spending is missing (it claims DoD is <0.01% of total spending)
- Missing spending for CIA, ODNI, and perhaps other agencies
- Inaccurate percentages given for “percent of total spending” for all the other agencies due to the above numbers’ absence from the denominator for federal spending
Here's why these issues are very problematic:
- This site is the authoritative site for federal spending. When journalists need numbers to site, this is the place to go to. Yet it currently overstates the percent of total spending for all federal agencies besides DoD and overstates the percent of spending for all budget functions except National Defense (while greatly understating those figures). This means that journalists might then broadcast inaccurate numbers. Federal spending is a highly politically charged issue, so it's crucially important that these summary percentages be right; I'd say this is a case where no percentages would be better than incorrect percentages.
- The site's statement that the DoD accounts for "Less than 0.01%" of total spending is very obviously wrong. This could lead viewers to then doubt the accuracy of the rest of the numbers on the site. I really don't want people going around saying "USASpending.gov is fake news." The true government skeptics out there might even say "the feds are trying to deceive me!!" (I know that you're not, of course.)
Until DoD (and CIA? and ODNI? and anyone else who’s missing) data is added, I would urge the following, in decreasing order of importance:
-
Putting a very prominent disclaimer at the top of the “spending by agency” and “spending by budget function” pages that says what data is missing. It doesn’t have to be pretty. An ugly disclaimer is much, much better than no disclaimer! Also, the DoD’s agency page and National Defense’s budget-function page should have this disclaimer too. Example:
-
A "missing data" page listing what agencies/types of spending are omitted from the current site, why that is the case, when that’s expected to be resolved, and (ideally) approximate numbers on how much those agencies spend. And link to this page from the disclaimer described in point 1 (above).
-
Either take away the “percent of total” percentages altogether (since they’re all incorrect), or have an asterisk on all of them indicating that they are all (except DoD) inflated due to the denominator being significantly smaller than it should be.
-
Remove the visualization (https://www.usaspending.gov/#/explorer/agency). The visualization style that was chosen is one whose specific purpose is to show how much of a whole is taken up by different parts. It’s fairly problematic to use that kind of a viz if at least two big chunks (DoD, CIA) are missing. If you don’t want to take it down, you could at least have the disclaimer in point 1 (above) and replace the percentages with dollar amounts.
-
Mention the missing data and link to the page from point 2 (above) on the About page in the Data Quality or Data Sources section: https://www.usaspending.gov/#/about.
You all have put so much hard work into this site and I truly appreciate it!! 😄 Making this site was a very complex task. And I otherwise like the site. I really don’t want all that work to be tarnished by the above issues. These issues can easily be addressed before you get the missing data.
I should also add — I am aware of the reason that DoD data is incomplete/mostly missing: https://usaspending-help.zendesk.com/hc/en-us/articles/360001249973-Spending-Explorer-Question-I-m-looking-for-data-from-the-Department-of-Defense-DoD-and-can-t-find-it-is-it-shown-here-
But:
- There's no disclaimer of the summary visualization and table
- There's no mention of this on the About page's Data Sources and Data Quality sections
- There's no mention of this on the FAQ about "Is there data or information I won’t find on the site?": https://usaspending-help.zendesk.com/hc/en-us/articles/360001410594-General-Question-Is-there-data-or-information-I-won-t-find-on-the-site-
- This is the 16th FAQ
The answer is totally legitimate and understandable! But I think it's fair to say that most users will not find this. Hence why I made the content/feature suggestions that I did.
Also, I'm still confused about why the CIA and ODNI appear to be omitted, as I haven't come across an explanation yet.
Again, I appreciate all the hard work that has gone into this site! I want it to be the best it can be, and also am deeply concerned about the figures that are effectively misleading.
Hi Greg.
Thank you for the feedback. We've gotten similar feedback from a few other users and are working to make the disclaimer more prominent and are considering some of your other suggestions.
Lisa