guibranco/BancosBrasileiros

[FEATURE] Grab taxes and levies from BCB for all banks

guibranco opened this issue · 2 comments

guibranco commented

Is your feature request related to a problem? Please describe.
Grab taxes and levies from the BCB page for all banks

Describe the solution you'd like
BCB publishes the taxes/levies for each bank based on ISPB at this page:

The 00000000 is the ISPB
The a after the ISPB in the corporate URL is how it identifies a corporate instead of a personal.

Call this page for each bank with ISPB and extract the information to a JSON file separated of banks files.

Add data from SFA - Open Finance

Documenation of the file format.

Contributing

Here is a step-by-step on how to add a new source of data to the merge tool:

  • Check out MergeTool.
  • Open VS, VS Code, Rider, or your favorite IDE / Code Editor for .NET projects.
  • Load the project (currently, it is in C# .NET 8.0).
  • Add the required URLs to the Constants.cs file.
  • Add a new enum item in the Source.cs file. Please use the source system acronyms whenever possible.
  • Add a new method in the Reader.cs called Load[NewSystemAcronym]. This should do all the heavy job of grabbing the information from the remote source.
  • Follow the other methods patterns to extract the PDF information from the file if the new source provides a PDF file.
  • Implements the data extraction the way you prefer if it is not a PDF file.
  • Add the RegExp to the Patterns.cs file if you need to use RegExp to extract data.
  • Add the new field(s) to the Bank.cs file.
  • In the Seeder.cs file, implement the method Merge[NewSystemAcronym] to merge the new data with the existing ones. I prefer to filter the data by ISPB and then Document to check for existing data. Rely on the existing list, DO NOT ADD new bank to the list if it is not present with COMPE, ISPB, Document, and Name at least. These are mandatory fields, if you have all this information, and you did not find the bank on the existing list, feel free to add it to the list. (Let me know this in the PR comment).
  • Call the new methods (Load[NewSystemAcronym]r and Merge[NewSystemAcronym]) in the AcquireData method inside the Program.cs file.
  • On the Writer.cs file, edit the following methods, mapping the new field(s):
    • SaveCsv
    • SaveMarkdown
    • SaveSql
  • Test it 🧪
  • Commit and submit a PR 🎉

Testing

  • You can run the application locally without submitting any changes to this repository.
  • Run how many times you need, it will only generate some files in the output directory inside result directory.
MauriciDmarc commented

corporate.txt
personal.txt

only need convert to json

  • 👍1
guibranco commented

Oh thanks, @MauriciDmarc but the idea is to have this inside the application, running on the daily basis, so it will be always updated, and also, in JSON/CSV/XML/etc formats like the bank's data. But again, thanks for the effort!