Create an equivalent of pandas LoadOptions for writing
ianbuss opened this issue · 0 comments
ianbuss commented
Please describe the feature you'd like to see
The serialization methods on Pandas DFs accept quite a few kwargs which we're not currently exposing. One example is the index=True|False
parameter which allows us to drop the DF index when writing to file.
Describe the solution you'd like
I'd like to see an equivalent, or something similar to, the https://astro-sdk-python.readthedocs.io/en/stable/autoapi/astro/dataframes/load_options/index.html called save_options
or export_options
which allows for fine tuning Pandas DF exports.
Acceptance Criteria
- All checks and tests in the CI should pass
- Unit tests (90% code coverage or more, once available)
- Integration tests (if the feature relates to a new database or external service)
- Example DAG
- Docstrings in reStructuredText for each of methods, classes, functions and module-level attributes (including Example DAG on how it should be used)
- Exception handling in case of errors
- Logging (are we exposing useful information to the user? e.g. source and destination)
- Improve the documentation (README, Sphinx, and any other relevant)
- How to use Guide for the feature (example)