Creating parquet conf table from a given csv file.
Clone the repo
Modify the local.setting.json
pointing to the storage account connection string and the container used.
The main project folder (<project_root>) can contain the following files:
- local.settings.json - Used to store app settings and connection strings when running locally. This file doesn't get published to Azure. To learn more, see local.settings.file.
- requirements.txt - Contains the list of Python packages the system installs when publishing to Azure.
- host.json - Contains global configuration options that affect all functions in a function app. This file does get published to Azure. Not all options are supported when running locally. To learn more, see host.json.
- .vscode/ - (Optional) Contains store VSCode configuration. To learn more, see VSCode setting.
- .venv/ - (Optional) Contains a Python virtual environment used by local development.
- Dockerfile - (Optional) Used when publishing your project in a custom container.
- tests/ - (Optional) Contains the test cases of your function app. For more information, see Unit Testing.
- .funcignore - (Optional) Declares files that shouldn't get published to Azure. Usually, this file contains .vscode/ to ignore your editor setting, .venv/ to ignore local Python virtual environment, tests/ to ignore test cases, and local.settings.json to prevent local app settings being published.
Each function has its own code file and binding configuration file (function.json).
Run the function locally by opening a terminal and running func host start
This is the URL:
Functions:
write_conf: [POST] http://localhost:7071/api/write_conf
Use your preffered plugin or application to POST a call:
POST http://localhost:7071/api/write_conf
{
"file_name": "my.parquet",
"file_partition": "d=4",
"config_file": "config_sample.csv"
}
The code will try and read the config_file
from the location specified in local.sttings.json
CONF_CONTAINER
and write it as parquet to the CONF_PARQUET/file_partition/file_name
under the same storage account.
-
To learn more about developing Azure Functions, please visit Azure Functions Developer Guide.
-
To learn more specific guidance on developing Azure Functions with Python, please visit Azure Functions Developer Python Guide.