A best practices guide for submitting spark applications, integration with hive metastore, security, storage options, debugging options and performance considerations.
Return to Live Docs.
The documentation is made available under the Creative Commons Attribution-ShareAlike 4.0 International License. See the LICENSE file.
The sample code within this documentation is made available under the MIT-0 license. See the LICENSE-SAMPLECODE file.
- Fork the repository
- Make your change and double-check the mkdocs.yml is updated accordingly.
- Install the MkDocs command tool and material theme if needed:
pip install mkdocs
pip install mkdocs-material # material theme- MkDocs comes with a built-in dev-server that lets you preview your documentation as you work on it. Make sure you're in the same directory as the
mkdocs.ymlconfiguration file, then run the command:
mkdocs serve- Open up http://127.0.0.1:8000/ in your browser, and you'll see the best practice website being displayed locally.
- Adjust your document changes in real time.
- When everything looks good and you're ready to deploy the change, run the command to build/compile the website content:
mkdocs build- This will refresh the directory
./site. Take a look inside the directory and make sure your changes are included.
ls site-
Commit change to github and send us a pull request.
-
With a repo admin permission, we can merge the pull request into the main branch.
-
Most importantly, as a repo admin, we must run the deploy command to copy the './site' content to 'gh-pages' branch and pushing to GitHub. Without this step, the website content won't be refreshed.
mkdocs gh-deploy