A Salt formula to install hadoop
Currently supports HDFS and YARN
- xml_hadoop custom serializer
- Python lxml
- Serializer external modules patch - saltstack/salt#46435
- slsutil.merge_all function - saltstack/salt#47679
hadoop:
hdfs:
name_node:
target: 'hdfs01*'
data_node:
target: 'hdfs*'
yarn:
resource_manager:
target: 'yarn01*'
node_manager:
target: 'yarn*'
Passing the bootstrap pillar tells the NameNode and ResourceManager to store their endpoints in SDB and tells the NameNode to format HDFS.
WARNING: Passing this option will reformat HDFS which is a destructive operation. Only do this if you are comfortable losing data.
salt '<tgt>' state.apply hadoop pillar='{"bootstrap": "True"}'
- hadoop
- Install Hadoop YARN and HDFS
- hadoop.install
- Install user and packages
- hadoop.hdfs
- Install Hadoop HDFS
- hadoop.hdfs.configure
- Manage hdfs-site.xml
- hadoop.hdfs.name_node
- Install Hadoop HDFS NameNode
- hadoop.hdfs.name_node.install
- Manage service and environment file, and format HDFS
- hadoop.hdfs.name_node.service
- Start the NameNode service
- hadoop.hdfs.name_node.store_name_node_endpoint
- Store the NameNode endpoint in SDB
- hadoop.hdfs.data_node
- Install Hadoop HDFS DataNode
- hadoop.hdfs.data_node.install
- Manage service and environment file
- hadoop.hdfs.data_node.service
- Start the NameNode service
- hadoop.yarn
- Install Hadoop YARN
- hadoop.yarn.configure
- Manage yarn-site.xml
- hadoop.yarn.resource_manager
- Install Hadoop YARN ResourceManager
- hadoop.yarn.resource_manager.install
- Manage service and environment file
- hadoop.yarn.resource_manager.service
- Start the ResourceManager service
- hadoop.yarn.resource_manager.store_resource_manager_endpoint
- Store the ResourceManager endpoint in SDB
- hadoop.yarn.node_manager
- Install Hadoop YARN NodeManager
- hadoop.yarn.node_manager.install
- Manage service and environment file
- hadoop.yarn.node_manager.service
- Start the NodeManager service