spatial-framework-for-hadoop

The Spatial Framework for Hadoop allows developers and data scientists to use the Hadoop data processing system for spatial data analysis.

For tools, samples, and tutorials that use this framework, head over to GIS Tools for Hadoop.

What's New

Hive Date type column is now supported in the JSON SerDe classes (as of v2.0.0 release).
Spatial Framework for Hadoop v2.0 is compatible with Geometry v2.0, Hive v2.3, and Jackson v2. Note: up-to-date releases may be available on Github but not on Maven Central.
Support for Hive-0.10 is dropped in v2.0, while adding support for Hive-2.3; Hive-0.11 should work at runtime but Hive-0.12+ is required for building from source.

Features

JSON Utilities - Utilities for interacting with JSON exported from ArcGIS
Javadoc
Hive Spatial - User-Defined Functions and SerDes for spatial analysis in Hive
UDF Documentation
JSON SerDe

Getting Started

Maven

Build as you would any other Mavenized repository. All dependencies are pulled automatically.

Ant

Ant build files are also available

At the root level of this repository, you can build a single jar with everything in the framework using Apache Ant. Alternatively, you can build a jar at the root level of each framework component (i.e., hive/build.xml).

The build files use Maven Ant Tasks for dependency management. You will need the jar in a place Ant can find it (i.e., ~/.ant/lib/maven-ant-tasks-2.1.3.jar).

Dependencies

Esri Geometry API for Java - Java geometry library for spatial data processing.

Requirements

Geometry 2.0
Hive 0.11 and above (see Hive Compatibility issues) (For building from source, Hive-0.12+ is required.)
Workflows calling MapReduce jobs require the location of the custom job to be run.
Custom MapReduce jobs that use the Esri Geometry API require that the developer has authored the job, (referencing the com.esri.geometry.* classes), and deployed the job Jar file to the Hadoop system, prior to the ArcGIS user submitting the workflow file.

Resources

Issues

Find a bug or want to request a new feature? Please let us know by submitting an issue.

Contributing

Esri welcomes contributions from anyone and everyone. Please see our guidelines for contributing

Licensing

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

A copy of the license is available in the repository's license.txt file.

gzqlovemxq/spatial-framework-for-hadoop