Airpal is a web-based, query execution tool which leverages Facebook's PrestoDB to make authoring queries and retrieving results simple for users. Airpal provides the ability to find tables, see metadata, browse sample rows, write and edit queries, then submit queries all in a web interface. Once queries are running, users can track query progress and when finished, get the results back through the browser as a CSV (download it or share it with friends). The results of a query can be used to generate a new Hive table for subsequent analysis, and Airpal maintains a searchable history of all queries run within the tool.
- Optional Access Control
- Syntax highlighting
- Results exported to a CSV for download or a Hive table
- Query history for self and others
- Saved queries
- Table finder to search for appropriate tables
- Table explorer to visualize schema of table and first 1000 rows
- Java 7 or higher
- MySQL database
- Presto 0.77 or higher
- S3 bucket (to store CSVs)
- Gradle 2.2 or higher
-
Build Airpal
We'll be using Gradle to build the back-end Java code and a Node.js-based build pipeline (Browserify and Gulp) to build the front-end Javascript code.
If you have
node
andnpm
installed locally, and wish to use them, simply run:./gradlew clean shadowJar -Dairpal.useLocalNode
Otherwise,
node
andnpm
will be automatically downloaded for you by running:./gradlew clean shadowJar
Specify Presto version by
-Dairpal.prestoVersion
:./gradlew -Dairpal.prestoVersion=0.85 clean shadowJar
-
Create a MySQL database for Airpal. We recommend you call it
airpal
and will assume that for future steps. -
Create a
reference.yml
file to store your configuration options.Start by copying over the example configuration,
reference.example.yml
.cp reference.example.yml reference.yml
Then edit it to specify your MySQL credentials, and your S3 credentials if using S3 as a storage layer (Airpal defaults to local file storage, for demonstration purposes).
-
Migrate your database.
java -Duser.timezone=UTC \ -cp build/libs/airpal-*-all.jar com.airbnb.airpal.AirpalApplication db migrate reference.yml
-
Run Airpal.
java -server \ -Duser.timezone=UTC \ -cp build/libs/airpal-*-all.jar com.airbnb.airpal.AirpalApplication server reference.yml
-
Visit Airpal. Assuming you used the default settings in
reference.yml
you can now open http://localhost:8081 to use Airpal. Note that you might have to change the host, depending on where you deployed it.
Note: To override the configuration specified in reference.yml
, you may
specify certain settings on the command line in the traditional Dropwizard
fashion,
like so:
java -Ddw.prestoCoordinator=http://presto-coordinator-url.com \
-Ddw.s3AccessKey=$ACCESS_KEY \
-Ddw.s3SecretKey=$SECRET_KEY \
-Ddw.s3Bucket=airpal \
-Ddw.dataSourceFactory.url=jdbc:mysql://127.0.0.1:3306/airpal \
-Ddw.dataSourceFactory.user=airpal \
-Ddw.dataSourceFactory.password=$YOUR_PASSWORD \
-Duser.timezone=UTC \
-cp build/libs/airpal-*-all.jar db migrate reference.yml
Airpal Version | Presto Versions Tested |
---|---|
0.1 | 0.77, 0.87 |
- Andy Kramolisch @andykram
- Harry Shoff @hshoff
- Josh Perez @goatslacker
- Spike Brehm @spikebrehm
- Stefan Vermaas @stefanvermaas