/lambda_gdal-python

Create a Lambda Function with gdal-python

Primary LanguagePython

Lambda gdal-python

Create a python Lambda function with GDAL python bindings

Related to RemotePixel.ca blog

I assume you know how to start an EC2 virtual server (with AMAZON Linux AMI) and to run some basics shell commands

Update and Install Python
sudo yum update
sudo yum install python27-devel python27-pip gcc libjpeg-devel zlib-devel gcc-c++ python-devel libpng-devel freetype-devel libcurl-devel
Create a directory where to devellop function
mkdir /home/ec2-user/lambda
mkdir /home/ec2-user/lambda/local
cd /home/ec2-user/lambda
Create a virtualenv and activate it
virtualenv env
source env/bin/activate
Install some Python modules
pip --no-cache-dir install numpy
Download and Install GDAL
wget https://github.com/OSGeo/proj.4/archive/4.9.2.tar.gz
tar -zvxf 4.9.2.tar.gz
cd proj.4-4.9.2/
./configure --prefix=/home/ec2-user/lambda/local
make
make install

cd ..

wget http://download.osgeo.org/gdal/1.11.3/gdal-1.11.3.tar.gz
tar -xzvf gdal-1.11.3.tar.gz
cd gdal-1.11.3
./configure --prefix=/home/ec2-user/lambda/local  \
    --with-geos=/home/ec2-user/lambda/local/bin/geos-config  \
    --with-static-proj4=/home/ec2-user/lambda/local \
    --with-curl \
    --with-python
make
make install
cd ..
Create you python function
vi /home/ec2-user/lambda/mylambdafunction.py
Edit your function

In order to get osgeo.gdal to load you new to set the GDAL_DATA path in the environment variable and load some shared libraries

#!/usr/bin/env python
# -*- coding: utf-8 -*-
import os
from ctypes import cdll

#Set environment variables and load shared libraries
path = os.path.dirname(os.path.realpath(__file__))
os.environ['GDAL_DATA'] = os.path.join(path, "local/share/gdal")
lib2 = cdll.LoadLibrary(os.path.join(path, 'local/lib/libproj.so.9'))
lib1 = cdll.LoadLibrary(os.path.join(path, 'local/lib/libgdal.so'))

#Import gdal modules
from osgeo import gdal

def processing_func(event, context):
    "This is my worker function that responds to an API getaway call"
    
    #Whatever you want
    
    return results
Creating a Deployment Package

You have create a zip file with everything you need (shared libraries and python packages)

First we need to move the osgeo package into the site-packages root directory

cd $VIRTUAL_ENV/lib64/python2.7/site-packages/GDAL-1.11.3-py2.7-linux-x86_64.egg
mv o* ../
mv g* ../
cd $VIRTUAL_ENV/..

We can now zip everything

zip -9 lambda.zip mylambdafunction.py
zip -r9 lambda.zip local/lib/libgdal.so
zip -r9 lambda.zip local/lib/libproj.so.9
zip -r9 lambda.zip local/share
cd $VIRTUAL_ENV/lib/python2.7/site-packages
zip -r9 /home/ec2-user/lambda/lambda.zip *
cd $VIRTUAL_ENV/lib64/python2.7/site-packages
zip -r9 /home/ec2-user/lambda/lambda.zip *

We're done you can now upload your lambda.zip in AWS S3 and call your worker (mylambdafunction.processing_func) using API gateway for example)

More