Fast and cost effective geospatial analysis pipeline with AWS lambda

Nov 21, 20180 likes110 views

This document discusses using AWS Lambda for geospatial analysis. It provides examples of serverless configuration files that define Lambda functions with Python 2.7 and 3.6 runtimes. It also describes a Dockerfile and build process for packaging Python dependencies within Lambda deployment packages to reduce their size and optimize startup performance.

Fast and cost effective geospatial
analysis pipeline with AWS lambda
© Civic Vision UG

I help civic advocacy groups
© Civic Vision UG

Available packages
boto3
botocore
docutils
jmespath
pip
python-dateutil
s3transfer
setuptools
six
© Civic Vision UG

service: foss4g_oceania
provider:
name: aws
runtime: nodejs6.10
functions:
hello:
handler: handler.hello
© Civic Vision UG

service: foss4g_oceania
provider:
name: aws
runtime: python2.7
package:
include:
- venv/lib/python2.7/site-packages
- analysis.py
exclude:
- venv
functions:
hello:
handler: handler.hello
© Civic Vision UG

import sys, os, json
here = os.path.dirname(os.path.realpath(__file__))
env_path = os.path.join(here, "./venv/lib/python2.7/site-packages/")
sys.path.append(env_path)
© Civic Vision UG

plugins:
- serverless-python-requirements
© Civic Vision UG

service: foss4g_oceania
provider:
name: aws
runtime: python3.6
package:
artifact: package.zip
functions:
hello:
handler: handler.hello
© Civic Vision UG

$FROM amazonlinux:latest # Install apt dependencies RUN yum install -y gcc gcc-c++ freetype-devel yum-utils findutils openssl-devel RUN yum -y groupinstall development RUN curl https://www.python.org/ftp/python/3.6.1/Python-3.6.1.tar.xz | tar -xJ && cd Python-3.6.1 && ./configure --prefix=/usr/local --enable-shared && make && make install && cd .. && rm -rf Python-3.6.1 ENV LD_LIBRARY_PATH=/usr/local/lib:$LD_LIBRARY_PATH # Install Python dependencies RUN pip3 install xxx--no-binary -t /tmp/vendored -U # Reduce Lambda package size to fit the 250Mb limit # Mostly based on https://github.com/jamesandersen/aws-machine-learning-demo RUN du -sh /tmp/vendored # This is the list of available modules on AWS lambda Python 3 # ['boto3', 'botocore', 'docutils', 'jmespath', 'pip', 'python-dateutil', 's3transfer', 'setuptools', 'six'] RUN find /tmp/vendored -name "*-info" -type d -exec rm -rdf {} + RUN rm -rdf /tmp/vendored/boto3/ RUN rm -rdf /tmp/vendored/botocore/ RUN rm -rdf /tmp/vendored/docutils/ RUN rm -rdf /tmp/vendored/dateutil/ RUN rm -rdf /tmp/vendored/jmespath/ RUN rm -rdf /tmp/vendored/s3transfer/ RUN rm -rdf /tmp/vendored/numpy/doc/ # Leave module precompiles for faster Lambda startup RUN find /tmp/vendored -type f -name '*.pyc' | while read f; do n=$(echo $f | sed 's/__pycache__///' | sed 's/.cpython-36//'); cp $f $n; done; RUN find /tmp/vendored -type d -a -name '__pycache__' -print0 | xargs -0 rm -rf RUN find /tmp/vendored -type f -a -name '*.py' -print0 | xargs -0 rm -f RUN du -sh /tmp/vendored COPY app /tmp/vendored/app # Create archive RUN cd /tmp/vendored && zip -r9q /tmp/package.zip * # Cleanup RUN rm -rf /tmp/vendored/ © Civic Vision UG$

# Install Python dependencies
RUN pip3 install xxx--no-binary -t /tmp/vendored -U
# Reduce Lambda package size to fit the 250Mb limit
# Mostly based on https://github.com/jamesandersen/aws-machine-learning-demo
RUN du -sh /tmp/vendored
© Civic Vision UG

Adapted from mapbox/landsat-tiler
© Civic Vision UG

all: build package
build:
docker build --tag fis-server:latest .
package:
docker run
-w /var/task/
--name fis
-itd
fis-server:latest
docker cp fis:/tmp/package.zip package.zip
docker stop fis
docker rm fis
deploy:
sls deploy
© Civic Vision UG

questions
Give me your two questions!
© Civic Vision UG

Resources available at
civicvision.de/f4goaws
© Civic Vision UG

This document provides an introduction to using Puppet including quick primers on its basic concepts like resources, classes, nodes, and data parameters. It also gives examples of useful Puppet commands and demonstrates how to configure sudoers permissions on different operating systems using Puppet manifests and inheriting classes. References for additional Puppet documentation and guides are provided at the end.

Pig workshopSudar Muthu

This document provides information about a Pig workshop being conducted by Sudar Muthu. It introduces Pig, describing it as a platform for analyzing large data sets. It outlines some key Pig components like the Pig Shell, Pig Latin language, libraries, and user defined functions. It also discusses why Pig is useful, highlighting aspects like its data flow model and ability to increase programmer productivity. Finally, it previews topics that will be covered in the workshop, such as loading and storing data, Pig Latin operators, and writing user defined functions.

37562259 top-consuming-processskumner

This document provides various SQL queries and UNIX commands that can be used to monitor and troubleshoot Oracle databases and the underlying UNIX environment. Some examples include: 1) A SQL query to find the SQL statement being executed by a specific OS process ID. 2) UNIX commands like ps and prstat to identify the top CPU consuming processes. 3) SQL queries to get session details like username and status for a specific OS process ID. 4) UNIX commands to locate Oracle files, find disk usage, schedule cron jobs, and more. 5) A link provided to a website with additional UNIX commands for database administrators.

Practical pigtrihug

This document provides an overview of the Pig Latin data flow language for Hadoop. It discusses why Pig is useful for increasing productivity and insulating users from complexity when working with MapReduce. The document provides examples of simple Pig Latin scripts for common tasks like filtering, joining, grouping and aggregation. It also covers performance considerations, user defined functions, common pitfalls, and recommendations.

15 map reduce on azurePatrick Bury

multi-line record grepRyoichi KATO

This document introduces mlr-grep, a tool for extracting entire multi-line records from files that contain a search pattern. It summarizes the motivations for such a tool and limitations of typical tools like grep, pcregrep and awk. It then describes the implementations of mlr-grep in AWK, Haskell and Python and their features, such as supporting boolean operators and timestamps for grouping log entries. The document provides examples of using mlr-grep and notes its capabilities like changing the record separator.

Find and zip filesMuqthiyar Pasha

GoLang & GoatCore Sebastian Pożoga

Golang and GoatCore were created to build web applications. The document provides steps to set up a development environment for Go and GoatCore, including installing dependencies like Mingw, Go, Git, Atom, and Postgres. It also demonstrates building and running a GoatCore project, and discusses features of Go like concurrency using channels and the runtime. GoatCore provides internationalization support and performs efficient JSON parsing by operating on byte-level payloads.

Git for beginnersVinh Nguyen

Building a DSL with GraalVM (VoxxedDays Luxembourg)Maarten Mulders

The Art of Command Line (2021)Kenta Yamamoto

The document discusses best practices for using the command line interface (CLI) efficiently. It provides examples of using common UNIX commands like grep, find, awk, sort, uniq, and xargs to analyze command history, filter output, run commands in parallel, and handle encoding. The document emphasizes building small, focused tools according to the UNIX philosophy and provides tips for common tasks like capturing command output, checking exit statuses, running subprocesses, and handling different operating systems. Recommendations are made for CLI productivity tools and the linter ShellCheck.

Mastering the Unix Command LineHoward Mao

Raspberry pi a la cfmlColdFusionConference

1) The document discusses running ColdFusion on a Raspberry Pi single board computer. It provides details on setting up the Pi with Raspbian Linux, Oracle Java, and the CommandBox CFML engine. 2) Examples are given of running simple CFML files and scripts to output the current date/time using CommandBox. Additional software like ContentBox is installed to create a basic website. 3) The document explores using the Pi's GPIO pins to blink an LED and discusses possibilities for creating an 8x8 LED matrix display with ColdFusion code. Shift registers are mentioned as a way to control more outputs with fewer pins.

How to adminyalegko

This document provides instructions for various tasks related to networking and system administration on Linux systems, including: 1) Configuring network interfaces using dhclient or manually assigning an IP address and route. 2) Installing and configuring OpenSSH for remote access and using SSH, SCP for secure file transfers. 3) Using common Linux commands like tcpdump, tshark, ps, kill, service to monitor network traffic, view processes, kill processes, and control services. 4) Additional instructions are provided for using shell commands like head, tail, awk, sed, grep to view logs and parse output from other commands. Guidance is given for capturing network traffic to PCAP files using tcp

Introduction to ns2ProfDrSharadSharma

This document provides an introduction and overview of Network Simulator 2 (NS2). It discusses the history and goals of NS2, the languages and protocols it supports, and how to work with and extend NS2. It describes how to create simple simulations using Tcl scripts, including defining nodes, links, traffic sources and sinks. It also provides an example of simulating wireless networks in NS2 using scripts to define node movement and the DSDV routing protocol.

Go初心者がGoでコマンドラインツールの作成に挑戦した話dcubeio

KubeCon EU 2016: Custom Volume PluginsKubeAcademy

Abstract: This talk will introduce you to the concept of Kubernetes Volume plugins. We will not only help you understand the basic concepts, but more importantly, using practical examples, we will show how you can develop your own volume plugins and contribute them back to the community of the OSS project as large as Kubernetes. We will conclude the talk by discussing various challenges one can come across when contributing to a high velocity OSS project of Kubernetes' size which can help you avoid the pain and enjoy the path. Sched Link: http://sched.co/6BYB

Beyond Lists - Functional Kats Conf Dublin 2015Phillip Trelford

This document summarizes Phillip Trelford's talk at the Functional Kats Conference in 2015. It discusses performance comparisons between different programming languages and data structures, including reactive extensions in C# and F# observables. It also covers topics like immutable vs mutable data structures and choosing pragmatic solutions that optimize for speed. Source code examples are provided for concepts like quicksort algorithms and higher-order functions.

Docker tips & tricksDharmit Shah

Phil Bartie QGIS PLPythonRoss McDonald

Gogo shelljwausle

Chapter21 separate-header-and-implementation-filesDeepak Singh

The document discusses separating a class definition into header and implementation files to make the class reusable. The class declaration is defined in a header file with a .h extension, member functions are defined in a cpp file with the same name and a .cpp extension, and a main.cpp file includes the header and contains the main function to test the class. This separation allows the class interface to be shared while hiding implementation details.

Ordered Record CollectionHadoop User Group

Catching Wire; An introduction to CBWire 4Ortus Solutions, Corp

Into the Box 2025 - Michael Rigsby We are continually bombarded with the latest and greatest new (or at least new to us) “thing” and constantly told we should integrate this or that right away! Keeping up with new technologies, modules, libraries, etc. can be a full-time job in itself. In this session we will explore one of the “things” you may have heard tossed around, CBWire! We will go a little deeper than a typical “Elevator Pitch” and discuss what CBWire is, what it can do, and end with a live coding demonstration of how easy it is to integrate into an existing ColdBox application while building our first wire. We will end with a Q&A and hopefully gain a few more CBWire fans!

Codingo Ltd. - Introduction - Mobile application, web, custom software develo...Codingo

EN: Codingo is a custom software development company providing digital solutions for small and medium-sized businesses. Our expertise covers mobile application development, web development, and the creation of advanced custom software systems. Whether it's a mobile app, mobile application, or progressive web application (PWA), we deliver scalable, tailored solutions to meet our clients’ needs. Through our web application and custom website creation services, we help businesses build a strong and effective online presence. We also develop enterprise resource planning (ERP) systems, business management systems, and other unique software solutions that are fully aligned with each organization’s internal processes. This presentation gives a detailed overview of our approach to development, the technologies we use, and how we support our clients in their digital transformation journey — from mobile software to fully customized ERP systems. HU: A Codingo Kft. egyedi szoftverfejlesztéssel foglalkozó vállalkozás, amely kis- és középvállalkozásoknak nyújt digitális megoldásokat. Szakterületünk a mobilalkalmazás fejlesztés, a webfejlesztés és a korszerű, egyedi szoftverek készítése. Legyen szó mobil app, mobil alkalmazás vagy akár progresszív webalkalmazás (PWA) fejlesztéséről, ügyfeleink mindig testreszabott, skálázható és hatékony megoldást kapnak. Webalkalmazásaink és egyedi weboldal készítési szolgáltatásaink révén segítjük partnereinket abban, hogy online jelenlétük professzionális és üzletileg is eredményes legyen. Emellett fejlesztünk egyedi vállalatirányítási rendszereket (ERP), ügyviteli rendszereket és más, cégspecifikus alkalmazásokat is, amelyek az adott szervezet működéséhez igazodnak. Bemutatkozó anyagunkban részletesen bemutatjuk, hogyan dolgozunk, milyen technológiákkal és szemlélettel közelítünk a fejlesztéshez, valamint hogy miként támogatjuk ügyfeleink digitális fejlődését mobil applikációtól az ERP rendszerig. https://codingo.hu/

Exchange Migration Tool- Shoviv SoftwareShoviv Software

The Shoviv Exchange Migration Tool is a powerful and user-friendly solution designed to simplify and streamline complex Exchange and Office 365 migrations. Whether you're upgrading to a newer Exchange version, moving to Office 365, or migrating from PST files, Shoviv ensures a smooth, secure, and error-free transition. With support for cross-version Exchange Server migrations, Office 365 tenant-to-tenant transfers, and Outlook PST file imports, this tool is ideal for IT administrators, MSPs, and enterprise-level businesses seeking a dependable migration experience. Product Page: https://www.shoviv.com/exchange-migration.html

Drawing Heighway’s Dragon - Part 4 - Interactive and Animated Dragon CreationPhilip Schwarz

More Related Content

What's hot (18)

multi-line record grepRyoichi KATO

Find and zip filesMuqthiyar Pasha

GoLang & GoatCore Sebastian Pożoga

Git for beginnersVinh Nguyen

Building a DSL with GraalVM (VoxxedDays Luxembourg)Maarten Mulders

The Art of Command Line (2021)Kenta Yamamoto

Mastering the Unix Command LineHoward Mao

Raspberry pi a la cfmlColdFusionConference

How to adminyalegko

Introduction to ns2ProfDrSharadSharma

Go初心者がGoでコマンドラインツールの作成に挑戦した話dcubeio

KubeCon EU 2016: Custom Volume PluginsKubeAcademy

Beyond Lists - Functional Kats Conf Dublin 2015Phillip Trelford

Docker tips & tricksDharmit Shah

Phil Bartie QGIS PLPythonRoss McDonald

Gogo shelljwausle

Chapter21 separate-header-and-implementation-filesDeepak Singh

Ordered Record CollectionHadoop User Group

multi-line record grepRyoichi KATO

Find and zip filesMuqthiyar Pasha

GoLang & GoatCore Sebastian Pożoga

Git for beginnersVinh Nguyen

Building a DSL with GraalVM (VoxxedDays Luxembourg)Maarten Mulders

The Art of Command Line (2021)Kenta Yamamoto

Mastering the Unix Command LineHoward Mao

Raspberry pi a la cfmlColdFusionConference

How to adminyalegko

Introduction to ns2ProfDrSharadSharma

Go初心者がGoでコマンドラインツールの作成に挑戦した話dcubeio

KubeCon EU 2016: Custom Volume PluginsKubeAcademy

Beyond Lists - Functional Kats Conf Dublin 2015Phillip Trelford

Docker tips & tricksDharmit Shah

Phil Bartie QGIS PLPythonRoss McDonald

Gogo shelljwausle

Chapter21 separate-header-and-implementation-filesDeepak Singh

Ordered Record CollectionHadoop User Group

Recently uploaded (20)

Catching Wire; An introduction to CBWire 4Ortus Solutions, Corp

Codingo Ltd. - Introduction - Mobile application, web, custom software develo...Codingo

Exchange Migration Tool- Shoviv SoftwareShoviv Software

Drawing Heighway’s Dragon - Part 4 - Interactive and Animated Dragon CreationPhilip Schwarz

Buy vs. Build: Unlocking the right path for your training techRustici Software

Investing in training technology is tough and choosing between building a custom solution or purchasing an existing platform can significantly impact your business. While building may offer tailored functionality, it also comes with hidden costs and ongoing complexities. On the other hand, buying a proven solution can streamline implementation and free up resources for other priorities. So, how do you decide? Join Roxanne Petraeus and Anne Solmssen from Ethena and Elizabeth Mohr from Rustici Software as they walk you through the key considerations in the buy vs. build debate, sharing real-world examples of organizations that made that decision.

wAIred_LearnWithOutAI_JCON_14052025.pptxSimonedeGijt

In today's world, artificial intelligence (AI) is transforming the way we learn. This talk will explore how we can use AI tools to enhance our learning experiences. We will try out some AI tools that can help with planning, practicing, researching etc. But as we embrace these new technologies, we must also ask ourselves: Are we becoming less capable of thinking for ourselves? Do these tools make us smarter, or do they risk dulling our critical thinking skills? This talk will encourage us to think critically about the role of AI in our education. Together, we will discover how to use AI to support our learning journey while still developing our ability to think critically.

Why CoTester Is the AI Testing Tool QA Teams Can’t IgnoreShubham Joshi

The QA landscape is shifting rapidly, and tools like CoTester are setting new benchmarks for performance. Unlike generic AI-based testing platforms, CoTester is purpose-built with real-world challenges in mind—like flaky tests, regression fatigue, and long release cycles. This blog dives into the core AI features that make CoTester a standout: smart object recognition, context-aware test suggestions, and built-in analytics to prioritize test efforts. Discover how CoTester is not just an automation tool, but an intelligent testing assistant.

Aligning Projects to Strategy During Economic UncertaintyOnePlan Solutions

cram_advancedword2007version2025final.pptahmedsaadtax2025

AI Agents with Gemini 2.0 - Beyond the ChatbotMárton Kodok

You will learn how to move beyond simple LLM calls to build intelligent agents with Gemini 2.0. Learn how function calling, structured outputs, and async operations enable complex agent behavior and interactions. Discover how to create purpose-driven AI systems capable of a series of actions. The demo covers how a chat message activates the agentic experience, then agents utilize tools to achieve complex goals, and unlock the potential of multi-agent systems, where they collaborate to solve problems. Join us to discover how Gemini 2.0 empowers you to create multi turn agentic workflows for everyday developers.

The-Future-is-Hybrid-Exploring-Azure’s-Role-in-Multi-Cloud-Strategies.pptxjames brownuae

As businesses are transitioning to the adoption of the multi-cloud environment to promote flexibility, performance, and resilience, the hybrid cloud strategy is becoming the norm. This session explores the pivotal nature of Microsoft Azure in facilitating smooth integration across various cloud platforms. See how Azure’s tools, services, and infrastructure enable the consistent practice of management, security, and scaling on a multi-cloud configuration. Whether you are preparing for workload optimization, keeping up with compliance, or making your business continuity future-ready, find out how Azure helps enterprises to establish a comprehensive and future-oriented cloud strategy. This session is perfect for IT leaders, architects, and developers and provides tips on how to navigate the hybrid future confidently and make the most of multi-cloud investments.

Multi-Agent Era will Define the Future of SoftwareIvo Andreev

The potential of LLMs is severely underutilized as they are much more capable than generating completions or summarizing content. LLMs demonstrate remarkable capabilities in reaching a level of reasoning and planning comparable to human abilities. Satya Nadella revealed his vision of traditional software being replaced by AI layer based on multi-agents. In this session we introduce agents, multi-agents, the agent stack with Azure AI Foundry Semantic Kernel, A2A protocol, MCP protocol and more. We will make first steps into the concept with a practical implementation.

Temas principales de GrafanaCON 2025 Grafana 12 y másImma Valls Bernaus

Únete con nosotros en un seminario web exclusivo mientras recapitulamos los aspectos más destacados de GrafanaCON 2025, nuestro evento comunitario más grande del año. Si no puedes asistir al evento en persona, esta es una excelente oportunidad para ponerte al día con lo último en Grafana y el ecosistema más amplio de monitorización de código abierto. También compartiremos ejemplos de paneles inspiradores y casos de uso de la comunidad, y repasaremos los momentos más destacados y los anuncios de la conferencia. Lo que aprenderás 1. Últimas novedades de Grafana 12: Explora las funciones de vanguardia que estamos desarrollando en nuestra última versión de Grafana. 2. Innovaciones en el Stack LGTM (Loki-Grafana-Tempo-Mimir): Descubre los aspectos más sobresalientes en los proyectos open source Loki, Mimir, Tempo, y más. 3. Logros de la comunidad: Aprende sobre los logros sobresalientes y contribuciones de los miembros de la comunidad de Grafana.

Applying AI in Marketo: Practical Strategies and ImplementationBradBedford3

Join Lucas Goncalves Machado, AJ Navarro and Darshil Shah for a focused session on leveraging AI in Marketo. In this session, you will: Understand how to integrate AI at every stage of the lead lifecycle—from acquisition and scoring to nurturing and conversion Explore the latest AI capabilities now available in Marketo and how they can enhance your campaigns Follow step-by-step guidance for implementing AI-driven workflows in your own instance Designed for marketing operations professionals who value clear, practical advice, you’ll leave with concrete strategies to put into practice immediately.

IObit Uninstaller Pro Crack {2025} Download FreeIobit Uninstaller Pro Crack

Shift Right Security for EKS Webinar SlidesAnchore

Container vulnerabilities don't stop at deployment. As your Kubernetes workloads scale across Amazon EKS clusters, maintaining continuous visibility becomes increasingly challenging, yet critically important. Anchore's Kubernetes Runtime Inventory delivers the real-time insights security and DevOps teams need to identify vulnerabilities, enforce policies, and maintain compliance in production environments. Join Anchore Customer Success Engineer Ty Henry and Bion Consulting Senior DevOps Engineer Baturay Ozcan for an in-depth technical demonstration and informational webinar.

File Viewer Plus 7.5.5.49 Crack Full Versionraheemk1122g

How to Create a Crypto Wallet Like Trust.pptxriyageorge2024

Looking to build a powerful multi-chain crypto wallet like Trust Wallet? AppcloneX offers a ready-made Trust Wallet clone script packed with essential features—multi-chain support, secure private key management, built-in DApp browser, token swaps, and more. With high-end security, customizable design, and seamless blockchain integration, this script is perfect for startups and entrepreneurs ready to launch their own crypto wallet. Check it out now and kickstart your Web3 journey with AppcloneX!

Logs, Metrics, traces and Mayhem - An Interactive Observability Adventure Wor...Imma Valls Bernaus

This is a hands-on introductory session on observability. Through an engaging text-based adventure, you'll learn to diagnose and resolve issues in your systems. This workshop covers essential observability tools —metrics, logs, and traces — and shows how to leverage them effectively for real-world troubleshooting and insights in your application. Bring your laptop for this session. Docker and git or a browser to run this on a killercoda playground are prerequisites. You can also work in pairs.

Deploying & Testing Agentforce - End-to-end with Copado - Ewenb ClarkPeter Caitens

Catching Wire; An introduction to CBWire 4Ortus Solutions, Corp

Codingo Ltd. - Introduction - Mobile application, web, custom software develo...Codingo

Exchange Migration Tool- Shoviv SoftwareShoviv Software

Drawing Heighway’s Dragon - Part 4 - Interactive and Animated Dragon CreationPhilip Schwarz

Buy vs. Build: Unlocking the right path for your training techRustici Software

wAIred_LearnWithOutAI_JCON_14052025.pptxSimonedeGijt

Why CoTester Is the AI Testing Tool QA Teams Can’t IgnoreShubham Joshi

Aligning Projects to Strategy During Economic UncertaintyOnePlan Solutions

cram_advancedword2007version2025final.pptahmedsaadtax2025

AI Agents with Gemini 2.0 - Beyond the ChatbotMárton Kodok

The-Future-is-Hybrid-Exploring-Azure’s-Role-in-Multi-Cloud-Strategies.pptxjames brownuae

Multi-Agent Era will Define the Future of SoftwareIvo Andreev

Temas principales de GrafanaCON 2025 Grafana 12 y másImma Valls Bernaus

Applying AI in Marketo: Practical Strategies and ImplementationBradBedford3

IObit Uninstaller Pro Crack {2025} Download FreeIobit Uninstaller Pro Crack

Shift Right Security for EKS Webinar SlidesAnchore

File Viewer Plus 7.5.5.49 Crack Full Versionraheemk1122g

How to Create a Crypto Wallet Like Trust.pptxriyageorge2024

Logs, Metrics, traces and Mayhem - An Interactive Observability Adventure Wor...Imma Valls Bernaus

Deploying & Testing Agentforce - End-to-end with Copado - Ewenb ClarkPeter Caitens

Fast and cost effective geospatial analysis pipeline with AWS lambda

20. service: foss4g_oceania provider: name: aws runtime: nodejs6.10 functions: hello: handler: handler.hello © Civic Vision UG

22. service: foss4g_oceania provider: name: aws runtime: python2.7 package: include: - venv/lib/python2.7/site-packages - analysis.py exclude: - venv functions: hello: handler: handler.hello © Civic Vision UG

23. import sys, os, json here = os.path.dirname(os.path.realpath(__file__)) env_path = os.path.join(here, "./venv/lib/python2.7/site-packages/") sys.path.append(env_path) © Civic Vision UG

26. service: foss4g_oceania provider: name: aws runtime: python3.6 package: artifact: package.zip functions: hello: handler: handler.hello © Civic Vision UG

27. FROM amazonlinux:latest # Install apt dependencies RUN yum install -y gcc gcc-c++ freetype-devel yum-utils findutils openssl-devel RUN yum -y groupinstall development RUN curl https://www.python.org/ftp/python/3.6.1/Python-3.6.1.tar.xz | tar -xJ && cd Python-3.6.1 && ./configure --prefix=/usr/local --enable-shared && make && make install && cd .. && rm -rf Python-3.6.1 ENV LD_LIBRARY_PATH=/usr/local/lib:$LD_LIBRARY_PATH # Install Python dependencies RUN pip3 install xxx--no-binary -t /tmp/vendored -U # Reduce Lambda package size to fit the 250Mb limit # Mostly based on https://github.com/jamesandersen/aws-machine-learning-demo RUN du -sh /tmp/vendored # This is the list of available modules on AWS lambda Python 3 # ['boto3', 'botocore', 'docutils', 'jmespath', 'pip', 'python-dateutil', 's3transfer', 'setuptools', 'six'] RUN find /tmp/vendored -name "*-info" -type d -exec rm -rdf {} + RUN rm -rdf /tmp/vendored/boto3/ RUN rm -rdf /tmp/vendored/botocore/ RUN rm -rdf /tmp/vendored/docutils/ RUN rm -rdf /tmp/vendored/dateutil/ RUN rm -rdf /tmp/vendored/jmespath/ RUN rm -rdf /tmp/vendored/s3transfer/ RUN rm -rdf /tmp/vendored/numpy/doc/ # Leave module precompiles for faster Lambda startup RUN find /tmp/vendored -type f -name '*.pyc' | while read f; do n=$(echo $f | sed 's/__pycache__///' | sed 's/.cpython-36//'); cp $f $n; done; RUN find /tmp/vendored -type d -a -name '__pycache__' -print0 | xargs -0 rm -rf RUN find /tmp/vendored -type f -a -name '*.py' -print0 | xargs -0 rm -f RUN du -sh /tmp/vendored COPY app /tmp/vendored/app # Create archive RUN cd /tmp/vendored && zip -r9q /tmp/package.zip * # Cleanup RUN rm -rf /tmp/vendored/ © Civic Vision UG

28. # Install Python dependencies RUN pip3 install xxx--no-binary -t /tmp/vendored -U # Reduce Lambda package size to fit the 250Mb limit # Mostly based on https://github.com/jamesandersen/aws-machine-learning-demo RUN du -sh /tmp/vendored © Civic Vision UG

30. all: build package build: docker build --tag fis-server:latest . package: docker run -w /var/task/ --name fis -itd fis-server:latest docker cp fis:/tmp/package.zip package.zip docker stop fis docker rm fis deploy: sls deploy © Civic Vision UG

Fast and cost effective geospatial analysis pipeline with AWS lambda

Recommended

More Related Content

What's hot (18)

Recently uploaded (20)

Fast and cost effective geospatial analysis pipeline with AWS lambda