/migrate-python2-appengine

This repo is for the codelabs (free, online, self-paced tutorials) showing developers how to migrate their Google App Engine applications from the Python 2 runtime to the 2nd generation Python3 App Engine or Cloud Run serverless container services. The repo for the code samples in the documentation are elsewhere: https://github.com/GoogleCloudPlatform/python-docs-samples/tree/master/appengine/standard/migration

Primary LanguagePythonApache License 2.0Apache-2.0

Python 2 App Engine app migration

To modern runtime, Cloud services, Python 3, and Cloud Run containers

Google App Engine (Standard) has undergone significant changes between the legacy and next generation platforms. To address this, we've created a set of codelabs (free, online, self-paced, hands-on tutorials) to show developers how to perform individual migrations they can apply to modernize their apps for the latest runtimes, with this repo managing the samples from those codelabs.

Codelabs begin with a "START" code base then walks developers through that migration step, resulting in a "FINISH" repo. If you made any mistakes along the way, you can always go back to START or compare your code with that in the FINISH folder to see the differences. Since another goal is to port to Python 3, some codelabs have a bonus section for that purpose.

NOTE: These migrations are only for those with Python 2 (2.7) App Engine apps.

  1. Python 3.x App Engine users: You're already on the next-gen platform, so there's no need for you to be here unless you help 2.x developers migrate.
  2. Python 2.5 App Engine developers: to revive apps on the original 2.5 runtime, deprecated in 2013 and shutdown in 2017, you must migrate from db to ndb before attempting these migrations.

Prerequisites

  • A Google account (G Suite accounts may require administrator approval)
  • A Google Cloud (GCP) project with an active billing account
  • Familiarity with operating system terminal/shell commands
  • Familiarity with developing & deploying Python 2 apps to App Engine
  • General skills in Python 2 and 3

Cost

App Engine is not a free service. While you may not have needed to enable billing in App Engine's early days, all applications now require an active billing account backed by a financial instrument (usually a credit card). Don't worry, App Engine (and other GCP products) still have an "Always Free" tier, and as long as you stay within those limits, you won't incur billing. Also check the App Engine pricing and quotas pages for more information.

Why

In App Engine's early days, users wanted Google to make the platform more flexible for developers and make their apps more portable. As a result, the team made significant changes to its 2nd-generation service which launched in 2017. As a result, all previously built-in services have been removed, and users can either choose from new standalone Cloud products as replacements or best-of-breed replacements in the broader developer community. Summary:

  • Legacy platform: Python 2 only, proprietary built-in services
  • Next generation: Python 3 only, external services, flexible platform

The key issue is that developers looking to port their applications to Python 3 have two huge hurdles to overcome, migrating from Python 2 to 3 and migrating from built-in services to alternatives. On top of this, direct replacements are not available for all built-in services; alternatives come in 3 flavors:

  1. Direct replacement: Legacy services which matured into their own Cloud products (e.g., App Engine Datastore is now Cloud Datastore)
  2. Partial replacement: Some aspects of legacy services (e.g., Cloud Tasks supports App Engine push tasks; for pull tasks, Cloud Pub/Sub is recommended; use of Cloud MemoryStore with REDIS as an alternative for Memcache)
  3. No replacement: No direct replacement available, so third-party or other tools recommended (e.g., Search, Images, Users, Email)

These are the challenges developers are facing, so the purpose of this content is to make this process more smooth and prescriptive. Review the runtimes chart to see the legacy services and current migration recommendation. The migration guide overview has more information.

NOTE: App Engine (Flexible) is a next-gen service but is not within the scope of these tutorials. Developers who are curious can compare App Engine Standard vs. Flexible.

Progression

All codelabs begin with code in a START repo folder and end with code in a FINISH folder, implementing a single migration. Upon completion, users should confirm their code (for the most part) matches what's in the FINISH folder. The baseline migration sample app is a barebones Python 2.7 App Engine app that uses the webapp2 web framework plus the ndb Datastore library. This is what's in the Module 0 repo folder (link below).

  1. From there (Module 0 is START), the Module 1 codelab migrates from the webapp2 web framework to Flask where the Module 1 repo folder is the FINISH point.

  2. The Module 2 codelab STARTs with the Module 1 code (yours or ours) and migrates away from ndb to Cloud NDB, and ends with the Module 2 (Python 2) FINISH repo folder. There's also has a bonus migration to Python 3, resulting in another FINISH repo folder. Once you've deployed this Python 3 code, your app is modernized and runs on the next-generation platform.

Of course, things aren't as simple in real life. Your Python 2 App Engine app may also be using Task Queues, Memcache, or many of the other original App Engine built-in services, so there will be additional migration modules (not all are available yet). With some exceptions, there's no specific order of what migrations you do next. It's just what you (or your apps) need. Here's full summary of what's currently available:

Migrations

The table below summarizes migration module resources currently available to developers along with a more detailed table of contents below. Be sure to check back for updates as more resources are planned.

Summary table

Module Topic Video Codelab START repo FINISH repo
0 Baseline app TBD N/A N/A Module 0 code (2.x)
1 Migrate to Flask TBD TBD Module 0 code (2.x) Module 1 code (2.x)
2 Migrate to Cloud NDB TBD TBD Module 1 code (2.x) Module 2 code (2.x) & code (3.x)
3 Migrate to Cloud Datastore TBD TBD Module 2 code (2.x) & code (3.x) Module 3 code (2.x) & code (3.x)
4 Migrate to Cloud Run with Docker TBD TBD Module 2 code (2.x) & Module 3 code (3.x) Module 4 code (2.x) & code (3.x)
5 Migrate to Cloud Run with Buildpacks TBD TBD Module 2 code (3.x) Module 5 code (3.x)
6 Migrate to Cloud Firestore TBD TBD Module 3 code (3.x) Module 6 code (3.x)
7 Add App Engine push tasks TBD TBD Module 1 code (2.x) Module 7 code (2.x)
8 Migrate to Cloud Tasks TBD TBD Module 7 code (2.x) Module 8 code (2.x)
9 Migrate to Python 3 (Cloud Datastore & Cloud Tasks v2) TBD TBD Module 8 code (2.x) Module 9 code (3.x)

Table of contents

If there is a logical codelab to do immediately after completing one, they will be designated as NEXT. Other recommended codelabs will be listed as RECOMMENDED, and the more optional ones will be labeled as OTHERS (and usually in some kind of priority order).

  • Module 1 codelab: Migrate from webapp2 to Flask

  • Module 2 codelab: Migrate from App Engine ndb to Cloud NDB

    • Required migration
      • Migration to Cloud NDB which is supported by Python 3 and the next-gen platform.
    • Python 2
    • Codelab bonus port to Python 3.x
    • RECOMMENDED:
      • Module 7 codelab - add App Engine (push) tasks
    • OTHERS (somewhat priority order):
      • Module 4 codelab - migrate to Cloud Run container with Docker
      • Module 5 codelab - migrate to Cloud Run container with Cloud Buildpacks
      • Module 3 codelab - migrate to Cloud Datastore

  • Module 7 codelab: Add App Engine (push) Task Queues to App Engine ndb Flask app

  • Module 8 codelab: Migrate from App Engine (push) Task Queues to Cloud Tasks v1

    • Required migration
      • Migration to Cloud Tasks which is supported by Python 3 and the next-gen platform.
      • Note this is only push tasks... pull tasks will be handled in a different codelab.
    • Python 2
    • NEXT: Module 9 codelab - migrate to Python 3

  • Module 9 codelab: Migrate a Python 2 Cloud NDB & Cloud Tasks app to a Python 3 Cloud Datastore app

    • Mixed migration recommendation
      • Migrating to Python 3 is required, but...
      • Migrating to Cloud Datastore is optional as Cloud NDB works on 3.x; it's to give you the experience of doing it
      • This codelab includes the migration in the Module 3 codelab, so skip it
    • Python 2
    • Python 3
    • RECOMMENDED:
      • Module 4 codelab - migrate to Cloud Run container with Docker
      • Module 5 codelab - migrate to Cloud Run container with Cloud Buildpacks

  • Module 4 codelab: Migrate from App Engine to Cloud Run with Docker

  • Module 5 codelab: Migrate from App Engine to Cloud Run with Cloud Buildpacks

  • Module 3 codelab: Migrate from Cloud NDB to Cloud Datastore

  • Module 6 codelab: Migrate from Cloud Datastore to Cloud Firestore

    • Highly optional migration (WARNING: infrequent/uncommon & "expensive" migration)
      • Requires new project & Datastore has better write performance (currently)
      • If you must have Firestore's Firebase features
    • Python 3 only
    • RECOMMENDED:
      • Module 7 codelab - add App Engine (push) tasks
    • OTHER OPTIONS (in somewhat priority order):
      • Module 4 codelab - migrate to Cloud Run container with Docker

Considerations for mobile developers

If your original app users does not have a user interface, i.e., mobile backends, etc., but still uses webapp2 for routing, some migration must still be completed. Your options:

Canonical code samples

References