ETL Script to upgrade webauthn DB

Question

ETL Script to upgrade webauthn DB

Closed this issue 9 years ago · 13 comments

Answer 1 · 2016-03-07T20:16:53.000Z

I suspect this is supposed to be project-specific ETL to fixup ERMrest databases that contain raw webauthn user IDs to instead contain the new denormalized user references we plan for this release. I think other staff would do that ETL, but @ljpearlman would help provide the core mapping table needed to convert the old ID to the new format.

Answer 2 · 2016-03-07T20:24:20.000Z

We've split that out into a different issue. This one is the database
used internally by webauthn itself.

               -- Laura

On 3/7/16 12:16 PM, Karl Czajkowski wrote:

I suspect this is supposed to be project-specific ETL to fixup ERMrest
databases that contain raw webauthn user IDs to instead contain the
new denormalized user references we plan for this release. I think
other staff would do that ETL, but @ljpearlman
https://github.com/ljpearlman would help provide the core mapping
table needed to convert the old ID to the new format.

—
Reply to this email directly or view it on GitHub
#19 (comment)
Bug from MailScannerWebBug

Answer 3 · 2016-03-15T00:19:17.000Z

Does this issue include getting a master mapping table of usernames and groupnames to the new representation? Or is that another issue we still need to create?

Answer 4 · 2016-04-04T16:59:36.000Z

@ljpearlman is this issue done? I am moving it to 2016-04 milestone. Please put it under review or close it if it is done.

Answer 5 · 2016-04-05T17:22:20.000Z

Sorry I'm not clear on what I am supposed to review here. Can you point me to some code or a revision or something?

Answer 6 · 2016-04-05T17:36:59.000Z

It's the stuff in https://github.com/informatics-isi-edu/webauthn/tree/globus_auth/webauthn2/scripts/globus_auth_upgrade (minus the script that you wrote).

Answer 7 · 2016-04-05T21:22:14.000Z

I understand this upgrade as an optional step to carry forward existing user information collected from past visits. Otherwise, the new system will start over with an empty user table and only collect user info for future visits.

Answer 8 · 2016-04-06T22:06:17.000Z

Just realized we need one of these for the database provider.

Answer 9 · 2016-04-06T22:11:36.000Z

Unless Mike needs it, I don't think it matters. We mostly use the database provider in transient deployments that would have no need to upgrade.

Answer 10 · 2016-04-13T03:05:02.000Z

There's now a script that will find the schema names in the ermrest and hatrac config files and upgrade schemas in place. I've also bumped the db version numbers so the script can see whether it's been run more than once. In https://github.com/informatics-isi-edu/webauthn/tree/globus_auth/webauthn2/scripts/v2_upgrade

Answer 11 · 2016-04-13T18:44:13.000Z

I don't have any plans to run this upgrade script so am reassigning to @bugacov to review when he applies these changes to his next dev VM

Answer 12 · 2016-04-13T19:53:36.000Z

@bugacov - you can run /usr/sbin/webauthn2-v2-upgrade as root instead of dropping database schemas and running ermrest-webauthn2-deploy. This will update both the ermrest and hatrac databases. If you're not using hatrac, you can use the --ermrest-only option to just update the ermrest db.

(By default, the script tries to update the hatrac db only if ~hatrac/webauthn_config.json exists. But if the config file exists and the hatrac database doesn't, you'll get an error message).

Answer 13 · 2016-04-20T21:25:01.000Z

We just tested this on gpcr-dev to upgrade to the globus_auth branch while remaining on the goauth provider config. Some minor bugs were found and a patch committed. I think we can call this complete.