dmwm/DBS

Migrate DBS client to port 8443

Closed this issue · 5 comments

vkuznet commented

In effort of machine tool migration we need to migrate DBS client to use port 8443 on cmsweb. Please test it and take care of this. Possibly it should be configured external with some default to 8443 or someone can overwrite it to use another port.

vkuznet commented

Yuyi, do you have any progress on this?

yuyiguo commented
bbockelm commented

Hi Yuyi,

Can you please review my pull request?

Brian

ticoann commented

@yuyiguo, Hi Yuyi, since there is already this issue. I am just updating this.
Aside why we need to change this (Valentin explain in group email).

  1. DBS client api need to convert any dbs url containing cmsweb.cern.ch or cmsweb-testbed.cern.ch to
    cmsweb.cern.ch:8443 and cmsweb-testbed.cern.ch:8433

Something like in WMAgent change below.
https://github.com/dmwm/WMCore/pull/8726/files

ticoann commented

Here is Valentin's explanation why we need to do this.

The in-depth details you can find in Brian's talk at CHEP, see [1, 2].

Here is a short summary:
- x509 as well as Globus Toolkit tools (used in my grid tools we use) will end
soon their support (according to their web site [3] by Jan 2018).
- in order to perform transition to new technology we need to separate
machine clients and web users. This is the task which spark this thread
discussion. It is as simple as moving machine tools to port 8443, while
leave all web clients (browser based) on default port 443.
- The new auth schema will be based on token technology in a similar way
as we authenticate ourselves with many web site via Facebook, Google, etc, see
[1] for details (slide End-Goal).
- Most likely web users will migrate to CERN SSO authentication (that's why
we need to separate them) while machine tools will migrate to SciTokens
or similar technology. The choices will be determined upon completion of
implementation (e.g. SciTokens) and integration tests (e.g. SSO with cmsweb)
- The SciTokens or similar technology will provide ability to authenticate
users and generate/provide long-live tokens (or renewed one) similar to what
we use with myproxy for our x509 CAs.
 - the token will be used in HTTP header and our tools will be able to
 decode it and use to authenticate and authorize users to perform certain tasks
 If you need concrete example please have a look at [4] and see [2].

Based on this description we (developers) will need to provide a way
to authenticate with 3rd party provider (SciTokens), obtain a token
and put it in HTTP header of the request. Most likely we'll develop
a single library for that and our tools will use it. Because tokens
are simple strings and will be used in HTTP header some work will need
to be done and again it is likely to be a common library. It would be
much easier to migrate all tools once we know what to migrate. And, for that
the proposal to identify and move machine tools to port 8443 is for.

The cmsweb infrastructure will also be changed from manual managed VMs to
kubernetes cluster and eventually migrated to CERN SSO for authentication.

Certainly, I would expect gradual migration, e.g.
- separate tools from web clients (port 8843 and 443)
- integrate CERN SSO into cmsweb infratructure
- move cmsweb to kubernetes cluster
- migrate grid tools to token authentication by gradually using x509 CAs for
obtaining a token and later replace it with OAuth2 way
Once current and new technologies will be working together outline
plan/schedule to perform final migration.

All tasks will be coordinated and performed by cmsweb (L2: Kate) and DMWM (L2:
Brian and Jean-Roch) groups. The concrete plan will be outlined once
we'll know and prove technology choice (SciTokens or similar) which is
now under development.

If I missed something, Brian may give further details.

Best,
Valentin.

[1] https://indico.cern.ch/event/587955/contributions/2936866/attachments/1683367/2705655/SciTokens-CHEP2018.pdf
[2] https://indico.cern.ch/event/587955/contributions/2936880/attachments/1682171/2702926/DataEcosystem-CHEP18.pdf
[3] https://gridcf.org/
[4] https://demo.scitokens.org/