ipfs/in-web-browsers

CID as a Subdomain

lidel opened this issue · 27 comments

lidel commented

Base32-encoded CIDv1 can be used in DNS subdomains. It requires additional setup server-side, but enables people to get a valid origin-based security perimeter for free.

This is a meta issue for tracking related notes and developments.

Zone Naming Convention

To make it easier for everyone, support for CID in subdomains comes with explicit zone naming convention:

https://<cidv1b32>.ipfs.<example.tld>

the CID will always be in a zone named ipfs (or ipns), e.g.
http://bafkreigh2akiscaildcqabsyg3dfr6chu3fgpregiymsck7e7aqa4s52zy.ipfs.dweb.link
– ipfs/ipfs-companion#526 (comment)

Nginx config

For now, this needs to be handled by reverse proxy.
An example of subdomain config for gateway at dweb.link:

if ($http_host ~ ^(.+)\.(ipfs|ipns)\.dweb\.link$) {
  set $ipfspath /$2/$1;
  rewrite "^(.*)$" $ipfspath$1 last;
}

Native config (go-ipfs >0.5.0)

Native support for CID in Host header ship with go-ipfs 0.5.0 and later.
It removes the need for Nginx config mentioned above, and can be enabled with this one-liner (dweb.link is the domain that should support $cid.ipfs.dweb.link):

$ ipfs config --json Gateway.PublicGateways '{
    "dweb.link": {
      "UseSubdomains": true,
      "Paths": ["/ipfs", "/ipns"]
    }
  }'

Length Limits

Subdomain needs to comply with length limits from RFC 1034:

Each node has a label, which is zero to 63 octets in length. #

To simplify implementations, the total number of octets that represent a
domain name (i.e., the sum of all label octets and label lengths) is
limited to 255. #

CIDv1(sha-256) in Base32 fits, but hashes with longer outputs can't be used as a single label.

Update: Potential solutions are discussed in:
ipfs/kubo#7318

mikeal commented

Is there a broader discussion already about how to square the existing origin based browser security model with the broader IPFS vision of a decentralized web?

This is a great approach in the short term but I'd like to get the ball rolling, if it isn't already, on how we're going to think about these security contexts down the road.

lidel commented

@mikeal historical discussions on origin-safe addressing can be found in #6.

"The four stages of the upgrade path for path addressing" from ipfs/specs#152 (comment) is a good summary of long discussion in ipfs/specs#152 that puts things in a wider context and lists some trade-offs. Subdomains are somewhere between stage 1 and 2. Looking back at that plan thru discussions we had with vendors I'd adjust ipfs:// to be mid to long term solution. Nobody wants to touch/change the way Origin is calculated. Everything depends on it.

If you are interested in synthesizing this in one place, there is a rough outline/draft of "Addressing on the Decentralized Web" document by Lars that I forked and started filling with my notes. Unfortunately I don't seem to have enough time/writing skills to get it where it needs to be: in Q3 want to add remainder of my updated notes and will be looking for a better champion for this document.

mikeal commented

@lidel Thanks! these are super helpful :)

Deleted user commented

Happy to help you out with any context/history questions. I'm having a hard time keeping up with github notifications nowadays, so make sure to tag me whenever you expect a response from me :)

mikeal commented

@lgierth are you going to be in San Francisco for dweb? I'm putting together a dinner with browser vendors and I'm sure this will come up. I was just talking to Alex Russell about how they've put together a new package format for browsers that uses PKI to sign resources that are then interpreted as being from specific domains. Super interesting and relevant to the work we're doing.

Deleted user commented

Nope unfortunately not -- there's a tight limit on how many people PL is sending and I didn't make it on the list.

Deleted user commented

Would it be helpful to condense all the existing docs and comments out there into one comprehensive doc though, before DWeb Summit? It might not be the pinnacle of technical writing excellence, but at least less confusing.

lidel commented

@mikeal @lgierth got the ball rolling in #92 – took first two stages of upgrade path and documented what can be used "today". I am 😴 now so its bit rough, but take a look in spare moment.

mikeal commented

For reference:

Of particular interest is the signed HTTP exchanges, which open the door to associating an origin with content that was not explicitly retrieved from that origin by the browser.

lidel commented

@mikeal webpackage seems to be also super relevant for:

burdakovd commented

I'd be really interested to see a public ipfs gateway using CID as subdomain: it opens the possibility to host dapps on ipfs for wide audience (who may not have special browser extension or local ipfs node), without risking all kinds of attacks stemming from the fact that all ipfs pages are hosted from the same origin.

It is exciting to see ipfs.dweb.link already supports that, however it doesn't have SSL working (i.e. it serves responses with ipfs.io certificate which is discarded due to domain name mismatch). ipfs.io on the other hand doesn't serve subdomains yet.

Is there any public gateway that supports that yet? Any timeline? I could certainly set up my own, but I would rather avoid that, as serving my own content from my own gateway defeats the purpose of ipfs in this case (that is to prove that the content being served can't be suddenly changed by me).

lidel commented

@burdakovd AFAIK dweb.link is not official yet (certificate is tracked in: ipfs/infra#391) , but the plan for that domain it to move current public gateway to it (from ipfs.io).

@lgierth Do you know if there will be an infra OKR for the move in Q4? or is it too soon?

  • 👍1
burdakovd commented

Until ipfs.io natively supports subdomains, I rolled out my own solution at dapps.earth, using base32 CIDv1 as subdomain (while converting to CIDv0 when querying backend if it was detected that the content was originally uploaded as CIDv0), e.g.:

  • 👍2
  • 🎉4
  • ❤️4
lidel commented

@burdakovd fantastic! Kudos for documenting what and why at https://dapps.earth 👍

  • ❤️1
MichaelMure commented

Would the path from an IPFS hash still work ?

In other word, how would you translate https://ipfs.io/ipfs/QmdmQXB2mzChmMeKY47C43LxUdg1NDJ5MWcKMKxDu7RgQm/195%20-%20Map%20of%20the%20Internet/195%20-%20Map%20of%20the%20Internet.png with the CID as subdomain ?

lidel commented

@MichaelMure Yes, but until we switch to base32 by default you need to manually convert CID encoding from base58 to base32.

Steps:

  1. Identify "root CID" (https://ipfs.io/ipfs/<root-cid>/sub/path)
  2. Convert <root-cid> to Base32 (eg. via ipfs cid base32 <root-cid> → <cidv1b32>)
  3. Open http://<cidv1b32>.ipfs.dweb.link/sub/path/

ps. for testing with your own domain, nginx can be set up like this:

server {
    server_name *.ipfs.foo.tld *.ipns.foo.tld;

    set $ipfspath "";
    if ($http_host ~ ^(.+)\.(ipfs|ipns)\.foo\.tld$) {
        set $ipfspath /$2/$1;
    }
    if ($ipfspath = '') {
        return 404;
    }

    rewrite "^(.*)$" $ipfspath$1;

    location / {
        proxy_set_header Host "";
        proxy_set_header X-Ipfs-Gateway-Prefix "";
        proxy_pass http://gateway;
}
  • 👍1
lidel commented

https://github.com/ipfs/ipfs/issues/337#issuecomment-503661705:

Cool update: Cloudflare officially supports CIDv1B32 in subdomains:
https://blog.cloudflare.com/continuing-to-improve-our-ipfs-gateway/#subdomain-based-gateway

Example: https://bafybeiemxf5abjwjbikoz4mc3a3dla6ual3jsgpdr4cjr3oz3evfyavhwq.cf-ipfs.com/wiki/

We will need to coordinate with them to add support for explicit .ipfs.cf-ipfs.com and .ipns.cf-ipfs.com when ipfs/kubo#5287 lands.

Bren2010 commented

Hey, I wanted to point out that since both of the workarounds listed for long hashes require using multiple labels, those URLs won't be able to support HTTPS. Wildcard certificates only match the first label and there are no double wildcards.

  • 👍3
ilyaigpetrov commented

For a long hashes is it possible to split hash into 2 parts, one is put as a DNS label, another is a prefix in the path, e.g., httpS://bafybeiemxf5abjwjbikoz4mc3a3dla6ual.cf-ipfs.com/3jsgpdr4cjr3oz3evfyavhwq/wiki.
I ask you if it's possible to make such split and preserve the desired sandboxing at the same time.

lidel commented

@ilyaigpetrov sadly no, only the first part (in subdomain) is used for sandboxing

ilyaigpetrov commented

@lidel, well, how about httpS://<short-hash-from-ipns-hash>.box.cf-ipfs.com/<full-length-ipns-hash>. It's just the matter of collisions probability I guess.

lidel commented

Collisions are not the only problem:

  • short and non-cryptographioc hash functions are blocked from being used in CID
  • putting libp2p-keyid (ipns hash) back on path changes the URL root, which breaks websites that use absolute links
  • bad UX (longer URL, introducing new hash that is not even a real CID)

Note: I don't think we will need HTTPS for long CIDs in subdomains any time soon. This discussion is mostly about future-proofing subdomain spec.

ilyaigpetrov commented

I want to try deploying such gateway with wildcard subdomains as cidv1b32 on localhost.
What is the status of current implementation and plans on it?
Will be nice if there are some repos with alpha implementations that reconcile with ipfs team views and plans.
If there is no any implementation then what stack should be used for implementing it myself?

lidel commented

Right now people implement support for subdomains at Nginx, as noted in example:

if ($http_host ~ ^(.+)\.(ipfs|ipns)\.dweb\.link$) {
  set $ipfspath /$2/$1;
  rewrite "^(.*)$" $ipfspath$1 last;
}

Next steps:

  • We want to support subdomains in go-ipfs itself but it is not implemented yet
    (tracked in ipfs/kubo#6498 – if you want to help with landing this sooner, PR would be amazing)
  • Things like http://<cidv1b32>.ipfs.localhost will work on some operating systems, but to ensure it works everywhere we need to add HTTP Proxy mode
    (tracked in ipfs/kubo#5982)
lidel commented

PSA: Native subdomain support will land in go-ipfs v0.5.0.

For now, one can try 0.5.0-rc1 from here or use Docker (ipfs/go-ipfs:v0.5.0-rc1).

https://github.com/ipfs/go-ipfs/blob/master/docs/config.md#gateway-recipes provides one-line example how to enable subdomain gateway on specific host (here: http://{cid}.ipfs.dweb.link)

$ ipfs config --json Gateway.PublicGateways '{
    "dweb.link": {
      "UseSubdomains": true,
      "Paths": ["/ipfs", "/ipns"]
    }
  }'

IPFS Companion v2.11.0 already supports this type of a gateway and redirects it to a local one.

lidel commented

go-ipfs 0.5.0 shipped with native support for subdomain gateways on hostnames defined in Gateway.PublicGateways configuration map.

Daemon configuration for hosting a public gateway can be found at:

lidel commented

I hoped to punt the problem of labls longer than 63 until we need to switch away from sha256 in CIDs, but we may need to solve this problem sooner than expected due to ED25519 keys being new default soon.

Subdomain support for CIDs longer than 63 is tracked in: ipfs/kubo#7318