SDK integration

[RangerMauve]

What do you think about eventually building dat-webext on top of the Dat SDK?

[sammacbeth]

That would be nice, the high level API looks nice. However I have a couple of reservations:

• The use of universal-dat-storage will likely add quite a bit of bloat to the bundle sizes. This is because 4 different random-access implementations (memory, idb, chrome, idb-mutable-file) will have to be included, though I would only use one. For these kind of cases I prefer the pattern of providing an implementation via options. This is a stylistic different between node, where unused dependencies don't cost much, and the web, where we still have to load all the unused code.
• The swarm instance is not exposed to consumers. I would like to be able to 1. Listen for swarm events to debug networking, and 2. access some internals in order to tweak and improve discovery performance.
• There are a couple of usages of node-only methods, like fs.readyFileSync. How will this work in the browser?

[RangerMauve]

Regarding universal-dat-storage, would you be okay with overwriting it using the browser field instead? The main use case was for the SDK to "just work" across platforms, so I'd like to keep the amount of required configuration down to a minimum. FWIW you can also provide storage via options.

What sort of debugging were you looking for? I'd like to figure out what the common use cases are in application and expose them explicitly with docs so that people won't need to access the swarm directly.

In the case of fs.readFileSync, it'll never get called with browser storage since browsers don't try to do a FS lookup.

[sammacbeth]

Regarding universal-dat-storage, would you be okay with overwriting it using the browser field instead? The main use case was for the SDK to "just work" across platforms, so I'd like to keep the amount of required configuration down to a minimum. FWIW you can also provide storage via options.

Yes, substituting the dependencies at build-time may be sufficient. The sdk requires synchronous storage creation, which may then require some extra boilerplate as this project uses random-access-mutable-file.

What sort of debugging were you looking for? I'd like to figure out what the common use cases are in application and expose them explicitly with docs so that people won't need to access the swarm directly.

Some examples:

• LIstening to swarm events is useful for checking networking is working as expected.
• I'm testing alternative discovery methods. I need access to the discovery instance in order to push peer events so that connections can be managed by discovery-swarm.
• Perhaps plugging extra discovery modules, like discovery-swarm-webrtc.
• Further connection management. I hope to run some tests soon with large swarm sizes (>1000 peers), and this may need some extra logic.

Some of these may require a hyperdiscovery fork anyway, meaning that you don't need to expose APIs, but just exposing the swarm instance should be sufficient for most things.

[RangerMauve]

Hmmm, it might not be worth it to bother with integrating with the SDK then until it has nice APIs for all those things built-in. 🤔 I think if you're managing the swarm and storage directly, the SDK wouldn't be bringing you anything new.

[RangerMauve]

Hmmm, would exposing the swarm and having an option for storage do the trick?

[RangerMauve]

I think your current API is even better than the dat-sdk for this use case since you already have backwards compatibility and all that good stuff. This can be reopened if we end up merging the two some day.