ipfs-rust/ipfs-embed

[noob] what does "no go-ipfs compatibility" mean?

akavel opened this issue · 8 comments

In the readme, I noticed the following disclaimer:

It does not aim at being compatible in any way with go-ipfs.

Could I kindly ask if you would fancy trying to help me understand what does this mean, for an IPFS noob? Most notably, does it mean:

  • (a) that files shared on IPFS by go-ipfs or js-ipfs are not visible/gettable from ipfs-embed, and reverse? (i.e. they are completely "separate ecosystems/networks")
  • (b) or, that the files are cross-visible, but just the "local on-disk database and contents of downloaded/shared files" is not compatible? i.e. in other words, they are "network compatible", but ipfs-embed is just not (and will not be) a drop-in replacement for go-ipfs and reverse?

TIA! ❤️

(edit: In my particular case, I'm interested in writing a small personal-use app for sharing photos over IPFS - more or less trying to rewrite https://github.com/akavel/catation in Rust - and wonder if that's currently something I could manage to achieve in Rust somehow)

You can configure ipfs-embed to work with go-ipfs, by setting the bootstrap nodes and using the unixfs crate and writing a little bit of code to hook it all up. Ipfs-embed works at a lower level of abstraction, suitable for embedding into different kinds of applications

You can configure ipfs-embed to work with go-ipfs, by setting the bootstrap nodes and using the unixfs crate and writing a little bit of code to hook it all up. Ipfs-embed works at a lower level of abstraction, suitable for embedding into different kinds of applications

Is there an example repo somewhere of this working? Go-ipfs compatibility seems like a common requirement, and it'd be nice to be able to reuse something proven to work. 😁

vmx commented

@RangerMauve not proven, but that worked for me. I wanted to create a proper example, but stopped in the middle. Here's some code that gets a block from an go-ipfs node that worked for me:

use futures::prelude::*;
use ipfs_embed::{Config, Ipfs, Multiaddr, PeerId, BitswapConfig};
use libipld::{store::StoreParams, Cid, IpldCodec};

#[derive(Debug, Clone)]
struct Sp;

impl StoreParams for Sp {
    type Hashes = libipld::multihash::Code;
    type Codecs = IpldCodec;
    const MAX_BLOCK_SIZE: usize = 1024 * 1024 * 4;
}

fn tracing_try_init() {
    tracing_subscriber::fmt()
        .with_env_filter(tracing_subscriber::EnvFilter::from_default_env())
        .init()
}

#[async_std::main]
async fn main() -> anyhow::Result<()> {
    tracing_try_init();
    let mut config = Config::default();
    config.network.bitswap = Some(BitswapConfig::default());
    let ipfs = Ipfs::<Sp>::new(config).await?;
    let peer: PeerId = "12D3KooWGK6RaVsSwA3QuTykaFKwtptDM9PjAM2n54cHpK8rYGSS".parse()?;
    let addr: Multiaddr = "/ip4/192.168.0.12/tcp/4001".parse()?;
    ipfs.dial_address(&peer, addr);

    let go_ipfs_root_cid: Cid = "QmQPeNsJPyVWPFDVHb77w8G42Fvo15z4bG2X8D2GhfbSXc".parse()?;

    let block = ipfs.fetch(&go_ipfs_root_cid, vec![peer]).await?;
    println!("got single block. len = {}", block.data(`).len());

    Ok(())
}

Please also note that you need to use current master and set the compat feature flag.

@vmx I tried your sample code since I'm interested with go-ipfs compatibility too, but got those errors:

ERROR ipfs_embed: block evicted too soon. use a temp pin to keep the block around.
ERROR libp2p_bitswap::behaviour: error inserting blocks Unsupported codec 112.
Error: Failed to retrive block QmQeizMfyTvv1z4AG3AxSgdxKaUTVkctABfHsYV4qsHFu1.

I understand that bitswap does not handle MerkleDAG protobuffer codec but don't get if there is an easy way for supporting it. Any advice will be appreciated.

Could I kindly ask if you would fancy trying to help me understand what does this mean, for an IPFS noob?

Well the intention behind it is don't spam my issue tracker, but since that doesn't work... You can enable dag-protobuf in libipld for the second error. For the first error make sure you pin the block.

@dvc94ch many thanks, it worked with the dag-pb feature. The first error disappeared at the same time (without pinning).

So not being able to decode meant that the cache system didn't work. You should be pinning stuff you want to keep around and not rely on the cache that may remove your block at any time

rkuhn commented

This looks like it was resolved.