tomeshnet/ipfs-live-streaming

ipfs pin add performance

Opened this issue · 3 comments

ipfs pin add takes about a minute, too slow for pinning 15 second ts chunks:
https://github.com/tomeshnet/ipfs-live-streaming/blob/master/terraform/ipfs-mirror/ipfs-pin.sh#L6

Our production endpoint live.mesh.world is currently running:

#!/bin/sh

IPFS_REF=$1

# Pin IPFS content
if [ -z "$(cat /home/ipfs/pin_log | grep $IPFS_REF)" ]; then
  echo $IPFS_REF >> /home/ipfs/pin_log
  wget -q "https://ipfs-gateway.mesh.world/ipfs/$IPFS_REF" -O "/tmp/$IPFS_REF"
  ipfs add /tmp/$IPFS_REF > /dev/null
  ipfs pin add $IPFS_REF > /dev/null   &
  rm -rf /tmp/$IPFS_REF

  mv /home/ipfs/pin_log /home/ipfs/pin_log.old
  tail -n 10 /home/ipfs/pin_log.old > /home/ipfs/pin_log
  rm -rf /home/ipfs/pin_log.old

fi

The ipfs-mirror instantiated by terraform ipfs-mirror-0.mesh.world pins with:

#!/bin/sh

IPFS_REF=$1

# Pin IPFS content
ipfs pin add /ipfs/$IPFS_REF

Problem continues with IPFS 0.4.22

a contact swarm connect was added to mirrors pointing at the main server. This seems to allow for quick ipfs add pin