Enable XCM assets on Pendulum

Question

Enable XCM assets on Pendulum

Closed this issue 10 months ago · 30 comments

Context

In order to move an asset from a parachain to Pendulum, the asset needs to be enabled, and opening an HRMP channel wouldn't be enough.
e.g. Enabling USDT on Pendulum from Asset Hub

Requirement

Based on the recent HRMP channels that were opened, and some of the upcoming ones, purpose of this ticket is to enable the following assets on Pendulum:

Asset	Origination	Asset ID	Decimals	Source
USDC	Asset Hub	1337	6	subscan
EQD	Equilibrium	6,648,164	9	Telegram
PDEX	Polkadex		12	subscan
EURC	Pendulum
BRZ	Moonbeam		18	subscan

USDC
EQD
PDEX
EURC
BRZ

Answer 1 · 2023-08-08T19:53:40.000Z

@annatekl Can you specify what EQ and EQD are? It looks to me like Equilibrium has only the native asset and nothing more. In this comment you write that EQ seems to be the native token and EQD its native decentralized stablecoin but I have problems to find where on the runtime the latter coin resides (stored and managed).

Answer 2 · 2023-08-09T08:14:20.000Z

I did checked there whitepaper and what I can see is:

EQ tokens are stored in user account balances within the Equilibrium blockchain's storage system.
Users can query their EQ balances using storage queries compliant with Polkadot.JS storage interfaces

https://www.npmjs.com/package/@equilab/api
Query storage
Storage queries are compliant with Polkadot.JS storage interfaces.

Get balances from storage method is using currencyFromU64 to decode asset u64 id into token name (eg 'eq')

import { currencyFromU64, u64FromCurrency } from "@equilab/api/equilibrium";

function getBalances(api: Api) {
return async function (account: string): Promise<{
ok: boolean;
lock?: string;
balances?: Map<string, string>;
}> {
const accountInfo = await api._api.query.system.account(account);
if (!accountInfo.data.isV0) return { ok: false };

const lock = accountInfo.data.asV0.lock.toString(10);

const balances = accountInfo.data.asV0.balance
  .toArray()
  .reduce(
    (acc, [id, balance]) =>
      acc.set(
        currencyFromU64(id),
        balance.isPositive
          ? balance.asPositive.toString(10)
          : `-${balance.asNegative.toString(10)}`,
      ),
    new Map<string, string>(),
  );

return { ok: true, lock, balances };

};
}
Successfull output looks like this:

{ ok: true, lock: '22000000000', balances: { 'eq' => '72000000000', 'dot' => '597056440465' } }
Balance has 1e9 decimals places and thus '72000000000' equals 72 eq.

the documentation doesn't explicitly detail the runtime storage of EQD, but seems like that EQD would be managed using mechanisms similar to those mentioned for EQ tokens

Answer 3 · 2023-08-09T11:41:49.000Z

confirmed by Equilibrium:
We have Assets pallet which is a storage of asset parameters.

All of our assets are 9 decimals
Balances are stored in a system pallet.

Asset Ids may be fetched using polkadot.js and the following script (Developer -> Javascript)

function hex2a(hexx) {
var hex = hexx.toString();
var str = '';
for (var i = 0; i < hex.length; i += 2)
str += String.fromCharCode(parseInt(hex.substr(i, 2), 16));
return str;
}

const assets = (await api.query.eqAssets.assets()).toJSON();

for (const asset of assets) {
const assetSymbol = hex2a(asset.id.toString(16)).padStart(5, ' ');
const assetId = asset.id.toString(10).padStart(13, ' ');
console.log(Asset: ${assetSymbol} assetId:${assetId});
}

Answer 4 · 2023-08-09T18:35:29.000Z

Okay, thank you, understood.

Two things to remark:

It seems like their parachain is closed source – at least their open source repo is very out of date (almost three years old) and does not contain this asset pallet yet. Can you ask where to find the open source repository of their chain, please @annatekl? This is crucial particularly so that we can see the index of their asset pallet.
This assets pallet is not the standard asset pallet but a pallet that they implemented, it is actually called eqAssets in their runtime.

Using their definition in the previous comment I can conclude that the assets ids are as follows:

EQ: asset id is 25,969
EQD: asset id is 6,648,164

Hence, I assume that these assets are defines as the following xcm multilocations:

MultiLocation {
  parents: 1,
  interior:
    X3(
      Parachain(<equilibrium paraId>),
      PalletInstance(<index of eqAssets>),
      GeneralIndex(<25969 or 6648164>),
    ),
  }

Answer 5 · 2023-08-10T14:19:28.000Z

@annatekl as mentioned by @TorstenStueber , let's consider adding tokens from parachains that we want to integrate with after Equilibrium too.

Answer 6 · 2023-08-10T14:51:55.000Z

@vadaynujra I guess that you specifically refer to Polkadex?

Answer 7 · 2023-08-10T15:10:43.000Z

@TorstenStueber this is correct:

EQ: asset id is 25,969
EQD: asset id is 6,648,164

Repo link

Answer 8 · 2023-08-10T15:19:27.000Z

Oh thanks for the link. This is really strange: the official Equilibrium website only links to this GitHub, which is a completely different GitHub organization and only contains their outdated parachain code.

Could be helpful to give them this hint so that they can change it.

Answer 9 · 2023-08-21T09:09:21.000Z

Hey team! Please add your planning poker estimate with Zenhub @adelarja @ashneverdawn @b-yap @ebma @TorstenStueber

Answer 10 · 2023-10-05T10:10:26.000Z

Consider adding:

USDC
EQD (already covered)
PDEX
EURC (natively issued)

Answer 11 · 2023-11-08T06:53:08.000Z

@ebma I have added the details for the assets (EURC is going to be issued on Pendulum), PDEX is a native token on polkadex so not sure about the id and it was not on the subscan, the same for BRZ - no details

Answer 12 · 2023-11-08T13:11:39.000Z

equilibrium asset info found in asset.rs

pallet-asset instance id found in lib.rs

para id:

Answer 13 · 2023-11-08T14:39:34.000Z

For the BRZ token, the multilocation should be the following:

{
  parents: 1,
  interior: {
    X3: [
      { 
        Parachain: 2004
      },
      {
        PalletInstance: 110
      },
      {
        AccountKey20: {
          key: '0xD65A1872f2E2E26092A443CB86bb5d8572027E6E'
        }
      }
    ]
  }
}

This is documented here, by inserting the address of this smart contract into the respective field.

Answer 14 · 2023-11-10T07:39:31.000Z

need Polkadex.

Answer 15 · 2023-11-10T07:49:33.000Z

@b-yap do you need asset id of polkadex?

Answer 16 · 2023-11-10T08:11:15.000Z

@annatekl yes the assetId of PDEX, or similar to what Marcel shared; which is as AccountKey20.

Though I think the pallet instance is 25 but please correct me if I'm wrong.

Answer 17 · 2023-11-13T09:56:27.000Z

@b-yap I found that they use a crate called xcm-helper which defines the conversion functions e.g. here. But I'm still not sure what the MultiLocation of their native asset would look like.
@pendulum-chain/product if we have some communication channel with them, can you please ask Polkadex what the MultiLocation of their native asset looks like?

Answer 18 · 2023-11-13T10:18:44.000Z

@b-yap @ebma I asked via telegram, they provided me with Multilocation: {parents: 1, interior: {x1:{parachain: 2040}}}

Answer 19 · 2023-11-14T11:14:05.000Z

for EURC, what are the ff. tickets that need to be resolved first?
https://github.com/pendulum-chain/tasks/issues/87
https://github.com/pendulum-chain/tasks/issues/128
#310
#331
#334

^ I'm naming a few, but I'm not quite sure
@annatekl

Answer 20 · 2023-11-14T11:50:57.000Z

The only real blocker I see is #342. Once that is merged, you should be able to configure EURC. Though we did not yet specify how we want the MultiLocation of our new CurrencyId::Token types to look like externally. We would also need to discuss this in the tech team. Do you have a suggestion for the MultiLocation @b-yap?

Answer 21 · 2023-11-14T13:33:07.000Z

@ebma I checked all the variants in Junction, and what makes sense (to me) is the GeneralIndex.
We could follow how Equilibrium does it, with Asset(<value>) where <value> came from fn from_bytes()

So it will be

MultiLocation {
	parents: 1,
	interior: X3(
		Parachain(id),
		PalletInstance(10),
		GeneralIndex(from_bytes(b"EURC"))
	),
}

and

MultiLocation {
	parents: 0,
	interior: X2(
		PalletInstance(10),
		GeneralIndex(from_bytes(b"EURC"))
	),
}

Ah but with Token(u64), Will EURC be Token(0) ? Token(1) ?

Answer 22 · 2023-11-15T18:03:13.000Z

Hmm interesting. I would put EURC as Token(0) but we can use anything.
For the MultiLocation, I'm not sure. First of all, we should probably change the number in PalletInstance to point to the Tokens pallet as 10 points to our Balances pallet which only makes sense for our native token.

Now that we point to our Tokens pallet, we still need to map the asset somehow to our CurrencyId enum. We should probably already think about how we can map any type of our CurrencyId enum to a MultiLocation so we should also consider Stellar-type assets in addition to the CurrencyId::Token().

We could use a combination of GeneralIndex and GeneralKey with the index pointing to the index of the type in our CurrencyId enum, with Stellar being index 2 and Token being index 4, and then putting the actual specifier into the GeneralKey.

WDYT @b-yap and also @pendulum-chain/devs?

Answer 23 · 2023-11-15T18:42:03.000Z

I think the solution of using the junction GeneralIndex and GeneralKey would work very well. Because even if we later would like to use GeneralIndex for other MultiLocations that do not fall into the category of the CurrencyId enum, we still have a very large space (u128) to represent them.
I imagine that for Stellar assets the key would be something like GeneralKey([from_bytes(b"USDT", b"Issuer..."].concat()) right?

Answer 24 · 2023-11-16T08:00:02.000Z

@gianfra-t The GeneralKey has a max data length of 32. b"USDT" + b"Issuer..." might be difficult.
Is it possible to have multiple GeneralKey junctions to exist? Although usage of the GeneralKey as pointed in the comment, must be avoided (if possible).

@ebma I'm using the mod latest, so this is I think the link of GeneralKey.
Is there a preference to use v2 rather than v3 immediately?

MultiLocation {
	parents: 0,
	interior: X3(
		PalletInstance(53), // 53 is Token pallet
		GeneralIndex(4),
		GeneralKey{ ?? }
	),
}

Let's sayEURC is Token(0); will the GeneralKey represent the "0" ? It makes sense for it to be 0.

Answer 25 · 2023-11-16T10:06:31.000Z

Is it possible to have multiple GeneralKey junctions to exist?

Yes we can use those multiple times.

Although usage of the GeneralKey as pointed in the comment, must be avoided (if possible).

I don't think we need to avoid it, the comment is probably meant to make people try using more specific items if possible. But there is no problem if we don't.

Is there a preference to use v2 rather than v3 immediately?

Not really, I just chose the first that I found 😅

Let's say EURC is Token(0); will the GeneralKey represent the "0" ? It makes sense for it to be 0.

Yeah we could do that.

The GeneralKey has a max data length of 32. b"USDT" + b"Issuer..." might be difficult.

So for Stellar assets we could also use two consecutive GeneralKeys like

interior: X4(
		PalletInstance(53), // 53 is Token pallet
		GeneralIndex(2),
		GeneralKey{ length: 4, // or 12 if it is an AlphaNum12
                                       data: code  
                 },
                 GeneralKey{ length: 32,
                                       data: issuer  
                 },

	),

Answer 26 · 2023-11-16T10:10:45.000Z

Here are a few unrelated thoughts.

@b-yap pointed out that using the value Token(0) to stand for EURC is not elegant. I agree and this was actually my misguiding input. However, I assume that this now already rolled out on Amplitude (?) and therefore too hard to change again (?)
Whether we use V2 or V3 should not matter. Usually V3 is just an extension of V2 and everything we an define as V2 would automatically internally converted to V3 if applicable. This conversion is transparent and we don't need to care about this (take this advice with care).
I agree with Marcel's previous comment. I think it's better to be explicit and therefore prefer to use multiple GeneralIndex and GeneralKey combinations.

Answer 27 · 2023-11-16T10:44:41.000Z

@TorstenStueber @ebma

However, I assume that this now already rolled out on Amplitude (?) and therefore too hard to change again (?)

-> #342 merged just recently.

Answer 28 · 2023-11-16T12:35:53.000Z

Oh, merged so recently that it is not part of the Amplitude release yet so that we could still change this.

Answer 29 · 2023-11-16T16:02:26.000Z

I would propose the following. Considering our current CurrencyId enum

pub enum CurrencyId {
	Native = 0_u8,
	XCM(u8),
	Stellar(Asset),
	ZenlinkLPToken(u8, u8, u8, u8),
	Token(u64),
}

Mappings

Native

We actually define a different MultiLocation for our Native asset, referring to the Balances pallet.

MultiLocation {
	parents: 0,
	interior: X1(
		PalletInstance(10),
	),
}

XCM

As these are all foreign assets, their MultiLocation is defined by the source chain.

Stellar

For StellarNative we might just leave out

MultiLocation {
	parents: 0,
         interior: X2(
		PalletInstance(53), // 53 is Token pallet
		GeneralIndex(2),
	),
}

For Stellar::AlphaNum4

MultiLocation {
	parents: 0,
        interior: X4(
		PalletInstance(53), // 53 is Token pallet
		GeneralIndex(2),
		GeneralKey{ length: 4,  data: code }, // eg. b"USDC"
                 GeneralKey{ length: 32, data: issuer  }, // the raw binary of the public key
	),

For Stellar::AlphaNum12

MultiLocation {
	parents: 0,
        interior: X4(
		PalletInstance(53), // 53 is Token pallet
		GeneralIndex(2),
		GeneralKey{ length: 12, data: code  }, // eg b"USDC\0\0\0\0\0\0\0\0"
                 GeneralKey{ length: 32, data: issuer },
	),

ZenlinkLPToken

We need to encode the four u8s

MultiLocation {
	parents: 0,
        interior: X4(
		PalletInstance(53), // 53 is Token pallet
		GeneralIndex(3),
		GeneralKey{ length: 4, data: [u8, u8, u8, u8]  }, // We can just concatenate them
	),

Token

MultiLocation {
	parents: 0,
        interior: X4(
		PalletInstance(53), // 53 is Token pallet
		GeneralIndex(4),
		GeneralIndex(index), // The index of our token in the token enum
	),
}

Answer 30 · 2023-11-16T16:36:12.000Z

@ebma Great, I like it.