bundler-spec-tests failing

[Abhimanyu121]

I have been trying to run bundler-spec-tests on this and they are failing, is this implementation a fully compliant implementation?
I am using eth-infinitism's bundler-spec-tests.

[ch4r10t33r]

Thanks for reporting the issue @Abhimanyu121 . May I know how are you running these tests locally? How many tests are failing. The latest test execution report is available here: https://www.erc4337.io/bundlers (https://github.com/eth-infinitism/bundler-test-executor). Skandha is non conformant to only 1 test.

[Abhimanyu121]

Hey, I am following these steps:

1. Running the local test instance of geth over docker with
   docker build -t geth .
docker run geth --network_mode=host
2. Building and running Skandha
   yarn build
./skandha --testingMode
   I am using the following config

  "networks": {
    "dev": {
      "entryPoints": [
        "0x5FF137D4b0FDCD49DcA30c7CF57E578a026d2789"
      ],
      "relayer": "0xpvtkey",
      "beneficiary": "0x5FF137D4b0FDCD49DcA30c7CF57E578a026d2789",
      "rpcEndpoint": "http://0.0.0.0:8545",
      "minInclusionDenominator": 10,
      "throttlingSlack": 10,
      "banSlack": 10
    }
  }
}

3. Using infinitism's bundler-spec-test from https://github.com/eth-infinitism/bundler-spec-tests
   executing the following
   pdm install && pdm run update-deps
arch -x86_64 pdm run pytest -rA -W ignore::DeprecationWarning --url http://0.0.0.0:14337/rpc --entry-point 0x5FF137D4b0FDCD49DcA30c7CF57E578a026d2789 --ethereum-node http://0.0.0.0:8545
4. I am getting the following output https://app.warp.dev/block/m93FqQmZEdqqUVvfvamVLJ

Can you please check once and let me know if I am doing something wrong.

[0xSulpiride]

@Abhimanyu121 I don't think the 0.0.0.0 IP address is correct, pls try using your local IP address

[Abhimanyu121]

Hey @marie-fourier I am able to hit both bundler and get at 0.0.0.0 as well as 127.0.0.1 and for both of them I am getting the same errors. I am also seeing logs at bundler side for getting different request that tests are making.

[0xSulpiride]

@Abhimanyu121 then it's most likely the issue with geth, try running geth-dev from test folder in the repo
cd test
docker-compose up geth-dev -d

[Abhimanyu121]

Hey, can you double-check the compose file?
Not able to connect to geth and bundler running inside containers.

[0xSulpiride]

@Abhimanyu121 compose file must be ok. You can have a look at how we run the tests - https://github.com/etherspot/bundler-test-executor/tree/master/launchers/skandha

[Abhimanyu121]

Hey, I tried a similar setup and got the following output
https://app.warp.dev/block/vNF7caAftUwhOA3PTkH8Ja

Steps:

1. Started a geth-dev instance.
2. Deployed entrypoint contract from account-abstraction repo
3. Compiled Skandha and executed the bin.
4. Finally triggered the tests, you can see the output in the above link.

[Abhimanyu121]

Hey, I tried forking that test executor and triggered the workflow and the test for most bundlers is failing, is there something I am missing, here is a link to the job execution.
https://github.com/Abhimanyu121/bundler-test-executor/actions/runs/4994721208/jobs/8945673718

[ch4r10t33r]

Hey, I tried forking that test executor and triggered the workflow and the test for most bundlers is failing, is there something I am missing, here is a link to the job execution. https://github.com/Abhimanyu121/bundler-test-executor/actions/runs/4994721208/jobs/8945673718

Hey @Abhimanyu121 Are you still facing this issue when you run the tests against our hosted bundler instance? or when you've configured your bundler to point to a testnet such as Sepolia/Mumbai?

[Abhimanyu121]

Tests are now working, thanks.

[saicharanmadina]

Tests are now working, thanks.

@Abhimanyu121 Could you please let us know how you were able to fix it? Also, please share the documentation on how to run the test cases. I'm facing similar issues