[Bug] Can not run full node with indexer-grpc configuration

[seiji0411]

🐛 Bug

I tried to run full node with indexer-grpc-fullnode configuration.

This is my fullnode.yaml

base:
    # Update this value to the location you want the node to store its database
    data_dir: "/opt/aptos/data"
    role: "full_node"
    waypoint:
        # Update this value to that which the blockchain publicly provides. Please regard the directions
        # below on how to safely manage your genesis_file_location with respect to the waypoint.
        from_file: "./waypoint.txt"

execution:
    # Update this to the location to where the genesis.blob is stored, prefer fullpaths
    # Note, this must be paired with a waypoint. If you update your waypoint without a
    # corresponding genesis, the file location should be an empty path.
    genesis_file_location: "./genesis.blob"

full_node_networks:
    - discovery_method: "onchain"
      # The network must have a listen address to specify protocols. This runs it locally to
      # prevent remote, incoming connections.
      listen_address: "/ip4/127.0.0.1/tcp/6180"
      network_id: "public"


api:
    enabled: true
    address: 127.0.0.1:8080

storage:
    enable_indexer: true
    
indexer_grpc:
    enabled: true
    address: 0.0.0.0:50051
    processor_task_count: 10
    processor_batch_size: 100
    output_batch_size: 100

I followed indexer-grpc configuration from https://github.com/aptos-labs/aptos-core/tree/main/ecosystem/indexer-grpc/indexer-grpc-fullnode

My fullnode succeed in fast syncing.
But soon, it was stuck with the unknown issue.
I tried to restart node again. I faced the following issue.

panicked at /home/ubuntu/aptos-core/storage/storage-interface/src/cached_state_view.rs:207:57:
Nov 28 20:44:41 apt-7950 bash[3743604]: Must succeed.: Other(\"AptosDB RocksDB Error: IO error: While open a file for random read: /opt/aptos/data/db/ledger_db/007596.sst: Too many open files\")"""
Nov 28 20:44:41 apt-7950 bash[3743604]: backtrace = """
Nov 28 20:44:41 apt-7950 bash[3743604]:    0:     0x63318a98f4fb - <unknown>
Nov 28 20:44:41 apt-7950 bash[3743604]:    1:     0x63318a98f482 - <unknown>
Nov 28 20:44:41 apt-7950 bash[3743604]:    2:     0x63318a9ab793 - <unknown>
Nov 28 20:44:41 apt-7950 bash[3743604]:    3:     0x63318a9ab6c4 - <unknown>
Nov 28 20:44:41 apt-7950 bash[3743604]:    4:     0x6331839cd52d - <unknown>
Nov 28 20:44:41 apt-7950 bash[3743604]:    5:     0x6331839ce620 - <unknown>
Nov 28 20:44:41 apt-7950 bash[3743604]:    6:     0x63318b08c570 - <unknown>
Nov 28 20:44:41 apt-7950 bash[3743604]:    7:     0x63318b08c2b2 - <unknown>
Nov 28 20:44:41 apt-7950 bash[3743604]:    8:     0x63318b08ada6 - <unknown>
Nov 28 20:44:41 apt-7950 bash[3743604]:    9:     0x63318b08bfe4 - <unknown>
Nov 28 20:44:41 apt-7950 bash[3743604]:   10:     0x63318b0b9bd5 - <unknown>
Nov 28 20:44:41 apt-7950 bash[3743604]:   11:     0x63318b0ba223 - <unknown>
Nov 28 20:44:41 apt-7950 bash[3743604]:   12:     0x6331867cd393 - <unknown>
Nov 28 20:44:41 apt-7950 bash[3743604]:   13:     0x633183f94a97 - <unknown>
Nov 28 20:44:41 apt-7950 bash[3743604]:   14:     0x633183f97274 - <unknown>
Nov 28 20:44:41 apt-7950 bash[3743604]:   15:     0x633183f29599 - <unknown>
Nov 28 20:44:41 apt-7950 bash[3743604]:   16:     0x633183f20639 - <unknown>
Nov 28 20:44:41 apt-7950 bash[3743604]:   17:     0x633183f2775d - <unknown>
Nov 28 20:44:41 apt-7950 bash[3743604]:   18:     0x633183f19116 - <unknown>
Nov 28 20:44:41 apt-7950 bash[3743604]:   19:     0x633183e8441d - <unknown>
Nov 28 20:44:41 apt-7950 bash[3743604]:   20:     0x633183ef7dd4 - <unknown>
Nov 28 20:44:41 apt-7950 bash[3743604]:   21:     0x633183f98a71 - <unknown>
Nov 28 20:44:41 apt-7950 bash[3743604]:   22:     0x633183f980c2 - <unknown>
Nov 28 20:44:41 apt-7950 bash[3743604]:   23:     0x633183f96dbd - <unknown>
Nov 28 20:44:41 apt-7950 bash[3743604]:   24:     0x63318406027f - <unknown>
Nov 28 20:44:41 apt-7950 bash[3743604]:   25:     0x63318ad98d0d - <unknown>
Nov 28 20:44:41 apt-7950 bash[3743604]:   26:     0x63318ad91ebf - <unknown>
Nov 28 20:44:41 apt-7950 bash[3743604]:   27:     0x63318ad91bb3 - <unknown>
Nov 28 20:44:41 apt-7950 bash[3743604]:   28:     0x63318ad91956 - <unknown>
...

System information

Please complete the following information:

• Aptos Core Version
  1.23.5 (mainnet)
• Rust Version
  1.78.0
• Computer OS
  Ubuntu

Additional context

I built with source code and used the following command

cargo run -p aptos-node --release -- -f ./fullnode.yaml