multi-node inference for llama3 70b

[Abolfazl-kr]

Check before submitting issues

• Make sure to pull the latest code, as some issues and bugs have been fixed.
• I have read the Wiki and FAQ section AND searched for similar issues and did not find a similar problem or solution
• Third-party plugin issues - e.g., llama.cpp, text-generation-webui, we recommend checking the corresponding project for solutions

Type of Issue

Model inference

Base Model

Others

Operating System

Linux

Describe your issue in detail

Hi,
Thanks for your effort and releasing llama3 training.
I have two gpu server with 8 T4 16GB GPU on each. 
how could i inference llama3 70b multi-node?
it needs more than 8 gpu so i need to use both gpu server.
thanks already for your response.

Dependencies (must be provided for code-related issues)

# Please copy-and-paste your dependencies here.

Execution logs or screenshots

# Please copy-and-paste your logs here.

[Abolfazl-kr]

can anybody please help me?

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your consideration.

[github-actions]

Closing the issue, since no updates observed. Feel free to re-open if you need any further assistance.