openai/shap-e

Offline use of this ML model

dkwdaft opened this issue · 3 comments

dkwdaft commented

hi, I wondered whether it is possible to use this machine learning model offline? If so, could I have some instructions detailing how to configure it correctly as all tutorials i have seen elsewhere use Google colab. What would be the recommended system requirements too? I would like to replicate the functionality in the code example below, but ideally use it without an Internet connection.




!git clone https://github.com/openai/shap-e.git
%cd shap-e

!pip install -e .



import torch

from shap_e.diffusion.sample import sample_latents
from shap_e.diffusion.gaussian_diffusion import diffusion_from_config
from shap_e.models.download import load_model, load_config
from shap_e.util.notebooks import create_pan_cameras, decode_latent_images, gif_widget




device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')




xm = load_model('transmitter', device=device)
model = load_model('text300M', device=device)
diffusion = diffusion_from_config(load_config('diffusion'))




batch_size = 4
guidance_scale = 15.0
prompt = "shark"

latents = sample_latents(
    batch_size=batch_size,
    model=model,
    diffusion=diffusion,
    guidance_scale=guidance_scale,
    model_kwargs=dict(texts=[prompt] * batch_size),
    progress=True,
    clip_denoised=True,
    use_fp16=True,
    use_karras=True,
    karras_steps=64,
    sigma_min=1e-3,
    sigma_max=160,
    s_churn=0,
)




render_mode = 'nerf' # you can change this to 'stf'
size = 128 # this is the size of the renders; higher values take longer to render.

cameras = create_pan_cameras(size, device)
for i, latent in enumerate(latents):
    images = decode_latent_images(xm, latent, cameras, rendering_mode=render_mode)
    display(gif_widget(images))







from shap_e.util.notebooks import decode_latent_mesh

for i, latent in enumerate(latents):
    t = decode_latent_mesh(xm, latent).tri_mesh()
    with open(f'shark_mesh_{i}.ply', 'wb') as f:
        t.write_ply(f)
    with open(f'shark_mesh_{i}.obj', 'w') as f:
        t.write_obj(f) `
4eJIoBek1 commented

Seems like this webui will be best for you: https://huggingface.co/spaces/hysts/Shap-E (you can gitclone it and run locally)

If youre using windows i have a portable version of webui (that is also works offline) https://huggingface.co/datasets/4eJIoBek/PAIT-Downloads/resolve/main/Shap-E-webui.7z

  • 🚀1
eliecer696 commented

i got one better you can use it in blender letting you see and edit the generated 3d models also it got something called prompt batching that let you generate 3d multiple models from one prompt or multiple models from multiple prompts https://devbud.gumroad.com/l/Shap-e