Only using 65% of GPU memory
tannercollin opened this issue ยท 6 comments
While running some benchmarks I noticed min-dalle was only using 65% of my 3070 Ti GPU's memory:
Here's the function I'm calling:
def run_dalle():
generate_image(
is_mega=True,
text='rich ducks playing poker',
seed=0,
grid_size=3,
top_k=256,
image_path='generated',
models_root='pretrained',
fp16=True,
)
After running it 50 times, each image takes 54.5 seconds on average to generate. It's running baremetal on 32x E5-2630 v3 threads and 64 GB RAM.
Is there a way to make it use more of the GPU? or am I reading this wrong? Thanks!
Also the GPU-Util field fluctuates between 0% and ~40%.
Doesn't seem too off. I use the bfloat16 and get a usage of 41% with 7.7 GB/ 8 GB VRAM. That's with a 3070 (non-Ti)
EDIT:
The time, however seems a little strange, as I am at 11 seconds per image
@78Alpha are you using the Mega model?
@78Alpha are you using the Mega model?
The default for the pip package
EDIT:
Checking back, it is the Mega Version. Will also try non-mega...
Non-mega went to about 9 seconds per image
I'm surprised there's only two seconds difference. Note that mega defaults as off, so you have to pass --mega
into the command line.
I was also getting around 10 seconds in the non-mega version.