Poor quality in responses and image recognition
apresence opened this issue · 1 comments
I'm using the 7b model on Ubuntu 22.04 running a RTX 4090 FE, Cuda 12.2 drivers, and I used the docker compose yaml. I also have an RTX 8000 in the system, but I have configured Docker to only enable the 4090 in the container:
devices:
- driver: nvidia
device_ids: ['0']
capabilities: [ gpu ]
Image recognition seems poor. I uploaded a picture of a MacBook and first it said it was an Acer running Windows, then it said it was Ubuntu with a welcome screen.
It will often reply "I'm sorry, but I'm unable to comply with that request" or similar to innocuous requests such as "describe this image" (of a cat).
Also, sometimes it will start repeating itself over and over until the token limit has been reached. I've attached a screenshot for reference.
OS/Config info:
# uname -a
Linux tau 6.5.0-41-generic #41~22.04.2-Ubuntu SMP PREEMPT_DYNAMIC Mon Jun 3 11:32:55 UTC 2 x86_64 x86_64 x86_64 GNU/Linux
# nvidia-smi
Wed Jun 26 03:45:59 2024
+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 535.183.01 Driver Version: 535.183.01 CUDA Version: 12.2 |
|-----------------------------------------+----------------------+----------------------+
| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|=========================================+======================+======================|
| 0 NVIDIA GeForce RTX 4090 On | 00000000:01:00.0 Off | Off |
| 0% 40C P5 69W / 450W | 14679MiB / 24564MiB | 0% Default |
| | | N/A |
+-----------------------------------------+----------------------+----------------------+
| 1 Quadro RTX 8000 On | 00000000:04:00.0 Off | Off |
| 33% 33C P8 12W / 260W | 3MiB / 49152MiB | 0% Default |
| | | N/A |
+-----------------------------------------+----------------------+----------------------+
+---------------------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=======================================================================================|
| 0 N/A N/A 61418 C python 14128MiB |
| 0 N/A N/A 61978 C /usr/bin/python3 540MiB |
+---------------------------------------------------------------------------------------+
Let me know if there are additional details I can provide. I'm also happy to test new code/settings or provide logs if it will be helpful.
This is not unexpected behavior. Asking less open ended questions about passed images should help.