So what is currently the best and easiest way to use an AMD GPU for reference I own a rx6700xt and wanted to run 13B model maybe superhot but I’m not sure if my vram is enough for that Since now I always sticked with llamacpp since it’s quiet easy to setup Does anyone have any suggestion?
Yes thank you for the information I really appreciate it! I decided to go for kobold.cpp for the meantime with CLBlast which works just overall way better than standart CPU inference. But im looking into the ROCm LLamacpp support which I am currently trying.