Issues with SSH keys

Hi Guys, I've been having a lot of cPHulk notifications recently of attempted logins that have been stopped. So I decided to buckle down on my security even more than I have done. I've got 2 DNSonly servers and one main server running WHM/cPanel, I've managed to disable password authentication on one of the DNSOnly servers and moved it over to an SSH key (for root by the way), but when I've done this on the other machines, it either doesn't recognise the password for the key as correct, or it works once and then never again. The password is 100% correct, I've deleted and removed maybe 10 keys trying to get this to work, it always comes back with this error: Enter passphrase for key '/Users/********/Desktop/*****_idrsa': root@***.uk: Permission denied (publickey,gssapi-keyex,gssapi-with-mic)
I find it a bit strange that this is happening on multiple servers, so what am I doing wrong? Or is something fishy going on here. EDIT: By the way I should say I've ensured the key permissions are 400 etc. it just won't let me login. This is an extended problem and brought about by the fact that I can no longer login to SSH and elevate to sudo through a wheel user that I have been using since day one. The password comes back as incorrect, even when I change the password directly in the terminal in WHM.