Websites Works fast after restarting FPM

I have dedicated server of ovh Intel Xeon D-1521 - 4c/8t - 2.4GHz /2.7GHz RAM: 16GB DDR4 ECC 2133MHz i am facing issue with my server response time normaly website takes 60 to 68 sec to connect every thing i have double checked my server load time every thing seems ok to me but when i restart php FPM websites loads fasts and then again after some time the same situation so i have to restart the fpm again any one can guide me what is the reason behind this