When testing a site's TTFB, it is high only when accessing the site via PHP content. When accessing the domain via a simple text file, the TTFB is low.
In almost all cases of TTFB being slow, it is due to the Web Application (your site code) and the time that it takes to process. The only way to speed up TTFB is to improve the website code itself or use additional caching mechanisms like Redis, Memcache, or otherwise (if the website code supports it).
Here are some troubleshooting steps and explanations:
Run a manual test direct to the server.
Make sure that Cloudflare is not in use to use a direct request.
Use a simple text file.
Replace "APACHE_PORT" with Apache's actual port number. The default port is port 80 unless there are already caching mechanisms or other web servers such as LightSpeed or NGNIX.
Replace "SERVER_IP" with the server's IP where the site is hosted.
Please note, these tests were conducted from a US-based workstation, and the test server was located in France.
cPAgent@cPanel.net [~] # time curl -o /dev/null -silent --resolve domain.tld:tld:APACHE_PORT:SERVER_IP https://doamin.tld:APACHE_PORT/cptestfile.txt
In the above test, the result was a very low TTFB.
Test again using the site's PHP content.
cPAgent@cPanel.net [~] # time curl -o /dev/null -silent --resolve domain.tld:APACHE_PORT:SERVER_IP https://domain.tld:APACHE_PORT
In the example above, the website with the PHP content takes nearly two full seconds longer.
Even when accessed the site through CloudFlare, there are similar processing delays when the site has to process your website's code.
[ Test file ]
cPAgent@cPanel.net [~] # time curl -o /dev/null -silent https://domain.tld/cptestfile.txt
[ PHP content ]
cPAgent@cPanel.net [~] # time curl -o /dev/null -silent https://domain.tld/
The above output reinforces the server itself and Cloudflare, is quick to respond. It is only when the website though the PHP code that the delays occur.
In both these examples, TTFB is seen once PHP processing is complete, so the TTFB is reported poorly for the site despite that TTFB for the server itself is very quick.
The only way to "speed up" PHP processing would be to improve the code itself or provide it with a faster processor. Even then, best practices for web applications are to use heavily caching mechanisms. Whether or not you can take advantage of those caching systems is dependent on your web application and if its designers have coded support for it. Only the developer of the site will know more information.