Prevent remote email from my own domain

How can I block incoming email from remote servers with "mail from: *@mydomain.com" header where mydomain is a domain hosted on my server? I tested this by executing telnet mydomain.com 25
from a different server and set mail from: admin@mydomain.com rcpt to: admin@mydomain.com
The e-mail is accepted. If I set the "rcpt to:" header to a remote domain, then the server properly responds with 550 not permitted to relay, because all users are required to authenticate to relay through my server, but I want all users to be forced to authenticate if the "mail from:" is a local address as well, regardless of recipient.