One way to solve this is to use the virtusertable to only accept mail for the specific valid users and reject mail for anyone else.
First list the valid users:
email@example.com firstname.lastname@example.org email@example.com firstname.lastname@example.org email@example.com firstname.lastname@example.org
Don't forget postmaster and abuse (which should go to you)
email@example.com postmaster firstname.lastname@example.org abuse
And then finally have a default entry which calls the error mailer:
@virt.dom error:5.1.1:"550 A-ha, you loose"
This last entry will reject mail to any address other than the 5 listed address.