Exim4 a powerful Send-Only MTA
MTA (or Mail Transport Agent) can prove helpful if You're hosting Your own server and do not want to pay someone for Your emails. If Properly configured, An MTA can be a good candidate for email transport at small scale for applications like WordPress which only really send emails for Password Reset or new user. It is a very low send volume application so we can use a self-hosted MTA. While I personally prefer a complete suite (e.g. mailcow) because it makes everything that much simple (we get GUI, Additional controls and ability to receive emails too.) but If Your usecase is Send-Only (e.g. You receive emails elsewhere), You can definitely put your VPS to good use.
Since exim isn't very resource hungry, it can easily run along your WordPress application Stack. You can install exim on ubuntu using command sudo apt install exim4
Once it is installed, You'll need some additional configuration on the server to make everything work as expected. Once the Package is installed, Run sudo dpkg-reconfigure exim4-config
It'll open a terminal UI to ask a few configuration parameters. Although the options are self explainatory, I've attached a few screenshots below for reference.
I've skipped a few non-essential settings. You can simply skip those by clicking OK. Most of the settings are self-explainatory so it shouldn't be much difficult to configure everything. The next thing to do here is to create credentials. This can be done using SASL auth. this process is very well explained on the Ubuntu Docs for exim. You can visit that here.
If You get stuck, feel free to discuss this on our forum at discuss.tekduke.com and I'll be happy to assist you with the configuration.