Unable to Telnet on port 25 outbound but inbound works

I have an Azure VM with SmarterMail installed on it. For the most part the mail server works fine except I do receive a 601 response sometimes. For example:

Failed Recipient: <email address>@<domain>

Reason: Remote host said: 601 Attempted to send the message to the following

ip's:

               216.82.249.212, 216.82.254.19, 216.82.254.20, 216.82.241.100,

               216.82.251.230, 85.158.139.103

I installed Telnet to test connectivity to these ip addresses. When I run telnet 216.82.249.212 25 I get a message that the connection cannot be established. However, if I run Telnet from the outside to connect to this VM it works perfect.

I have an endpoint configured to allow traffic on port 25 to this VM. I have double and triple checked that I have port 25 open in both directions on the firewall as well.

I am at a total loss as to what the issue it.