Low response time from 1 physical server in Azure

We have a database running as an IIS application on a virtual machine in Azure. In reality there are two physical servers. When I open the dashboard tab on manage.windowsazure.com for this VM, I see in a graph that response time from one of the servers is normal (about one second), from the second one - too low (about 7 seconds). The response time from the second server rose yesterday and now everything seems to be stable.

Is there a chance that the problem is in that specific physical server? Or how can we investigate this issue? Thanks in advance.