Degraded performance of VMs as compared to Web App Service.

We are using Web App Service to host our websites. We are planning to move to VMs having improved performace.

However after testing site performance we found that sites hosted on VMs are showing delay in response as compared to site hosted in Web App Service.

1) Web App Service Details:
Size: S2, 3.5 GB memory A=Series Compute,200 Total ACU
Mirror enviornment response time: 6.69 sec

2) VMs enviornment details:
Size : Standard E2s v3 (2 vcpus, 16 GiB memory)
POC environment response time : 8.03 sec

Please Note: Both environment are using same code and pointing to same database
Question: How can I achieve improved performance in VM environment