Load Balanced Endpoints connection timeout

I'm trying to set up a cloud service consisting of three virtual machines. I basically follow this article: hanselman.com/blog/HowToSetupALoadBalancedWebFarmOfVirtualMachinesLinuxOrOtherwiseOnWindowsAzureCommandLine.aspx

My VM's are linux boxes, running nginx on port 80 that proxies to some node apps. They have no firewall setup yet. I configured my load balanced endpoints using the following command, it deviates a bit from the article mentioned above:

azure vm endpoint create-multiple uyb-app-farm 80:80:tcp:true:http-traffic:http:80:/host

I do this for each vm. I tried to do the same using the azure portal. But the result is the same. I can see the successful connections by the Load Balancer Agent in my webserver logs and I can connect to it using localhost and the actual IP address of the virtual machine (when I'm ssh'd into the box) but not when curling the load balanced ip. This is my output of showing the vm details:

azure vm show uyb-app-farm  
info:    Executing command vm show+ Getting virtual machines                                                     
data:    DNSName "uyb-app-farm.cloudapp.net"
data:    Location "West Europe"
data:    VMName "uyb-app-farm"
data:    IPAddress "100.90.190.14"
data:    InstanceStatus "ReadyRole"
data:    InstanceSize "Small"
data:    InstanceStateDetails ""
data:    OSVersion ""
data:    Image "uyb-app-image"
data:    OSDisk HostCaching "ReadWrite"
data:    OSDisk DiskName "uyb-app-farm-uyb-app-farm-0-201404250948260961"
data:    OSDisk MediaLink "http://portalvhdspzc1vcnsgcrl7.blob.core.windows.net/vhds/45jrybql.kbv201404250948250789.vhd"
data:    OSDisk SourceImageName "uyb-app-image"
data:    OSDisk OS "Linux"
data:    DataDisks ""
data:    Network Endpoints 0 LoadBalancedEndpointSetName "http-app"
data:    Network Endpoints 0 LocalPort "80"
data:    Network Endpoints 0 Name "HTTP"
data:    Network Endpoints 0 Port "80"
data:    Network Endpoints 0 LoadBalancerProbe Port "80"
data:    Network Endpoints 0 LoadBalancerProbe Protocol "tcp"
data:    Network Endpoints 0 LoadBalancerProbe IntervalInSeconds "15"
data:    Network Endpoints 0 LoadBalancerProbe TimeoutInSeconds "31"
data:    Network Endpoints 0 Protocol "tcp"
data:    Network Endpoints 0 Vip "137.117.246.59"
data:    Network Endpoints 0 EnableDirectServerReturn "true"
data:    Network Endpoints 1 LocalPort "22"
data:    Network Endpoints 1 Name "ssh"
data:    Network Endpoints 1 Port "22"
data:    Network Endpoints 1 Protocol "tcp"
data:    Network Endpoints 1 Vip "137.117.246.59"
data:    Network Endpoints 1 EnableDirectServerReturn "false"
info:    vm show command OK

So what do I miss. I looked into the docs, but I think I did everything that is said there. Are there any other ways how I could debug this issue? Anything I forget?