It's likely one of two issues:
1) DNS
2) Your desktop's routing table
My specific case (also Ubuntu 14.04) turned out to be routing tables. That's what I go through below.
To factor out if DNS is a problem, can you successfully ping an IP from inside your container?
docker run -i -t ubuntu:14.04 /bin/bash
root@44445bfefc4e:/# ping 8.8.8.8
PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data.
64 bytes from 8.8.8.8: icmp_seq=1 ttl=54 time=76.1 ms
64 bytes from 8.8.8.8: icmp_seq=2 ttl=54 time=75.6 ms
If that works and your containers still can't reach out while you're connected to the VPN, look at your routes.
Disconnect from the VPN and inspect your routes with route. Here's my output as an example:
Destination Gateway Genmask Flags Metric Ref Use Iface
default DD-WRT 0.0.0.0 UG 0 0 0 wlan0
172.17.0.0 * 255.255.0.0 U 0 0 0 docker0
192.168.1.0 * 255.255.255.0 U 9 0 0 wlan0
192.168.122.0 * 255.255.255.0 U 0 0 0 virbr0
In there, you'll see Docker's network (172.17.0.0).
Now, connect to your VPN and re-issue the command. Your mileage may vary, but what I found was a duplicate entry in the route table:
172.17.0.0 * 255.255.0.0 U 0 0 0 vpn0
172.17.0.0 * 255.255.0.0 U 0 0 0 docker0
The server was pushing a duplicate route!
In my case, I didn't need those routes to successfully navigate the VPN, so I found a way of disabling them. I use OpenVPN, so I drilled down in the settings in the dialog and checked the 'Ignore automatically obtained routes'.

That image is from this blog post.
Once I checked that and reconnected to the VPN, I no longer had the duplicate entry and my Docker containers were able to connect to the Internet and to hosts inside the VPN.