If you are using a single video, check the bit rate of the file. A high bit rate could lead to substantial buffering time even on high-quality connections.
The recommended method is to use Adaptive Video Sets. Make sure that there are not too many options configured. The default preset uses five: two for mobile, two for tablet, and one for desktop. If you have modified the default, check the quality settings of the device you are testing against. For example, if you have too high a bit rate selected for phones, it can't work optimally.
Here is the Help page with more information on AVS:
You can troubleshoot specific network issues with Scene7 Tech Support.