Adobe recommends implementing a client-side buffer even though it tends to increase the latency in video playback.

Unfavorable network conditions (for example, client bandwidth degradation or loss of connectivity) can affect playback quality. These conditions include frequent pauses, player disconnects, and rebuffering. The player struggles to keep enough data available for smooth, uninterrupted playback under these conditions. Buffering enables the player to maintain seemless playback and to compensate for unfavorable network conditions. Implementing client-side buffering causes delay in video playback as the player tries to preload the buffer before starting playback. The client NetStream.bufferTime default value is 0.1 seconds. This setting causes the player to buffer for 100 milliseconds before it starts playing. Increasing the NetStream.bufferTime increases the perceived latency but the gain in quality is worth it. 

Set initial client buffer sizes relative to content. For example, small values (0.1 seconds to 2 seconds) are sufficient for most live applications. Larger values (3 seconds to 5 seconds) are suitable for video on-demand applications. Consider doubling the initial buffer size to handle H.264 container overhead when streaming HD content. For dynamic streaming, a 6-10 second client-side buffer is considered optimal.