Video and screen sharing are fundamental methods of delivering your message with efficiency and impact. Microsoft Teams is focussing on optimizing these capabilities to provide quality, dependable, and resilient experiences regardless of content type or network constraints.
Video quality
In situations where not enough bandwidth is available for the highest quality video, the encoder must make a trade-off between better picture quality versus smoother frame rate. To optimize the user experience, this trade-off must consider aspects of the source content we are trying to encode. Sharing screen content such as documents with small fonts requires optimizing the spatial resolution for the document to remain legible. Alternatively, if the user shares their desktop while playing a video, it’s important to optimize for a high frame rate, otherwise the shared video will not look smooth. Since shared screen content is often a mix of both scenarios, it’s challenging whether to prioritize high spatial quality or smooth motion. Some apps ask the end-user to choose by exposing a setting for this, which can be a cumbersome and potentially confusing step.
Machine Learning
To make it easier for the end-user, Teams uses machine learning to understand the characteristics of the content the user is sharing to ensure participants experience the highest video quality in constrained bandwidth scenarios. If we detect static content such as documents or slide decks, we optimize for readability. If motion is detected, then we instead optimize for a smooth playback experience. This detection runs continuously on the user’s Teams client to switch easily back and forth. To ensure that this experience is great across devices, the classifier is optimized for hardware offloading and can run in real-time with minimal impact on CPU and GPU usage.
Bandwidth
In addition, when users share content with their webcam enabled, Teams balances video and shared content requirements by adaptively allocating bandwidth based on the needs of each content type. We have recently improved this capability by using machine learning to train models using different network characteristics and source content. This enables Teams to transmit video at higher quality and frame rate while maintaining the screen-sharing experience while utilizing the same bandwidth.
Source: Microsoft