Today consumers are increasingly demanding massive volumes of content across a plethora of networks and devices – managed and unmanaged – expecting consistent quality, service and uninterrupted delivery.

As a result a robust QoE framework is essential for high quality content delivery and customer expectation management. In fact, businesses which fail to deliver superior quality experiences may find themselves edged out of the race for innovation and growth.

Ensuring QoE requires a comprehensive measurement technique – a set of tools and enablers that can objectively map QoE standards / models. The challenge here is that several QoE aspects cannot be easily quantified (and are subjective in nature), requiring a more assimilative intellectual approach.

Ideally, QoE blueprints are built on robust end-user testing scenarios; this however is a highly time consuming and cost intensive exercise. This paper addresses the above concerns, highlighting effective QoE measurement guidelines. It describes the various tools that could be employed to simplify QoE assurance, ensuring accuracy and objectivity. It also discusses the large-scale benefits driving QoE adoption and takes a concentrated look at QoE models for Video-on-Demand (VoD).

QoE Measurement Challenges

QoE is rapidly emerging as a game-changing strategy – a unique and differentiated approach towards improving consumer experiences and service quality.

However, for a QoE mechanism to succeed, one needs an error-free measuring index that computes the various factors involved. For example: Understanding the criticality of Adaptive Bit-rates (ABRs), consumer movements across bandwidths, and quality expectations.

Additionally, with businesses catering to a wide range of customers - across multiple social and usage demographics - QoE measurement requires multiple strategic models, depending on varying circles of influence for each segment.

Below indicates a high-level circle of influence blueprint for QoE Assurance, designed specifically for VoD / OTT providers:

Devices and Applications

• Application - Direct influence in driving improvement
• Device - Minimal influence in driving improvement for standard devices; for specific devices influence may be higher

Network

• Service Provider - Minimal influence unless piping self-owned; ISP partnership significantly enhances influence and improves quality

Services

• Direct influence on driving improvement - With scalability and content delivery as key measures, robust impact and QoE influence
• Content provider to also be considered in this category - Large-scale content availability for multiple networks / devices is of critical importance

Customer Perception

• Minimal influence by marketing aids for perception management
•  Direct influence on application / device / service issue management

QoE Assurance as an Approach - Creating an Effective Model

As a key innovator at your organization, how do you leverage the rising importance of QoE?

What’s the best approach towards engineering a QoE model that’s efficient, delivers advanced insights and balances the many quantifiable and qualitative factors affecting consumer preference?

With a rapidly evolving product and service ecosystem (new devices hitting the market and internet bandwidth penetration expanding across developing markets), it is imperative to continuously enhance QoE measurement / monitoring standards and techniques.

The QoE Assurance approach architects a smart and well-calculated model building a composite perspective of customer experience, while also fortifying attention to accuracy, detailing and precision.

Identification & Feedback

• Identify “Critical for QoE” (KQoEI) parameters
• Launch surveys, monitor user forums and evaluate analytics reports

Setup - Measurement Tool Selection

• QoE analysis set-up installation (tools, technologies and methodologies)
•  Parameter identification and measurement
•  Tools, emulators and simulators for multiple scenario mapping

Accurate Benchmarking

• Benchmark identified parameters with:

1. Industry standard best practices

2. Market analysis and competition

 Focused Priortization

• Priortized parameters to be addressed
•  Categorize the impact, cost and perception to calculate ROI

Robust Implementation

• Implement identified improvements in 4 quadrants
•  Devices, apps, services to be directly upgraded as per key learnings

Effective Measurement & Strategy Rollout

• Execute measurements utilizing defined set-up
•  Employ surveys, user-forms, analytic
• Comprehensive strategic roll-out

Consistent Measurement Standards Upgradation

• Continuous improvement of monitoring techniques
•  Regular overhaul of QoE pathways to keep abreast of market tendencies / device launches / consumer expectations

As you begin constructing your QoE Assurance model, “Content Quality Rating Levels” must be clearly defined - these in turn, would also work as a part of the essential KQoEI parameters.

The following are some of the levers you could use to devise a content rating system:

MOS - Mean Opinion Score

 DMOS - Differential Mean Opinion Score

 MPQM - Moving Picture Quality Metrics

 VSTQ - Video Service Transmission Quality

 VSPQ - Video Service Picture Quality

 VSAQ - Video Service Audio Quality

 MDI - Media Delivery Index V Factor

 JND - Just Noticeable Difference

SSCQE - Single Stimulus Continuous Quality Evaluation

Key Aspects Underlining QoE Assurance

What is the foundation of an end-to-end QoE model? While you go about enunciating your QoE Assurance Roadmap, these aspects will help maximize your QoE outcomes and drive business efficiencies:

Ensure Objective QoE Measurements

•  Deploy tools for multi-condition objective QoE measurements.
•  Modernize MOS (Mean Opinion Score) usage. The most commonly used metric to quantify media quality was previously used via manual intervention. Today, a plethora of tools are available digitizing the process and optimizing timeframes.
•  Undertake network condition simulations using network emulators to deliver an end-to-end perspective. Scenario mapping should include past quality-related incidents.
•  Measure QoE parameters in varying external environments via network emulation tools to enable real-life approach and conditions analysis.

Boost Service Delivery and Operational Efficiency

•  Enable analytics and multi-CDN solutions for load requirement analysis at a particular time, thereby allowing multiple CDN usage, shedding load and ensuring better quality for users.
•  Extend service reliability, availability and scalability analysis to help ascertain maximum load bearing capability, and incremental load-bearing capacity for a temporary situation (Eg, a major sports event).
•  Focus on technology areas such as SDN / NFV to help improve device content delivery.

Enhance User Experience and Device Coverage

• Initiate seamless and similar UX across devices.
•  Maintain consistent device coverage analysis and identify cross platform issues.
•  Migrate to / validate popular platforms as determined by marketing department.

Optimize Performance

•  Improve device software performance.
•  Expand understanding through partnership with QoE tool vendors.
•  Enable in-house point solution for Cloud services performance analysis.

Implement Costs-Managed Delivery

•  Articulate flexible business models for different activities

1.  Outcome based model for cloud service performance analysis

2.  Per cycle model for device coverage of application

3.  Predictive cost model for engineering (development and validation)

QoE Measurements for Video - Challenges and Solutions

An effective flowchart for video QoE monitoring raises a number of challenges – fluctuating / degrading content quality during creation, transcoding / encoding, transmission or playback / rendering.

A simpler alternative is to compare input and output video to develop a comparative picture. A few considerations remain: for static content scenarios ”Full-Reference models” are easier to build while in the case of more complex Adaptive streaming solutions, a ”No-Reference approach“ becomes essential.

In brief here are the referencing alternatives you could select:

Full- Referencing (FR) Objective Testing

Compares the Reference files / segments in server with the processed / streamed file on the user’s side

No-reference (NR) Objective Testing

Analyzes only processed sequences - more suited to in-production video services

Reduced Reference (RR) Objective Testing

Extracts specific information from the reference stream - for utilization during processed stream analysis

QoE Assurance - Benefits and Advantages

Some of the broader effects and outcomes of a structured QoE validation approach are as follows:

Customer Loyalty and Retention: This is particularly true in the case of unmanaged devices / networks - a QoE framework helps to identify issues early - critical issues are unearthed in simulated environments even before product field-trials or launches.

Calibrate Costs Management: QoE measurement plans require efficient cost management blueprints. A QoE validation approach helps to articulate a desired QoE level vs. costs evaluation matrix for each individual operation (costs evaluated include engineering / validation, tools, infrastructure and marketing). As a result, companies have prefect clarity on expenditure considerations and can easily determine the size of their investment based on the specific targeted QoE outcome.

Consistent Developmental Pathway: The most important takeaway from a QoE approach is the increase of consumer expectations. Upon achieving a level of service excellence customers demand a continuous improvement pipeline. This indicates the need to regularly upgrade QoE models and revitalize deployment where “Automation” will, no doubt, play an invaluable role.

The Way Forward

As companies begin to steadily embrace content consumption across a plethora of unmanaged networks and devices, and advanced OTT Video Delivery gains more importance, a cost-effective QoE Assurance model will be critical - ensuring objective measurement with acceptable levels of accuracy, rationalizing implementation architectures.

Additionally, consistent issue identification, progressive automation and an effective tool-box (network tools for simulation and video / audio tools for artefact recognition) will hugely influence the gradual maturity of QoE Assurance models.

As organizations push harder to deliver unmatched quality, QoE models will be at the forefront of a new-era of consumer understanding and synchronized delivery. Guiding the momentum will be state-of-the-art innovation and non-subjective measurement techniques.