Interest in artificial intelligence (AI) is growing day-by-day. This is confirmed by the ubiquitous presence of AI algorithms in a plethora of applications. Gianni Rosa Gallina, Senior Software Engineer at Deltatre, delivered an interesting speech at Codemotion Milan 2018, where he showed how and where AI can be used in the development of an OTT multimedia platform. In this post we consider the OTT platform described by Gianni Rosa Gallina as a case study for understanding which kinds of facilities can be provided by AI.
Why an OTT platform
OTT (over-the-top) is a term used in broadcasting and technology business reporting to refer to audio, video, and other media transmitted via the Internet as a standalone product. Thus, the architecture of an OTT platform may involve several actors, including CMS, third parties, data, end-user apps, and so on. Gianni Rosa Gallina showed the architecture of the Deltatre’s OTT platform for sport events, which is depicted by the following pictures.
For every component of this architecture, there is an area where AI can be used. All these areas are highlighted in green, and the following discussion will focus on the AI tools which have been used to support such areas.
Customer Support
Customers represent one of the main actors involved in an OTT platform. Their interaction with the platform can be measured by looking at the tickets they open, since they represent potential issues and are an important index for customer satisfaction. Of course, it is crucial to understand the level of satisfaction, and even better if this can be predicted. To this end, Deltatre adopted Zendesk Satisfaction Prediction, a tool that uses several metrics (e.g. how long the customer waited for a reply, how long a ticket has been open, ticket content, level of effort needed to solve a ticket) in order to predict bad customer satisfaction ratings before they happen. It uses sophisticated machine learning algorithms that serves as a predictor model.
Another interesting AI-based Zendesk product (also included by Deltatre to improve their OTT platform) is Content Cues, which automatically review incoming tickets, recognise them and then classify them. This is crucial for speeding up the process for replying to customers, and consequently avoiding potential satisfaction issues.
Analytics and Monitoring
Deltatre also uses Conviva Video AI Platform, another tool that uses AI in order to (indirectly) detect potential issues in consumer experience. In particular, this tool is used for quickly tracking down exactly what entity in the end-to-end streaming hierarchy might be responsible for problems in service delivery. This automatic evaluation is done based on video-based metrics, like bandwidth or video quality. Conviva Video AI Platform can also be used for making content recommendations, or in order to understand device usage.
Platform Monitoring
In the architecture depicted above, end-user apps have a crucial role. And according to the recent trend in app development, they are probably built using web technologies. Consequently, understanding potential performance problems in web applications is mandatory, since it would help in increasing the global customer satisfaction level. Deltatre opted for Smart Detection, a feature of Application Insights (from the Microsoft Azure platform) that automatically warns in case of potential performance problems in web applications. It performs proactive analysis of the telemetry sent by the web app to Application Insights. If there is a sudden rise in failure rates, or abnormal patterns in client or server performance, an alert is sent.
Content Editing
The OTT platform that we are analysing also includes a CMS. So it is important to support content creators in the process of content generation. Deltatre developed a smart editor, which in turns exploits some interesting features provided by the Microsoft Cognitive Services. In particular, the editor analyses the text, and applies sentiment analysis algorithms in order to support the user with suggestions. Moreover, the smart editor also exploits a Bing search API in order to combine information written by the user with suggestions found on the Internet that can be included in the text in order to enrich it. And all the process is in real time, during the content generation.
Video Indexing
Since an OTT platform is mainly based on multimedia content, being able to automatically analyse videos is essential. Deltatre provides a video content indexer tool (namely Diva) that exploits Microsoft Video Indexer. This tool automatically extracts spoken words, faces, characters and emotional information from the video. Such metadata is then used to build engaging experiences with recommendations, highlight clips and interactive videos.
Diva also uses the Computer Vision API and Face API from Microsoft Cognitive Services, in order to detect faces (in combination with the Microsoft Video Indexer) and texts (OCR) in the videos.
Recommendation
If you think about Netflix or similar platforms, one of the winning factors is recommendation. You usually have an incredibly large amount of content, but consequently you are pretty much unable to decide which movie or series to watch. It is no surprise that Deltatre also included a recommendation engine in its OTT platform, and this is another example of how embedding AI is vital.
In this case, they needed tools for training models with users’ habits, so they opted for PyTorch and fast.ai. The former is an open source deep learning framework for Python, while the latter simplifies the training process of neural networks using modern best practices. Using both these tools, Deltatre is now able to provide accurate recommendation to their customers, even within a very large database of content.
Next Gen Content
We touched upon the content generation process when describing how Deltatre used AI for supporting content editing. Gianni Rosa Gallina described an entire new area where they used different AI and computer vision algorithms for creating what they defined next generation content. In particular, he showed a prototype implemented by researchers from University of Washington, Facebook and Google (see this link for a showcase), aimed at transforming a monocular video of a soccer game into a moving 3D reconstruction. This can be particularly useful to provide additional interactive solutions to end users, especially for an OTT platform that provides sport-related media content. Thus it comes as no surprise that Deltatre are also looking at such kinds of solutions for their products.
Conclusion
The Deltatre’s OTT platform that we analysed here is just an example, particularly complex in fact, to understand how AI applications are everywhere, being pervasively embedded in many contexts. Using AI and machine learning algorithms is actually becoming easier, thanks to this great availability of tools and libraries to embed in our applications. You just need to start practising with some of these tools: using AI has never been so accessible to any programmer.
