In advance of the Data Science Salon in NYC on September 27, we asked our speakers to shed some light on how Artificial Intelligence and Machine Learning are impacting one of America’s largest industries. These data science practitioners have been working in the field from 3 years to over a decade and bring a wealth of experience and perspectives to the conversation around AI and Machine Learning in media and entertainment.
What trends are you seeing in machine learning and AI in media and entertainment?
In Media and Entertainment, there are already many applications of Machine Learning and AI hiding in plain sight. But this is a recent development, says Ryan McCabe, Senior Data Scientist at Spotify.
“Machine learning has moved past the high-school sex phase (everyone talking about it, but no one doing it), to a much broader implementations across industries including media.”
The demand for data remains vast, says Friederike Schüür, Senior Data Scientist and Researcher at Fast Forward Labs. “Across industries, organizations are looking to make their data work for them. Data science and machine learning provides them with the tools to do so. Within media and entertainment, the bulk of the data work today is dedicated to audience understanding; who reads, listens to, and watches your media and entertainment content? Audience insights help inform … anything, from business strategy to marketing and content creation.”
There are many ways that ML can be used to turbocharge the user experience. “Machine learning can help tag your image or text archive,” says Schüür, “for a better search experience. Recommender systems surface relevant content to readers, listeners, watchers. As creators develop new content, embedding based recommender systems can autosuggest image assets for articles, for example, or help surface image assets without copyright limitations given a target image. Sequence-to-sequence learning can translate not only from German to English but also from one writing style to another allowing your content to reach distinct audiences more easily. Generally, representation learning, at the heart of neural networks, allows us to make use of more unstructured data, such as image and text data, more easily unlocking novel, exciting use cases.”
Whitely and Capelo strongly agree. Says Christopher Whitely, Senior Director of Applied Analytics at Comcast, “There is a trend towards using machine learning models to deliver the most relevant content to consumers to keep them engaged, whether that is programming that they might watch or advertisements that are of interest to them.” Luis Capelo, Head of Data Products at Forbes adds,“There is a strong trend for personalization and for optimization of conversions. Personalization efforts are related to improving the experience of users and increasing their value by increasing the probability that users see multiple pages (or read multiple articles). These come typically in the form of recommendation algorithms that suggest another article to read based on the articles that a user has read in the past” This kind of personalized service is the new normal in Media and Entertainment and has become an extension of a brand’s identity and ability to drive business.
Expect the personalization trend to continue, says McCabe. “What I expect to see in the upcoming years is seeing ML being applied to older industries far away from the tech/media world we live in.” Personalization is important in sales and conversion optimization too, something that quite a few data scientists are already working on. Says Capelo, “There I see many of my colleagues creating solutions that increase the probability for users to subscribe. These solutions involve the creation of models that help identify a set of users that are likely to contact a subscription and figure out what is the best time to reach out to them.”
What are some of the biggest challenges data scientists face today in your industry?
The major challenge facing Data Scientists in Media and Entertainment mirrors the challenge facing data scientists across the board. Whitely suggests that, “data scientists have a challenge first understanding the business problems in the industry, and then often it is a challenge to obtain and access the right data sets for their work. It is also a challenge to establish the infrastructure and inter-divisional communication so that projects don’t always need to start from scratch.” Capelo says that this is largely because, “there are a number of managerial-related questions that get in the way of many data science teams.”The trend of isolating teams is bad for productivity. “Data science teams need to identify a question and then work to answer it. This challenge occurs when the results from investigations are the one and only output from those teams. That is, when data science teams generate insights without associated actions. Actions increase the probability for implementing the findings of an investigation and generate value for a business. These actions — and not only insights — are the real value added by data science teams. I think that a lot of organizations in the industry are struggling to allow their data science teams to have control over the implementation of insights. I believe that allowing for such direct control greatly increases the team’s value and productivity.”
In the end, there’s plenty of reason for optimism. McCabe says, “Data science is a new field comprised of a bunch of really smart and typically young people. As individuals we are fine.” But he also warns that, “as a whole, we have to really be cognizant of which directions we push our own field and if they are in the best interest of its long-term success.
What’s next in AI in Media and Entertainment?
When it comes to the future of AI in Media and Entertainment, the experts agree: it’s all about content, content, content. “In addition to continued advances in content personalization, there will be more integration across media experiences, such as digital video and traditional television, and widespread usage of digital assistants to aid content discovery,” says Whitely. “Eventually, AI will be used for the creation of content and helping drive immersive virtual reality experiences.” Eyal Pfeifel, CTO and Co-founder of imperson agrees, “We see AI becoming a new form of entertainment media through VR and AR technologies offering fully immersive experiences with intelligent avatars.”
Some of these initiatives are already underway. Capelo is, “excited about AI that help authors create excellent content. At Forbes, we are developing a new publishing platform, Bertie, that works as an AI assistant to writers (you can read more about it here). Bertie learns many things from writers, such as their writing style, the topics they typically write about, and so on. Bertie then makes suggestions aimed at improving the quality of that given piece, which also increase the article’s performance after it has been published. Our ambition is to make Bertie an assistant that writers can relate to, working alongside authors to create great stories.”
How’s AI changing Media and Entertainment?
There’s no question that AI is quickly changing not only how we consume content, but also deepening interactive possibilities with consumers. “AI has significantly changed the media landscape — both in the way that audiences discover and engage with content, and also in the way that content is created and distributed to audiences, says Amy Yu, Senior Director of Product Strategy and Data Science at Viacom. “Currently, algorithms not only influence what audiences see on different platforms, but also the content that is being created in the first place. Although media has traditionally been a data-driven business, AI is fundamentally changing both audience behavior and the creative process.” Pfeifel says this goes even further.
“By offering fully interactive capabilities, media and entertainment is changing from a “watch only” medium, to being fully interactive. We also see entertainment extend from the theatres into our day-to-day lives.”
These changes resonate in the home as well, says Whitely. “Video-on-demand on a television or device is increasingly the medium of choice to consume premium content, except for certain live events such as the Olympics, Superbowl, or hit “watercooler” shows. Understanding who the loyal viewers of a given piece of content truly are and communicating with them is increasingly becoming essential for programmer success.”
And AI is also increasingly essential for success for data workers in the industry at large. “AI has opened up new opportunities within media and entertainment industry and created new roles in that require technical and quantitative skills,” says Jennifer Shin, CEO of 8Path Solutions. We can expect to see major job growth in this area over the coming years.
How does this compare to data science in other industries?
“Media & entertainment has many of the same challenges as other industries,” says Whitely, “requiring effective big data engineering, data quality and stewardship efforts, attention to data governance and data privacy, and usage of the most appropriate Machine Learning models for a given problem space.” Yu agrees, “the data engineering challenges that we encounter within the media/entertainment space are definitely relatable for any industry that deals with large scale datasets — especially for data that includes any type of user generated content. Creating sustainable production pipelines to harness the relevant datasets is a challenge in data wrangling and is central to building the advanced analytics and tools that are used for decision making within the team. “
There are additional challenges that may be familiar to data scientists working in other industries. “Just about every industry is trying to find data scientists and data engineers, but many companies hire by posting jobs that include a wish list of qualifications that is not necessary or required for the position and in many cases, the hiring manager doesn’t even possess,” says Shin. “Media and entertainment started hiring for AI and ML positions a bit later than other industries and I hear a lot of new data scientists compare it against companies like Google and Facebook without realizing they’re comparing apples and oranges. Data science positions in media and entertainment are not equivalent to those offered in technology companies and while this may be a deterrent to some, there are a lot of unsolved problems you can work on that are interesting and relatable to those who are outside of your industry.” Similar Machine Learning opportunities and challenges are encountered in the retail industry for example. Learn more of them in this complete guide by Tryolabs!
But not everyone feels that data science in media and entertainment are essentially analogous to other industries. McCabe says, “Data collection is vastly different in our industry compared to where the vast majority of the world’s economic production occurs. We have already made the first step in this IoT space, Bosch and GE have sensors in everything to collect data and predict… well everything. However, as data science penetrates more industries — folks will have to get creative in how they collect data.”
What data science, machine learning tools and techniques are your favorite?
“The fun is in finding the best way to mix and match the techniques and measure the success of each one,” says Shin. “When I try to solve a problem, I have a lot of ideas about what I think might work, but there are so many factors that can change how well these ideas perform in the real world and there are times where I’ll figure out a better approach after I start the project. I enjoy discovering new aspects about existing techniques as I apply it to real problems — even the most ‘boring’ technique can be fascinating when applied to the right problem.”
Even so, there are a few clear standout winners in our speaker’s toolbox.
“Advances in the cloud such as AWS’ S3, Glue and Athena are making it easier to wrangle data quickly, and machine learning libraries like Python’s scikit learn and Spark’s MLLib are making it easier to apply advanced models to big data sets,”says Whitely.
Capelo has a different perspective — “I am a big fan of the Python programming language and its associated data science packages (pandas, numpy, scipy, and Jupyter Lab). I also really like working with Keras and also with TensorFlow (although, I’ve heard really good things about PyTorch). Lastly, I’m a big fan of the PostgreSQL project and its amazing extension ecosystem that include cstore_fdw(from Citus Data), TimescaleDB, and PipelineDB — all really great projects.”
No matter the tools you use in your workflow, one thing is for certain — AI and Machine Learning will continue to push innovation, communication, and interactivity to new levels in media and entertainment.
Watch this video:https://www.youtube.com/watch?v=Z5vxRC8dMvs