Meet Pearl. World's first AI jury member that will join the Mixx Awards 2017.
ANTWERP. During the summer, DDB teamed up with Faction XYZ to research and create a first version of an AI-based jury member that will co-judge the creativity during the Mixx Awards 2017.
The MIXX awards are celebrating their 10th anniversary this year. The international advertising sector has received a lot of criticism on the process of its award shows. That was an ideal opportunity to rethink how juries are composed. This first AI experiment on judging will investigate if creativity is measurable and how AI might be able to help the judging system. Named after Radia Perlman, one of the prominent female internet pioneers, the result is Pearl. To let people get used to the idea, Pearl will judge a new and separate category this year.
People claim juries are inefficient. They say that jury members grow tired and inattentive, are influenced, biased and sometimes inexperienced. It is hard to control these factors. Unless you use a completely neutral, never tiring jury member that is ultra-smart. Like one with Artificial Intelligence for example.
An integrated AI partnership
Because of the technical complexity of building an Artificial Intelligence algorithm like Pearl, MIXX requested the help of the Belgian company Faction XYZ. They specialise in integrating state of the art artificial intelligence techniques like natural language modelling and deep learning.
Pearl was fed with all international MIXX case studies from the last decade to analyse the texts, videos, results and even music. This surely makes her the most experienced advertising jury member on the planet. For analysing text, a form of deep learning called Long-Short Term Memory was used. Audio was modelled by scoring things like happiness, danceability and acoustics. For analysing video content, Microsoft gave Faction XYZ early preview access to their Video Indexer service, which extracts insights from videos using artificial intelligence. The scores and findings from those three sub-models (text, audio and video) were put into the final machine learning model that combined them to come up with a prediction.
Pearl has taught herself to model how humans think and has already reached an accuracy level of about 76% (AUC). Of course, we keep working to increase this. But in other words, if you feed her a new case, she is already pretty good at being able to predict whether it will be a winner or not, and even what colour of award you might expect. This is beyond our expectations, but it seems that creativity is more or less measurable after all.
Pearl is already giving us great insights into what is important in an award-winning idea. She has made links between narrative styles and awards, between choice of instruments and awards, and she also has understood that the number of impressions doesn’t mean that much, while number of shares is far more important.
There is room for improvement though, as she needs to be able to better recognize humor and a creative idea.
Pearl is still learning and her accuracy will continue to improve. Pearl can identify the originality of your campaign, but is still learning to recognize if an original idea is also meaningful. We think Pearl will keep on learning and will play an even more significant role in the coming years.
What PEARL is showing us so far
* PEARL doesn't seem to be impressed by the numbers of impressions you ran. She prefers qualitative metrics such as ‘trending topic’ and ‘earned media’. If your campaign has had more than 100.000 shares, don’t hesitate to share it with the jury.
* Being mentioned by dedicated ad press is no guarantee to win. Being mentioned by BBC & CNN, however, does give more credibility to your case.
* Upbeat feel good music is so much overused that it’s being perceived as cliché and generic.
* The use of a voice-over has a positive effect on the outcome.
* When it comes to new technologies, buzzwords have the most success in the year after they peak.
* Case movies of winning campaigns featured 25% more suits than non-winning cases.
Faktion is a Eurasian A.I. technology provider that enables organizations with a vision and budget to build competitive cognitive capabilities for a future where data science and machine learning fulfill a pivotal role in the rapidly transforming economic reality of our clients. We build machine and deep learning algorithms in the space of computer vision, natural language and sensor data for the telco, automotive, retail and financial industry.