Our client's global operations are spread across ten offices in five countries, and they use a closed enterprise suite, called Brandwatch, to analyze digital marketing and social networks. It provides them with good insights and metrics for the English language, by periodically crawling several social networks, blogs, and other websites in search of keywords.
However, Brandwatch does not perform at the same level of accuracy when switching to other languages such as Spanish, Portuguese, or Italian. From a large set of Brandwatch features, our customer remains focused on its NLP (Natural Language Processing) capabilities and insights discovery. Since they provide global services to their multinational customers, they wanted to find a way to support this type of analysis in a multi-language setting.
Even though the client is not a small company –they have more than 500 employees and its own IT department–, they were struggling to develop a custom solution to generate insights beyond the existing Brandwatch limitations.
After analyzing the requirements and agreeing on a proposed implementation, Allata quickly assembled a team composed of a Data Architect, a Data Engineer, and a Data Scientist to implement a Natural Language Processing & Understanding (NLP & NLU) platform for multiple languages. This platform has the capacity to reveal a more accurate set of insights on Social Network Analysis, based on extracted data from Brandwatch plus other data sources such as Social Networks, blog sites, forums, newspapers, etc.
KPIs and metrics stored in a database are useless for an analyst without a flexible Dashboard for rendering infocharts, so our team also implemented and deployed a custom dashboard based on the open-source BI tool Superset. The multi-user and auto-scalable UI was a containerized solution on AWS ECS.
Below is a diagram of the overall solution architecture:
The data flows over a comprehensive NLP pipeline, with every step following strict data life cycle practices: labeling and storing data and resulting KPIs in aggregated and raw formats. This enables traceability, which allows our customer to rebase the historical data with different business rules and threshold configurations, for even more accurate insights.
The implemented solution allows the client to interpret and understand the impact of each of their campaigns on the LATAM Spanish-speaking market, making use of Artificial Intelligence for Natural Language Understanding.
Also, the solution was designed to easily support new languages, making it readily expansible.