Text Cleanser

Digital Contact’s Text Cleanser: Pioneering AI-Driven News Processing

One of our earliest ventures into Machine Learning and AI was the development of a powerful text cleansing system for Digital Contact. As a company specializing in real-time news analysis and financial insights, they needed a way to intelligently filter and clean incoming news articles by removing unnecessary content, such as advertisements and irrelevant metadata, while preserving valuable information.

AI-Powered Content Processing

We built a robust Natural Language Processing (NLP) pipeline that combined multiple techniques to analyze and clean text dynamically. Digital Contact’s in-house team, although highly technical, lacked the specialist AI expertise to implement such a system, so we were brought in to develop a tailored solution.

Key Features & Technologies

• Named Entity Recognition (NER): Used to identify and retain key entities such as company names, locations, and people while filtering out redundant information.
• Text Summarization: Implemented lexical similarity techniques to distill articles down to their essential components.
• Lexical & Semantic Analysis: Identified common advertising phrases and non-news elements for removal.
• Stemming & Lemmatization: Ensured a uniform text format for better processing across Digital Contact’s tech stack.

A Core Part of Digital Contact’s Ecosystem

The Text Cleanser became a fundamental component of Digital Contact’s news processing pipeline. The system significantly improved the quality of financial insights derived from raw news data by delivering high-accuracy text filtering and automated summarisation.

This project not only demonstrated our AI expertise but also solidified our long-term relationship with Digital Contact, as we continued to enhance and evolve their data processing capabilities.