Contify uses different AI technologies for various purposes throughout the entire content processing pipeline in the platform. We use a AI technologies such as open-source models (NER), in-house-developed ML models (de-duplication and tagging), third-party APIs such as OpenAI GPT (summarisation).
In addition, it is not possible for any AI technology to accurately understand the subjective meaning of text, even with the latest Generative AI models using LLMs. Therefore, we include human intervention wherever necessary. Below is a list of sample AI applications within the Contify platform.
Content Sourcing
Identify the type of source: Is it a company website, a news website, or a regulator's website
It is important to accurately identify the 'Type of Source' because the information and how it should be processed for generating insights depend on the type of source. Contify uses a custom-trained ML model supported by a meta description of the source and custom rules to identify the source type.
Identify which pages to monitor on a website
A company usually posts strategic updates in different sections, such as press releases, announcements, news, media, etc. However, the challenge of working with internet data is that there are no standards that are followed by all websites. For example, companies could post press releases within their blog's section.
Identify whether website integrations are working fine or should be reviewed
Contify determines whether a website sourcing has any problem or not, depending on the change in the publishing patterns.
Identify the content published date
It seems like a trivial problem, but accurately identifying the date of an article can be quite challenging, particularly because there are no standard date formats. They have to be identified through the sources.
Identify which part of the webpage has the headline and which part has the text of the article
A web page has hundreds of HTML tags. Any of those tags can have the text of the article. Contify searches for all of these tags and identify the one with the text to retrieve it.
Content Processing
Removing irrelevant and non-business information
Data Cleaning with ML Reject Model: Contify's Reject Model, powered by the BERT ML architecture, acts as a sophisticated filter, removing irrelevant information and ensuring data integrity. Pre-trained ML models are used to identify and remove non-business information such as Sports, Entertainment, Politics, etc.
Named Entity Recognition (nouns such as companies, locations, persons, etc.)
Entity Recognition with ner-base-large: Contify pinpoints critical elements within your data such as Companies, Locations, People using the advanced ner-base-large model. This in-depth analysis unlocks valuable insights hidden within the text.
Disambiguation: Is it "Apple" the fruit, or "Apple" the company?
Aboutness: Is it about the company or just a passing mention of the company name?
Classification: Tagging of business metadata such as topics, industry, etc.
The classification is based on three layers: Rules, Machine Learning, and Human Curation. The human-curated data is used to continuously update ML models.
Through powerful ML models, we show the user the rationale behind each assigned tag, offering a deeper understanding of your data (Why am I seeing this Tag).
Custom tagging
Custom tag ML models leverage multi-label classification techniques. A dedicated model to identify and categorise content by custom topics of individual clients.
Grouping of similar and duplicate information
Group duplicates and similar information published on different sources. Similar information is identified not only based on the words, but also the company. Similar content is clustered together to deliver unique content.
Multilingual Content Processing
Contify processes several languages. Employing an in-house translation model, we seamlessly translate content across 40+ languages into English.
Content Analysis
Extraction of Key Highlights, Summarisation, and Quotes
Extracts important text along with business data (facts) from a news story, and auto-summarises based on extractive summarisation techniques. This is done using a combination of NLP and ML models. Also extracts the quotes of leadership from the article.Generation of Knowledge Graph using LLMs
Contify uses Large Language Models (LLMs) to create Knowledge Graphs, which help identify important relationships between organisations or business events. Contify can accurately pinpoint these connections to bring out relevant insights.
Derive Insights using LLMs using a proprietary Key Intelligence Questions based methodology
Key Intelligence Questions (KIQs) aid in gathering and aligning valuable facts and insights according to stakeholder needs. Contify employs LLMs to provide answers to these essential questions. Through LLMs, Contify can identify various types of information and fill knowledge gaps that might otherwise remain unnoticed. Since the information is structured and vetted, it minimises LLM hallucinations as well.Contify Copilot
The Copilot, Contify's newest addition, offers users a simple and convenient way to find answers to their questions. Through a familiar chat interface, users can effortlessly access all the information on the platform that they want. This is based on an Retrieval-Augmented Generation (RAG) based architecture to provide highly accurate responses to user queries from their specific data.