The NVIDIA Jarvis framework simplifies the process of training and deploying Conversational AI systems making it much easier to build applications with Automatic Speech Recognition, Text-to-Speech, and Natural Language Understanding capabilities — Introduction The capabilities of Artificial Intelligence technology have improved rapidly over the past few years. Natural Language Processing (NLP), in particular, has seen incredible improvements as a result of the Transformer architecture and its derivates. Current state-of-the-art systems are capable of nearly human-level (or beyond human-level in some cases!) …

The mT5 model is pre-trained on over a hundred different languages. Let’s see how we can leverage this to train a bilingual translation model for a low-resource language — Sinhalese. — mT5 is a multilingual Transformer model pre-trained on a dataset (mC4) containing text from 101 different languages. The architecture of the mT5 model (based on T5) is designed to support any Natural Language Processing task (classification, NER, question answering, etc.) by reframing the required task as a sequence-to-sequence task.

Cross-lingual, zero-shot training with mT5 — Training an mT5 model in English and using it with other languages! — The original T5 (Text-To-Text Transfer Transformer) model achieved state-of-the-art performance on a variety of NLP benchmarks by leveraging a unified text-to-text format and a gigantic training dataset (C4). With the unified text-to-text approach, all downstream tasks were reframed such that both the input and the output of the model are…

Real-life data is messy, complex, and hard to understand! Let’s see how we can make it a little simpler! — One of my favourite approaches to dealing with large, complex problems is to break them down into smaller, more manageable sub-problems. This makes it easier to focus on the important bits without being overwhelmed by the details. At Skil.AI, we often deal with enormous, complicated datasets which can seem quite…

John Smith. J Smith. Smith, John. How to find if this John is the same as that John! — Have you ever searched for a contact on your phone and come up with several duplicate or near-duplicate entries? (I’m fairly certain this isn’t just me!) For me, this tends to happen when I forget that I already have a particular contact saved and create a new one for a…

Paraphrasing is the act of expressing something using different words while retaining the original meaning. Let’s see how we can do it with BART, a Sequence-to-Sequence Transformer Model. — Introduction BART is a denoising autoencoder for pretraining sequence-to-sequence models. BART is trained by (1) corrupting text with an arbitrary noising function, and (2) learning a model to reconstruct the original text.

How to tune your hyperparameters with Simple Transformers for better Natural Langauge Processing. — The goal of any Deep Learning model is to take in an input and generate the correct output. The nature of these inputs and outputs, which can vary wildly from application to application, depends on the specific job that the model should perform. For example, a dog breed classification model…

The T5 Transformer can perform any NLP task. It can perform multiple tasks, at the same time, with the same model. Here’s how! — The T5 (Text-To-Text Transfer Transformer) model was the product of a large-scale study (paper) conducted to explore the limits of transfer learning. It builds upon popular architectures like GPT, BERT, and RoBERTa(to name only a few) models that utilized Transfer Learning with incredible success. While BERT-like models can be fine-tuned…

The T5 Transformer frames any NLP task as a text-to-text task enabling pre-trained models to easily learn new tasks. Let’s teach the old dog a new trick! — I’ve been itching to try the T5 (Text-To-Text Transfer Transformer) ever since it came out way, way back in October 2019 (it’s been a long couple of months). I messed around with open-sourced code from Google a couple of times, but I never managed to get it to work properly…