Published in Towards AI·Nov 13, 2021Member-onlyFinding the Needle in the Haystack: How to Train a Dense Passage RetrieverLet’s see how we can train a model to perform dense-passage retrieval with Transformer models using Simple Transformers. — Introduction to Passage Retrieval Passage retrieval is a conceptually simple task where a system has to retrieve the most relevant passage(s) given an input query. Open-domain question answering is a common use case of passage retrieval. Here, the system has access to a large corpus of candidate contexts (passages that may contain the answer…Data Science5 min readData Science5 min read
Published in Towards Data Science·Feb 26, 2021Member-onlyA Quick Overview of NVIDIA Jarvis — An End-to-End Framework for Conversational AIThe NVIDIA Jarvis framework simplifies the process of training and deploying Conversational AI systems making it much easier to build applications with Automatic Speech Recognition, Text-to-Speech, and Natural Language Understanding capabilities — Introduction The capabilities of Artificial Intelligence technology have improved rapidly over the past few years. Natural Language Processing (NLP), in particular, has seen incredible improvements as a result of the Transformer architecture and its derivates. Current state-of-the-art systems are capable of nearly human-level (or beyond human-level in some cases!) performance in…Artificial Intelligence5 min readArtificial Intelligence5 min read
Published in Towards Data Science·Jan 4, 2021Member-onlyHow to Train an mT5 Model for Translation With Simple TransformersThe mT5 model is pre-trained on over a hundred different languages. Let’s see how we can leverage this to train a bilingual translation model for a low-resource language — Sinhalese. — mT5 is a multilingual Transformer model pre-trained on a dataset (mC4) containing text from 101 different languages. The architecture of the mT5 model (based on T5) is designed to support any Natural Language Processing task (classification, NER, question answering, etc.) by reframing the required task as a sequence-to-sequence task.Machine Learning8 min readMachine Learning8 min read
Published in Towards Data Science·Dec 16, 2020Member-onlyGoing Global —How to Multi-Task in Multiple Languages with the mT5 TransformerCross-lingual, zero-shot training with mT5 — Training an mT5 model in English and using it with other languages! — The original T5 (Text-To-Text Transfer Transformer) model achieved state-of-the-art performance on a variety of NLP benchmarks by leveraging a unified text-to-text format and a gigantic training dataset (C4). With the unified text-to-text approach, all downstream tasks were reframed such that both the input and the output of the model are…Artificial Intelligence10 min readArtificial Intelligence10 min read
Published in Skil-AI·Oct 26, 2020Member-onlyDivide and Conquer — How to Deal With Complex DatasetsReal-life data is messy, complex, and hard to understand! Let’s see how we can make it a little simpler! — One of my favourite approaches to dealing with large, complex problems is to break them down into smaller, more manageable sub-problems. This makes it easier to focus on the important bits without being overwhelmed by the details. At Skil.AI, we often deal with enormous, complicated datasets which can seem quite…Data Science4 min readData Science4 min read
Published in Skil-AI·Oct 11, 2020Member-onlyDealing with DuplicatesJohn Smith. J Smith. Smith, John. How to find if this John is the same as that John! — Have you ever searched for a contact on your phone and come up with several duplicate or near-duplicate entries? (I’m fairly certain this isn’t just me!) For me, this tends to happen when I forget that I already have a particular contact saved and create a new one for a…Machine Learning3 min readMachine Learning3 min read
Published in Towards Data Science·Aug 5, 2020Member-onlyBART for Paraphrasing with Simple TransformersParaphrasing is the act of expressing something using different words while retaining the original meaning. Let’s see how we can do it with BART, a Sequence-to-Sequence Transformer Model. — Introduction BART is a denoising autoencoder for pretraining sequence-to-sequence models. BART is trained by (1) corrupting text with an arbitrary noising function, and (2) learning a model to reconstruct the original text.Artificial Intelligence8 min readArtificial Intelligence8 min read
Published in Towards Data Science·Jul 13, 2020Member-onlyHyperparameter Optimization for Optimum Transformer ModelsHow to tune your hyperparameters with Simple Transformers for better Natural Langauge Processing. — The goal of any Deep Learning model is to take in an input and generate the correct output. The nature of these inputs and outputs, which can vary wildly from application to application, depends on the specific job that the model should perform. For example, a dog breed classification model…Data Science16 min readData Science16 min read
Published in Towards Data Science·Jun 19, 2020Member-onlyThe Guide to Multi-Tasking with the T5 TransformerThe T5 Transformer can perform any NLP task. It can perform multiple tasks, at the same time, with the same model. Here’s how! — The T5 (Text-To-Text Transfer Transformer) model was the product of a large-scale study (paper) conducted to explore the limits of transfer learning. It builds upon popular architectures like GPT, BERT, and RoBERTa(to name only a few) models that utilized Transfer Learning with incredible success. While BERT-like models can be fine-tuned…Artificial Intelligence8 min readArtificial Intelligence8 min read
Published in Towards Data Science·May 22, 2020Member-onlyAsking the Right Questions: Training a T5 Transformer Model on a New taskThe T5 Transformer frames any NLP task as a text-to-text task enabling pre-trained models to easily learn new tasks. Let’s teach the old dog a new trick! — I’ve been itching to try the T5 (Text-To-Text Transfer Transformer) ever since it came out way, way back in October 2019 (it’s been a long couple of months). I messed around with open-sourced code from Google a couple of times, but I never managed to get it to work properly…Artificial Intelligence12 min readArtificial Intelligence12 min read
