Deeplearning

Table of contents We can remove the hardcoded part by replacing it with a loop Improving our RNN First thing first, let’s solve the resetting of hidden state Creating Multilayer RNN Exploding and disappearing Activations If we have data like: (from the example of previous blog) X – Y 'one', '.', 'two' – '.' '.', 'three', '.' – 'four' Where 3 words are used as input to predict 1 word from a vocabulary as an output, we can create a neural network architecture that takes three words as input and returns a prediction of the probability for each possible next word in the vocabulary. ...

Here are the main steps for language modeling: Tokenization– Converting Text into list of words (creating the vocab) Numericalization– Converting each word in vocab to number, by replacing them with their indices (simple!) The next steps are: Language Model Data Creation (X&Y) and Language Model Creation Jargon: Token One element of a list created by the tokenization process. It could be a word, part of a word (a subword), or a single character. ...

A language model is a model that is trained to guess the next word in a text (having read the ones before). This kind of task is called self-supervised learning. Jargon: self supervised learning Training a model using labels that are embedded in the independent variable, rather than requiring external labels. Self supervised learning is not usually used for the model that is trained directly, but instead is used for pretraining a model used for transfer learning. Self supervised learning is used to train a base model, and that base model is used to train different model for specific task like text classification!1 ...

Alright who cares about the follow up blog, here is a notion notes to save me. This is the notes for tweaking Random forest on a tabular dataset to squeeze some performance, I tried this on a Kaggle compe and it worked out pretty badly. Score of something like 2.xx. Maybe I Will improve it when time comes. zzz As we already initialized the embeddings of our users and products for example, We take the result of the embedding lookup and concatenate those activations together. This gives us a matrix that we can then pass through linear layers and nonlinearities in the usual way. ...

Table of contents Collaborative Filtering A Tabular Dataset -> Movie Recommendation System? Embeddings? Speed up the calculation of scores Why sharp curves of learning weights = overfitting Weight Decay or L2 Regularization Movie Recommendation System with Embeddings (MovieLens Dataset) Direction and Distance of embeddings Otakus are kinda poison to our embeddings I will write a follow up blog about Regression and Random Forest, today completes the half of the first month of 2024, I learned lots of stuffs that I am interested and participated on a Kaggle compe, predicing Energy consumption and production using solar panels, just to try what I learned and now I am one step closer to understanding the “Attention”. I will continue this journey and still be writing this blog for future me like creating notes, maybe in more readable form. ...

Table of contents Hyperparameters I coded the Titanic dataset in a simple MLP Regression and Summarizing Loss functions Source Multi label classification refers to the problem of identifying the categories of objects in images that may not contain exactly one type of object. So each data can have either single or multiple label(s). Example, a image has car, bicycle, person, tree. Why we cant use softmax and NLL loss? ...

Table of contents Understanding Softmax Logarithm Finding a right Learning Rate - A technique Unfreezing & Transfer Learning Unfreezing? Discriminative learning rate To learn the foundation very clearly, I coded MLP, from scratch and trained MNIST dataset. (it was a 3 vs 7 model, a binary classifier). For that I used a Linear function in each neuron, and Relu as activation. and for the final layer I used Sigmoid. I wanted to expand this model from just a binary classifier to multi-class classifier (where each instance belongs to one and only one class) I learned about Softmax activation that can be used in the final layer and then creating a loss function for MNIST model. ...

A artificial neural network can learn (almost) anything, and so its called a universal function approximator. To understand how it works, we need to know function. Function, let’s say f(x) is just a system of inputs and outputs, a number in, a number out. x -> f(x) -> y We give a input x, and it outputs y. We can plot all the functions on a graph, where it gives an output for an input. What is important is, if you know a function you can always calculate the output (y) for a given input (x). ...