Ahlad Kumar’s YouTube channel

If you were on YouTube trying to learn about variational autoencoders (VAEs) as I was, you might have come across Ahlad Kumar’s series on the topic. In his second video (embedded above), he explained KL divergence which we will later see is in fact a building block of the loss function in the VAE. This article aims to bridge ideas in probability theory as you may have learnt in school to those in the video. We will then re-look at the proof for KL divergence between 2 multivariate Gaussians (a.k.a normal distributions).

Note: This topic requires…

In this article, I will be sharing my takeaways from reading Background Matting: The World is Your Green Screen by Sengupta et.al.

Background Matting: The World is Your Green Screen (Sengupta et.al.)

Visual effects in movies make use of a green screen to superimpose computer-generated landscapes into the background. That’s green-screen matting.

Writing this article after reading Fixing the train-test resolution discrepancy (Touvron et. al). I will include the relevant sections of the paper in brackets. I have also used images from the paper.

The current best practice for image preprocessing to train image classifiers looks a little something like this: (Section 1)

train_transform = transforms.Compose([
transforms.RandomResizedCrop(224),
transforms.RandomHorizontalFlip(),
transforms.ToTensor(),
transforms.Normalize((0.5,0.5,0.5),(0.5,0.5,0.5))
])
test_transform = transforms.Compose([
transforms.Resize(256),
transforms.CenterCrop(224),
transforms.ToTensor(),
transforms.Normalize((0.5,0.5,0.5),(0.5,0.5,0.5))
])

We take different image preprocessings steps at train and test time. And here’s why. …

Greetings! In this article, I will be consolidating my thoughts on Jeremy Howard’s implementation of Kaiming Init from fastai’s Lesson 9. When I encountered this implementation, I had a few troubles in my understanding:

  • How does the code relate to the math from the paper?
  • What is gain(a)?
  • Why is he multiplying std by √3 to get bound?
  • How does he derive fan_in and fan_out?

Code vs Paper

The code for Kaiming Init in the fully connected layer (as presented in Lesson 8) is very different from that in the convolutional layer. I guess this was what threw me off a bit.

#…

gordonlim

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store