I'm creating animations and instructional videos about neural networks. Check out my Patreon and YouTube channel!

Convolution

The Basic Algorithm

Watch the companion YouTube video: Fundamental Algorithm of Convolution in Neural Networks. The process of convolution in neural networks with a 3x3 kernel size

Padding

Companion video: Convolution Padding - Neural Networks
The process of convolution in neural networks with a 3x3 kernel size The process of convolution in neural networks with a 3x3 kernel size and a padding of 'SAME', i.e., 1 pixel on the top, bottom, left, and right.
No Padding AKA "Valid" [1,1,1,1] Padding AKA "Same"

Stride

Companion video: Stride - Convolution in Neural Networks
Stride 1Stride 2
The process of convolution in neural networks with a 3x3 kernel size The process of convolution in neural networks with a 3x3 kernel size and a stride of 2x2
No Padding AKA "Valid"; Stride of 1 No Padding AKA "Valid"; Stride of 2
The process of convolution in neural networks with a 3x3 kernel size, a stride of 2x2, and a padding of 'SAME', i.e., 1 pixel on the top, bottom, left, and right.
[1,1,1,1] Padding AKA "Same"; Stride of 1 [1,1,1,1] Padding AKA "Same"; Stride of 2

Groups, Depthwise, and Depthwise-Separable

Watch the companion YouTube video: Groups, Depthwise, and Depthwise-Separable Convolution (Neural Networks).
The process of convolution in neural networks with a 3x3 kernel size The process of convolution in neural networks with a 3x3 kernel size and 2 groups
1 Group 2 Groups
The process of convolution in neural networks with a 3x3 kernel size and 8 groups making it a depthwise convolution A depthwise convolution layer followed by a pointwise convolution layer making the combined process a depthwise-separable convolution
Depthwise (8 Groups) Depthwise-separable (8 Groups followed by pointwise)

Pixel Shuffle

Watch the companion YouTube video: Pixel Shuffle - Changing Resolution with Style

2x2 Block Size

Pixel shuffle animation in neural networks with a block size of 2x2 Pixel unshuffle animation in neural networks with a block size of 2x2
2x2 Pixel Shuffle 2x2 Pixel Unshuffle
Pixel shuffle and unshuffle in neural networks with a block size of 2x2 looping animation
2x2 Pixel Shuffle/Unshuffle Loop

3x3 Block Size

Pixel shuffle animation in neural networks with a block size of 3x3 Pixel unshuffle animation in neural networks with a block size of 3x3
3x3 Pixel Shuffle 3x3 Pixel Unshuffle
Pixel shuffle and unshuffle in neural networks with a block size of 3x3 looping animation
3x3 Pixel Shuffle/Unshuffle Loop

Licensed under the MIT License