The following gist will be helpful for the latter part of this article: Inspecting ggml_cont Recently, I’ve been playing around with GGML. While doing so, I was looking through the examples, and I saw this in mnist_common.cpp: 1 2 3 dense_in = ggml_reshape_2d(model.ctx_compute, ggml_cont(model.ctx_compute, ggml_permute(model.ctx_compute, dense_in, 1, 2, 0, 3)), (MNIST_HW/4)*(MNIST_HW/4)*(MNIST_CNN_NCB*2), model.nbatch_physical); This was on line 362. It preceded a dense matrix multiplication and addition for a fully-connected layer. It’s pretty clear what ggml_reshape_2d does....
A Short GGML Tutorial
I recently wanted to learn about GGML, a tensor and machine learning library behind popular open source versions of LLaMA and Whisper. Trying to find good resources on GGML is hard, so I thought I’d write up some preliminary notes for anyone looking to get started. Before reading the rest of this, please consider the following resources, especially the first link – the examples in the GGML repository are a great starting point:...
Credible Intervals versus Confidence Intervals
Suppose you have data $X_i | \mu, \sigma^2 \sim N(\mu, \sigma^2)$, for $i = 1, …, n$, i.e, $n$ normally distributed data points with mean $\mu$ and variance $\sigma^2$ (or standard deviation $\sigma$). Assume you don’t know $\sigma^2$, which is usually the case anyway. You could estimate $\mu \approx \bar{X}$ and call it a day, but you probably want some measure of uncertainty, i.e, an interval estimate. Note: Throughout this article, I use the convention that uppercase variables are random variables and lowercase variables are fixed (observed) quantities....
Reverse Mode Autodifferentiation Explained
Reverse Mode Autodifferentiation Explained This article is my attempt to explain reverse mode autodifferentiation to myself and hopefully to anyone else that finds this useful. (Link to notebook) Why autodifferentiation? The reason we prefer autodifferentiation over symbolic differentiation is due to its efficiency and simplicity. Instead of writing out explicit derivatives or parsing complex expressions and finding their symbolic derivatives, we can just compute a derivative at a particular value directly with the help of autodifferentiation....
Why convolutions are effective
Convolutional neural networks have seen great success in computer vision tasks. However, why is this architecture so effective? This article hopes to elucidate the apparent efficacy of convolutional networks in many computer vision tasks. We’ll approach this by training a convolutional network on the Fashion MNIST dataset. (Link to notebook). A brief look at the dataset First, we make some necessary imports: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 import torch import torch....
Optimizing Matrix Multiplication with Zig
I recently started playing with the Zig programming language and wanted to try it out for its speed. And what better way to do that than to try optimizing matrix multiplication? Since there are a plethora of resources to understand how to multiply matrices efficiently (see the Resources section below), I won’t be doing anything intense in this article (though maybe in the future I will). The naive matrix multiplication algorithm is given below in Zig:...