I've put up a new blog post that aims to distill the ideas behind the neural tangent kernel that is making waves in recent theoretical deep learning research. A large portion of the talks in the recent Workshop on Theory of Deep Learning(https://www.math.ias.edu/wtdl) at the Institute for Advanced Study were based on ideas related to the neural tangent kernel. This is a slightly long post, as it involves a fair bit of math (you can skip some of the proofs though). A bit of linear algebra background is necessary to fully grasp what is going on here, but I hope that my visualizations can help with that.
I've written a new blog post (https://rajatvd.github.io/Factor-Graphs/) on an awesome visualization tool that I recently came across -- factor graphs. Initially, I encountered them in the context of message passing on graphical models, but soon realized that they are useful in more general contexts.
This is the first post in a series that covers the basics and mainly focuses on understanding how factor graphs work as a visualization tool, along with a cool example of a visual proof using them. In future posts, I plan to cover algorithms like message passing and belief propagation using this visualization framework.
I made a blog post a while back on Generating Words From Embeddings. It's a simple project which aims to create new meaningful words by generating them character by character, conditioned on a word embedding.
Now, I finally got around to making a simple colab notebook (https://colab.research.google.com/drive/1f_vW0k8YyoyiPIgX7eH...) which makes it very easy to play around with the model and sample new words in a matter of minutes. I'd love to see what weird and interesting words you encounter when messing around with it!
Also, I made this quite a while back, so I only experimented with a simple decoder RNN (GRU/LSTM). Given the leaps and bounds by which NLP research has grown since then, it might be worth trying out more models (perhaps transformers) and seeing if they can generate qualitatively more pleasing words.
I've put up a new blog post that aims to distill the ideas behind the neural tangent kernel that is making waves in recent theoretical deep learning research. A large portion of the talks in the recent Workshop on Theory of Deep Learning(https://www.math.ias.edu/wtdl) at the Institute for Advanced Study were based on ideas related to the neural tangent kernel. This is a slightly long post, as it involves a fair bit of math (you can skip some of the proofs though). A bit of linear algebra background is necessary to fully grasp what is going on here, but I hope that my visualizations can help with that.
Code for the experiments and animations: https://github.com/rajatvd/NTK
Feedback and suggestions are welcome!