A list of projects that I’ve been up to, separate from my research.
AWD-LSTM / ULMFiT
A reproduction of two papers: Regularizing and Optimizing LSTM Language Models (Merity et al., 2017) and Universal Language Model Finetuning for Text Classification (Howard & Ruder, 2018).
PyTorch Language Modeling
Reproductions of key RNN-based language modeling architectures. Currently includes Weight-tied LSTMs (Inan et al., 2016; Press & Wolf, 2016) and Recurrent Highway Networks (Zilly et al., 2016). Currently a work-in-progress adding in more models.