Enhancing Non-Markov Reinforcement Learning to Solve Complex Environments

In recent years, sequence models have become one of the most researched topics in computer science. They are some of the most useful models in computer science, with applications ranging from weather forecasting, video and text generation, to voice recognition. Despite their wide-spread use, the theoretical understanding of machine learning-based sequence models is limited. This project consists of the creation of a novel benchmark designed by the principles of cascades and formal language theory to empirically test the performance of such models. From this we obtained evidence proving to what degree different sequence models are capable of length generalisation, insights into how different models represent languages, and prove that whilst Transformers are currently the most popular form of sequence model, other model types still have their use cases.