Comparing CNNs, RNNs and Transformers on sequential data
You can choose among 2 options:
Temporal data is everywhere. Being able to leverage sequential data is thus an important challenge. In this course, we saw three types of neural networks: CNNs, RNNs and Transformers. The goal of this project is to present a rigorous quantitative comparison of the performance of each of the three approaches to handle temporal data.
The broad question you will have to answer in this project is: How do the three types of architectures compare on a given sequential dataset? You will select different architectures and hyperparameters for the three classes of models in order to provide a rigorous and fair comparison. Such scientific comparative study is very valuable when starting a project dealing with a new dataset. Comparing different approaches fairly and rigorously will help you pick the best one.
In this project, you will have to compare CNN, RNN and Transformer models on the task of binary movie review sentiment classification, i.e. given a textual review of a movie, predicting whether the given review is positive or negative. You will use the Large Movie Review Dataset. From the previous link, you can download the dataset and access a README to learn about the structure of the dataset folder.
The dataset contains unsupervised samples. You should not use them, but only supervised train and test sets. And also, remember what we saw in the course about the importance of building and using a proper validation set!
The dataset also provides review scores. You don’t have to use those, only treat this as a binary classification problem (classification with 2 classes: positive and negative).
You can’t use any pre-trained model. Everything should be trained from scratch!
At the beginning of the project, you should spend some time on the literature. Each member in the group should select and read at least one research paper that is relevant and presents a method that could be applied to your problem. Such related work should be presented in your written report.
In the project, you will need to convert textual reviews to vectors. You are asked to compare different approaches to do so. This comparison will be presented in the final report, along with the details of your data pre-processing pipeline.
As already mentionned, the core question you will try to answer is the following: What works best between CNNs, RNNs and Transformers on this particular dataset? You should not compare only one model of each, but rather pick different variants and hyperparameters to provide a fair and rigorous comparison.