How to classify movement data (time series) in real time

Question

I have some movement data sampled over a time series. I am trying to classify the movements in real time as either smooth or shaky. For example, as the movement is smooth it is classified as smooth until it suddenly becomes shaky and is then classified as so.

I am having some serious trouble doing this. Aside from using an RNN and feeding it subsets of the time series data to classify (which I do not have enough data to train), does anybody know of any time series classification algorithms? I plan on feeding subsets of the time series into the algorithm with a single label for training and prediction.

Any help is greatly appreciated.

Answer 1

Supervised learning for classification in machine learning trains a model in order to determine which distribution a certain novel instance belongs to. Model complexity is directly correlated with the performance of a specific algorithm given a set number of instances.

This means you should always start with understanding your data before selecting a model.

Your dataset

Your dataset is comprised of instances $x_i \in \mathbb{R}^{300}, i \in {1, ..., n}$ and $10 < n < 50$.

This is a very small number of instances $n$. Usually I recommend to have ten times more instances than features for a two class classification problem. Furthermore, less than $50,000$ instances is not sufficient for training any deep learning model. You will need to consider using traditional machine learning algorithms (SVM, KNN, Random Forests, etc.). Lastly, the class imbalance in your dataset will introduce some bias, this will lead to poorer results.

All hope is not lost, we can still do something to solve this problem!

Feature selection

The first step which I would suggest is to consider performing some feature reduction. You want to map your $X \in \mathbb{R}^{300}$ to a lower dimensional set $\mathbb{R}^{p}$ where $p<300$. However, in removing features you will evidently be losing information, so you want to be able to learn what are the relevant features to your decision smooth/shaky. For example, when classifying cat/dog, the weight may be a highly informative feature thus its contribution in the decision should be highly considered, whereas the number of feet would not provide any useful information.

Some techniques to do this include PCA, LDA, and theres others as well. You can try them, its one line in scikit-learn.

Choosing an algorithm

Due to the heavy class imbalance, I would suggest that classification based algorithms will perform poorly. In contrast, I would attempt to use anomaly detection algorithms. These techniques will learn the distribution of your nominal set (smooth) and attempt to detect when signals differ significantly from this distribution and will then classify them as anomalous (shaky).

Many of these techniques exist, please refer to the following answers:

Using time series data from a sensor for ML

How to train model to predict events 30 minutes prior, from multi-dimensionnal timeseries

Answer 2

I have to wonder why you are classifying entire instances as either smooth or shaky? Would it be possible to model each instance as a hidden Markov model (HMM) with two states, smooth vs shaky? Your task would be to model the transition probabilities and estimate the current state of the instances. Do instances ever transition from shaky back to smooth?

Estimating the model parameters can be heavy work, but you have a small dataset. I recently ran an analysis similar to this using the msm package in R. I had around 3000 instances of varying lengths (typically < 20 each). The code ran quickly (under a minute but noticeable wait).

If the effect is strong -- i.e. the measurements really change when you go from "smooth" to "shaky", you should still be able to estimate the parameters even with the "class imbalance" that you have. If not, you could take a subset of the data that achieves better balance (or better, randomly select subsets of the smooth data and bootstrap the parameter estimates). The transition probabilities are conditional on the state you are in, so you can change the overall proportion of being in one state or another without harm to your analysis. Of course, global estimates of how long the system remains in one state would need to be adjusted in light of population prevalence.

If the transition to the shaky state is anticipated by changes in the observed movement data, then you need to consider one of those Poisson intensity models where the intensity function depends on the observations. Check out R package PtProcess and the references therein. It offers a variety of models, which could meet your needs.