Multivariate time series forecasting - model selection

Question

I have historical data on customer contracts. I know the date a customer terminated their contract and the date they notified of this termination. For example, a customer could end their contract in March-2024 and notify about it in January 2024. I have applied some transformations to the data and created a time series that looks something like:

Date	N_contract_ends	notification_1	notification_2	...	notification_12	Move_outs
2018-01	100	20	10		30	90
2018-02	300	60	40		10	270
...	...	...	...	...	...	...
2024-05	843	0	0		230	790

N_contract_ends: is the number of possible contract-ends in that month (this is the upper limit on the number of terminations for that month).

notification_i: the months of advance notice given. So in the first row, we had a total of 100 possible terminations, 90 of those ended up terminating. And out of those 90, 20 people gave 1 months notice, 10 gave 2 months notice etc.

I want to forecast Move_outs. I have made a somewhat successful model with Prophet where I include N_contract_ends and some of the notification_i columns as additional regressors. Note that I do not use all of the columns because they always add up to the outcome variable, Move_outs. The data is quite cyclical. I see large spikes towards the end of the year.

The interesting part of this problem is that in the last row (May of this year), we can see that 230 customers notified us a year ago. So I already know some of the notification_i columns. But I do not know notification_3, notification_2 and notification_1 because those are still in the future (notification_1 for example would be customers that notify of termination in April 2024).

And it is this information I want to make use of when forecasting. The fact that I already know of some terminations. When using Prophet I experimented with only including notification_i, where i >= 4. The model seems to work ok but I was wondering if there are any other, more suitable methods I should consider.

Answer 1

are these the raw data? what does the raw data look like? you may be missing many important features, especially if you have individual data (higher granularity) and you are grouping everything.

from what i can tell you a few useful points based on what you provided so far.

1. use time series cross validation techniques like outlined here

2. since Prophet can handle missing values, you want to simulate this when using your cross validation. you can NAN out known values to simulate missing values, using a NAN mask. so for example of you want to predict next months values you can use all the values, but when predicting 3 months out you won't have access to this, you you can simulate this by NAN-ing out notification_1 and notification_2 (or impute using some other metric etc). I am not sure how this can be handled in the Prophet model but in a design matrix you would just use a sliding NAN mask to achieve this.

3. I am not super familiar with Prophet and I would assume it already handles autocorrelation, but it wouldn't hurt to test adding in lag features from previous months and especially from previous years for the same month to account for seasonality directly.

4. you should not use 0 to represent missing values like the 2024-05 notification_1. using 0 you are telling the model that it is a fact that you have heard the numbers for 2024-05 and that number is 0 notification which is not correct. use None or NAN or equivalent

5. you also might consider transforming your data to reflect a percentage of the possible move out. so divide your entire table but the N_contract_ends value (and obviously remove this from the table). this will give you a set range between 0 and 1 in terms of percent of move out from possible move out and will scale your features nicely too. I am not sure this will work better but would be worth trying.

it is really hard to solve problems like this without more detail. there are almost always domain relevant rules and exceptions that affect how you would implement your model. for example ,what are these contracts? is there a penalty for canceling? is a financial penalty? is it large for the income level of the customer? does that penalty change if you cancel without notice or if you cancel 6 months out vs 1 month out? are these contracts related to social engagement where you might expect interactions with time of year or holidays? is it required that a customer notify before terminating the contract, what happens if they don't? is this data for contracts with renting apartments and that's why it says move out?

this problems is nebulus because you have in theory a highly predictive feature which essentially amounts to some form of agreement/promise that some event is going to take place, and you are trying to predict if that event will take place or not. now if you have more information like income level, credit score, on-time payment rate, zip code etc. then you could model something where you account for individual differences in your data to fit a more intricate relationships but as it stands right now you are using autocorrelation/ seasonality baked into the prophet model along with some type of 'promise' that an event will take place, and trying to see if 'promises by random people made in june occur at higher fidelity than promises of random people in january' so you are limited in what you can actually expect.

to be clear I imagine you will get pretty good results because you have these things. I just think that (outside of autocorrelation/seasonality) you are essentially modeling a simple probability of each notification