Feature Interactions vs Feature Importances

Question

What are the differences between Feature Interactions and Feature Importances?

Answer 1

My perspective:

Feature Importance: How did the feature contribute to the prediction? This will be the coefficient if it is a linear regression with IID data. If you are in decision trees you can look at SHAP or other feature attributions methods.

Feature Interaction: How does a feature depend on another? A model makes a prediction based on two features, we can decompose the prediction into four terms: a constant term, a term for the first feature, a term for the second feature and a term for the interaction between the two features. The interaction between two features is the change in the prediction that occurs by varying the features after considering the individual feature effects.

Lets say our model is like:

y = a + b·x_1 + c·x_2 + d·x_1·x_2  

And x1 and x2 are IID.

The feature interaction will be the product x_1 · x_2

In the following graph, you can see the relationship between Age and Sex changes when the Age increases value for a given dataset.