So your query is a comparison of linear regression vs. random forest's model-derived importance of variables.
The lasso finds linear regression model coefficients by applying regularization. A popular approach to rank a variable's importance in a linear regression model is to decompose $R^2$ into contributions attributed to each variable. But variable importance is not straightforward in linear regression due to correlations between variables. Refer to the document describing the PMD method (Feldman, 2005) in the references below.
Another popular approach is averaging over orderings (LMG, 1980). The LMG works like this:
- Find the semi-partial correlation of each predictor in the model, e.g. for variable a we have: $SS_a/SS_{total}$. It implies how much would $R^2$ increase if variable $a$ were added to the model.
- Calculate this value for each variable for each order in which the variable gets introduced into the model, i.e. {$a,b,c$} ; {$b,a,c$} ; {$b,c,a$}
- Find the average of the semi-partial correlations for each of these orders. This is the average over orderings.
The random forest algorithm fits multiple trees, each tree in the forest is built by randomly selecting different features from the dataset. The nodes of each tree are built up by choosing and splitting to achieve maximum variance reduction. While predicting on the test dataset, the individual trees output is averaged to obtain the final output. Each variable is permuted among all trees and the difference in out of sample error of before and after permutation is calculated. The variables with highest difference are considered most important, and ones with lower values are less important.
The method by which the model is fit on the training data is very different for a linear regression model as compared to random forest model. But both models don't contain any structural relationships between the variables.
Regarding your query about non-linearity of the dependent variable: The lasso is essentially a linear model which will not be able to give good predictions for an underlying non-linear processes, as compared to tree based models. You should be able to check this by verifying the models performance over a set-aside test set, if the random forest performs better, the underlying process may be non-linear. Alternatively, you could include variable interaction effects and higher order variables created using a, b, and c in the lasso model and verify if this model performs better as compared to a lasso with only a linear combination of a, b and c. If it does, then the underlying process might be non-linear.
References:
