Dealing with feature vectors of variable length

Question

How does one deal with a feature vector that can vary in size?

Let's say per object, I calculate 4 features. In order to solve a certain regression problem, I may have 1, 2, or more of these objects (no more than 10). Thus, the feature vector is 4*N in length. How is this normally addressed?

The objects represent physical objects (e.g. other people) w.r.t. an observer. For a time slice, an object can be placed laterally, longitudinally, have some speed and some heading (4 features). Trying to solve: where should a person feel most comfortable. In some cases there is only 1 object, but there can be 2 or more.

Disclaimer: I have limited knowledge on ML approaches. I had classes in college years ago and took Andrew Ng's ML course online as a refresher but otherwise not up to speed. A starting place to look is appreciated.

Answer 1

First of all welcome to the community!

About the question I would say there are misunderstandings. You say you calculate 4 features "per objects". It means that every data point (object) is described with 4 features. So the length of features are not different. Please note that I say this because "you calculate features for your samples/data points/objects" so every object here is a point in 4-dimensional space. If you mean something else please correct me.

In other words, you have a data matrix with some rows (number of objects) and 4 columns.

More general about the whole concept. No, this is not a valid approach to create standard data. In standard data the number of columns are always the same because you can not study objects if they are defined in different spaces. Many ML algorithms also work with standard data which means you can not use them if the size of feature vectors vary. More conceptual so to speak, the size of features you calculate for objects can not vary. How? I want to extract features from faces. I say eye color, distance between eyes and distance between ears. I have 3 features and for any person who comes to my study I calculate the same set of features. How can the size be different if the feature extraction process is standard?

But in non-standard data you may end up with such a case. In this case you may standardize the features. E.g. as a baby example, imagine different graphs with different number of nodes and edges. You can describe each graph with its average degree, average path length and number of nodes. Then each graph is described with 3 features and you can feed it into an algorithm.

But if you don't want to standardize the data (because it usually has loss of information) you need to find tailored analytic methods for which you need to explain more about your data.

I hope it helped!

Good Luck!

Answer 2

OK, it sounds like you are trying to encode the position and velocity of a variable number of objects into each training example.

One way to do this is with two feature planes per training example as channel inputs to a convolutional neural network. The first feature plane encodes the x-component of velocity for each object at that object's position, and the second does the same for the y-component.

Since you said positions are with respect to the observer, I'll assume the observer is always in the same location, so there is no need to add any additional information to the inputs.

I'll also assume that two objects cannot be in the same place.

For example let's say you're in a 3x3 world and you have objects at (0,1) and (2,2) with velocities (3,2) and (1,7) respectively. This input example could be encoded as:

[ 0 0 0
  3 0 0
  0 0 1 ]

[ 0 0 0
  2 0 0
  0 0 7 ]

As you can see the input size is always the same regardless of how many objects are present. More objects will just lead to more non-zero entries.

I suggested a conv net since they work especially well with spatial problems, but if you want to try a vanilla neural network first then you can flatten your input shape from (m,3,3,2) to (m,12).