Remove duplicate objects in JSON array with jq based on specific criteria

Question

I have the following data where I want entire objects removed based on duplicate values of the "run" key, while keeping the object with the largest "startTime" number:

{
  "data": {
    "results": [
      {
        "event": {
          "biking": {
            "startTime": 12,
            "id": "a",
            "run": "x"
          }
        },
        "displayName": "Alex"
      },
      {
        "event": {
          "biking": {
            "startTime": 10,
            "id": "b",
            "run": "x"
          }
        },
        "displayName": "Adam"
      },
      {
        "event": {
          "biking": {
            "startTime": 11,
            "id": "c",
            "run": "y"
          }
        },
        "displayName": "Aaron"
      }
    ]
  }
}

I've been trying to finagle unique with jq but can't quite get what I want. My intended result is this:

{
  "data": {
    "results": [
      {
        "event": {
          "biking": {
            "startTime": 12,
            "id": "a",
            "run": "x"
          }
        },
        "displayName": "Alex"
      },
      {
        "event": {
          "biking": {
            "startTime": 11,
            "id": "c",
            "run": "y"
          }
        },
        "displayName": "Aaron"
      }
    ]
  }
}

I was trying to use unique because I want to keep only 1 of each "run": ids where in a larger list I might have three x, two y, and four z. I'd want to keep one x, y, and z in this case based on the largest "startTime".

What have you tried so far - Show us your attempt to code this please. — Always Helping, Jul 15 '21 at 03:32
The question has been tagged with both the jq and jquery tags, but these have almost nothing to do with each other. If you want a jQuery solution, please change the title; otherwise, please remove the jquery tag. Thank you. — peak, Jul 15 '21 at 04:55

Harsh Gundecha · Answer 1 · 2021-07-15T04:43:26.367

Here's how it can be done using reducer.

const input = {
  "data": {
    "results": [{
        "event": {
          "biking": {
            "startTime": 12,
            "id": "a",
            "run": "x"
          }
        },
        "displayName": "Alex"
      },
      {
        "event": {
          "biking": {
            "startTime": 10,
            "id": "b",
            "run": "x"
          }
        },
        "displayName": "Adam"
      },
      {
        "event": {
          "biking": {
            "startTime": 11,
            "id": "c",
            "run": "y"
          }
        },
        "displayName": "Aaron"
      }
    ]
  }
};
const output = {
  data: {}
};
output.data.results = Object.values(input.data.results.reduce((r, o) => {
  r[o.event.biking.run] =
    (r[o.event.biking.run] &&
      r[o.event.biking.run].event.biking.startTime > o.event.biking.startTime) ? r[o.event.biking.run] : o
  return r
}, {}));
console.log(output);

Credits -> https://stackoverflow.com/a/56901839/8057127

score 0 · Accepted Answer · answered Jul 15 '21 at 04:45

0

Here's a straightforward jq solution:

.data.results |=
  (group_by(.event.biking.run)
   | map(max_by(.event.biking.startTime)))

It uses group_by to group by "run", and then max_by to select the desired event.

answered Jul 15 '21 at 04:45

peak

105,803
17
152
177

This is exactly what I was looking for - thanks! – tsimmons Jul 22 '21 at 20:27

score 0 · Answer 3 · answered Jul 16 '21 at 16:03

0

This would work

const result = [
  ...new Map(
    obj.data.results.map((item) => [item["event"]["biking"]["run"], item])
  ).values()
];

DEMO

This is based on helprjs removeDuplicates.

answered Jul 16 '21 at 16:03

gareth Slinn

11
1

Remove duplicate objects in JSON array with jq based on specific criteria

3 Answers3