Using jq to copy field from one object to a later object in a JSON array

Question

I have a series of events represented in a JSON array. I want to move data in the fields of earlier events to a later event. Is there a way to do this with jq?

Here is an example of what I'm looking for.

Input:

[
  {"verb":"set", "object":"parameter1", "value":1},
  {"verb":"set", "object":"parameter2", "value":2},
  {"verb":"run", "object":"simulation"},
  {"verb":"set", "object":"parameter1", "value":10},
  {"verb":"set", "object":"parameter2", "value":20},
  {"verb":"run", "object":"simulation"}
]

Here is my desired output:

[
  {"verb":"set", "object":"parameter1", "value":1},
  {"verb":"set", "object":"parameter2", "value":2},
  {"verb":"run", "object":"simulation", "value":{
    "parameter1":1, "parameter2":2}
  },
  {"verb":"set", "object":"parameter1", "value":10},
  {"verb":"set", "object":"parameter2", "value":20},
  {"verb":"run", "object":"simulation", "value":{
    "parameter1":10, "parameter2":20}
  }
]

It seems like I could use variables to do this, but I don't understand how to tell jq to associate the 1 and 2 with the first "run" event and 10 and 20 with the second "run" event.

Answer 1

Using jq:

reduce .[] as $elem (
        [[],{}];  # [[final array],{temporary object}]

        if $elem.verb == "set"
        then
                [
                        first + [$elem],
                        last + { ($elem.object): $elem.value }
                ]
        elif $elem.verb == "run"
        then
                [
                        first + [ $elem + { value: last } ],
                        {}
                ]
        end
) |
first

This uses a reduce() call, which is like a loop over the elements in the array and which accumulates data in the initially empty structure [[],{}]. The first part of this structure will hold the the final array that we are interested in, while the last part of the structure holds an object that we temporarily add parameter values to.

In the reduce loop, we test whether the current element is a set verb. If so, we update our accumulation structure by appending the element to the first array and setting the parameter key to the given value in the temporary object.

If the element is a run verb, we assume the accumulated parameter values should be added to the current element with the value key, and we clear the temporary object.

At the very end, we extract only the array.

first is the same as .[0], and last is the same as .[-1] (or .[1] in an array that only has two elements).

Testing:

$ jq -f script.jq file   
[
  {
    "verb": "set",
    "object": "parameter1",
    "value": 1
  },
  {
    "verb": "set",
    "object": "parameter2",
    "value": 2
  },
  {
    "verb": "run",
    "object": "simulation",
    "value": {
      "parameter1": 1,
      "parameter2": 2
    }
  },
  {
    "verb": "set",
    "object": "parameter1",
    "value": 10
  },
  {
    "verb": "set",
    "object": "parameter2",
    "value": 20
  },
  {
    "verb": "run",
    "object": "simulation",
    "value": {
      "parameter1": 10,
      "parameter2": 20
    }
  }
]

---

I see Stéphane Chazelas' answer, which happens to be a blow-by-blow Perl equivalent of this answer (they are not based on each other; some problems just have a minimal set of solutions, and we happen to pick the same solution in different languages). Their %params hash is my "temporary object", i.e. the second part of the accumulator structure.

Answer 2

If using perl instead is an option, it could be something like:

perl -MJSON::PP -l -0777 -pe '
  BEGIN {
    $j = JSON::PP->new->indent->indent_length(2);
  }

  my $array = $j->decode($_);
  my %params;
  for (@$array) {
    if ($_->{verb} eq "set") {
      $params{$_->{object}} = $_->{value};
    } elsif ($_->{verb} eq "run") {
      $_->{value} = {%params};
      undef %params;
    }
  }
  $_ = $j->encode($array)' file.json