Mastering jq: Advanced Pipes, Functions, and JSON Format Transformations

Pipes and Functions

The article introduces advanced jq usage by showing how the pipe operator ( |) can chain multiple filters, similar to Unix pipelines. For example, .name.first and .name | .first produce identical results by passing the output of the left filter to the right filter.

Combining functions with pipes enables powerful queries. Using keys retrieves the set of keys from a JSON object:

cat FunTester.json | jq '. | keys'
[
  "article",
  "name"
]

The length function returns the size of strings, arrays, or objects. Examples:

cat FunTester.json | jq '. | keys | length'
2

If the input is a string, length returns the character count.

For arrays, it returns the number of elements (or size).

For objects, it returns the number of key‑value pairs (or size).

These functions can be combined with comparison operators, e.g., checking if an array has more than one element:

cat FunTester.json | jq '. | keys | length > 1'
true

The select function acts like a filter similar to SQL's WHERE clause. It can pick objects matching a condition:

cat FunTester.json | jq '.article[] | select(.author == "tester2")'
{
  "author": "tester2",
  "title": "performanceTest"
}

Complex boolean expressions can also be used after select, though the article does not enumerate them.

Format Conversion

The second part shows how to construct new JSON structures and convert them to other formats using jq.

Creating a simple key‑value pair where the key is derived from existing data:

cat FunTester.json | jq '{(.article[0].title): "FunTester"}'
{
  "ApiTest": "FunTester"
}

When the key expression is dynamic, it must be wrapped in parentheses. If the value is an expression, parentheses are not required. Example of using a value expression to embed an entire object:

cat FunTester.json | jq '{(.article[0].title): .article}'
{
  "ApiTest": [
    {"author": "tester1", "title": "ApiTest"},
    {"author": "tester2", "title": "performanceTest"}
  ]
}

The map function extracts a specific field from each element of an array to build a new array:

cat FunTester.json | jq '{(.article[0].title): (.article | map(.title))}'
{
  "ApiTest": ["ApiTest", "performanceTest"]
}

When map receives constant arguments, it repeats those constants for each element, producing a non‑deduplicated list:

cat FunTester.json | jq '{(.article[0].title): (.article | map("FunTester1","FunTester2"))}'
{
  "ApiTest": ["FunTester1","FunTester2","FunTester1","FunTester2"]
}

The join function concatenates array elements into a string, similar to Java's StringUtils.join(). Example combining map and join:

cat FunTester.json | jq '{(.article[0].title): (.article | map("FunTester1","FunTester2") | join("-"))}'
{
  "ApiTest": "FunTester1-FunTester2-FunTester1-FunTester2"
}

These examples illustrate how jq can be used not only for querying but also for reshaping JSON data into new structures or plain text formats.