JSON

JQ is a new GREP

Michael Mikhailidi

• 3 min read
JQ is a new GREP
Photo by Greg Rakozy / Unsplash

If you are a cloud system administrator, security analyst, or data scientist, you already use this marvelous command-line tool. If you don't, it's time to learn something new and improve your scripting capabilities.  

If you think that JQ stands for JSON Query, you guess it right. It is a command-line JSON query tool that extracts and transforms data from JSON sources. We live at the Big Data More down, when more and more systems and services produce and consume JSON formatted information. You can name any shred of information that comes to your mind:  messages, metadata, log entries, anything that other systems may consume.

Unfortunately,  I cant consume JSON directly, and I prefer formatted tables and well-formatted emails. And from time to time, you need to use one particular value from the three-page output. So without any further ado, let's go through a few examples.

Most likely, you don't have the jq utility installed on your VM, so install it or and your system administrator to install it for you:

$ sudo apt-get install jq -y
Install jq utility on Ubuntu

Since I use my Always Free OCI instance, the most natural JSO source would be a compute instance metadata, and I want to keep it small, so let's start with VNICs.

Query network interface metadata
  1. Make your JSON output much more readable.  
curl -s -H "Authorization: Bearer Oracle" \
  http://169.254.169.254/opc/v2/vnics | jq
Query all elements
JQ: Formatted output. 

The query result shows a single-item list, and it matches the input. Because the default query is '.' (period), which in human words means "select the current object."  

2. Select the first object in the list. Remember, we start the index with 0. 

curl -s -H "Authorization: Bearer Oracle" \
  http://169.254.169.254/opc/v2/vnics | jq '.[0]'
Query the first element in the list
The first item on the list

3. Let's query some key values, for example, the CIDR block for this instance. We do it the same way ad the first object  

curl -s -H "Authorization: Bearer Oracle" \
http://169.254.169.254/opc/v2/vnics | \
jq '.[0].subnetCidrBlock'
Query a single key.

4. You can request multiple keys  and use them to produce new objects

curl -s -H "Authorization: Bearer Oracle" \
 http://169.254.169.254/opc/v2/vnics | \
 jq '.[0].privateIp, .[0].virtualRouterIp, .[0].subnetCidrBlock'
Query multiple keys. 
Query multiple keys response

5. Let's get the same result but more conventional, with the join function. 

curl -s -H "Authorization: Bearer Oracle" \ 
http://169.254.169.254/opc/v2/vnics | \
jq '[.[0].privateIp, .[0].virtualRouterIp, .[0].subnetCidrBlock] |join(", ")'
Join query results into a string
Joined string result

Now you get the idea of how powerful this tool is. To unlock its full potential, check some additional resources: