How to retrieve values from json output in Bash

Written by - 0 comments

Published on - last updated on October 15th 2021 - Listed in Linux Coding Shell


Sure, one could get a json output with curl and then do all kinds of grepping, sedding and awking. But there is actually a cool way, to get the value directly by accessing the json objects. The magic is done by jshon

Note/Update October 15th 2021: This article about jshon was written in 2016. I have meanwhile discovered jq and switched to it, mainly because jq is under ongoing development compared to jshon.

Fortunately this (cli-)program doesn't need to be compiled manually, in Ubuntu it can be downloaded from the official repos:

$ sudo apt-get install jshon

Now let's do some json'ing. Let's take the ElasticSearch cluster stats for example. As you may know, you can get the current ES cluster statistics and information from the following URL: http://your.es.node:9200/_cluster/stats (for a human readable output you can use http://your.es.node:9200/_cluster/stats?human&pretty). 

The json output is separated into several objects. You can use jshon to list them:

$ curl "http://elasticsearch.local:9200/_cluster/stats" -s | jshon -k
timestamp
cluster_name
status
indices
nodes

As you see above, I directly accessed the ES URL and piped the output to jshon. The -k parameter returns a list of keys. In this case we got 5 keys.

The value I am looking for is within the "indices" key, so I first display the values of the full key with the -e parameter (returns json value):

$ curl "http://elasticsearch.local:9200/_cluster/stats" -s | jshon -e indices
{
 "count": 13,
 "shards": {
  "total": 61,
  "primaries": 61,
  "replication": 0.0,
  "index": {
   "shards": {
    "min": 1,
    "max": 5,
    "avg": 4.6923076923076925
   },
   "primaries": {
    "min": 1,
    "max": 5,
    "avg": 4.6923076923076925
   },
   "replication": {
    "min": 0.0,
    "max": 0.0,
    "avg": 0.0
   }
  }
 },
 "docs": {
  "count": 935249,
  "deleted": 196434
 },
 "store": {
  "size": "11.9gb",
  "size_in_bytes": 12860502682,
  "throttle_time": "3.7h",
  "throttle_time_in_millis": 13374212
 },
 "fielddata": {
  "memory_size": "7.9mb",
  "memory_size_in_bytes": 8328212,
  "evictions": 0
 },
 "filter_cache": {
  "memory_size": "78.1mb",
  "memory_size_in_bytes": 81991016,
  "evictions": 72989502
 },
 "id_cache": {
  "memory_size": "0b",
  "memory_size_in_bytes": 0
 },
 "completion": {
  "size": "0b",
  "size_in_bytes": 0
 },
 "segments": {
  "count": 453,
  "memory": "17.9mb",
  "memory_in_bytes": 18873642,
  "index_writer_memory": "0b",
  "index_writer_memory_in_bytes": 0,
  "index_writer_max_memory": "190.1mb",
  "index_writer_max_memory_in_bytes": 199421333,
  "version_map_memory": "0b",
  "version_map_memory_in_bytes": 0,
  "fixed_bit_set": "3.2mb",
  "fixed_bit_set_memory_in_bytes": 3448160
 },
 "percolate": {
  "total": 0,
  "get_time": "0s",
  "time_in_millis": 0,
  "current": 0,
  "memory_size_in_bytes": -1,
  "memory_size": "-1b",
  "queries": 0
 }
}

So this is the full output, but not the single value of what I actually wanted. To do this, the subkeys within the "indices" key can be shown and used the same way:

$ curl "http://elasticsearch.local:9200/_cluster/stats" -s | jshon -e indices -k
percolate
shards
count
store
docs
fielddata
filter_cache
id_cache
completion
segments

So now we got a nice list of the subkeys which we can access directly with an additional -e:

$ curl "http://elasticsearch.local:9200/_cluster/stats" -s | jshon -e indices -e store
{
 "size": "11.9gb",
 "size_in_bytes": 12860502682,
 "throttle_time": "3.7h",
 "throttle_time_in_millis": 13374212
}

And to get the final single value I wanted from the start, this can be retrieved by using yet another -e parameter:

$ curl "http://elasticsearch.local:9200/_cluster/stats" -s | jshon -e indices -e store -e "size_in_bytes"
12860502682


Add a comment

Show form to leave a comment

Comments (newest first)

No comments yet.

RSS feed

Blog Tags:

  AWS   Android   Ansible   Apache   Apple   Atlassian   BSD   Backup   Bash   Bluecoat   CMS   Chef   Cloud   Coding   Consul   Containers   CouchDB   DB   DNS   Database   Databases   Docker   ELK   Elasticsearch   Filebeat   FreeBSD   Galera   Git   GlusterFS   Grafana   Graphics   HAProxy   HTML   Hacks   Hardware   Icinga   Influx   Internet   Java   KVM   Kibana   Kodi   Kubernetes   LVM   LXC   Linux   Logstash   Mac   Macintosh   Mail   MariaDB   Minio   MongoDB   Monitoring   Multimedia   MySQL   NFS   Nagios   Network   Nginx   OSSEC   OTRS   Office   PGSQL   PHP   Perl   Personal   PostgreSQL   Postgres   PowerDNS   Proxmox   Proxy   Python   Rancher   Rant   Redis   Roundcube   SSL   Samba   Seafile   Security   Shell   SmartOS   Solaris   Surveillance   Systemd   TLS   Tomcat   Ubuntu   Unix   VMWare   VMware   Varnish   Virtualization   Windows   Wireless   Wordpress   Wyse   ZFS   Zoneminder