How to handle JSON decode error in Python script with try and except

Written by - 1 comments

Published on - Listed in Coding Python


The following Python code is supposed to handle JSON, stored in the "output" variable:

data = json.loads(output)

However when the "output" does not contain a valid JSON format (e.g. empty, broken syntax, etc.) the following error is raised:

Traceback (most recent call last):
  File "/home/ck/Git/python_script/python_script.py", line 99, in <module>
    data = json.loads(output)
  File "/usr/lib/python3.10/json/__init__.py", line 346, in loads
    return _default_decoder.decode(s)
  File "/usr/lib/python3.10/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib/python3.10/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

So we see the json.decoder throwing a JSONDecodeError.

Catching errors with try + except

A common way in Python scripts is to use try + except to catch errors. The "try" part runs the code you want to run, in this case the json.loads line. The "except" part handles a potential error of the try part.

Let's use the JSONDecodeError seen in the previous output:

try:
    data = json.loads(output)
except JSONDecodeError as e:
    print("Error: Unable to decode JSON, Error: {}. Manually run command ({}) to verify JSON output.".format(e, cmd))

But when I ran the script, this caused yet another error:

$ python3 python_script.py
Traceback (most recent call last):
  File "/home/ck/Git/python_script/python_script.py", line 100, in <module>
    data = json.loads(output)
  File "/usr/lib/python3.10/json/__init__.py", line 346, in loads
    return _default_decoder.decode(s)
  File "/usr/lib/python3.10/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib/python3.10/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/ck/Git/python_script/python_script.py", line 101, in <module>
    except JSONDecodeError as e:
NameError: name 'JSONDecodeError' is not defined

The new NameError shows that JSONDecodeError doesn't actually exist. Huh?

JSONDecodeError = ValueError ?

After looking for this error, an answer on StackOverflow mentions that JSONDecodeError is not the "real error type", but rather inherited from "ValueError":

Since simplejson.decoder.JSONDecodeError actually inherits from ValueError [...]

Let's try ValueError in the except part then:

try:
    data = json.loads(output)
except ValueError as e:
    print("Error: Unable to decode JSON, Error: {}. Manually run command ({}) to verify JSON output.".format(e, cmd))

And this works quite nicely now:

$ python3 python_script.py
Error: Unable to decode JSON, Error: Expecting value: line 1 column 1 (char 0). Manually run command (['cat', 'output.json']) to verify JSON output.

But even though this now works, this is not the proper solution yet!

JSONDecodeError works (if named correctly)

Luckily my toot (a post on Mastodon) received a few comments and hints and it turns out that JSONDecodeError is indeed an error which can be used in the exception handling. However the error handler JSONDecodeError does not exist in the "global" namespace of the Python script and needs to be called from the json module:

try:
    data = json.loads(output)
except json.JSONDecodeError as e:
    print("Error: Unable to decode JSON, Error: {}. Manually run command ({}) to verify JSON output.".format(e, cmd))

This works and returns the same output as using ValueError:

python3 python_script.py
Error: Unable to decode JSON, Error: Expecting value: line 1 column 1 (char 0). Manually run command (['cat', 'output.json']) to verify JSON output.

Because the goal of the try+except error handling is to catch invalid JSON data, the json.JSONDecodeError should be preferred over ValueError.


More recent articles:

RSS feed

Blog Tags:

  AWS   Android   Ansible   Apache   Apple   Atlassian   BSD   Backup   Bash   Bluecoat   CMS   Chef   Cloud   Coding   Consul   Containers   CouchDB   DB   DNS   Database   Databases   Docker   ELK   Elasticsearch   Filebeat   FreeBSD   Galera   Git   GlusterFS   Grafana   Graphics   HAProxy   HTML   Hacks   Hardware   Icinga   Icingaweb   Icingaweb2   Influx   Internet   Java   KVM   Kibana   Kodi   Kubernetes   LVM   LXC   Linux   Logstash   Mac   Macintosh   Mail   MariaDB   Minio   MongoDB   Monitoring   Multimedia   MySQL   NFS   Nagios   Network   Nginx   OSSEC   OTRS   Office   PGSQL   PHP   Perl   Personal   PostgreSQL   Postgres   PowerDNS   Proxmox   Proxy   Python   Rancher   Rant   Redis   Roundcube   SSL   Samba   Seafile   Security   Shell   SmartOS   Solaris   Surveillance   Systemd   TLS   Tomcat   Ubuntu   Unix   VMWare   VMware   Varnish   Virtualization   Windows   Wireless   Wordpress   Wyse   ZFS   Zoneminder   


Update cookies preferences