Read and Use JSON in Bash with jq
Bash, unfortunately, doesn’t ship with a command that can work with JSON natively. In this lesson, we’ll learn how to read and do basic queries on JSON with jq, an installable command line tool that makes working with JSON in bash really easy. We'll pipe the JSON output of the Github API to jq to extract values.
Note: jq has to be installed. On macOS, the easiest way to do this is to run brew install jq. To view install instructions for all platforms, see https://stedolan.github.io/jq/download/.
Instructor: [00:00] I'm going to echo out some JSON here. I'm going to pipe it to JQ. I'm doing this because usually JQ is used as a filter and usually you pipe JSON to it for processing. While you don't normally use echo with JQ, it's useful for testing things out.
[00:14] Let's see how React sets the property of foo here. How do we get this one, two, three? We just use this dot notation. .foo will access the D.foo property. You can see that worked. The same works for nested values, so we can do A.B.
[00:33] Note that I'm wrapping this JQ selector in single quotes. That's because there's sometimes special symbols in these that bash will interpret before JQ gets a chance to see it. Let's combine JQ with curl to extract the JSON value. I'm going to do a curl against the Github API and I'm going to see how many stars the React package has.
[00:55] First let's see what the JSON payload looks like. Actually, if you just pipe to JQ without any selector it will just pretty print the JSON for you. We scroll up here we can see we have star gazers count. That's what we're looking for. We just want to extract this value.
[01:11] Let's copy that. Then if we run this again we'll just do .stargazers_count. That should get it. Then I'm going to add the silent flag here to curl to silence some of its output. Let's run that. We just get that number.
[01:29] Let's see how we iterate through a list. For that we're going to do . and then (). What this does is it takes each element in the array and separates it onto a new line, which may not seem very useful at first but it makes it useful by bash in a for loop or with a command like X args.
[01:54] This array value iterator syntax can be combined with the property value access syntax like that. That will get us just these IDs from that JSON. Using the Github API again, let's do a search and pull the name property for each result.
[02:10] I have my curl here. For my query I'm going to look up service worker. We'll pipe that to JQ. The JSON payload from that is going to return an object with an items property. That's going to be the array of the results. Then I'm going to get the name property from that. Cool. We can see that these are the results for that query.
Hrmm, fails in zsh
echo ['1,2,3'] | jq '.[]'
zsh: no matches found: [1,2,3]
It should be echo '[1,2,3]' | jq '.[]' with the single quotes surrounding the [].
Hrmm, fails in zsh
echo ['1,2,3'] | jq '.[]'
zsh: no matches found: [1,2,3]
It should be echo '[1,2,3]' | jq '.[]' with the single quotes surrounding the [].
Both work in iTerm2.
I had to wrap the url into the double quotes, I was getting curl -s https://api.github.com/search/repositories?q=service+worker zsh: no matches found: https://api.github.com/search/repositories?q=service+worker otherwise
Hrmm, fails in zsh