/gojsonq.github.io

A simple Go package to Query over JSON Data

gojsonq

A simple Go package to Query over JSON Data

gojsonq-logo

Installation

Install the package using

$ go get github.com/thedevsaddam/gojsonq

Usage

To use the package import it in your *.go code

import "github.com/thedevsaddam/gojsonq"

Let's see a quick example:

package main

import "github.com/thedevsaddam/gojsonq"

const json = `{"name":{"first":"Tom","last":"Hanks"},"age":61}`

func main() {
	name := gojsonq.New().JSONString(json).Find("name.first")
	println(name.(string)) // Tom
}

Another example:

package main

import (
	"fmt"

	"github.com/thedevsaddam/gojsonq"
)

const json = `{"city":"dhaka","type":"weekly","temperatures":[30,39.9,35.4,33.5,31.6,33.2,30.7]}`

func main() {
	avg := gojsonq.New().JSONString(json).From("temperatures").Avg()
	fmt.Println(avg) // 33.471428571428575
}

API for different queries

Sample data (data.json) 
{
   "name":"computers",
   "description":"List of computer products",
   "prices":[2400, 2100, 1200, 400.87, 89.90, 150.10],
   "names":["John Doe", "Jane Doe", "Tom", "Jerry", "Nicolas", "Abby"],
   "items":[
      {
         "id":1,
         "name":"MacBook Pro 13 inch retina",
         "price":1350
      },
      {
         "id":2,
         "name":"MacBook Pro 15 inch retina",
         "price":1700
      },
      {
         "id":3,
         "name":"Sony VAIO",
         "price":1200
      },
      {
         "id":4,
         "name":"Fujitsu",
         "price":850
      },
      {
         "id":null,
         "name":"HP core i3 SSD",
         "price":850
      }
   ]
}

Following API examples are shown based on the sample JSON data given above. To get a better idea of the examples see that JSON data first.

List of API:

File(path)

This method takes a JSON file path as argument for further queries.

res := gojsonq.New().File("./data.json").From("items").Get()
fmt.Printf("%#v\n", res)

JSONString(json)

This method takes a valid JSON string as argument for further queries.

res := gojsonq.New().JSONString("[19, 90.9, 7, 67.5]").Sum()
fmt.Printf("%#v\n", res)

Reader(io.Reader)

This method takes an io.Reader as argument to read JSON data for further queries.

strReader := strings.NewReader("[19, 90.9, 7, 67.5]")
res := gojsonq.New().Reader(strReader).Avg()
fmt.Printf("%#v\n", res)

Get()

This method will execute queries and will return the resulted data. You need to call it finally after using some query methods. See usage in the above example

Find(path)

  • path -- the path hierarchy of the data you want to find.

You don't need to call Get() method after this. Because this method will fetch and return the data by itself.

caveat: You can't chain further query methods after it. If you need that, you should use From() method.

example:

Let's say you want to get the value of 'items' property of your JSON Data. You can do it like this:

items := gojsonq.New().File("./data.json").Find("vendor.items");
fmt.Printf("%#v\n", items)

If you want to traverse to more deep in hierarchy, you can do it like:

item := gojsonq.New().File("./data.json").Find("vendor.items.[0]");
fmt.Printf("%#v\n", item)

From(path)

  • path (optional) -- the path hierarchy of the data you want to start query from.

By default, query would be started from the root of the JSON Data you've given. If you want to first move to a nested path hierarchy of the data from where you want to start your query, you would use this method. Skipping the path parameter or giving '.' as parameter will also start query from the root Data.

Difference between this method and Find() is that Find() method will return the data from the given path hierarchy. On the other hand, this method will return the Object instance, so that you can further chain query methods after it.

Example:

Let's say you want to start query over the values of 'items' property of your JSON Data. You can do it like this:

jq := gojsonq.New().File("./data.json").From("items").Where("price", ">", 1200)
fmt.Printf("%#v\n", jq.Get())

If you want to traverse to more deep in hierarchy, you can do it like:

jq := gojsonq.New().File("./data.json").From("vendor.items").Where("price", ">", 1200)
fmt.Printf("%#v\n", jq.Get())

Select(properties)

  • properties -- The properties which you want to get in final results. It is similar to Only but the only difference is you can chain Select in any where. To get a clear idea see the example below:

Example

jq := gojsonq.New().File("./data.json").From("items").Select("id", "name").WhereNotNil("id")
fmt.Printf("%#v\n", jq.Get())

Output

[]interface {}{
    map[string]interface {}{"id":1, "name":"MacBook Pro 13 inch retina"},
    map[string]interface {}{"id":2, "name":"MacBook Pro 15 inch retina"},
    map[string]interface {}{"id":3, "name":"Sony VAIO"},
    map[string]interface {}{"id":4, "name":"Fujitsu"},
}

Note: You can also select nested property and use alias to the property, see the example below:

jq := gojsonq.New().File("./data.json").From("users").Select("id", "user.name as uname", "user.followers")
fmt.Printf("%#v\n", jq.Get())

where(key, op, val)

  • key -- the property name of the data.

  • val -- value to be matched with. It can be a int, string, bool or slice - depending on the op.

  • op -- operator to be used for matching. The following operators are available to use:

    • = : For equality matching

    • eq : Same as =

    • != : For not equality matching

    • neq : Same as !=

    • <> : Same as !=

    • > : Check if value of given key in data is Greater than val

    • gt : Same as >

    • < : Check if the value of given key in data is Less than val

    • lt : Same as <

    • >= : Check if the value of given key in data is Greater than or Equal of val

    • gte : Same as >=

    • <= : Check if the value of given key in data is Less than or Equal of val

    • lte : Same as <=

    • in : Check if the value of given key in data is exists in given val. val should be a Slice of int/float64/string.

    • notIn : Check if the value of given key in data is not exists in given val. val should be a Slice of int/float64/string.

    • startsWith : Check if the value of given key in data starts with (has a prefix of) the given val. This would only work for String type data and exact match.

    • endsWith : Check if the value of given key in data ends with (has a suffix of) the given val. This would only work for String type data and exact match.

    • contains : Check if the value of given key in data has a substring of given val. This would only work for String type data and loose match.

    • strictContains : Check if the value of given key in data has a substring of given val. This would only work for String type data and exact match.

example:

Let's say you want to find the 'items' who has price greater than 1200. You can do it like this:

jq := gojsonq.New().File("./data.json").From("items").Where("price", ">", 1200)
fmt.Printf("%#v\n", jq.Get())

You can add multiple where conditions. It'll give the result by AND-ing between these multiple where conditions.

jq := gojsonq.New().File("./data.json").From("items").Where("price", ">", 500).Where("name","=", "Fujitsu")
fmt.Printf("%#v\n", jq.Get())

You can also compare nested property for Where query like:

jq := gojsonq.New().Reader(r.Body).From("users").Where("address.city", "=", "LA")
fmt.Printf("%#v\n", jq.Get())

OrWhere(key, op, val)

Parameters of OrWhere() are the same as Where(). The only difference between Where() and OrWhere() is: condition given by the OrWhere() method will OR-ed the result with other conditions.

For example, if you want to find the 'items' with id of 1 or 2, you can do it like this:

jq := gojsonq.New().File("./data.json").From("items").Where("id", "=", 1).OrWhere("id", "=", 2)
fmt.Printf("%#v\n", jq.Get())

WhereIn(key, val)

  • key -- the property name of the data
  • val -- it should be a Slice of int/float64/string

This method will behave like where(key, "in", val) method call.

WhereNotIn(key, val)

  • key -- the property name of the data
  • val -- it should be a Slice of int/float64/string

This method will behave like Where(key, "notIn", val) method call.

WhereNil(key)

  • key -- the property name of the data

This method will behave like Where(key, "=", nil) method call.

WhereNotNil(key)

  • key -- the property name of the data

This method will behave like Where(key, "!=", nil) method call.

WhereEqual(key, val)

  • key -- the property name of the data
  • val -- it should be a int/float64/string

This method will behave like Where(key, "=", val) method call.

WhereNotEqual(key, val)

  • key -- the property name of the data
  • val -- it should be a int/float64/string

This method will behave like Where(key, "!=", val) method call.

WhereStartsWith(key, val)

  • key -- the property name of the data
  • val -- it should be a String

This method will behave like Where(key, "startsWith", val) method call.

WhereEndsWith(key, val)

  • key -- the property name of the data
  • val -- it should be a String

This method will behave like where(key, "endsWith", val) method call.

WhereContains(key, val)

  • key -- the property name of the data
  • val -- it should be a String

This method will behave like Where(key, "contains", val) method call.

WhereStrictContains(key, val)

  • key -- the property name of the data
  • val -- it should be a String

This method will behave like Where(key, "strictContains", val) method call.

Limit(val)

  • val -- the value should be a integer number greater than Zero

example:

Let's say you want to get 2 items from the 'items' node. You can do it like this:

jq := gojsonq.New().File("./data.json")
res := jq.From("items").Limit(2).Get()
fmt.Printf("%#v\n", res)

Sum(property)

  • property -- the property name of the data.

example:

Let's say you want to find the sum of the 'price' of the 'items'. You can do it like this:

jq := gojsonq.New().File("./data.json").From("items")
fmt.Printf("%#v\n", jq.Sum("price"))

If the data you are aggregating is a slice of int/float, you don't need to pass the 'property' parameter. See example below:

jq := gojsonq.New().File("./data.json").From("prices")
fmt.Printf("%#v\n", jq.Sum())

Count()

It will return the number of elements in the collection/object.

example:

Let's say you want to find how many elements are in the 'items' property. You can do it like:

jq := gojsonq.New().File("./data.json").From("items")
fmt.Printf("%#v\n", jq.Count())

// or count properties of an object
jq := gojsonq.New().File("./data.json").From("items.[0]")
fmt.Printf("%#v\n", jq.Count())

Max(property)

  • property -- the property name of the data

example:

Let's say you want to find the maximum of the 'price' of the 'items'. You can do it like this:

jq := gojsonq.New().File("./data.json").From("items")
fmt.Printf("%#v\n", jq.Max("price"))

If the data you are querying a slice of int/float, you don't need to pass the 'property' parameter. See example below:

jq := gojsonq.New().File("./data.json").From("prices")
fmt.Printf("%#v\n", jq.Max())

Min(property)

  • property -- the property name of the data

example:

Let's say you want to find the minimum of the 'price' of the 'items'. You can do it like this:

jq := gojsonq.New().File("./data.json").From("items")
fmt.Printf("%#v\n", jq.Min("price"))

If the data you are querying a slice of int/float, you don't need to pass the 'property' parameter. See detail example:

jq := gojsonq.New().File("./data.json").From("prices")
fmt.Printf("%#v\n", jq.Min())

Avg(property)

  • property -- the property name of the data

example:

Let's say you want to find the average of the 'price' of the 'items'. You can do it like this:

jq := gojsonq.New().File("./data.json").From("items")
fmt.Printf("%#v\n", jq.Avg("price"))

If the data you are querying a slice of int/float, you don't need to pass the 'property' parameter. See detail example:

jq := gojsonq.New().File("./data.json").From("prices")
fmt.Printf("%#v\n", jq.Avg())

First()

It will return the first element of the collection.

example:

jq := gojsonq.New().File("./data.json").From("items")
fmt.Printf("%#v\n", jq.First())

Last()

It will return the last element of the collection.

example:

jq := gojsonq.New().File("./data.json").From("prices")
fmt.Printf("%#v\n", jq.Last())

Nth(index)

  • index -- index of the element to be returned.

It will return the nth element of the collection. If the given index is a positive value, it will return the nth element from the beginning. If the given index is a negative value, it will return the nth element from the end.

example:

jq := gojsonq.New().File("./data.json").From("items")
fmt.Printf("%#v\n", jq.Nth(2))

GroupBy(property)

  • property -- The property by which you want to group the collection.

example:

Let's say you want to group the 'items' data based on the 'price' property. You can do it like:

jq := gojsonq.New().File("./data.json").From("items")
fmt.Printf("%#v\n", jq.GroupBy("price").Get())

You can also group data using nested property:

jq := gojsonq.New().Reader(r.Body).From("users").GroupBy("address.city")
fmt.Printf("%#v\n", jq.Get())

Sort(order)

  • order -- If you skip the 'order' property the data will be by default ordered as ascending. You need to pass 'desc' as the 'order' parameter to sort the data in descending order.

Note: This method should be used for Slice. If you want to sort an Array of Objects you should use the SortBy() method described later.

example:

Let's say you want to sort the 'prices/names' data. You can do it like:

jq := gojsonq.New().File("./data.json").From("prices")
fmt.Printf("%#v\n", jq.Sort().Get())

// or sort array of strings in descending order
jq := gojsonq.New().File("./data.json").From("names")
fmt.Printf("%#v\n", jq.Sort("desc").Get())

SortBy(property, order)

  • property -- You need to pass the property name on which the sorting will be done.
  • order -- If you skip the 'order' property the data will be by default ordered as ascending. You need to pass 'desc' as the 'order' parameter to sort the data in descending order.

Note: This method should be used for Array of Objects. If you want to sort a plain Array you should use the Sort() method described earlier.

example:

Let's say you want to sort the 'price' data of 'items'. You can do it like:

jq := gojsonq.New().File("./data.json").From("items")
fmt.Printf("%#v\n", jq.SortBy("price").Get())

// or in descending order
jq := gojsonq.New().File("./data.json").From("items")
fmt.Printf("%#v\n", jq.SortBy("price", "desc").Get())

You can also sort data using nested property:

jq := gojsonq.New().Reader(r.Body).From("users").SortBy("address.city")
fmt.Printf("%#v\n", jq.Get())

Reset()

Reset the queries with the original data so that you can query again. See the example below:

jq := gojsonq.New().File("./data.json")

res1 := jq.Where("price", ">", 900).From("items").Sum("price")

// got our first result, now reset the instance and query again
res2 := jq.Reset().From("prices").Max()
fmt.Printf("Res1: %#v\nRes2: %#v\n", res1, res2)

Only(properties)

  • properties -- The properties which you want to get in final results. To get a clear idea see the example below:

Example

jq := gojsonq.New().File("./data.json").From("items").WhereNotNil("id")
fmt.Printf("%#v\n", jq.Only("id", "price"))

Output

[]interface {}{
    map[string]interface {}{"id":1, "price":1350},
    map[string]interface {}{"id":2, "price":1700},
    map[string]interface {}{"id":3, "price":1200},
    map[string]interface {}{"id":4, "price":850},
}

Pluck(property)

  • property -- The property by which you want to get an array.

Only returns a plain array of values for the property, you can't chain further method to Pluck. To get a clear idea see the example below:

Example

jq := gojsonq.New().File("./data.json").From("items")
fmt.Printf("%#v\n", jq.Pluck("price"))

Output

[]interface {}{1350, 1700, 1200, 850, 850}

Macro(operator, QueryFunc)

Query matcher can be written as macro and used multiple time for further queries. Lets' say we don't have weak match for WhereStartsWith, we can write one. See the example below:

jq := gojsonq.New().File("./data.json").From("items")
jq.Macro("WM", func(x, y interface{}) (bool, error) { // WM is our weak match operator
    xs, okx := x.(string)
    ys, oky := y.(string)
    if !okx || !oky {
        return false, fmt.Errorf("weak match only support string")
    }

    return strings.HasPrefix(strings.ToLower(xs), strings.ToLower(ys)), nil
})
jq.Where("name", "WM", "mac")
fmt.Printf("%#v\n", jq.Get())

Copy()

It will return a complete clone of the Object instance. Note Copy method is very useful when working concurrently. You can copy the instance for multiple goroutine

Example

jq := gojsonq.New().File("./data.json")
for i := 0; i < 10; i++ {
    go func(j *gojsonq.JSONQ) {
        fmt.Printf("Sum: %#v\n", j.From("items").Sum("price"))
    }(jq.Copy())

    go func(j *gojsonq.JSONQ) {
        fmt.Printf("Min: %#v\n", j.From("prices").Min())
    }(jq.Copy())
}
time.Sleep(time.Second)

Errors()

Return a list of errors occurred when processing the queries, it may help to debug or understand what is happening.

Error()

Return the first occurred error, you can check any error occurred when processing the queries.

Example

jq := gojsonq.New().File("./invalid-file.xson")
res := jq.Get()
err := jq.Error() // if err != nil do something, may show the error list using jq.Errors() method
fmt.Printf("Error: %v\nResult: %#v\n", err, res)

If you like the idea don't forget to put a Star on the repository, it inspires author to do some more for the package

Share your thoughts, idea or any bug report by creating an issue