Name

cirql — Public documentation for the project.

cirql is a Go library for a small jq-style pipeline language over decoded JSON: Parse compiles a query string into a Pipeline, and Pipeline.Run threads a JSON value through a chain of |-separated transform stages.

Source: gomatic/cirql
API reference: pkg.go.dev/github.com/gomatic/cirql

Install

go get github.com/gomatic/cirql

Usage

cirql.Parse compiles a query; Pipeline.Run executes it over a []value.Value (or single value) from gomatic/go-json, returning the result set.

package main

import (
	"encoding/json"
	"fmt"

	value "github.com/gomatic/go-json"

	"github.com/gomatic/cirql"
)

func main() {
	p, err := cirql.Parse(`filter .stars > 1000 | map { name: .name, stars: .stars } | sort .stars desc | limit 2`)
	if err != nil {
		panic(err)
	}
	in := []value.Value{
		map[string]value.Value{"name": "a", "stars": int64(500)},
		map[string]value.Value{"name": "b", "stars": int64(3000)},
		map[string]value.Value{"name": "c", "stars": int64(2000)},
	}
	out, err := p.Run(in)
	if err != nil {
		panic(err)
	}
	for _, row := range out {
		enc, _ := json.Marshal(row)
		fmt.Println(string(enc))
	}
	// {"name":"b","stars":3000}
	// {"name":"c","stars":2000}
}

Deterministic time

The now() builtin reads epoch seconds from an injectable clock, so time-dependent queries are deterministic under test. Pass cirql.WithClock to Parse:

p, _ := cirql.Parse(`map { t: now() }`, cirql.WithClock(func() int64 { return 1234 }))

Language

A query is one or more stages joined by |. The result set flows left to right; each stage receives the previous stage’s output.

Transform stages

Stage	Form	Effect
`map`	`map { k: expr, ... }`	Project each object 1:1 into a new object.
`flatMap`	`flatMap { k: expr, ... }`	Like `map`, but list-valued mappings fan out into one row per element (scalars repeat; the fanout is the longest list).
`filter`	`filter expr`	Keep objects whose expression is truthy.
`reduce`	`reduce <op> (expr)?`	Aggregate the set into a single element.
`sort`	`sort expr [asc\|desc]`	Reorder by a key expression (stable; ascending by default).
`limit`	`limit N`	Truncate to at most `N` elements.
`uniq`	`uniq [expr]`	Deduplicate by key expression, or by whole value when no key is given.

Reduce operators: count, sum, min, max, avg, first, last, group_by, collect. Numeric aggregates (sum/avg/min/max) return null on empty input; count returns the element count; collect gathers values into a list; group_by buckets objects into an object keyed by the argument’s value.

Expressions

Field access with .name (and . for the whole object), $name variables, string/number/boolean/null literals, arithmetic (+ - * / %), comparison (== != > >= < <=), boolean (&& || !), parentheses, and function calls.

Builtins: length, keys, values, type, toInt, toFloat, toString, upper, lower, trim, split, join, contains, startsWith, now, flatten, distinct, coalesce.

Design

The top-level cirql package is a thin façade that wires together the sub-packages: the ANTLR-generated grammar (pkg/dialect/cirql) parses a query into a typed AST, stage.Build maps each AST stage to an executor, and pipeline.RunStages threads the normalized input through them. Errors are matchable sentinels: Parse returns the dialect’s ErrParse on a syntax error, and stage.ErrStageUnsupported when a query uses a source stage this build does not execute.

The grammar also defines source stages (query, http, file, stdin) that parse here but are executed elsewhere: stdin is the identity stage (its input is the pipeline’s input), while query/http/file report ErrStageUnsupported until the sources sub-project provides them.