kiok Key Features
Discover the core capabilities of the workflow orchestration platform.
DAGs as Code
Define workflows in YAML, Python, or Java — all compile once to a single DagSpec. Beyond shell, python, and http, built-in Livy (Spark), Trino, and Ontul operators orchestrate external engine jobs directly.
Flexible Scheduling
Cron schedules and on-demand triggers, with catchup to recover the intervals that passed while the cluster was down. It resumes from the DAG’s last run and creates one run per missed interval, each stamped with the interval it stands for rather than the moment it was created, so a recovered run carries the time its data belongs to. A long gap is capped — the most recent intervals are recovered and the dropped range is logged, because releasing a week of runs at once is another outage, not a recovery. #{ } date macros slide the date on every run, so a DAG compiled once executes with the right date each time.
Distributed Execution & Failover
A master/worker cluster runs tasks in parallel across per-worker task slots, and driver failover keeps runs alive through worker loss — without re-running completed tasks.
Git Deployment & Connections
Auto-sync DAGs from multiple git repositories (including private ones) or upload bundles for air-gapped clusters. Credentials live in an encrypted connection store, used only via ${conn}/${secret} references — never exposed in DAGs or git.
Built-in Security
Native KMS envelope encryption secures all cluster state at rest — metadata, IAM, connections, logs — and AWS IAM-style policies control access down to individual DAGs.
Observability & Operations
Watch live run graphs, streaming job logs, metrics, and cluster topology in the built-in admin UI, with incremental S3 backup and restore for production-grade operations.
Maintenance Mode — Hold the Scheduler Still
For the moments when work starting underneath you is the problem — restoring a backup, rotating keys — the cluster stops creating work for itself. The cron scheduler creates no runs, pending runs are not dispatched to workers, git sync stops registering DAG definitions, and manual triggers are refused with 503 and Retry-After. Runs already executing are not killed: cancel the ones you need stopped, and that path stays open precisely so you can. Backups and driver-failure detection keep running. Cron firings inside the window are skipped rather than replayed on exit — releasing a night of runs at once would be an outage of its own — and every skipped firing is logged. The setting lives in the replicated RocksDB store, so a leader restarted mid-window, or a follower that wins the election mid-window, comes up already paused.
Use Cases
Data Pipeline Orchestration
Schedule and run ETL/ELT and analytics jobs with Livy (Spark), Trino, and Ontul operators, driven by task dependencies.
Job Automation
Automate recurring batch jobs reliably across a scalable worker fleet.
Workflow as Code
Version-control workflows in git and deploy them to the cluster with a single push.
Reliable Operations
Backup and restore, leader failover, and encrypted state for production-grade reliability.
Considering kiok for your workflows?
Author. Schedule. Orchestrate.
Run your workflows on a distributed, secure orchestration platform.