CloudQuery Week: Destination Hardening and a Contribution Policy Shift


CloudQuery, the Go based ELT tool that syncs cloud APIs into databases, shipped 31 commits this week covering 160 files. Two themes dominate: a coordinated hardening pass across almost every destination plugin, driven by a monorepo security audit, and a policy change that closes the repository to external pull requests. Both matter to operators who sync into shared infrastructure.

The bulk of the security work targets how destination plugins build SQL statements and queries. A series of commits from the same author, all filed within two days, closes the same class of problem in plugin after plugin:

  • ClickHouse: identifiers escaped and table names bound as parameters in the query builder. The commit carries !, so expect a major version bump for that plugin.
  • MySQL: backticks in identifiers escaped in the destination writer.
  • Snowflake, DuckDB, SQLite, and the SQLite Python destination each got quoting fixes.
  • BigQuery now binds embeddings row IDs and validates table names before use.
  • Neo4j quotes Cypher identifiers, Elasticsearch validates table names before using them as index targets, and file and object store destinations keep table name paths inside the configured prefix.

The root cause is structural. Source plugins derive table and column names from cloud API resource names and field names. Those names flow into destination write paths where some were interpolated directly into SQL or query strings instead of being bound or quoted. A hostile or malformed field name from a source API could reach statement text. The fix pattern is consistent: quote or escape identifiers, bind values where the driver supports it, and validate names against an allowlist before use. New test files like table_name_escape_test.go in the ClickHouse plugin suggest the coverage is there to keep it fixed.

One more breaking change deserves its own mention: the Kafka destination now verifies broker TLS certificates by default. Previously you could end up writing cloud inventory data over an unverified TLS connection, which defeats the point of encrypting it. The commit is 78916db.

This is a default flip, so the failure mode is operational, not silent. If your brokers use self signed certificates or an internal CA, the next plugin upgrade will start failing the TLS handshake. You will need to point the plugin at your CA bundle explicitly. Winners: anyone syncing into a shared Kafka cluster over untrusted networks. Losers: anyone who was silently relying on the old behavior and now has a config change to make. That is the right tradeoff, and the ! in the commit prefix marks it correctly.

A quieter fix addresses a data loss path in syncs without state checkpoints. delete-stale removes rows for tables that no longer appear in a sync. The problem: if a table’s writes failed, the table may legitimately be missing from the write results, and delete-stale would then wipe rows that still have valid data. The fix in b9472a9 skips delete-stale for tables with failed writes in that mode.

If you run incremental syncs without state checkpoints and you have seen rows vanish after a partial failure, this is probably the explanation. Worth noting the tradeoff: skipping the delete leaves stale rows behind until a clean sync, so downstream consumers should treat the destination as eventually consistent during failures.

The most recent commit changes how the project accepts contributions. The policy commit, together with a rewritten CONTRIBUTING.md and a new AGENTS.md, states that only collaborators and invited external contributors can open pull requests. Everyone else contributes through issues, and the bug report template now requires reproduction steps: exact commands, configuration, and ideally a failing test. The stated reasons are supply chain risk from external pull requests and faster fixes from well specified issues.

The practical consequence for users is simple. If you hit a bug, spend the time on a minimal reproduction before filing. A cloudquery --version output, a redacted config, and the failing command will get you further than a patch ever would under this policy. It is a defensible position for a monorepo of this size, though it does move more review load onto the core team.

The rest of the window is routine: aws-sdk-go-v2 monorepo bumps, Google API and Azure storage module updates, and two CLI releases. The CLI moved to v6.51.0 and then quickly to v6.51.1, carrying the destination fixes above plus a plugin-pb-go update. The cli/go.mod file was the most touched file in the window, which tells you where the release machinery sits.

  • Check which destination plugins you use and whether their ! marked fixes bump major versions. Read each migration note before upgrading your sync configs.
  • If you run the Kafka destination, audit your broker certificate setup now, before the upgrade forces the issue.
  • If you run syncs without state checkpoints with delete-stale enabled, verify your tables no longer lose rows on partial write failures, and expect a follow up on checkpointed sync behavior.

The repo lives at cloudquery/cloudquery on the main branch.