Storage Commands

KalamDB supports multiple storage backends for cold-tier data. These commands manage storage configuration, health checks, and maintenance.

For practical table-level storage selection (STORAGE_ID, USE_USER_STORAGE, and verification queries), see Storage ID Usage.

CREATE STORAGE


CREATE STORAGE <storage_id>
  TYPE '<filesystem|s3|gcs|azure>'
  [NAME '<storage_name>']
  [DESCRIPTION '<description>']
  [PATH '<path>']
  [BUCKET '<bucket_or_s3_url>']
  [REGION '<region>']
  [BASE_DIRECTORY '<path_or_url>']
  [SHARED_TABLES_TEMPLATE '<template>']
  [USER_TABLES_TEMPLATE '<template>']
  [CREDENTIALS '<json_credentials>']
  [CONFIG '<json_config>'];

Examples

Local filesystem:


CREATE STORAGE local
  TYPE 'filesystem'
  PATH './data';

Amazon S3:


CREATE STORAGE s3_prod
  TYPE 's3'
  BUCKET 'my-bucket'
  REGION 'us-west-2'
  CREDENTIALS '{"access_key_id":"...","secret_access_key":"..."}';

Google Cloud Storage:


CREATE STORAGE gcs_prod
  TYPE 'gcs'
  BUCKET 'my-gcs-bucket'
  CREDENTIALS '{"service_account_key":"..."}';

Azure Blob Storage:


CREATE STORAGE azure_prod
  TYPE 'azure'
  BUCKET 'my-container'
  CREDENTIALS '{"account_name":"...","account_key":"..."}';

ALTER STORAGE


ALTER STORAGE <storage_id>
  [SET NAME '<new_name>']
  [SET DESCRIPTION '<new_description>']
  [SET SHARED_TABLES_TEMPLATE '<new_template>']
  [SET USER_TABLES_TEMPLATE '<new_template>']
  [SET CONFIG '<json_config>'];

DROP STORAGE


DROP STORAGE <storage_id>;
DROP STORAGE IF EXISTS <storage_id>;

SHOW STORAGES


SHOW STORAGES;

STORAGE CHECK

Verify storage backend health:


STORAGE CHECK <storage_id>;
STORAGE CHECK <storage_id> EXTENDED;

STORAGE FLUSH

Force flushing data from hot tier (RocksDB) to cold tier (Parquet):


-- Flush a specific table
STORAGE FLUSH TABLE <namespace>.<table_name>;
 
-- Flush all tables in a namespace
STORAGE FLUSH ALL IN <namespace>;
STORAGE FLUSH ALL IN NAMESPACE <namespace>;
 
-- Flush everything
STORAGE FLUSH ALL;

STORAGE COMPACT

Merge and optimize cold-tier Parquet segments:


-- Compact a specific table
STORAGE COMPACT TABLE <namespace>.<table_name>;
 
-- Compact all tables in a namespace
STORAGE COMPACT ALL IN <namespace>;
STORAGE COMPACT ALL IN NAMESPACE <namespace>;
 
-- Compact everything
STORAGE COMPACT ALL;

SHOW MANIFEST

Display the storage manifest (schema + segment index):


SHOW MANIFEST;