DB Compact Utility

Objective

For customers managing F5 or ADC devices in AppViewX, device backups often consume significant database space, risking high disk usage and potential outages. Even after backups are deleted and archive limits are reduced, MongoDB retains allocated space, anticipating future data use. To mitigate storage buildup, we:
  1. Re-sync database replicas to keep only valid data.
  2. Run targeted cleanup/compact commands on specific collections.

To automate regular compaction, we’ve set up a Kubernetes cronjob that periodically executes the compact command on the database collections.

Implementation Overview

AppViewX has created a configurable cronjob for automated MongoDB cleanup. The job frequency can be set during installation, and it performs db.compact() on all MongoDB instances. Since compaction is only allowed on secondary nodes by default, the cronjob follows a process to compact each secondary, then promotes the primary to secondary for compaction, and finally restores it to its primary role.

Implementation Details:
  1. Health Check: Verifies all MongoDB instances are healthy and ensures the cleanup user exists.
  2. Database State Validation: Confirms each database’s status (PRIMARY, SECONDARY, or ARBITER).
  3. Replication Sync Check: Checks that all secondaries are within one hour of the primary; if not, cleanup is paused.
  4. Compaction on Secondaries: Executes `db.compact()` on each secondary, followed by a re-election to compact the former primary.
  5. Restore Configuration: Restores the original primary setup after compaction.

Any errors trigger alerts in the AppViewX GUI infrastructure alert section for quick response.

Installing the Cronjob

To install the db-compact cronjob:

  1. Download the db-compact tar from the release portal - https://release.appviewx.com/downLoadArtifact?id=1437
  2. Copy and extract the tar into the appviewx_kubernetes/yaml directory.
  3. Navigate to the appviewx_db_compact folder.
  4. Trigger the run.sh and provide the input according to the required configuration.
    Example:
  5. Create a job manually to test:
    kubectl create job -from=cronjob/db-compact -n avx-jobs