Skip to main content

Helper Command Line Utilities

Xylok includes many helper scripts in the install location (/opt/xylok) by default. These are generally just a few lines long and just call into the Xylok Manager, possibly after copying some files around.

backup-db.sh​

Creates a Postgres dump of the database to a given location. For example:

cd /opt/xylok/backups /opt/xylok/backup-db.sh important_backup.dump

This script accounts for using the pass-through directory by creating the backup in /_passthrough with the basename of the supplied path, then moving it on the host to the full specified path.

See Backups for more information.

backup-cleanup.sh​

Removes database .dump backup files in the backup directory older than a specified number of days. For example, to remove files older than 30 days: cd /opt/xylok ./backup-cleanup.sh 30

See Backups for more information.

create-superuser.sh​

note

This script does NOT account for the pass-through directory.

This script will create a new superuser for Xylok and can be used to recover from a lost-password situation. generate-report.sh

This script builds a JSON-log version of the individual findings spreadsheet. It takes two arguments--a path to write the file to and a client UUID. This format is intended to be used with a log monitoring system such as Splunk. Each line of the output is a JSON object representing a single finding within the given client.

import-data.sh​

This script allows for bulk-importing data into Xylok. It is used internally by the installer most commonly, but may be used to import data if desired. It should be passed a list of files to import. For example, /opt/xylok/import-data.sh --ignore-errors ~/Downloads/*.xylok would import all Xylok results files in the user's Downloads directory.

It will iterate over every file in the directory importing them. By default, any errors will cause the import to stop. To ignore errors, include the --ignore-errors flag (/opt/xylok/import-data.sh --ignore-errors *).

As with normal imports, post processing is run and automatic analysis is not run by default. To change these, use the --skip-pp and --auto-analyze flags. This script accounts for using the pass-through directory by copying all supplied paths to _passthrough and calling xylok pt fix to correct the permissions. The import is then performed using the corrected paths. If this is process fails, then the import can instead be accomplished manually:

  1. Copy the desired files to /opt/xylok/_passthrough manually
  2. Run /opt/xylok/xylok pt fix to correct permissions
  3. Run /opt/xylok/run-management-cmd.sh import_data <paths to import>

periodic-backup.sh​

This script uses backup-db.sh to create a dated backup in /opt/xylok/backups. Typically this is used from a cron job to periodically create a backup. See Backups for more information.

periodic-report.sh​

This script uses generate-report.sh to create a dated report in /opt/xylok/finding-reports. It must be given a client UUID to generate the report for.

Typically this is used from a cron job to periodically create a JSON log report for ingest into a log server. For example, the following crontab entry will create such a report once a week at midnight:

0 * * * 1 /opt/xylok/periodic-report.sh <UUIDofclient>

This cron job must be run by a user with access to Docker--root or any user in the "docker" group. If using Podman, it should generally be run by the user who started the Xylok server.

This script accounts for the pass-through directory.

pgcli.sh​

This script spawns a connection to Postgres, the database used by Xylok. Unless instructed by Xylok personnel, this should not be used to modify data.

rediscli.sh​

This script spawns a connection to Redis, the in-memory cached used by Xylok. Unless instructed by Xylok personnel, this should not be used.

restore-db.sh​

This script can restore a Postgres database dump (created with backup-db.sh) over top of existing data. For example:

cd /opt/xylok/backups /opt/xylok/restore-db.sh important_backup.dump

Running this script will wipe out all existing data.

This script accounts for the pass-through directory by copying the supplied file to /opt/xylok/_passthrough, correcting permissions, then restoring the database using the internal pass-through directory.

run-management-cmd.sh​

This script runs a Django management command inside the Xylok worker container, which can be used for some low-level management functionality and is used as a helper by other scripts in /opt/xylok/. Unless instructed by Xylok personnel, this should generally not be used. There are two exceptions to this:

  • Resetting User Passwords: run-mangement-cmd.sh changepassword <username>
  • Creating a new super user: run-management-cmd.sh createsuperuser

shell.sh​

This script spawns in a shell inside the Xylok worker container, which can be used for some low-level management functionality. Unless instructed by Xylok personnel, this should generally not be used.

start.sh​

danger

As of v2021.09+, Xylok uses systemd to manage execution. You should use systemctl start xylok rather than this script directly.

stop.sh​

danger

As of v2021.09+, Xylok uses systemd to manage execution. You should use systemctl start xylok rather than this script directly.

versions.sh​

Retrieves the version numbers for all of Xylok's major Docker images. Both the core utility in the image (i.e., Postgres, Redis, etc) will be shown as well as the OS used as the base of that image. Currently, all Xylok container images are based on Alpine Linux. Xylok must be running for this script to work correctly.