Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Haze

Hazy with a chance of clouds.

haze is a tool that provides an easy way to set up Nextcloud test instances with a choice of PHP version, database server, optional s3 or LDAP setup and much more.

Quickstart

  • Grab a binary from the Codeberg releases and place it in your $PATH

  • Create a file ~/.config/haze/haze.toml with the following content:

    sources_root = "/path/to/nextcloud/sources"

  • Start a basic Nextcloud instance:

    haze start

  • Navigate to the address that is provided in the output.

See the setup, configuration and usage for a more details.

Setup

Requirements

  • Docker

haze is built around docker containers and manages containers using the docker socket. Using Podman with docker compatibility might also work, but is untested.

Installation

Configuration

Create a file ~/.config/haze/haze.toml with the following options:

sources_root = "/path/to/nextcloud/sources"

See the configuration section for more options.

Test the Setup

Quick examples

  • Start a basic Nextcloud instance:

    haze start

  • Navigate to the address that is provided in the output.

Usage

Quick examples

  • Start a Nextcloud instance with postgresql, and s3 primary storage:

    haze start pgsql s3

  • Start a Nextcloud instance with sqlite, php 8.3 and an smb external storage:

    haze start 8.3 smb

  • Run specific units test against an oracle database

    haze test oracle apps/dav/tests/unit/Connector/Sabre

Managing instances

Starting an instance

haze start [--name <name>] [--detach] [database] [php-version] [services] [vX.Y.Z]

Where database is one of sqlite, mysql, mariadb, pgsql or oracle with an optional version (e.g. pgsql:12), defaults to sqlite. And php-version is one of 8.0, 8.1, 8.2, 8.3, 8.4 or 8.5, defaults to the maximum version support by the current Nextcloud version.

You can specify a version number (e.g. v32.0.2) to use the sources from a release instead of using the local sources.

Use --name <name> to give the instance a specific name instead of a randomly generated one. For example:

Use --detach to give the instance its own sources instead of sharing sources_root with all other instances.

See the services documentation for a list of available services.

List running instances

haze

or

haze list

Stop an instance

haze [match] stop

Remove all unpinned running instances

haze clean

Pin an instance

haze [match] pin

Pinned instances will not be removed by haze clean.

Unpin an instance

haze [match] unpin

Run commands in a temporary instance

haze shell [database] [php-version] [services] [cmd]

This will create an instance, run the provided command, and cleanup the instance.

If no cmd is specified it will launch bash

Run tests in a new instance

haze test [database] [php-version] [services] [phpunit version] [path]

Where path is a file or folder to run PHPUnit in, relative to the sources root.

This will create a fresh instance, run the PHPUnit tests, and clean up the created instance.

Interacting with running instances

The following commands run against the most recently started instance by default and allow optionally providing a match to select a specific instance by its name.

Open an instance in the browser

haze [match] open

Execute a command on an instance

haze [match] [service] [cmd]

If no cmd is specified it will launch bash

If a service name or db is provided, the command will be in the container of the service or database.

If no cmd is specified it will launch bash

Execute an occ command on an instance

haze [match] occ [cmd]

Connect to the database on an instance

haze [match] db

Show the logs of an instance

haze [match] logs

Edit a file in an instance with the local $EDITOR

haze [match] edit <path>

Where <path> is the path of a file inside the container, for example config/config.php.

Reload the PHP configuration of an instance

haze [match] reload

The PHP configuration can edit changed with haze edit /config/php.ini

Run a command with instance environment variables set

haze [match] env <cmd> [args]

Runs the provided command with NEXTCLOUD_URL, DATABASE_URL and REDIS_URL environment variables set for the matched instance.

This is intended to run a local push daemon against an instance.

Git tools

Haze provides a couple of utilities to make working with many git repositories for apps easier.

Checkout a branch for all local apps

haze git checkout [branch]

Checks out the branch in all git repositories within the apps folder.

Defaults to the branch matching the current checked out server versions (e.g. master or stable33).

master and main can be used interchangeably.

Pull remote changes for all local apps

haze git pull

Performs a pull in all git repositories within the apps folder.

Update the container images

haze update

Instances with their own sources

By default every instance shares the sources configured as sources_root, so all instances always run the same code. An instance started with --detach gets its own git worktree of sources_root instead, created in worktree_dir, which lets you run several instances on different branches at the same time.

haze start --name my-fix --detach

The worktree is checked out with a detached head. Every app in one of the app_directories that gets enabled during setup receives its own worktree as well, other apps keep running from the shared app directory.

The worktrees are removed together with the instance, by haze stop or haze clean.

Configuration

Configuration is loaded from ~/.config/haze/haze.toml.

The minimum required configuration needed to get started is just the sources_root options.

The full list of supported options is:

sources_root

The local path of the Nextcloud sources. This options is required.

Type: string

app_directories

A list of additional app directory paths to look for apps into

Default []

Type: list of strings

work_dir

The location where haze keeps it’s temporary files and caches

Default: /tmp/haze

Type: string

worktree_dir

The location to store git worktrees when using instances with detached sources.

Default: <work_dir>/worktrees

Type: string

auto_setup

Options for automatically setting up the newly created Nextcloud instance.

Examples:

[auto_setup]
username = "foo"
password = "bar"
enable_apps = ["files_external"]
disable_apps = ["contacts"]
post_setup = [
    "occ group:add test",
]
config = { "enforce_theme" = "dark" }

[auto_setup]
enabled = false

auto_setup.enabled

Whether to automatically setup Nextcloud inside a created instance.

Default: true

Type: boolean

auto_setup.username

The username to use for the admin account during auto setup.

Default: admin

Type: string

auto_setup.password

The password to use for the admin account during auto setup.

Default: admin

Type: string

auto_setup.enabled_apps

Extra apps to enable after auto setup

Default: []

Type: list of strings

auto_setup.disabled_apps

Apps to disabled after auto setup

Default: []

Type: list of strings

auto_setup.post_setup

Commands to execute after auto setup

Default: []

Type: list of strings

auto_setup.config

System configuration options to set before auto setup

Default: {}

Type: Object

volume

Additional files or directories to bind-mount into instances.

Any number of volume options can be configured

Examples:

[[volume]]
source = "/tmp/haze-shared"
target = "/shared"
create = true

[[volume]]
source = "/home/me/Downloads"
target = "/Downloads"
read_only = true

volume.source

The source path on the host

Type: string

volume.target

The target path inside the container

Type: string

volume.create

Create the source directory on the host if it doesn’t exist already.

Default: false

Type: boolean

volume.read_only

Whether to mount the file or directory as read only

Default: false

Type: boolean

preset

Configured presets that can be used when creating instances.

Any number of presets can be configured.

Example:

[[preset]]
name = "groupfolders"
apps = ["groupfolders"]
commands = [
    "occ groupfolders:create gf",
    "occ groupfolders:group 1 admin read write share delete"
]

preset.name

The name of the preset, this is used when creating instances to select the preset.

Type: string without whitespace

preset.apps

A list of apps to enable when the preset is used.

Default: []

Type: list of strings

preset.commands

A list of commands to run post-setup when the preset is used.

Default: []

Type: list of strings

proxy

Configuration for the haze proxy. Only required when using the proxy.

[proxy]
address = "haze.example.com"
listen = "/run/haze/haze.sock"
https = true
cert = "/path/to/haze.test.crt"
key = "/path/to/haze.test.key"

proxy.listen

The IP and port or Unix socket for the proxy to listen on.

Required when the proxy is enabled.

Type: string

Examples:

listen = "/run/haze/haze.sock"

listen = "127.0.0.1:8080"

proxy.address

The base domain the proxy is accessible on.

Required if the proxy is enabled.

Type: string

proxy.https

Whether to enable built-in HTTPS support for the proxy.

Default: false

Type: boolean

proxy.cert

The path of the PEM encoded certificate chain to use for HTTPS.

Required if HTTPS is enabled for the proxy.

Type: string.

proxy.cert

The path of the PEM encoded private key to use for HTTPS.

Required if HTTPS is enabled for the proxy.

Type: string.

Proxy

By default, instances can be accessed by their IP. In order to get more memorable URLs and allow supporting HTTPS. haze comes with a builtin reverse proxy to allow using a wildcard domain.

Setup

  • Setup a DNS record for *.haze.example.com and haze.example.com pointing to your development machine.
  • Set the proxy configuration with your domain and desired listen endpoint.
  • Set up a service to run haze proxy in the background as your own user. A SystemD user service is recommended (see haze.service for an example).
  • If you’re already running a reverse proxy, configure your reverse proxy of choice to proxy *.haze.example.com and haze.example.com to the proxy’s listen endpoint.
  • (Optionally) setup HTTPS for the proxy.

Configuration

Add the following configuration to the haze.toml configuration file:

[proxy]
address = "haze.example.com" # the base domain for the proxy to use
listen = "127.0.0.1:8080" # the port+ip to listen on
# listen = "/var/run/haze/haze.sock" # or a unix socket path

Without a reverse proxy

If you have no other http(s) servers on your development machine, you can setup things without a reverse proxy.

Simply configure the proxy to listen on port 80 (or 443 when using HTTPS).

Binding to port 80 or443 as a regular user requires either giving the haze binary the net_bind_service capability with sudo setcap cap_net_bind_service=+ep $(which haze) (this will have to be done every time your upgrade haze) or configure your system to allow unprivileged users to bind on the low port numbers. Using sysctl net.ipv4.ip_unprivileged_port_start=80 and writing

net.ipv4.ip_unprivileged_port_start=80

to /etc/sysctl.d/bind.conf to make it persistent across reboot.

With a reverse proxy

If you’re already have other http(s) services listening on your machine, you’ll probably want to setup a reverse proxy to allow them to all be served on your machine.

The setup for this will depend on your reverse proxy of the choice, the following example configuration is for nginx.

upstream haze-handler {
    server unix:/var/run/haze/haze.sock;
}

server {
    listen 80;
    server_name *.haze.example.com;

    location / {
            proxy_pass http://haze-handler;
            proxy_set_header Host $host;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header X-Forwarded-Proto $scheme;
    }
}

Usage

When the proxy is configured, generated URLs for the instances will use a subdomain of the configured domain, e.g. the rolling-bees instance will be available at rolling-bees.haze.example.com. Additionally, haze.example.com will automatically point to the last created instance.

Additionally, the proxy allows access to the service containers trough either <instance id>-<service id>.haze.example.com for a specific instance, or <service-id>.haze.example.com for the last created instance. For example rolling-bees-mail.haze.example.com will give access to the smtp4dev web interface of the rolling-bees instance.

DNS

Since the domain name used for the instance is dynamic, a wildcard DNS record is required.

With your domain’s DNS provider

If you own a domain you would like to use, you can create a wildcard domain within the DNS settings of your DNS provider. For example creating a record an A record for *.haze.example.com with a value of 127.0.0.1 and a similar one for haze.example.com.

With a local dnsmasq

If you do not own a “real” domain for using with haze, you can setup dnsmasq locally to achieve the same goal instead.

How to install and enable dnsmasq will depend on your Linux distribution of choice and should be documented by it’s documentation.

Once setup, a configuration line like

address=/haze.local/172.0.0.1

should be enough to point haze.local and *.haze.local to your local host.

HTTPS

The proxy can be setup to enable using HTTPS to access the running instances. Besides the warm and fuzzy feeling of knowing that nobody can snoop on the traffic that is happening completely local inside your machine. Accessing the page over HTTPS is required for some JavaScript features (such as service workers), as they are only available in “secure contexts”.

Getting a wildcard certificate

Since the domain name used for the instance is dynamic, a wildcard certificate is required.

Let’s encrypt

Let’s encrypt allows getting trusted wildcard certificates for free if you can use DNS validation.

How to setup DNS validation will depend on the specifics of the DNS provider and ACME client. This lists some DNS providers and supported ACME clients.

Self signed

You can also create a self-signed wildcard certificate using a tool like mkcert. This certificate will not be trusted by your browser and tools like curl, but you can add manually add it to the trusted certificates on your system, or bypass the certificates warning in the browser/curl every time.

# Generate local wildcard certificate
mkcert -cert-file <path-to-your-certificates>haze.example.com.crt -key-file <path-to-your-certificates>haze.example.com.key '*.haze.example.com'

Using the certificate

Without reverse proxy

The haze proxy can serve over HTTPS directly, to enable that add the following to the [proxy] section of your haze.toml.

https = true
cert = "/path/to/haze.example.com.crt"
key = "/path/to/haze.example.com.key"

You might also want to change the port it’s listening on to 443.

With a reverse proxy

This depends on what reverse proxy you have setup. The following example is for nginx.

upstream haze-handler {
    server unix:/run/haze/haze.sock;
}

server {
    listen 80;
    listen 443 ssl;
    http2 on;
    server_name *.haze.example.com;

    ssl_certificate     <path-to-your-certificates>/haze.example.com.crt;
    ssl_certificate_key <path-to-your-certificates>/haze.example.com.key;

    location / {
            proxy_pass http://haze-handler;
            proxy_set_header Host $host;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header X-Forwarded-Proto $scheme;
    }
}

Services

The following service options are available:

  • s3: set up an S3 server and configure to Nextcloud to use it as primary storage.
    • s3s: enable TLS for the S3 setup.
    • s3mb: enable multi-bucket S3 setup.
    • s3m: enable multi-instance S3 setup.
  • ldap: set up an LDAP server.
  • saml: set up Authentik as a SAML IDP.
  • oidc: set up Authentik as an OIDC IDP.
  • scim: set up Authentik as a SCIM server.
  • office: set up a Nextcloud Office server.
  • onlyoffice setup an OnlyOffice document server.
  • push set up client push.
  • smb: set up a samba server for external storage use.
  • dav: set up a WebDAV server for external storage use.
  • sftp: set up a SFTP server for external storage use.
  • sftp-key: set up a SFTP server for external storage use with public key authentication.
  • kaspersky: set up a Kaspersky scan engine server in HTTP mode. ( Requires manually setting up the image)
  • kaspersky-icap: setup a Kaspersky scan engine server in ICAP mode.
  • clamav: set up a local clam av scanner in executable mode.
  • clamav-socket: set up a clam av scanner in socket mode.
  • clamav-icap: set up a clam av scanner in ICAP mode.
  • clamav-icap-tls: set up a clam av scanner in ICAP mode with TLS encryption.
  • oc: start an ownCloud instance in the same network.
  • imaginary: start an Imaginary service and configure it for preview generation.
  • mail: start a smtp4dev server and configure it the mail server.
  • webhook start a webhook tester
  • redis: start a separate container for Redis.
  • redis-tls: connect to Redis over TLS.
  • <path to app.tar.gz>: by specifying the path to an app package this package will be extracted into the apps. directory of the new instance (overwriting any existing app code). This can be used to quickly test a packaged app.
  • The name of any configured preset.

Federation

Multiple instances can reach each other by using their instance name as domain name to allow for testing federation between instances.

For example, if you have a rover-beavers and splendid-couch instance, you can create a federated share from the rover-beavers instance to http://admin@splendid-couch.

If the proxy is setup with HTTP, you can use HTTP between the instances by using the full proxy URLs.

For example sharing to admin@splendid-couch.haze.example.com.

Haze scripts

Haze scripts combine a set of instance options and a script to run in the instance.

Haze scripts are intended to way to create automated ways of running more complex tests are setting up more complex instances.

A script contains of 3 paths

  1. An optional shebang line setting haze as the interpreter, e.g. #! /usr/bin/env -S haze script
  2. A shebang line setting the options for the instance creation as the interpreter, e.g. #! haze shell pgsql s3
  3. The rest that is ran as a script inside the created instance.

The first shebang is set, the script can be ran directly. Else it needs to be run with haze script [path-to-script].

For example, the following script will create an instance with postgresql and s3 primary storage. Then creates a new file, gets the file id, read the object of the file from S3, validate that it contains the expected contents, and cleanup the instance.

#! #! /usr/bin/env -S haze script
#! haze shell pgsql s3

echo 'test' | occ file:put '-' /admin/files/test.txt
FILE_ID=$(occ info:file /admin/files/test.txt | grep fileid: | grep -oE '[0-9]+')
OBJECT_CONTENTS=$(occ file:object:get urn:oid:$FILE_ID -)
if [ "$OBJECT_CONTENTS" == 'test' ]; then
    echo "object contains expected contents"
else
    echo "object does not contains expected contents!"
fi

Script modes

Scripts can be either shell or start scripts.

shell scripts will automatically remove the created instance once the script is done, just like haze shell will. The intended use case for this is performing some tests in the created instance without leaving state behind.

start scripts on the other hand will keep the instance, like haze start will. The intended use case for this is creating more complex instances without having to configure a dedicated preset.

The script mode is determined based on the shebang line. The mode set in a script can be overridden by running the script with haze script [shell|start] path-to-script.sh.

Using different interpreters

By default, the script contents are executed as either bash script, or nu script based on the extension.

You can overwrite the interpreter used to execute the script by adding an extra #! line to the script, for example:

#! /usr/bin/env -S haze script
#! haze shell pgsql s3
#! php -f
<?php
print("Hello");

Note that the interpreter used needs to already be available inside the haze container.

Xdebug

To use Xdebug running in a haze instance with your IDE for debugging, you need to tell you IDE how to properly map the path from the container to the host. The IDE debugger configuration usually looks like:

	{
		"label": "PHP: Debug server within docker",
		"adapter": "Xdebug",
		"request": "launch",
		"port": 9003,
		"pathMappings": {
			"/var/www/html": "<sources_root_configured_in_haze.toml>",
			"/var/www/html/apps-extra": "<app_directories_configured_in_haze.toml>",
		},
	},

This would have to be adapted for detached instances.

Debugging all requests

By default, Xdebug is configured to only run for requests that containing the trigger (e.g. from using the Xdebug browser extension, or adding ?XDEBUG_SESSION_START=1 to the URL).

To enable Xdebug for all requests:

  • Un-comment the lines in haze [cloud-id] edit /config/php.ini
  • Then run haze reload [cloud-id]

FrankenPHP (experimental)

You can have Nextcloud run on FrankenPHP by starting the instance with the franken-php option.

Caddy’s admin metrics are then accessible through http://<cloud-id>-franken-php.haze.test. With ember you can run:

ember  --addr http://<cloud-id>-franken-php.haze.test