..
  Licensed under the Apache License, Version 2.0 (the "License"); you may
  not use this file except in compliance with the License. You may obtain
  a copy of the License at

          http://www.apache.org/licenses/LICENSE-2.0

  Unless required by applicable law or agreed to in writing, software
  distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
  WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
  License for the specific language governing permissions and limitations
  under the License.


=======================
Using Continuous Audit
=======================

Continuous audits allow Watcher to continuously monitor and optimize your
OpenStack infrastructure based on predefined schedules or intervals. This guide
demonstrates how to set up and use continuous audits with the dummy strategy,
which is useful for testing, development, and understanding the continuous
audit workflow. However, this doc is valid for any other combination of
strategy and goal.

Overview
========

A continuous audit differs from a oneshot audit in that it runs repeatedly
at specified intervals. It supports both time-based intervals
(in seconds) and cron-like expressions for more complex scheduling patterns.

The dummy strategy is a test strategy that doesn't perform actual optimization
but creates sample actions (nop and sleep) to demonstrate the complete audit
workflow. It's ideal for:

- Testing continuous audit functionality
- Development and debugging
- Learning how Watcher works

Prerequisites
=============

Before setting up continuous audits, ensure:

1. Watcher services are running and configured properly
2. You have administrator access to OpenStack


You can verify the services are running:

.. code-block:: bash

  $ openstack optimize service list
  +----+-------------------------+------------+--------+
  | ID | Name                    | Host       | Status |
  +----+-------------------------+------------+--------+
  |  1 | watcher-decision-engine | controller | ACTIVE |
  |  2 | watcher-applier         | controller | ACTIVE |
  +----+-------------------------+------------+--------+


Continuous Audit State Machine
==============================

You can view the Audit state machine diagram in the Watcher documentation:
`Audit State Machine`_

.. _Audit State Machine: https://docs.openstack.org/watcher/latest/architecture.html#audit-state-machine


Transitions:

- An audit is created and enters the **PENDING** state.

- When the scheduled time arrives, a **PENDING** audit becomes **ONGOING**.

- A continuous audit remains in the **ONGOING** state across executions.
  It does not switch to **SUCCEEDED** after each run.

- If an execution fails, the audit transitions to **FAILED** and is no longer
  executed.

- Each execution produces a new action plan. When a new action plan is created
  by the same continuous audit, previous **RECOMMENDED** action plans are moved
  to **CANCELLED**. Only the latest action plan remains in **RECOMMENDED**.

- An administrator can **CANCEL** an audit that is **PENDING** or **ONGOING**.

- An administrator can **SUSPEND** an **ONGOING** audit.

- A **SUSPENDED** audit can be resumed by an administrator, at which point it
  becomes **ONGOING** again.

- An administrator can **DELETE** an audit only when its state is
  **SUCCEEDED**, **FAILED**, or **CANCELLED**.

.. note::

   You can enable the auto-trigger option if you want to automatically apply action
   plans generated by continuous audits as soon as they are created.
   Depending on the environment, continuous audits are often good candidates for
   auto-trigger.


Create a Continuous Audit
--------------------------

Create a continuous audit that will run at regular intervals. You can specify
the interval in seconds or use cron-like expressions.

Using Time Interval (seconds)
------------------------------

This example creates a continuous audit that runs every 5 minutes indefinitely
(300 seconds):

.. code-block:: bash

  $ openstack optimize audit create \
    --goal dummy \
    --strategy dummy \
    --audit_type CONTINUOUS \
    --interval 300 \
    --name "continuous-dummy-5min"
  +---------------+--------------------------------------+
  | Field         | Value                                |
  +---------------+--------------------------------------+
  | UUID          | 7607cf57-ea05-4e1a-b8d7-34e570f95132 |
  | Name          | continuous-dummy-5min                 |
  | Created At    | 2025-08-12T07:26:18.496536+00:00     |
  | Updated At    | None                                 |
  | Deleted At    | None                                 |
  | State         | PENDING                              |
  | Audit Type    | CONTINUOUS                           |
  | Parameters    | {'para1': 3.2, 'para2': 'hello'}     |
  | Interval      | 300                                  |
  | Goal          | dummy                                |
  | Strategy      | dummy                                |
  | Audit Scope   | []                                   |
  | Auto Trigger  | False                                |
  | Next Run Time | None                                 |
  | Hostname      | None                                 |
  | Start Time    | None                                 |
  | End Time      | None                                 |
  | Force         | False                                |
  +---------------+--------------------------------------+

.. note::

   If you don't enable `notification-based`_ cluster model synchronization,
   make sure the audit interval is longer than the cluster data model
   collector period configured in the ``watcher.conf``:

   .. code-block:: ini

      [watcher_cluster_data_model_collectors.baremetal]
      period = 3600

      [watcher_cluster_data_model_collectors.compute]
      period = 3600

      [watcher_cluster_data_model_collectors.storage]
      period = 3600

   Otherwise, outdated cluster model may be used during audit execution,
   leading to suboptimal or incorrect optimization decisions.

.. _`notification-based`: https://docs.openstack.org/watcher/latest/configuration/configuring.html#configure-nova-notifications

Using Cron Expression
----------------------

For more complex scheduling, you can use cron-like expressions. This example
runs the audit every hour at the 15-minute mark:

.. code-block:: bash

  $ openstack optimize audit create \
    --goal dummy \
    --strategy dummy \
    --audit_type CONTINUOUS \
    --interval "15 * * * *" \
    --name "continuous-dummy-hourly"
  +---------------+--------------------------------------+
  | Field         | Value                                |
  +---------------+--------------------------------------+
  | UUID          | 9cbce4f1-eb75-405a-8f4e-108eb08fdd0a |
  | Name          | continuous-dummy-hourly              |
  | Created At    | 2025-08-12T07:32:31.469309+00:00     |
  | Updated At    | None                                 |
  | Deleted At    | None                                 |
  | State         | PENDING                              |
  | Audit Type    | CONTINUOUS                           |
  | Parameters    | {'para1': 3.2, 'para2': 'hello'}     |
  | Interval      | 15 * * * *                           |
  | Goal          | dummy                                |
  | Strategy      | dummy                                |
  | Audit Scope   | []                                   |
  | Auto Trigger  | False                                |
  | Next Run Time | None                                 |
  | Hostname      | None                                 |
  | Start Time    | None                                 |
  | End Time      | None                                 |
  | Force         | False                                |
  +---------------+--------------------------------------+

Time Constraints via start_time and end_time
--------------------------------------------

We can limit when the continuous audit runs by setting start and end times
in a time-interval schedule. The interval can passed in seconds or cron expression.

.. note::

   Start and End Time are interpreted in the timezone configured on the host where the
   Watcher Decision Engine service is running. We can provide ``start_time`` and
   ``end_time`` in ISO 8601 format, for example ``'2025-08-13T14:30:00'``.


The example below creates a continuous audit that runs from 12:00 to 13:00
with a 5 minute interval.

.. code-block:: bash

  $ openstack optimize audit create \
    --goal dummy \
    --strategy dummy \
    --audit_type CONTINUOUS \
    --interval 300 \
    --start-time "$(date -d 'today 12:00' +%Y-%m-%dT%H:%M:%S)" \
    --end-time "$(date -d 'today 13:00' +%Y-%m-%dT%H:%M:%S)" \
    --name "continuous-dummy-5min"
  +---------------+--------------------------------------+
  | Field         | Value                                |
  +---------------+--------------------------------------+
  | UUID          | dadd279b-1e3d-4c38-aba6-4a730a78589b |
  | Name          | continuous-dummy-5min                |
  | Created At    | 2025-08-12T08:36:42.924460+00:00     |
  | Updated At    | None                                 |
  | Deleted At    | None                                 |
  | State         | PENDING                              |
  | Audit Type    | CONTINUOUS                           |
  | Parameters    | {'para1': 3.2, 'para2': 'hello'}     |
  | Interval      | 300                                  |
  | Goal          | dummy                                |
  | Strategy      | dummy                                |
  | Audit Scope   | []                                   |
  | Auto Trigger  | False                                |
  | Next Run Time | None                                 |
  | Hostname      | None                                 |
  | Start Time    | 2025-08-12T12:00:00                  |
  | End Time      | 2025-08-12T13:00:00                  |
  | Force         | False                                |
  +---------------+--------------------------------------+

Monitoring Continuous Audit Execution
======================================

Create a continuous audit
--------------------------

Create a continuous audit with 5 second interval:

.. code-block:: bash

  $ openstack optimize audit create \
    --goal dummy \
    --strategy dummy \
    --audit_type CONTINUOUS \
    --interval 5 \
    --name "continuous-dummy-5sec"
  +---------------+--------------------------------------+
  | Field         | Value                                |
  +---------------+--------------------------------------+
  | UUID          | 7d1f1961-41a6-47ae-a94a-cf5e43174fbd |
  | Name          | continuous-dummy-5sec                |
  | Created At    | 2025-08-12T09:27:33.592575+00:00     |
  | Updated At    | None                                 |
  | Deleted At    | None                                 |
  | State         | PENDING                              |
  | Audit Type    | CONTINUOUS                           |
  | Parameters    | {'para1': 3.2, 'para2': 'hello'}     |
  | Interval      | 5                                    |
  | Goal          | dummy                                |
  | Strategy      | dummy                                |
  | Audit Scope   | []                                   |
  | Auto Trigger  | False                                |
  | Next Run Time | None                                 |
  | Hostname      | None                                 |
  | Start Time    | None                                 |
  | End Time      | None                                 |
  | Force         | False                                |
  +---------------+--------------------------------------+

Once created, the continuous audit will be automatically scheduled and executed
by the Watcher Decision Engine. You can monitor its progress:

Check Audit Status
------------------

.. code-block:: bash

  $ openstack optimize audit show 7d1f1961-41a6-47ae-a94a-cf5e43174fbd
  +---------------+--------------------------------------+
  | Field         | Value                                |
  +---------------+--------------------------------------+
  | UUID          | 7d1f1961-41a6-47ae-a94a-cf5e43174fbd |
  | Name          | continuous-dummy-5sec                |
  | Created At    | 2025-08-12T09:27:34+00:00            |
  | Updated At    | 2025-08-12T09:28:28+00:00            |
  | Deleted At    | None                                 |
  | State         | ONGOING                              |
  | Audit Type    | CONTINUOUS                           |
  | Parameters    | {'para1': 3.2, 'para2': 'hello'}     |
  | Interval      | 5                                    |
  | Goal          | dummy                                |
  | Strategy      | dummy                                |
  | Audit Scope   | []                                   |
  | Auto Trigger  | False                                |
  | Next Run Time | 2025-08-12T09:28:33                  |
  | Hostname      | chkumar-devstack-1                   |
  | Start Time    | None                                 |
  | End Time      | None                                 |
  | Force         | False                                |
  +---------------+--------------------------------------+

.. note::

   The *Next Run Time* is the next time the audit will run. It is calculated based on the
   interval and the start and end times.


List Generated Action Plans
---------------------------

Each execution of the continuous audit generates a new action plan:

.. code-block:: bash

  $ openstack optimize actionplan list --audit 7d1f1961-41a6-47ae-a94a-cf5e43174fbd
  +--------------------------------------+--------------------------------------+-------------+
  | UUID                                 | Audit                                | State       |
  +--------------------------------------+--------------------------------------+-------------+
  | b301dd17-a139-4a45-ade2-b2c2ddf006ef | 7d1f1961-41a6-47ae-a94a-cf5e43174fbd | CANCELLED   |
  | 22a5bc60-adef-447a-aa27-731b4f5f7ee3 | 7d1f1961-41a6-47ae-a94a-cf5e43174fbd | RECOMMENDED |
  +--------------------------------------+--------------------------------------+-------------+

.. note::

   In continuous audits, when a new action plan is generated, previous
   RECOMMENDED action plans are automatically set to CANCELLED state to
   avoid conflicts.


Manage Continuous Audits
========================

Stop a Continuous Audit
------------------------

To stop a continuous audit, update its state:

.. code-block:: bash

  $ openstack optimize audit update 550e8400-e29b-41d4-a716-446655440000 replace state=CANCELLED


Modify Audit Interval
---------------------

You can change the interval of a running continuous audit:

.. code-block:: bash

  $ openstack optimize audit update 550e8400-e29b-41d4-a716-446655440000 replace interval=900

The Decision Engine will automatically reschedule the audit with the new
interval.

Modify End Time
---------------

You can change the end time of a running continuous audit:

.. code-block:: bash

  $ openstack optimize audit update 550e8400-e29b-41d4-a716-446655440000 replace end_time=2025-08-12T14:00:00

Delete a Continuous Audit
--------------------------

In order to delete a continuous audit, the audit state must be
SUCCEEDED, FAILED, or CANCELLED.
An audit with PENDING or ONGOING state cannot be deleted.

To delete an ongoing or pending continuous audit, update its state to
CANCELLED:

.. code-block:: bash

  $ openstack optimize audit update 550e8400-e29b-41d4-a716-446655440000 replace state=CANCELLED


Then, delete the audit:

.. code-block:: bash

  $ openstack optimize audit delete 550e8400-e29b-41d4-a716-446655440000



Configuration Reference
========================

Continuous Audit Intervals
---------------------------

**Numeric Intervals (seconds):**

- Minimum recommended: 60 seconds
- Common values: 300 (5 min), 600 (10 min), 1800 (30 min), 3600 (1 hour)

**Cron Expressions (5 format fields):**

See the `POSIX crontab specification <https://pubs.opengroup.org/onlinepubs/9799919799/utilities/crontab.html>`_.

- ``0 * * * *``: Every hour at minute 0
- ``*/15 * * * *``: Every 15 minutes
- ``0 9-17 * * 1-5``: Every hour during business hours (9 AM - 5 PM, Mon-Fri)
- ``30 2 * * *``: Daily at 2:30 AM


Decision Engine Configuration
-----------------------------

The continuous audit polling interval is configured in ``watcher.conf``:

.. code-block:: ini

  [watcher_decision_engine]
  # Interval for checking continuous audits (seconds)
  continuous_audit_interval = 30

Spec Linked with Continuous Audit
=================================

- `Watcher Continuous Optimization <https://specs.openstack.org/openstack/watcher-specs/specs/newton/implemented/continuously-optimization.html>`_
- `Cron-based continuous audits <https://specs.openstack.org/openstack/watcher-specs/specs/pike/implemented/cron-based-continuous-audits.html>`_
- `Add the start and end time for CONTINUOUS audit <https://specs.openstack.org/openstack/watcher-specs/specs/stein/implemented/add-start-end-time-for-continuous-audit.html>`_

