Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 25 additions & 0 deletions content/en/architecture/ci-operator.md
Original file line number Diff line number Diff line change
Expand Up @@ -774,6 +774,9 @@ There are few extra fields that can be configured to control if or when the test
* `skip_if_only_changed` Set a regex to skip triggering the job when all the changes in the pull request match (see the documentation link above).
* `always_run` Set to `false` to disable automatic triggering on every PR. This deaults to `true` (run on every PR) unless `run_if_changed` or `skip_if_only_changed` is set.
* `optional` Set to `true` to make the job not block merges.
* `max_concurrency` Set the maximum number of concurrent runs of this job.
* `disable_rehearsal` Set to `true` to prevent this test from being picked up for rehearsals.
* `slack_reporter` Configure [Slack reporting](/how-tos/notification/) for this test with `channel`, `job_states_to_report`, and `report_template`.

**Note:** `run_if_changed`, `skip_if_only_changed`, and `always_run: true` are mutually exclusive.

Expand Down Expand Up @@ -829,6 +832,28 @@ tests:
Note that the build farms used to execute jobs run on UTC time, so time-of-day based `cron` schedules must be set with
that in mind.

## Prowgen Configuration

The `prowgen` stanza in `ci-operator` configuration controls Prow job generation behavior. These fields replace the
deprecated `.config.prowgen` file.

{{< highlight yaml >}}
prowgen:
private: true # mark jobs as hidden and mount git credentials for private repos
expose: true # make private jobs visible in Deck
disable_rehearsals: true # prevent all tests in this config from being rehearsed
enable_secrets_store_csi_driver: true # use CSI Secrets Store for multi-stage credentials
{{< / highlight >}}

The `ci-operator` config fields take precedence over `.config.prowgen` when both are set. If a field is omitted in the
`ci-operator` config, the `.config.prowgen` value is used as a fallback.

**Note:** Repositories in the `openshift-priv` organization are automatically treated as `private: true`.

{{< alert title="Note" color="info" >}}
The `.config.prowgen` file is deprecated. All configuration should be migrated to the `prowgen` stanza in the `ci-operator` configuration file. The `.config.prowgen` fallback is still supported but will be removed in the future.
{{< /alert >}}

## Referencing Images

As `ci-operator` is OpenShift-native, all `images` used in a test workflow are stored as `ImageStreamTags`. The following
Expand Down
13 changes: 8 additions & 5 deletions content/en/architecture/private-repositories.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,15 +14,18 @@ At that time, a Test Platform member will assist with creating the proper config
2. When not configured to use a private `deck` instance, the logs and artifacts executed for the private repository
will be publicly available on the `deck`. **Only set up such jobs when you are absolutely sure your jobs would not leak any sensitive information.**

To allow the CI jobs to access a private repo, drop the following file to the directory in `openshift/release` holding the
`ci-operator` configuration for your repository (usually `ci-operator/config/$org/$repo`):
To allow the CI jobs to access a private repo, add the `prowgen` stanza to your `ci-operator` configuration file:

`.config.prowgen`
{{< highlight yaml >}}
private: true # allows the source to be cloned from the private repo, without setting this: jobs will not succeed
expose: true # allows the jobs to be displayed on the `deck` instance in which they are configured
prowgen:
private: true # allows the source to be cloned from the private repo, without setting this: jobs will not succeed
expose: true # allows the jobs to be displayed on the `deck` instance in which they are configured
{{< / highlight >}}

{{< alert title="Note" color="info" >}}
The `.config.prowgen` file is deprecated. All prowgen configuration should be moved into the `ci-operator` configuration file under the `prowgen` stanza.
{{< /alert >}}

## `openshift-priv` organization

The `openshift-priv` organization holds private forks of selected repositories. The purpose of these forks is to allow
Expand Down
25 changes: 16 additions & 9 deletions content/en/how-tos/contributing-openshift-release.md
Original file line number Diff line number Diff line change
Expand Up @@ -135,25 +135,32 @@ Handcrafted jobs can opt to be rehearsable by including this label.
All presubmits and periodics generated by `make jobs` have this label by default. Generated postsubmits will not contain
it because generated postsubmits are used for promoting `images`.

In order to opt-out of rehearsals for generated presubmits and periodics, prowgen must be configured to disable rehearsals.
To do this your org or repo's `.config.prowgen` file can be configured to either disable **all** rehearsals:
In order to opt-out of rehearsals for generated presubmits and periodics, rehearsals can be disabled in the `ci-operator`
configuration file. To disable rehearsals for **all** tests in a configuration, use the `prowgen` stanza:

{{< highlight yaml >}}
rehearsals:
disable_all: true
prowgen:
disable_rehearsals: true
{{< / highlight >}}

Or to disable them for specific tests:
To disable rehearsals for a **specific** test, use the `disable_rehearsal` field on the test:

{{< highlight yaml >}}
rehearsals:
disabled_rehearsals:
- unit # only disable rehearsals on the job named "unit"
tests:
- as: "unit"
commands: "make test"
container:
from: "bin"
disable_rehearsal: true
{{< / highlight >}}

Either of the preceding configurations will result in the `pj-rehearse.openshift.io/can-be-rehearsed: "true"` label not
Either of the preceding configurations will result in the `pj-rehearse.openshift.io/can-be-rehearsed: "true"` label not
being added to the affected jobs when running `make jobs`.

{{< alert title="Note" color="info" >}}
The `.config.prowgen` file `rehearsals` section is deprecated. The `rehearsals.disable_all` and `rehearsals.disabled_rehearsals` fields in `.config.prowgen` are still supported as a fallback, but new configuration should use the `ci-operator` config fields above.
{{< /alert >}}

{{< alert title="Warning" color="warning" >}}
Jobs that are marked as `hidden: true` will never be rehearsed, regardless of the presence of the `pj-rehearse.openshift.io/can-be-rehearsed: "true"` label.
{{< /alert >}}
Expand Down
121 changes: 45 additions & 76 deletions content/en/how-tos/notification.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,101 +3,70 @@ title: "Set Up Slack Alerts for Job Results"
description: How to set up alerting to Slack for a CI job.
---

We can set up Slack alerts for job results by defining `reporter_config` in the job definition. This is possible for all job types: `periodics`, `presubmits`, and `postsubmits`.
`Prowgen` is [capable](https://docs.ci.openshift.org/how-tos/notification/#setting-up-slack-notifications-with-prowgen) of adding this to the job definition.
We can set up Slack alerts for job results by configuring the `slack_reporter` field on individual tests in the `ci-operator` configuration file. This is possible for all job types: `periodics`, `presubmits`, and `postsubmits`.

{{< alert title="Warning" color="warning" >}}
Currently, it is still possible to manually add the `reporter_config` to periodic jobs. This functionality is deprecated, and using `Prowgen` is preferred.
Once a significant number of jobs have migrated to use `Prowgen`, the ability to manually edit the periodics will be removed.
{{< /alert >}}
## Setting up Slack notifications

To add Slack reporting for a test, add the `slack_reporter` field to the test in your `ci-operator` configuration:

```yaml
reporter_config:
slack:
channel: '#forum'
tests:
- as: "unit"
commands: "make test-unit"
container:
from: "bin"
slack_reporter:
channel: "#my-channel"
job_states_to_report:
- failure
report_template: Job {{.Spec.Job}} failed.
- error
report_template: 'Job {{.Spec.Job}} ended with {{.Status.State}}. <{{.Status.URL}}|View logs>'
```

For example, by the above snippet, a Slack alert will be sent out to `#forum` channel when there is a failure of the job. The alert is formatted by `report_template`.
### Configuration fields

* The channel has to be in [coreos.slack.com](https://coreos.slack.com/).
* The channel has to be public. If it is not then the `@prow` bot has to be added to it otherwise it won't be able to properly post messages.
* The state in `job_states_to_report` has to be a valid Prow job state. See [upstream documentation](https://pkg.go.dev/sigs.k8s.io/prow/pkg/apis/prowjobs/v1#ProwJobState).
* The value of `report_template` is a [Go template](https://golang.org/pkg/text/template/) and it will be applied to the Prow job instance. The annotations such as `{{.Spec.Job}}` will be replaced by the job name when the alert is received in Slack. See [upstream documentation](https://pkg.go.dev/sigs.k8s.io/prow/pkg/apis/prowjobs/v1#ProwJob) for more fields of a Prow job. Note that no alerts will be sent out if the template is broken, e.g., cannot be parsed or applied successfully.
* `channel` (**required**) The Slack channel to report to (e.g., `#my-channel`).
* `job_states_to_report` (optional) Which job states trigger a report. Defaults to `["failure", "error"]` if not set. Must be valid [Prow job states](https://pkg.go.dev/sigs.k8s.io/prow/pkg/apis/prowjobs/v1#ProwJobState).
* `report_template` (optional) A [Go template](https://golang.org/pkg/text/template/) for the Slack message. Applied to the [ProwJob](https://pkg.go.dev/sigs.k8s.io/prow/pkg/apis/prowjobs/v1#ProwJob) instance. If not set, a default template is used. No alerts will be sent if the template cannot be parsed or applied.

## Setting up Slack notifications with Prowgen
`Prowgen` is the tool that is used to generate `ProwJobs` from `ci-operator` configuration. It is possible to instruct `Prowgen` to add a `reporter_config` to specific jobs.
Doing so requires creating or updating the repo (or organization) in question's `.config.prowgen` file. This YAML configuration file is stored in the repo's `ci-operator` folder.
For example, `ci-tools` configuration file is found at `ci-operator/config/openshift/ci-tools/.config.prowgen`. The following example illustrates how to utilize this config to add slack notifications for jobs:

```yaml
slack_reporter:
- channel: "#slack-channel"
job_states_to_report: #Accepts any ProwJob status
- success
- failure
- error
report_template: '{{if eq .Status.State "success"}} :rainbow: Job *{{.Spec.Job}}*
ended with *{{.Status.State}}*. <{{.Status.URL}}|View logs> :rainbow: {{else}}
:volcano: Job *{{.Spec.Job}}* ended with *{{.Status.State}}*. <{{.Status.URL}}|View
logs> :volcano: {{end}}'
job_names: # Listing of job names (ci-operator's 'as' field) that this configuration applies to
- unit
- upload-results
- lint
excluded_variants: # It is possible to exclude notifications from specific variants, for example, when you don't require them from older releases
- some-old-release
```
### Requirements

{{< alert title="Note" color="info" >}}
Note that it is possible to include multiple `slack_reporter` entries in this config, but each `job_name` should only be included in, at most, one.
{{< /alert >}}
* The channel must be on [coreos.slack.com](https://coreos.slack.com/).
* The channel must be public, or the `@prow` bot must be added to it.

{{< alert title="Note" color="info" >}}
It is possible to add slack reporting for the `images` postsubmit job by adding the "images" entry to `job_names`.
{{< /alert >}}
### Example with defaults

Once the configuration is added, simply use the `make jobs` target to generate the new job definitions containing the `reporter_config`.
When only `channel` is provided, defaults are applied for `job_states_to_report` and `report_template`:

### Slack Reporter Configuration
```yaml
tests:
- as: "e2e"
commands: "make test-e2e"
container:
from: "bin"
slack_reporter:
channel: "#my-team"
```

The `slack_reporter` configuration supports flexible job matching with multiple approaches:
This will report on `failure` and `error` states using a default template.

1. **Exact matching** with `job_names`: Matches job names exactly
2. **Pattern matching** with `job_name_patterns`: Matches job names using regular expressions
3. **Exclusion** with `excluded_job_patterns`: Excludes jobs matching regex patterns (similar to `excluded_variants`)
## Deprecated: `.config.prowgen` Slack configuration

**Matching Priority:**
- Exclusions are checked first - if a job matches any `excluded_job_patterns`, it's excluded regardless of other rules
- Exact matches in `job_names` take precedence over pattern matches in `job_name_patterns`
- This allows for more flexible job matching while maintaining compatibility with existing configurations
{{< alert title="Warning" color="warning" >}}
The `.config.prowgen` file `slack_reporter` section is deprecated. Slack reporting should be configured directly on each test using the `slack_reporter` field as shown above. The `.config.prowgen` fallback is still supported but will be removed in the future.
{{< /alert >}}

**Examples:**
The legacy `.config.prowgen` approach used a separate file (`ci-operator/config/$org/$repo/.config.prowgen`) with pattern-based job matching:

```yaml
slack_reporter:
# Example 1: Mixed exact and pattern matching
- channel: "#my-team"
job_names: ["unit", "images"] # Exact matches
job_name_patterns: ["^e2e-.*"] # Regex patterns

# Example 2: Broad matching with exclusions (similar to excluded_variants)
- channel: "#ops-team"
job_name_patterns: [".*"] # Match all jobs
excluded_job_patterns: # But exclude these patterns
- ".*-skip$" # Jobs ending with "-skip"
- "^nightly-.*" # Jobs starting with "nightly-"
- ".*-flaky-.*" # Jobs containing "-flaky-"

# Example 3: Combining all approaches
- channel: "#dev-team"
job_names: ["critical-test"] # Always include this specific job
job_name_patterns: ["^unit-.*"] # Include all unit tests
excluded_job_patterns: [".*-slow$"] # But exclude slow tests
- channel: "#slack-channel"
job_states_to_report:
- failure
- error
job_names:
- unit
- e2e
```

The `excluded_job_patterns` approach is often simpler than complex inclusion patterns, especially when you want to match most jobs in a repository but exclude specific types.

The `SlackReporterConfig` is provided for [reference](https://github.com/openshift/ci-tools/blob/6810ce942bbe25a06c092af8098fd2d071604a04/pkg/config/load.go#L49-L57).
This is now replaced by the per-test `slack_reporter` field in the `ci-operator` configuration, which is simpler and keeps all configuration in one place.