Connect Metadata Test Results

The operator certification pipeline, whether using the local test or hosted offering, performs a series of tests all of which must pass before you can successfully publish your operator.

Each of the tests performed by the certification pipeline is outlined below.

Operator Metadata Linting

In order to pass this test both of the commands below need to be run and yield no errors.

operator-sdk bundle validate ./bundle

Operator OLM Deployment

In order to pass this test, the operator must deploy successfully from the Operator Lifecycle Manager. Keep in mind that even when an operator is successfully deployed, there could still be non-fatal errors hidden in the operator's pod log, so prior testing on OpenShift is essential.

Operator Scorecard Tests

Operator-Scorecard-Results

===== Test: operator-scorecard-tests =====

 Basic Tests:
	Spec Block Exists: 1/1
	Status Block Exists: 1/1
	Writing into CRs has an effect: 0/1
OLM Tests:
	Provided APIs have validation: 0/0
	Owned CRDs have resources listed: 2/2
	CRs have at least 1 example: 1/1
	Spec fields with descriptors: 8/8
	Status fields with descriptors: 1/1

Total Score: 74%

In order to certify you must pass the first two Basic Tests: Spec Block Exists, and Status Block Exists. Passing the third basic test (Writing into CRs has an effect) requires adding the scorecard-proxy container to the operator deployment, which is not desired in a production operator and is therefore not required for certification.

If you're building either a Helm or Ansible operator using the SDK, then the CR Status is managed automatically, and there is no extra coding required. However, if you're building an operator in Go (using the SDK) or some other framework, you'll need to make certain that you're updating the Status field of the CR with relevant info regarding the running state of the application. See the Golang Gotchas section on Writing to the Status Subresource.

Passing the OLM Tests portion of the scorecard is not a requirement for certification, as currently the CR Spec and Status Descriptor fields don't populate the OpenShift UI.

The scorecard does not test the operator's functionality besides ensuring that it updates the CR status field. You are required to fully test your operator in OpenShift 4.

Operator Image Source

You must have a published operator image source in order to publish your operator. The base image must be RHEL 7 or UBI. Refer to this blog and the FAQ for more information on the Universal Base Image (UBI).

An example showing an operator that has passed all tests, and is ready to be published