Visibility

This guide provides a comprehensive overview of Temporal Visibility.

The term Visibility, within the Temporal Platform, refers to the subsystems and APIs that enable an operator to view Workflow Executions that currently exist within a Cluster.

Standard Visibility

Standard Visibility, within the Temporal Platform, is the subsystem and APIs that list Workflow Executions by a predefined set of filters.

Open Workflow Executions can be filtered by a time constraint and either a Workflow Type, Workflow Id, or Run Id.

Closed Workflow Executions can be filtered by a time constraint and either a Workflow Type, Workflow Id, Run Id, or Execution Status (Completed, Failed, Timed Out, Terminated, Canceled, or Continued-As-New).

Advanced Visibility

Advanced Visibility, within the Temporal Platform, is the subsystem and APIs that enable the listing, filtering, and sorting of Workflow Executions through a custom SQL-like List FilterWhat is a List Filter?

A List Filter is the SQL-like string that is provided as the parameter to an Advanced Visibility List API.

Learn more.

To use Advanced Visibility, your Temporal Cluster must be integrated with ElasticsearchHow to integrate Elasticsearch into a Temporal Cluster

To integrate Elasticsearch with your Temporal Cluster, edit the persistence section of your development.yaml configuration file and run the index schema setup commands.

Learn more. We highly recommend operating a Temporal Cluster with Elasticsearch for any use case that spawns more than just a few Workflow Executions. Elasticsearch takes on the Visibility request load, relieving potential performance issues.

List Filter

A List Filter is the SQL-like string that is provided as the parameter to an Advanced VisibilityWhat is Advanced Visibility?

Advanced Visibility, within the Temporal Platform, is the subsystem and APIs that enable the listing, filtering, and sorting of Workflow Executions through an SQL-like query syntax.

Learn more List API. List Filter Search AttributeWhat is a Search Attribute?

A Search Attribute is an indexed name used in List Filters to filter a list of Workflow Executions that have the Search Attribute in their metadata.

Learn more names are case sensitive, and each List Filter is scoped by a single NamespaceWhat is a Namespace?

A Namespace is a unit of isolation within the Temporal Platform

Learn more.

A List Filter that uses a time range has a resolution of 1 ns on Elasticsearch 7+ and 1 µs for SQL.

Supported operators

A List Filter contains Search AttributeWhat is a Search Attribute?

A Search Attribute is an indexed name used in List Filters to filter a list of Workflow Executions that have the Search Attribute in their metadata.

Learn more names, Search Attribute values, and the following supported operators:

• =, !=, >, >=, <, <=
• AND, OR, ()
• BETWEEN ... AND
• IN
• ORDER BY

The ORDER BY operator is supported only when Elasticsearch is used as the Visibility store. Additionally, custom Search Attributes of the Text type cannot be used in ORDER BY clauses.

Partial string match

The = operator works like CONTAINS to find Workflows with Search Attributes that contain a specific word.

For example, if you have a Search Attribute Description of Text type with the value of "The quick brown fox jumps over the lazy dog", searching for Description=quick or Description=fox will successfully return the Workflow. However, partial word searches such as Description=qui or Description=laz will not return the Workflow. This is because Elasticsearch's tokenizer is configured to return complete words as tokens.

Efficient API usage

An Advanced List Filter API may take longer to respond if it is retrieving a large number of Workflow Executions (over 10,000).

Paginate the results with the ListWorkflow API by using the page token to retrieve the next page; continue until the page token is null/nil.

List Filter examples

The following is a List Filter set with tctltctl workflow list

How to list open or closed Workflow Executions using tctl.

Learn more:

WorkflowType = "main.YourWorkflowDefinition" and ExecutionStatus != "Running" and (StartTime > "2021-06-07T16:46:34.236-08:00" or CloseTime > "2021-06-07T16:46:34-08:00")

When used, a list of Workflows that meet the following conditions are returned:

• The Workflow Type is set to main.YourWorkflowDefinition.
• The Workflow isn't running.
• The Workflow either started after "2021-06-07T16:46:34.236-08:00" or closed after "2021-06-07T16:46:34-08:00".

More List Filter examples have been provided below.

WorkflowId = '<workflow-id>'

WorkflowId = '<workflow-id>' or WorkflowId = '<another-workflow-id>'

WorkflowId IN ('<workflow-id>', '<another-workflow-id>')

WorkflowId = '<workflow-id>' order by StartTime desc

WorkflowId = '<workflow-id>' and ExecutionStatus = 'Running'

WorkflowId = '<workflow-id>' or ExecutionStatus = 'Running'

WorkflowId = '<workflow-id>' and StartTime > '2021-08-22T15:04:05+00:00'

ExecutionTime between '2021-08-22T15:04:05+00:00' and '2021-08-28T15:04:05+00:00'

ExecutionTime < '2021-08-28T15:04:05+00:00' or ExecutionTime > '2021-08-22T15:04:05+00:00'

order by ExecutionTime

order by StartTime desc, CloseTime asc

order by CustomIntField asc

Search Attribute

A Search Attribute is an indexed field used in a List FilterWhat is a List Filter?

A List Filter is the SQL-like string that is provided as the parameter to an Advanced Visibility List API.

Learn more to filter a list of Workflow Executions that have the Search Attribute in their metadata.

If a Temporal ClusterWhat is a Temporal Cluster?

A Temporal Cluster is the Temporal Server paired with persistence.

Learn more does not have Elasticsearch integratedHow to integrate Elasticsearch into a Temporal Cluster

To integrate Elasticsearch with your Temporal Cluster, edit the persistence section of your development.yaml configuration file and run the index schema setup commands.

Learn more, but a Workflow Execution is spawned and tagged with Search Attributes, no errors occur. However, you won't be able to use Advanced VisibilityWhat is Advanced Visibility?

Advanced Visibility, within the Temporal Platform, is the subsystem and APIs that enable the listing, filtering, and sorting of Workflow Executions through an SQL-like query syntax.

Learn more List APIs and List Filters to find and list the Workflow Execution.

When using Continue-As-NewWhat is Continue-As-New?

Continue-As-New is the mechanism by which all relevant state is passed to a new Workflow Execution with a fresh Event History.

Learn more or a Temporal Cron JobWhat is a Temporal Cron Job?

A Temporal Cron Job is the series of Workflow Executions that occur when a Cron Schedule is provided in the call to spawn a Workflow Execution.

Learn more, Search Attributes are carried over to the new Workflow Run by default.

Search Attributes maximums

Default total maximum number of Search Attribute keys per Temporal Cluster is 100.

Default single Search Attribute value size limit is 2 KB.

Total Search Attribute size: 40 KB

Default Search Attributes

A Temporal Cluster that is integrated with Elasticsearch has a set of default Search Attributes already available. These Search Attributes are created when the initial index is created.

NAME	TYPE	DEFINITION
WorkflowType	Keyword	The type of Workflow.
WorkflowId	Keyword	Identifies the Workflow Execution.
ExecutionStatus	Keyword	The current state of the Workflow Execution.
StartTime	Datetime	The time at which the Workflow Execution started.
CloseTime	Datetime	The time at which the Workflow Execution completed.
ExecutionTime	Datetime	Same as StartTime for the most cases but different for cron Workflows and retried Workflows. For them it is the time at which the Workflow Execution actually begin running.
RunId	Keyword	Identifies the current Workflow Execution Run.
ExecutionDuration	Int	The time needed to run the Workflow Execution. Available only for closed Workflows.
HistoryLength	Int	The number of events in the history of Workflow Execution. Available only for closed Workflows.
StateTransitionCount	Int	The number of times that Workflow Execution has persisted its state. Available only for closed Workflows.
TaskQueue	Keyword	Task Queue used by Workflow Execution.
TemporalChangeVersion	Keyword	If Workflow versioning is enabled, list of change/version pairs will be stored here.
BinaryChecksums	Keyword	List of binary Ids of Workers that run the Workflow Execution.
BatcherNamespace	Keyword	Used by internal batcher to indicate the Namespace where batch operation was applied to.
BatcherUser	Keyword	Used by internal batcher to indicate the user who started the batch operation.

• All default Search Attributes are reserved and read-only. (You cannot create a custom one with the same name or alter the existing one.)

• ExecutionStatus values correspond to Workflow Execution Statuses: Running, Completed, Failed, Canceled, Terminated, ContinuedAsNew, TimedOut.

• StartTime, CloseTime, and ExecutionTime are stored as dates but are supported by queries that use either EpochTime in nanoseconds or a string in RFC3339Nano format (such as "2006-01-02T15:04:05.999999999Z07:00").

• ExecutionDuration is stored in nanoseconds but is supported by queries that use integers in nanoseconds, Golang duration format, or "hh:mm:ss" format.

• CloseTime, HistoryLength, StateTransitionCount, and ExecutionDuration are present only in a Closed Workflow Execution.

• ExecutionTime can differ from StartTime in retry and cron use cases.

Custom Search Attributes

Custom Search Attributes can be added to a Temporal Cluster by using tctl search-attribute create. Adding a Search Attribute makes it available to use with Workflow Executions within that Cluster.

There is no hard limit on the number of attributes you can add. However, we recommend enforcing the following limits:

• Number of Search Attributes: 100 per Workflow
• Size of each value: 2 KB per value
• Total size of names and values: 40 KB per Workflow

note

Due to Elasticsearch limitations, you can only add Search Attributes. It is not possible to rename Search Attributes or remove them from the index schema.

The temporalio/auto-setup Docker image uses a pre-defined set of custom Search Attributes that are handy for testing. Their names indicate their types:

• CustomBoolField
• CustomDatetimeField
• CustomDoubleField
• CustomIntField
• CustomKeywordField
• CustomTextField

Types

Search Attributes must be one of the following types:

• Bool
• Datetime
• Double
• Int
• Keyword
• Text

Note:

• Double is backed up by scaled_float Elasticsearch type with scale factor 10000 (4 decimal digits).

• Datetime is backed up by date type with milliseconds precision in Elasticsearch 6 and date_nanos type with nanoseconds precision in Elasticsearch 7.

• Int is 64-bit integer (long Elasticsearch type).

• Keyword and Text types are concepts taken from Elasticsearch. Each word in a Text is considered a searchable keyword. For a UUID, that can be problematic because Elasticsearch indexes each portion of the UUID separately. To have the whole string considered as a searchable keyword, use the Keyword type. For example, if the key ProductId has the value of 2dd29ab7-2dd8-4668-83e0-89cae261cfb1:

  • As a Keyword it would be matched only by ProductId = "2dd29ab7-2dd8-4668-83e0-89cae261cfb1.
  • As a Text it would be matched by ProductId = 2dd8, which could cause unwanted matches.

• The Text type cannot be used in the "Order By" clause.

• How to view Search Attributes using tctl

Search Attributes as Workflow Execution metadata

To actually have results from the use of a List FilterWhat is a List Filter?

A List Filter is the SQL-like string that is provided as the parameter to an Advanced Visibility List API.

Learn more, Search Attributes must be added to a Workflow Execution as metadata. How to do this entirely depends on the method by which you spawn the Workflow Execution:

• How to set Search Attributes as Workflow Execution metadataHow to set custom Search Attributes
  
  After you've created custom Search Attributes in your Cluster (using `tctl` or the Cloud UI), you can set the values of the custom Search Attributes when starting a Workflow.
  
  Learn more