Get started with streams

A stream stores ordered data points, typically time-series data. The list below provides a tour of the key features related to streams. After you create streams, you can search for them, analyze trends over time, organize them with metadata, and reshape returned data with stream views. Streams are stored in SDS, where types define data structure and indexes define ordering and query behavior.

For additional information about stream best practices, see Manage streams.

1. Sequential Data Store

   The Sequential Data Store (SDS) is a streaming database optimized for storing sequential data, typically time-series data. It can store any data that is indexed by an ordered sequence. For each namespace that is created, an SDS instance or SDS resources is created. Use SDS to store, retrieve, and analyze data.

2. Search queries

   From the Sequential Data Store, use the Search for Streams field to find streams that you want to work with. For more information on using the field and specifying search criteria, see Search queries.

3. Create a trend session

   Using a Trend session, you can look for pattens in your streams over time. Trend sessions display property values from your streams with a given time range. You can view multiple streams in a Trend session, and you can view all property data for each stream.

4. Add a type

   A type defines the shape and structure of events and how to associate events within a stream of data. A type is comprised of at least two properties. One property serves as the primary index, most commonly a timestamp or DateTime. In addition, it has one or more additional properties called value properties that describe the data in each stream event. Each value property can have a different property type. A wide variety of property types are supported.

   Additional documentation:

   • Types best practices

   • Learn more about types in the Developer Guide

5. Create a metadata rule

   When possible, you should explicitly include metadata when you create streams. However, when that is not possible, you can use metadata rules to leverage a consistent naming pattern for streams to embed metadata.

   Metadata, or data about data, is a collection of attributes that stream instances of a stream type are expected to provide. The type and units of measure for the value can be defined. Metadata enriches sequential data in CONNECT data services and it logically silos and contextualizes data. It supports data analysis, visualization, organization, and search capabilities.

   A metadata rule is a user-defined stream name pattern in which each part is assigned a metadata type. Metadata rules capture any streams currently stored in a namespace, as well as matching streams that are subsequently added to the namespace.

   Additional documentation:

   • Stream metadata rules

   • Learn more about metadata rules in the API Reference

6. Stream views

   A stream view is a logical overlay that enables you to customize your view of streaming data so it is most useful to you. While you cannot change the properties of types, stream views enable you to create a view of a stream, so it appears as if you had changed the type. You create a stream view by choosing a source and target type, and then defining mappings between the properties of the two types. The source type is the type associated with the stream. The target type includes the properties you want to include in the stream view. In effect, you can remove, rename, or add properties without altering the original stream type.