Type Classes¶

In Substrait, the “class” of a type, not to be confused with the concept from object-oriented programming, defines the set of non-null values that instances of a type may assume.

Implementations of a Substrait type must support at least this set of values, but may include more; for example, an i8 could be represented using the same in-memory format as an i32, as long as functions operating on i8 values within [-128..127] behave as specified (in this case, this means 8-bit overflow must work as expected). Operating on values outside the specified range is unspecified behavior.

Simple Types¶

Simple type classes are those that don’t support any form of configuration. For simplicity, any generic type that has only a small number of discrete implementations is declared directly, as opposed to via configuration.

Compound Types¶

Compound type classes are type classes that need to be configured by means of a parameter pack.

User-Defined Types¶

User-defined type classes are defined as part of simple extensions. An extension can declare an arbitrary number of user-defined extension types. Once a type has been declared, it can be used in function declarations.

For example, the following declares a type named point (namespaced to the associated YAML file) and two scalar functions that operate on it.

types:
  - name: "point"

scalar_functions:
  - name: "lat"
    impls:
      - args:
        - name: p
        - value: u!point
    return: fp64
  - name: "lon"
    impls:
      - args:
        - name: p
        - value: u!point
    return: fp64

Handling User-Defined Types¶

Systems without support for a specific user-defined type: * Cannot generate values of the type. * Cannot implement functions operating on the type. * May support consuming and emitting values of the type without modifying them.

Communicating User-Defined Types¶

Specifiers of user-defined types may provide additional structure information for the type to assist in communicating values of the type to and from systems without built-in support.

For example, the following declares a point type with two i32 values named longitude and latitude:

types:
  - name: point
    structure:
      longitude: i32
      latitude: i32

The name-type object notation used above is syntactic sugar for NSTRUCT<longitude: i32, latitude: i32>. The following means the same thing:

name: point
structure: "NSTRUCT<longitude: i32, latitude: i32>"

The structure field of a type is only intended to inform systems that don’t have built-in support for the type about how they can create and transfer values of that type to systems that do support the type.

The structure field does not restrict or bind the internal representation of the type in any system.

As such, it’s currently not possible to “unpack” a user-defined type into its structure type or components thereof using FieldReferences or any other specialized record expression; if support for this is desired for a particular type, this can be accomplished with an extension function.

Literals¶

Literals for user-defined types can be represented in one of two ways: * Using protobuf Any messages. * Using the structure representation of the type.

Compound User-Defined Types¶

User-defined types may be turned into compound types by requiring parameters to be passed to them. The supported “meta-types” for parameters are data types (like those used in LIST, MAP, and STRUCT), booleans, integers, enumerations, and strings. Using parameters, we could redefine “point” with different types of coordinates. For example:

name: point
parameters:
  - name: T
    description: |
      The type used for the longitude and latitude
      components of the point.
    type: dataType

or:

name: point
parameters:
  - name: coordinate_type
    type: enumeration
    options:
      - integer
      - double

or:

name: point
parameters:
  - name: LONG
    type: dataType
  - name: LAT
    type: dataType

We can’t specify the internal structure in this case, because there is currently no support for derived types in the structure.

The allowed range can be limited for integer parameters. For example:

name: vector
parameters:
  - name: T
    type: dataType
  - name: dimensions
    type: integer
    min: 2
    max: 3

This specifies a vector that can be either 2- or 3-dimensional. Note however that it’s not currently possible to put constraints on data type, string, or (technically) boolean parameters.

Similar to function arguments, the last parameter may be specified to be variadic, allowing it to be specified one or more times instead of only once. For example:

name: union
parameters:
  - name: T
    type: dataType
variadic: true

This defines a type that can be parameterized with one or more other data types, for example union<i32, i64> but also union<bool>. Zero or more is also possible, by making the last argument optional:

name: tuple
parameters:
  - name: T
    type: dataType
    optional: true
variadic: true

This would also allow for tuple<>, to define a zero-tuple.