Spark Connect for Scala 3
API
Generated with
Spark Connect for Scala 3/org.apache.spark/org.apache.spark.sql/TypedColumn

TypedColumn

org.apache.spark.sql.TypedColumn
class TypedColumn[-T, U] extends Column

A Column with an Encoder for its values, produced by Column.as[U]. Passing typed columns to Dataset.select returns a typed Dataset[U] (or a tuple Dataset). Mirrors org.apache.spark.sql.TypedColumn.

Attributes

Graph
Supertypes
class Column
class Object
trait Matchable
class Any

Members list

Value members

Concrete methods

override def name(alias: String): TypedColumn[T, U]

Gives the typed column a name (alias), keeping its encoder.

Gives the typed column a name (alias), keeping its encoder.

Attributes

Definition Classes

Inherited methods

def %(other: Any): Column

Attributes

Inherited from:
Column
def &&(other: Any): Column

Attributes

Inherited from:
Column
def *(other: Any): Column

Attributes

Inherited from:
Column
def +(other: Any): Column

Attributes

Inherited from:
Column
def -(other: Any): Column

Attributes

Inherited from:
Column
def /(other: Any): Column

Attributes

Inherited from:
Column
def <(other: Any): Column

Attributes

Inherited from:
Column
def <=(other: Any): Column

Attributes

Inherited from:
Column
def <=>(other: Any): Column

Attributes

Inherited from:
Column
def =!=(other: Any): Column

Attributes

Inherited from:
Column
def ===(other: Any): Column

Attributes

Inherited from:
Column
def >(other: Any): Column

Attributes

Inherited from:
Column
def >=(other: Any): Column

Attributes

Inherited from:
Column
def alias(alias: String): Column

Attributes

Inherited from:
Column
def and(other: Column): Column

Attributes

Inherited from:
Column
def apply(extraction: Any): Column

Extract an array element by index, a map value by key, or a struct field.

Extract an array element by index, a map value by key, or a struct field.

Attributes

Inherited from:
Column
def as[U](using enc: Encoder[U]): TypedColumn[Any, U]

Gives this column an Encoder, producing a TypedColumn usable with the typed Dataset.select overloads.

Gives this column an Encoder, producing a TypedColumn usable with the typed Dataset.select overloads.

Attributes

Inherited from:
Column
def as(alias: String, metadata: Metadata): Column

Attributes

Inherited from:
Column
def as(alias: Symbol): Column

Attributes

Inherited from:
Column
def as(aliases: Array[String]): Column

Attributes

Inherited from:
Column
def as(aliases: Seq[String]): Column

Attributes

Inherited from:
Column
def as(alias: String): Column

Attributes

Inherited from:
Column
def asc: Column

Attributes

Inherited from:
Column
def asc_nulls_first: Column

Attributes

Inherited from:
Column
def asc_nulls_last: Column

Attributes

Inherited from:
Column
def between(lowerBound: Any, upperBound: Any): Column

True if lowerBound <= this <= upperBound.

True if lowerBound <= this <= upperBound.

Attributes

Inherited from:
Column
def bitwiseAND(other: Any): Column

Attributes

Inherited from:
Column
def bitwiseOR(other: Any): Column

Attributes

Inherited from:
Column
def bitwiseXOR(other: Any): Column

Attributes

Inherited from:
Column
def cast(to: String): Column

Attributes

Inherited from:
Column
def cast(to: DataType): Column

Attributes

Inherited from:
Column
def contains(other: Any): Column

Attributes

Inherited from:
Column
def desc: Column

Attributes

Inherited from:
Column
def desc_nulls_first: Column

Attributes

Inherited from:
Column
def desc_nulls_last: Column

Attributes

Inherited from:
Column
def divide(other: Any): Column

Attributes

Inherited from:
Column
def dropFields(fieldNames: String*): Column

Drops fields of a struct column by name; a no-op for fields that do not exist.

Drops fields of a struct column by name; a no-op for fields that do not exist.

Attributes

Inherited from:
Column
def endsWith(literal: String): Column

Attributes

Inherited from:
Column
def endsWith(other: Column): Column

Attributes

Inherited from:
Column
def eqNullSafe(other: Any): Column

Attributes

Inherited from:
Column
def equalTo(other: Any): Column

Attributes

Inherited from:
Column
override def equals(obj: Any): Boolean

Compares the receiver object (this) with the argument object (that) for equivalence.

Compares the receiver object (this) with the argument object (that) for equivalence.

Any implementation of this method should be an equivalence relation:

  • It is reflexive: for any instance x of type Any, x.equals(x) should return true.
  • It is symmetric: for any instances x and y of type Any, x.equals(y) should return true if and only if y.equals(x) returns true.
  • It is transitive: for any instances x, y, and z of type Any if x.equals(y) returns true and y.equals(z) returns true, then x.equals(z) should return true.

If you override this method, you should verify that your implementation remains an equivalence relation. Additionally, when overriding this method it is usually necessary to override hashCode to ensure that objects which are "equal" (o1.equals(o2) returns true) hash to the same scala.Int. (o1.hashCode.equals(o2.hashCode)).

Value parameters

that

the object to compare against this object for equality.

Attributes

Returns

true if the receiver object is equivalent to the argument; false otherwise.

Definition Classes
Column -> Any
Inherited from:
Column
def explain(extended: Boolean): Unit

Prints the expression to the console: the SQL-like form, or the full protobuf if extended.

Prints the expression to the console: the SQL-like form, or the full protobuf if extended.

Attributes

Inherited from:
Column
def geq(other: Any): Column

Attributes

Inherited from:
Column
def getField(fieldName: String): Column

Attributes

Inherited from:
Column
def getItem(key: Any): Column

Attributes

Inherited from:
Column
def gt(other: Any): Column

Attributes

Inherited from:
Column
override def hashCode(): Int

Calculates a hash code value for the object.

Calculates a hash code value for the object.

The default hashing algorithm is platform dependent.

Note that it is allowed for two objects to have identical hash codes (o1.hashCode.equals(o2.hashCode)) yet not be equal (o1.equals(o2) returns false). A degenerate implementation could always return 0. However, it is required that if two objects are equal (o1.equals(o2) returns true) that they have identical hash codes (o1.hashCode.equals(o2.hashCode)). Therefore, when overriding this method, be sure to verify that the behavior is consistent with the equals method.

Attributes

Returns

the hash code value for this object.

Definition Classes
Column -> Any
Inherited from:
Column
def ilike(literal: String): Column

Attributes

Inherited from:
Column
def isInCollection(values: Iterable[_]): Column

(Java-specific) True if the column's value is in the given collection.

(Java-specific) True if the column's value is in the given collection.

Attributes

Inherited from:
Column
def isInCollection(values: Iterable[_]): Column

True if the column's value is in the given collection.

True if the column's value is in the given collection.

Attributes

Inherited from:
Column
def isNaN: Column

Attributes

Inherited from:
Column
def isNotNull: Column

Attributes

Inherited from:
Column
def isNull: Column

Attributes

Inherited from:
Column
def isin(ds: Dataset[_]): Column

True if the column's value is in the result of the single-column (or, for a struct(...) column, multi-column) subquery ds. Builds an IN (subquery) expression, which needs a Spark 4.1+ server.

True if the column's value is in the result of the single-column (or, for a struct(...) column, multi-column) subquery ds. Builds an IN (subquery) expression, which needs a Spark 4.1+ server.

Attributes

Inherited from:
Column
def isin(list: Any*): Column

True if the column's value is in the given list.

True if the column's value is in the given list.

Attributes

Inherited from:
Column
def leq(other: Any): Column

Attributes

Inherited from:
Column
def like(literal: String): Column

Attributes

Inherited from:
Column
def lt(other: Any): Column

Attributes

Inherited from:
Column
def minus(other: Any): Column

Attributes

Inherited from:
Column
def mod(other: Any): Column

Attributes

Inherited from:
Column
def multiply(other: Any): Column

Attributes

Inherited from:
Column
def notEqual(other: Any): Column

Attributes

Inherited from:
Column
def or(other: Column): Column

Attributes

Inherited from:
Column
def otherwise(value: Any): Column

Provides the default (ELSE) value for a CASE expression.

Provides the default (ELSE) value for a CASE expression.

Attributes

Inherited from:
Column
def outer(): Column

Marks this column as an outer reference for a correlated subquery or a lateral join's correlated right side. Spark Connect resolves outer references on the server, so (as in the Apache Spark Connect client) this does not change the expression sent to the server.

Marks this column as an outer reference for a correlated subquery or a lateral join's correlated right side. Spark Connect resolves outer references on the server, so (as in the Apache Spark Connect client) this does not change the expression sent to the server.

Attributes

Inherited from:
Column
def over(): Column

Attributes

Inherited from:
Column
def over(window: WindowSpec): Column

Attributes

Inherited from:
Column
def plus(other: Any): Column

Attributes

Inherited from:
Column
def rlike(literal: String): Column

Attributes

Inherited from:
Column
def startsWith(literal: String): Column

Attributes

Inherited from:
Column
def startsWith(other: Column): Column

Attributes

Inherited from:
Column
def substr(startPos: Int, len: Int): Column

Attributes

Inherited from:
Column
def substr(startPos: Column, len: Column): Column

Attributes

Inherited from:
Column
override def toString: String

A SQL-like rendering of this column, e.g. (a + 1) AS b.

A SQL-like rendering of this column, e.g. (a + 1) AS b.

Attributes

Definition Classes
Column -> Any
Inherited from:
Column
def transform(f: Column => Column): Column

Applies f to this column, for chaining custom transformations.

Applies f to this column, for chaining custom transformations.

Attributes

Inherited from:
Column
def try_cast(to: String): Column

Casts to the type given as a SQL string, returning null when not possible.

Casts to the type given as a SQL string, returning null when not possible.

Attributes

Inherited from:
Column
def try_cast(to: DataType): Column

Casts to to, returning null instead of failing when the cast is not possible.

Casts to to, returning null instead of failing when the cast is not possible.

Attributes

Inherited from:
Column
def unary_!: Column

Attributes

Inherited from:
Column
def unary_-: Column

Attributes

Inherited from:
Column
def when(condition: Column, value: Any): Column

Adds a branch to a CASE expression started by functions.when.

Adds a branch to a CASE expression started by functions.when.

Attributes

Inherited from:
Column
def withField(fieldName: String, col: Column): Column

Adds or replaces a field in a struct column by name.

Adds or replaces a field in a struct column by name.

Attributes

Inherited from:
Column
def ||(other: Any): Column

Attributes

Inherited from:
Column

Deprecated and Inherited methods

def !==(other: Any): Column

Attributes

Deprecated
true
Inherited from:
Column
Generated with