Encoder

org.apache.spark.sql.Encoder

See theEncoder companion object

trait Encoder[T] extends Serializable

Converts between JVM values of type T and the Row representation produced and consumed by Spark Connect. Encoders are derived at compile time for primitives, Option, collections, maps, tuples, and case classes, so df.as[Person] and spark.createDataset(people) work with no server-side closures.

Encoders intentionally do NOT cover Dataset.map/flatMap/groupByKey/reduce/mapPartitions: those evaluate a JVM closure per row on the server (the same mechanism as a UDF), which Spark Connect for Scala 3 does not support.

Attributes

Companion: object
Graph
Supertypes: trait Serializable

class Object

trait Matchable

class Any

Members list

Value members

Abstract methods

Runtime class tag for T, used to materialize typed arrays from collect().

Attributes

The Spark data type of T.

Attributes

Concrete methods

Whether T admits null / None.

Attributes

In this article

Generated with