Spark Connect for Scala 3
API
Generated with
Spark Connect for Scala 3/org.apache.spark/org.apache.spark.sql/SparkSession/Builder

Builder

org.apache.spark.sql.SparkSession.Builder
class Builder

Builder for SparkSession.

Attributes

Graph
Supertypes
class Object
trait Matchable
class Any

Members list

Value members

Concrete methods

def appName(name: String): Builder
def apply(): Builder

Returns this builder, so that SparkSession.builder() reads like Apache Spark.

Returns this builder, so that SparkSession.builder() reads like Apache Spark.

Attributes

def config(key: String, value: String): Builder
def config(key: String, value: Boolean): Builder
def config(key: String, value: Long): Builder
def config(key: String, value: Double): Builder
def config(map: Map[String, Any]): Builder

Sets several configurations at once.

Sets several configurations at once.

Attributes

def config(map: Map[String, Any]): Builder

(Java-specific) Sets several configurations at once.

(Java-specific) Sets several configurations at once.

Attributes

def create(): SparkSession

Builds a brand-new session.

Builds a brand-new session.

Attributes

def getOrCreate(): SparkSession

Returns the active session if present, otherwise builds and registers a new one.

Returns the active session if present, otherwise builds and registers a new one.

Attributes

def remote(connectionString: String): Builder

Configures the connection from a Spark Connect connection string (sc://...).

Configures the connection from a Spark Connect connection string (sc://...).

Attributes

def userAgent(agent: String): Builder

Sets the user agent reported to the server.

Sets the user agent reported to the server.

Attributes

Deprecated methods

def enableHiveSupport(): Builder

Has no effect over Spark Connect (the server decides its catalog implementation). Kept for source compatibility with Apache Spark.

Has no effect over Spark Connect (the server decides its catalog implementation). Kept for source compatibility with Apache Spark.

Attributes

Deprecated
true
def master(master: String): Builder

Has no effect over Spark Connect (the server decides the master); use remote to choose the server. Kept for source compatibility with Apache Spark.

Has no effect over Spark Connect (the server decides the master); use remote to choose the server. Kept for source compatibility with Apache Spark.

Attributes

Deprecated
true
Generated with