org.apache.spark.sql.connect.client
Members list
Type members
Classlikes
Encodes local rows into an Apache Arrow IPC stream so that the client can ship local data to the server as a LocalRelation (used by SparkSession.createDataFrame).
Encodes local rows into an Apache Arrow IPC stream so that the client can ship local data to the server as a LocalRelation (used by SparkSession.createDataFrame).
This is the symmetric ENCODE counterpart to SparkResult (which DECODES the Arrow batches returned by the server). The Spark-type-to-Arrow-type mapping and the per-cell value conversions here mirror the decode logic in SparkResult.getValue so that a value round-trips faithfully.
Modelled on the same-author Ruby reference (spark_connect/arrow.rb, from_rows / build_arrow_schema / arrow_field_type).
Attributes
- Supertypes
-
class Objecttrait Matchableclass Any
- Self type
-
ArrowSerializer.type
Configures the retry mechanism used by GrpcRetryHandler. Ported from Apache Spark's org.apache.spark.sql.connect.client.RetryPolicy.
Configures the retry mechanism used by GrpcRetryHandler. Ported from Apache Spark's org.apache.spark.sql.connect.client.RetryPolicy.
Value parameters
- backoffMultiplier
-
multiplicative base of the exponential backoff.
- canRetry
-
predicate deciding whether an error is retryable.
- initialBackoff
-
start of the exponential backoff.
- jitter
-
random value in
[0, jitter]added to the backoff. - maxBackoff
-
cap on the exponential backoff.
- maxRetries
-
maximum number of retries (None = unbounded).
- maxServerRetryDelay
-
cap on the server-provided retry delay.
- minJitterThreshold
-
minimum backoff at which jitter is added.
- name
-
policy name (for diagnostics).
- recognizeServerRetryDelay
-
honor a server-provided
RetryInfo.retry_delay.
Attributes
- Companion
- object
- Supertypes
-
trait Serializabletrait Producttrait Equalsclass Objecttrait Matchableclass AnyShow all
Attributes
- Companion
- class
- Supertypes
-
trait Producttrait Mirrorclass Objecttrait Matchableclass Any
- Self type
-
RetryPolicy.type
Renders rows as the familiar df.show() table, client-side, matching the layout of Dataset.showString in Apache Spark (horizontal and vertical). Rendering locally from collected rows keeps show() deterministic and exactly consistent with collect().
Renders rows as the familiar df.show() table, client-side, matching the layout of Dataset.showString in Apache Spark (horizontal and vertical). Rendering locally from collected rows keeps show() deterministic and exactly consistent with collect().
The caller passes up to numRows + 1 rows; the extra row (if present) only triggers the "only showing top N rows" footer and is not displayed.
Attributes
- Supertypes
-
class Objecttrait Matchableclass Any
- Self type
-
ShowString.type
The low-level client that talks to a Spark Connect server over gRPC.
The low-level client that talks to a Spark Connect server over gRPC.
It owns the gRPC channel, the (stable) session id, the user context and the user agent, and exposes the four core RPCs used by the public API: ExecutePlan, AnalyzePlan, Config and Interrupt.
Attributes
- Companion
- object
- Supertypes
-
class Objecttrait Matchableclass Any
Attributes
- Companion
- class
- Supertypes
-
class Objecttrait Matchableclass Any
- Self type
-
SparkConnectClient.type