org.apache.spark.sql.catalog

The catalog interface for inspecting and managing databases, tables, functions, and the query cache.

Methods that return rows (listDatabases, listTables, ...) return a DataFrame; predicate methods return a Boolean; command-like methods return Unit. Mirrors PySpark's Catalog.

 spark.catalog.listTables().show()
 spark.catalog.tableExists("my_table")  // => true

Attributes

Supertypes: class Object

trait Matchable

class Any

A database in the catalog, as returned by Catalog.getDatabase / Catalog.listDatabases.

Value parameters

catalog: name of the catalog the database belongs to (may be null on older servers).
description: description of the database.
locationUri: path (in the form of a uri) to data files.
name: name of the database.

Attributes

Supertypes: trait Serializable

trait Product

trait Equals

class Object

trait Matchable

class Any
Show all

A function in the catalog, as returned by Catalog.getFunction / Catalog.listFunctions.

Value parameters

catalog: name of the catalog the function belongs to (may be null on older servers).
className: the fully qualified class name of the function.
description: description of the function.
isTemporary: whether the function is a temporary function.
name: name of the function.
namespace: the namespace the function belongs to; null/empty for temporary functions.

Attributes

Supertypes: trait Serializable

trait Product

trait Equals

class Object

trait Matchable

class Any
Show all

A table or view in the catalog, as returned by Catalog.getTable / Catalog.listTables.

Value parameters

catalog: name of the catalog the table belongs to (may be null on older servers).
description: description of the table.
isTemporary: whether the table is a temporary view.
name: name of the table.
namespace: the namespace (database) the table belongs to; empty for temporary views.
tableType: type of the table (e.g. MANAGED, EXTERNAL, VIEW, TEMPORARY).

Attributes

Supertypes: trait Serializable

trait Product

trait Equals

class Object

trait Matchable

class Any
Show all

org.apache.spark.sql.catalog

Members list

Type members

Classlikes

Attributes

Value parameters

Attributes

Value parameters

Attributes

Value parameters

Attributes