Client API Reference¶

Client¶

class sjsclient.client.Client(endpoint, auth=None)[source]

Bases: object

Client for Spark Job Server

App¶

class sjsclient.app.App(manager, attrs=None)[source]

Bases: sjsclient.base.Resource

An app is a spark application.

name = None: Name of the App

time = None: App creation time

class sjsclient.app.AppManager(client)[source]

Bases: sjsclient.base.ResourceManager

Manage App resources.

base_path = 'binaries'

create(name, app_binary, app_type='java')[source]

Create an app.

Parameters:	name – Descriptive name of application app_binary – Application binary app_type – App type, for example java or python, default: java
Return type:	`App`

delete(name)[source]

Delete a specific App.

Parameters:	name – The name of the `App` to delete.

get(name)[source]

Get a specific App.

Parameters:	name – The name of the `App` to get.
Return type:	`App`

list()[source]: Lists Apps.

resource_class: alias of App

class sjsclient.app.AppType[source]

Bases: object

A helper class that contains app types

JAVA = 'java'

PYTHON = 'python'

static get_header(app_type)[source]

Job¶

class sjsclient.job.Job(manager, attrs=None)[source]

Bases: sjsclient.base.Resource

A Spark job.

classpath = None: Main java class path

context = None: Context name

delete()[source]: Delete job.

duration = None: Time taken by the job to finish

get_config()[source]: Get job configuration.

jobId = None: Job ID

result = None: Response from Spark.

status = None: Jobs status

class sjsclient.job.JobConfig[source]

Bases: dict

A Spark job config dictionary.

class sjsclient.job.JobManager(client)[source]

Bases: sjsclient.base.ResourceManager

Manage Job resources.

base_path = 'jobs'

create(app, class_path, conf=None, ctx=None, sync=False)[source]

Create a Spark job.

Parameters:	app – Instance of `App` class_path – Main class path of spark job. conf – Configuration json ctx – Instance of `Context` sync – Set to True for synchronous job creation
Return type:	`Job`

delete(job_id)[source]

Delete a specific Job.

Parameters:	job_id – The jobId of the `Job` to get.

get(job_id)[source]

Get a specific Job. This returns more information than create.

Parameters:	job_id – The jobId of the `Job` to get.
Return type:	`Job`

get_config(job_id)[source]

Get job configuration.

Parameters:	job_id – The jobId of the `Job` to get.
Return type:	`JobConfig`

resource_class: alias of Job

class sjsclient.job.JobStatus[source]

Bases: object

A Helper class that contains the job status

ERROR = 'ERROR'

FINISHED = 'FINISHED'

RUNNING = 'RUNNING'

Context¶

class sjsclient.context.Context(manager, attrs=None)[source]

Bases: sjsclient.base.Resource

A Spark context.

delete()[source]: Delete context.

class sjsclient.context.ContextManager(client)[source]

Bases: sjsclient.base.ResourceManager

Manage Context resources.

base_path = 'contexts'

create(name, params=None)[source]

Create a Spark context.

Parameters:	name – Descriptive name of context params – Dictionary of context parameters
Return type:	`Context`

delete(name)[source]

Delete a specific Context.

Parameters:	name – The name of the `Context` to delete.

get(name)[source]

Get a specific Context.

Parameters:	name – The name of the `Context` to get.
Return type:	`Context`

resource_class: alias of Context