job module

class mavis.schedule.job.ArrayJob(stage, task_list, **kwargs)

Bases: mavis.schedule.job.Job

Class for dealing with array jobs. Jobs with many tasks

Parameters:task_list (list or int) – the ids of tasks in the job array

complete_stamp(task_ident)

returns the path to the expected complete stamp

copy_with_tasks(task_list)

flatten()

get_task(task_ident)

returns a task by task id

has_task(task_ident)

logfile(task_ident)

returns the path to the logfile with job name and job id substituted into the stdout pattern

remove_task(task_ident)

reset()

tasks

class mavis.schedule.job.Job(stage, output_dir, stdout=None, job_ident=None, name=None, dependencies=None, script=None, created_at=None, status='NOT SUBMITTED', status_comment='', **options)

Bases: object

Parameters:

• stage (str) – the mavis pipleine stage this job belongs to
• job_ident (int) – the job number/id according to the scheduler being used
• output_dir (str) – path to the output directory where logs/stamps for this job will be written
• name (str) – the job name according to the scheduler being used
• dependencies (list of Job) – list of jobs which must complete for this job to run
• stdout (str) – basename of the file to write std output to
• script (str) – path to the script which contains the commands for the job
• created_at (int) – the time stamp for when the job was created (created != submitted)
• status (JOB_STATUS) – The current (since last checked) status of the job
• status_comment (str) – the comment which describes the status, generally this is used for reporting errors from the log file or failed dependencies (SLURM)
• options (**dict) – override default options specified by OPTIONS

complete_stamp()

returns the path to the expected complete stamp

display_name

Used for identifying this job in an ini config file

flatten()

logfile()

returns the path to the logfile with job name and job id substituted into the stdout pattern

reset()

class mavis.schedule.job.LogFile(filename, status, message=None)

Bases: object

stores information about the log status

Parameters:

• filename (str) – path to the logfile
• status (LogFile.STATUS) – the status of the logfile
• message (str) – the message parsed from the logfile. Generally this is an error from the log

STATUS = MavisNamespace(COMPLETE='COMPLETE', CRASH='CRASH', EMPTY='EMPTY', INCOMPLETE='INCOMPLETE')

The status of the job based on parsing of the logfile

Type:MavisNamespace

classmethod parse(filename)

given a file parse to see if it looks like a complete log file (contains run time), was truncated, or reported an error

class mavis.schedule.job.Task(array_job, task_ident)

Bases: object

complete_stamp()

logfile()

reset()

class mavis.schedule.job.TorqueArrayJob(stage, task_list, **kwargs)

Bases: mavis.schedule.job.ArrayJob

Parameters:task_list (list or int) – the ids of tasks in the job array

complete_stamp(task_ident)

returns the path to the expected complete stamp

logfile(task_ident)

returns the path to the logfile with job name and job id substituted into the stdout pattern