Skip to content

mavis.schedule.scheduler

class mavis.schedule.scheduler.Scheduler

Class responsible for methods interacting with the scheduler

mavis.schedule.scheduler.Scheduler.__init__()

def __init__(self, concurrency_limit=None, remote_head_ssh=''):

Args

  • concurrency_limit (int): the maximum allowed concurrent processes. Defaults to one less than the total number available
  • remote_head_ssh

mavis.schedule.scheduler.Scheduler.command()

Wrapper to deal with subprocess commands. If configured and not on the head node currently, will send the command through ssh

def command(self, command, shell=False):

Args

  • command (list or str): the command can be a list or a string and is passed to the subprocess to be run
  • shell

Returns

  • str: the content returns from stdout of the subprocess

mavis.schedule.scheduler.Scheduler.submit()

submit a job to the scheduler

def submit(self, job):

Args

  • job

mavis.schedule.scheduler.Scheduler.update_info()

update the information about the job from the scheduler

def update_info(self, job):

Args

  • job

mavis.schedule.scheduler.Scheduler.format_dependencies()

returns a string representing the dependency argument

def format_dependencies(self, job):

Args

  • job

class mavis.schedule.scheduler.SlurmScheduler

inherits Scheduler

Class for formatting commands to match a SLURM scheduler system SLURM docs can be found here https://slurm.schedmd.com

mavis.schedule.scheduler.SlurmScheduler.submit()

runs a subprocess sbatch command

def submit(self, job):

Args

  • job (Job): the job to be submitted

mavis.schedule.scheduler.SlurmScheduler.parse_sacct()

parses content returned from the sacct command

@classmethod
def parse_sacct(cls, content):

Args

  • content (str): the content returned from the sacct command

mavis.schedule.scheduler.SlurmScheduler.parse_scontrol_show()

parse the content from the command: scontrol show job 

@classmethod
def parse_scontrol_show(cls, content):

Args

  • content (str): the content to be parsed

mavis.schedule.scheduler.SlurmScheduler.update_info()

Pull job information about status etc from the scheduler. Updates the input job

def update_info(self, job):

Args

  • job (Job): the job to be updated

mavis.schedule.scheduler.SlurmScheduler.cancel()

cancel a job

def cancel(self, job, task_ident=None):

Args

  • job (Job): the job to be cancelled
  • task_ident (int): the task id to be cancelled (instead of the entire array)

mavis.schedule.scheduler.SlurmScheduler.format_dependencies()

returns a string representing the dependency argument

def format_dependencies(self, job):

Args

  • job (Job): the job the argument is being built for

class mavis.schedule.scheduler.SgeScheduler

inherits Scheduler

Class for managing interactions with the SGE scheduler

mavis.schedule.scheduler.SgeScheduler.parse_qacct()

parses the information produced by qacct

@classmethod
def parse_qacct(cls, content):

Args

  • content (str): the content returned from the qacct command

mavis.schedule.scheduler.SgeScheduler.parse_qstat()

parses the qstat content into rows/dicts representing individual jobs

@classmethod
def parse_qstat(cls, content, job_id):

Args

  • content (str): content returned from the qstat command
  • job_id

mavis.schedule.scheduler.SgeScheduler.submit()

runs a subprocess sbatch command

def submit(self, job):

Args

  • job (Job): the job to be submitted

mavis.schedule.scheduler.SgeScheduler.update_info()

runs a subprocess scontrol command to get job details and add them to the current job

def update_info(self, job):

Args

  • job (Job): the job information is being gathered for

mavis.schedule.scheduler.SgeScheduler.cancel()

cancel a job or a specific task of an array job

def cancel(self, job, task_ident=None):

Args

  • job (Job): the job to cancel
  • task_ident (int): if specified, will cancel the given task instead of the whole array or job

mavis.schedule.scheduler.SgeScheduler.format_dependencies()

returns a string representing the dependency argument

def format_dependencies(self, job):

Args

  • job

class mavis.schedule.scheduler.TorqueScheduler

inherits SgeScheduler

Class for managing interactions with the Torque scheduler

mavis.schedule.scheduler.TorqueScheduler.format_dependencies()

returns a string representing the dependency argument

def format_dependencies(self, job):

Args

  • job

mavis.schedule.scheduler.TorqueScheduler.parse_qstat()

parses the qstat content into rows/dicts representing individual jobs

@classmethod
def parse_qstat(cls, content):

Args

  • content (str): content returned from the qstat command

mavis.schedule.scheduler.TorqueScheduler.submit()

runs a subprocess qsub command

def submit(self, job):

Args

  • job (Job): the job to be submitted

mavis.schedule.scheduler.TorqueScheduler.update_info()

runs a subprocess scontrol command to get job details and add them to the current job

def update_info(self, job):

Args

  • job (Job): the job information is being gathered for

mavis.schedule.scheduler.TorqueScheduler.cancel()

cancel a job

def cancel(self, job, task_ident=None):

Args

  • job (Job): the job to be cancelled
  • task_ident (int): if specified then a single task will be cancelled instead of the whole job or array

mavis.schedule.scheduler.time_format()

Converts a total seconds to a str format "H:M:S"

def time_format(total_seconds):

Args

  • total_seconds

mavis.schedule.scheduler.consecutive_ranges()

Given a list of integers, return a list of ranges

def consecutive_ranges(numbers):

Args

  • numbers

Examples

>>> consecutive_ranges([1, 2, 3, 4, 5, 9, 10, 14, 18])
[(1, 5), (9, 10), (14, 14), (18, 18)]