Input / Output - papermill 2.7.0 documentation

class papermill.iorw.ABSHandler¶

Bases: object

listdir(path)¶

pretty_path(path)¶

read(path)¶

write(buf, path)¶

class papermill.iorw.ADLHandler¶

Bases: object

listdir(path)¶

pretty_path(path)¶

read(path)¶

write(buf, path)¶

class papermill.iorw.GCSHandler¶

Bases: object

RATE_LIMIT_RETRIES = 3¶

RETRY_DELAY = 1¶

RETRY_MAX_DELAY = 4¶

RETRY_MULTIPLIER = 1¶

listdir(path)¶

pretty_path(path)¶

read(path)¶

write(buf, path)¶

class papermill.iorw.GithubHandler¶

Bases: object

listdir(path)¶

pretty_path(path)¶

read(path)¶

write(buf, path)¶

class papermill.iorw.HDFSHandler¶

Bases: object

listdir(path)¶

pretty_path(path)¶

read(path)¶

write(buf, path)¶

class papermill.iorw.HttpHandler¶

Bases: object

classmethod listdir(path)¶

classmethod pretty_path(path)¶

classmethod read(path)¶

classmethod write(buf, path)¶

class papermill.iorw.LocalHandler¶

Bases: object

cwd(new_path)¶: Sets the cwd during reads and writes

listdir(path)¶

pretty_path(path)¶

read(path)¶

write(buf, path)¶

class papermill.iorw.NoDatesSafeLoader(stream)¶

Bases: SafeLoader

yaml_implicit_resolvers = {'': [('tag:yaml.org,2002:null', re.compile('^(?: ~\n |null|Null|NULL\n | )$', re.VERBOSE))], '!': [('tag:yaml.org,2002:yaml', re.compile('^(?:!|&|\\*)$'))], '&': [('tag:yaml.org,2002:yaml', re.compile('^(?:!|&|\\*)$'))], '*': [('tag:yaml.org,2002:yaml', re.compile('^(?:!|&|\\*)$'))], '+': [('tag:yaml.org,2002:float', re.compile('^(?:[-+]?(?:[0-9][0-9_]*)\\.[0-9_]*(?:[eE][-+][0-9]+)?\n |\\.[0-9][0-9_]*(?:[eE][-+][0-9]+)?\n |[-+]?[0-9][0-9_]*(?::[0-5]?[0-9])+\\.[0-9_]*\n , re.VERBOSE)), ('tag:yaml.org,2002:int', re.compile('^(?:[-+]?0b[0-1_]+\n |[-+]?0[0-7_]+\n |[-+]?(?:0|[1-9][0-9_]*)\n |[-+]?0x[0-9a-fA-F_]+\n |[-+]?[1-9][0-9_]*(?::[0-5]?[0-9]), re.VERBOSE))], '-': [('tag:yaml.org,2002:float', re.compile('^(?:[-+]?(?:[0-9][0-9_]*)\\.[0-9_]*(?:[eE][-+][0-9]+)?\n |\\.[0-9][0-9_]*(?:[eE][-+][0-9]+)?\n |[-+]?[0-9][0-9_]*(?::[0-5]?[0-9])+\\.[0-9_]*\n , re.VERBOSE)), ('tag:yaml.org,2002:int', re.compile('^(?:[-+]?0b[0-1_]+\n |[-+]?0[0-7_]+\n |[-+]?(?:0|[1-9][0-9_]*)\n |[-+]?0x[0-9a-fA-F_]+\n |[-+]?[1-9][0-9_]*(?::[0-5]?[0-9]), re.VERBOSE))], '.': [('tag:yaml.org,2002:float', re.compile('^(?:[-+]?(?:[0-9][0-9_]*)\\.[0-9_]*(?:[eE][-+][0-9]+)?\n |\\.[0-9][0-9_]*(?:[eE][-+][0-9]+)?\n |[-+]?[0-9][0-9_]*(?::[0-5]?[0-9])+\\.[0-9_]*\n , re.VERBOSE))], '0': [('tag:yaml.org,2002:float', re.compile('^(?:[-+]?(?:[0-9][0-9_]*)\\.[0-9_]*(?:[eE][-+][0-9]+)?\n |\\.[0-9][0-9_]*(?:[eE][-+][0-9]+)?\n |[-+]?[0-9][0-9_]*(?::[0-5]?[0-9])+\\.[0-9_]*\n , re.VERBOSE)), ('tag:yaml.org,2002:int', re.compile('^(?:[-+]?0b[0-1_]+\n |[-+]?0[0-7_]+\n |[-+]?(?:0|[1-9][0-9_]*)\n |[-+]?0x[0-9a-fA-F_]+\n |[-+]?[1-9][0-9_]*(?::[0-5]?[0-9]), re.VERBOSE))], '1': [('tag:yaml.org,2002:float', re.compile('^(?:[-+]?(?:[0-9][0-9_]*)\\.[0-9_]*(?:[eE][-+][0-9]+)?\n |\\.[0-9][0-9_]*(?:[eE][-+][0-9]+)?\n |[-+]?[0-9][0-9_]*(?::[0-5]?[0-9])+\\.[0-9_]*\n , re.VERBOSE)), ('tag:yaml.org,2002:int', re.compile('^(?:[-+]?0b[0-1_]+\n |[-+]?0[0-7_]+\n |[-+]?(?:0|[1-9][0-9_]*)\n |[-+]?0x[0-9a-fA-F_]+\n |[-+]?[1-9][0-9_]*(?::[0-5]?[0-9]), re.VERBOSE))], '2': [('tag:yaml.org,2002:float', re.compile('^(?:[-+]?(?:[0-9][0-9_]*)\\.[0-9_]*(?:[eE][-+][0-9]+)?\n |\\.[0-9][0-9_]*(?:[eE][-+][0-9]+)?\n |[-+]?[0-9][0-9_]*(?::[0-5]?[0-9])+\\.[0-9_]*\n , re.VERBOSE)), ('tag:yaml.org,2002:int', re.compile('^(?:[-+]?0b[0-1_]+\n |[-+]?0[0-7_]+\n |[-+]?(?:0|[1-9][0-9_]*)\n |[-+]?0x[0-9a-fA-F_]+\n |[-+]?[1-9][0-9_]*(?::[0-5]?[0-9]), re.VERBOSE))], '3': [('tag:yaml.org,2002:float', re.compile('^(?:[-+]?(?:[0-9][0-9_]*)\\.[0-9_]*(?:[eE][-+][0-9]+)?\n |\\.[0-9][0-9_]*(?:[eE][-+][0-9]+)?\n |[-+]?[0-9][0-9_]*(?::[0-5]?[0-9])+\\.[0-9_]*\n , re.VERBOSE)), ('tag:yaml.org,2002:int', re.compile('^(?:[-+]?0b[0-1_]+\n |[-+]?0[0-7_]+\n |[-+]?(?:0|[1-9][0-9_]*)\n |[-+]?0x[0-9a-fA-F_]+\n |[-+]?[1-9][0-9_]*(?::[0-5]?[0-9]), re.VERBOSE))], '4': [('tag:yaml.org,2002:float', re.compile('^(?:[-+]?(?:[0-9][0-9_]*)\\.[0-9_]*(?:[eE][-+][0-9]+)?\n |\\.[0-9][0-9_]*(?:[eE][-+][0-9]+)?\n |[-+]?[0-9][0-9_]*(?::[0-5]?[0-9])+\\.[0-9_]*\n , re.VERBOSE)), ('tag:yaml.org,2002:int', re.compile('^(?:[-+]?0b[0-1_]+\n |[-+]?0[0-7_]+\n |[-+]?(?:0|[1-9][0-9_]*)\n |[-+]?0x[0-9a-fA-F_]+\n |[-+]?[1-9][0-9_]*(?::[0-5]?[0-9]), re.VERBOSE))], '5': [('tag:yaml.org,2002:float', re.compile('^(?:[-+]?(?:[0-9][0-9_]*)\\.[0-9_]*(?:[eE][-+][0-9]+)?\n |\\.[0-9][0-9_]*(?:[eE][-+][0-9]+)?\n |[-+]?[0-9][0-9_]*(?::[0-5]?[0-9])+\\.[0-9_]*\n , re.VERBOSE)), ('tag:yaml.org,2002:int', re.compile('^(?:[-+]?0b[0-1_]+\n |[-+]?0[0-7_]+\n |[-+]?(?:0|[1-9][0-9_]*)\n |[-+]?0x[0-9a-fA-F_]+\n |[-+]?[1-9][0-9_]*(?::[0-5]?[0-9]), re.VERBOSE))], '6': [('tag:yaml.org,2002:float', re.compile('^(?:[-+]?(?:[0-9][0-9_]*)\\.[0-9_]*(?:[eE][-+][0-9]+)?\n |\\.[0-9][0-9_]*(?:[eE][-+][0-9]+)?\n |[-+]?[0-9][0-9_]*(?::[0-5]?[0-9])+\\.[0-9_]*\n , re.VERBOSE)), ('tag:yaml.org,2002:int', re.compile('^(?:[-+]?0b[0-1_]+\n |[-+]?0[0-7_]+\n |[-+]?(?:0|[1-9][0-9_]*)\n |[-+]?0x[0-9a-fA-F_]+\n |[-+]?[1-9][0-9_]*(?::[0-5]?[0-9]), re.VERBOSE))], '7': [('tag:yaml.org,2002:float', re.compile('^(?:[-+]?(?:[0-9][0-9_]*)\\.[0-9_]*(?:[eE][-+][0-9]+)?\n |\\.[0-9][0-9_]*(?:[eE][-+][0-9]+)?\n |[-+]?[0-9][0-9_]*(?::[0-5]?[0-9])+\\.[0-9_]*\n , re.VERBOSE)), ('tag:yaml.org,2002:int', re.compile('^(?:[-+]?0b[0-1_]+\n |[-+]?0[0-7_]+\n |[-+]?(?:0|[1-9][0-9_]*)\n |[-+]?0x[0-9a-fA-F_]+\n |[-+]?[1-9][0-9_]*(?::[0-5]?[0-9]), re.VERBOSE))], '8': [('tag:yaml.org,2002:float', re.compile('^(?:[-+]?(?:[0-9][0-9_]*)\\.[0-9_]*(?:[eE][-+][0-9]+)?\n |\\.[0-9][0-9_]*(?:[eE][-+][0-9]+)?\n |[-+]?[0-9][0-9_]*(?::[0-5]?[0-9])+\\.[0-9_]*\n , re.VERBOSE)), ('tag:yaml.org,2002:int', re.compile('^(?:[-+]?0b[0-1_]+\n |[-+]?0[0-7_]+\n |[-+]?(?:0|[1-9][0-9_]*)\n |[-+]?0x[0-9a-fA-F_]+\n |[-+]?[1-9][0-9_]*(?::[0-5]?[0-9]), re.VERBOSE))], '9': [('tag:yaml.org,2002:float', re.compile('^(?:[-+]?(?:[0-9][0-9_]*)\\.[0-9_]*(?:[eE][-+][0-9]+)?\n |\\.[0-9][0-9_]*(?:[eE][-+][0-9]+)?\n |[-+]?[0-9][0-9_]*(?::[0-5]?[0-9])+\\.[0-9_]*\n , re.VERBOSE)), ('tag:yaml.org,2002:int', re.compile('^(?:[-+]?0b[0-1_]+\n |[-+]?0[0-7_]+\n |[-+]?(?:0|[1-9][0-9_]*)\n |[-+]?0x[0-9a-fA-F_]+\n |[-+]?[1-9][0-9_]*(?::[0-5]?[0-9]), re.VERBOSE))], '<': [('tag:yaml.org,2002:merge', re.compile('^(?:<<)$'))], '=': [('tag:yaml.org,2002:value', re.compile('^(?:=)$'))], 'F': [('tag:yaml.org,2002:bool', re.compile('^(?:yes|Yes|YES|no|No|NO\n |true|True|TRUE|false|False|FALSE\n |on|On|ON|off|Off|OFF)$', re.VERBOSE))], 'N': [('tag:yaml.org,2002:bool', re.compile('^(?:yes|Yes|YES|no|No|NO\n |true|True|TRUE|false|False|FALSE\n |on|On|ON|off|Off|OFF)$', re.VERBOSE)), ('tag:yaml.org,2002:null', re.compile('^(?: ~\n |null|Null|NULL\n | )$', re.VERBOSE))], 'O': [('tag:yaml.org,2002:bool', re.compile('^(?:yes|Yes|YES|no|No|NO\n |true|True|TRUE|false|False|FALSE\n |on|On|ON|off|Off|OFF)$', re.VERBOSE))], 'T': [('tag:yaml.org,2002:bool', re.compile('^(?:yes|Yes|YES|no|No|NO\n |true|True|TRUE|false|False|FALSE\n |on|On|ON|off|Off|OFF)$', re.VERBOSE))], 'Y': [('tag:yaml.org,2002:bool', re.compile('^(?:yes|Yes|YES|no|No|NO\n |true|True|TRUE|false|False|FALSE\n |on|On|ON|off|Off|OFF)$', re.VERBOSE))], 'f': [('tag:yaml.org,2002:bool', re.compile('^(?:yes|Yes|YES|no|No|NO\n |true|True|TRUE|false|False|FALSE\n |on|On|ON|off|Off|OFF)$', re.VERBOSE))], 'n': [('tag:yaml.org,2002:bool', re.compile('^(?:yes|Yes|YES|no|No|NO\n |true|True|TRUE|false|False|FALSE\n |on|On|ON|off|Off|OFF)$', re.VERBOSE)), ('tag:yaml.org,2002:null', re.compile('^(?: ~\n |null|Null|NULL\n | )$', re.VERBOSE))], 'o': [('tag:yaml.org,2002:bool', re.compile('^(?:yes|Yes|YES|no|No|NO\n |true|True|TRUE|false|False|FALSE\n |on|On|ON|off|Off|OFF)$', re.VERBOSE))], 't': [('tag:yaml.org,2002:bool', re.compile('^(?:yes|Yes|YES|no|No|NO\n |true|True|TRUE|false|False|FALSE\n |on|On|ON|off|Off|OFF)$', re.VERBOSE))], 'y': [('tag:yaml.org,2002:bool', re.compile('^(?:yes|Yes|YES|no|No|NO\n |true|True|TRUE|false|False|FALSE\n |on|On|ON|off|Off|OFF)$', re.VERBOSE))], '~': [('tag:yaml.org,2002:null', re.compile('^(?: ~\n |null|Null|NULL\n | )$', re.VERBOSE))]}¶

class papermill.iorw.NoIOHandler¶

Bases: object

Handler for output_path of None - intended to not write anything

listdir(path)¶

pretty_path(path)¶

read(path)¶

write(buf, path)¶

class papermill.iorw.NotebookNodeHandler¶

Bases: object

Handler for input_path of nbformat.NotebookNode object

listdir(path)¶

pretty_path(path)¶

read(path)¶

write(buf, path)¶

class papermill.iorw.PapermillIO¶

Bases: object

The holder which houses any io system registered with the system. This object is used in a singleton manner to save and load particular named Handler objects for reference externally.

get_handler(path, extensions=None)¶

Get I/O Handler based on a notebook path

Parameters:

path (str or nbformat.NotebookNode or None)
extensions (list of str, optional) – Required file extension options for the path (if path is a string), which will log a warning if there is no match. Defaults to None, which does not check for any extensions

Raises:

PapermillException – If a valid I/O handler could not be found for the input path:

Return type:

I/O Handler

listdir(path)¶

pretty_path(path)¶

read(path, extensions=['.ipynb', '.json'])¶

register(scheme, handler)¶

register_entry_points()¶

reset()¶

write(buf, path, extensions=['.ipynb', '.json'])¶

class papermill.iorw.S3Handler¶

Bases: object

classmethod listdir(path)¶

classmethod pretty_path(path)¶

classmethod read(path)¶

classmethod write(buf, path)¶

class papermill.iorw.StreamHandler¶

Bases: object

Handler for Stdin/Stdout streams

listdir(path)¶

pretty_path(path)¶

read(path)¶

write(buf, path)¶

papermill.iorw.fallback_gs_is_retriable(e)¶

papermill.iorw.get_pretty_path(path)¶

papermill.iorw.list_notebook_files(path)¶

Returns a list of all the notebook files in a directory.

papermill.iorw.load_notebook_node(notebook_path)¶

Returns a notebook object with papermill metadata loaded from the specified path.

Parameters:: notebook_path (str) – Path to the notebook file.
Returns:: nbformat.NotebookNode

papermill.iorw.local_file_io_cwd(path=None)¶

papermill.iorw.read_yaml_file(path)¶

Reads a YAML file from the location specified at ‘path’.

papermill.iorw.write_ipynb(nb, path)¶

Saves a notebook object to the specified path. :param nb_node: Notebook object to save. :type nb_node: nbformat.NotebookNode :param notebook_path: Path to save the notebook object to. :type notebook_path: str

Input / Output¶

papermill.iorw¶