Nornir Service Parse Task¤

task api name: parse

The Nornir Service Parse Task is an integral part of NorFab's Nornir service, designed to facilitate the parsing and extraction of valuable information from network device outputs. This task provides network automation and developer engineers with powerful tools to transform raw command outputs into structured data, enabling more efficient network management and automation workflows.

Key features of the Nornir Service Parse Task include:

TextFSM Parsing: This task allows you to use TextFSM templates to parse command outputs into structured data. TextFSM is a powerful text processing tool that uses templates to define how to extract data from unstructured text. By leveraging TextFSM, you can convert complex command outputs into easily readable and processable data formats, which can then be used for further analysis or automation tasks.
TTP Parsing: The Template Text Parser (TTP) is a robust parsing tool supported by the Nornir Service Parse Task. TTP allows you to define templates for parsing text data, similar to TextFSM, but with additional flexibility and features. Using TTP, you can extract specific information from command outputs and transform it into structured data, making it easier to integrate with other systems and processes.
NAPALM Getters: The Nornir Service Parse Task leverages NAPALM getters to retrieve and parse structured data directly from network devices. NAPALM getters are pre-defined methods that extract specific pieces of information from devices, such as interface details, routing tables, ARP tables, and more.

The Nornir Service Parse Task is essential for network automation and developer engineers who need to process and analyze large volumes of network data. By transforming raw command outputs into structured data, you can automate complex workflows, generate insightful reports, and ensure that your network devices are configured and operating correctly.

This document also includes a reference for the NorFab shell commands related to the Nornir parse task, detailing the available options and parameters. These commands provide granular control over the parsing tasks.

NORFAB Nornir Parse Shell Reference¤

NorFab shell supports these command options for Nornir parse task:

nf#man tree nornir.parse
root
└── nornir:    Nornir service
    └── parse:    Parse network devices output
        ├── napalm:    Parse devices output using NAPALM getters
        │   ├── timeout:    Job timeout
        │   ├── workers:    Filter worker to target, default 'all'
        │   ├── add_details:    Add task details to results
        │   ├── run_num_workers:    RetryRunner number of threads for tasks execution
        │   ├── run_num_connectors:    RetryRunner number of threads for device connections
        │   ├── run_connect_retry:    RetryRunner number of connection attempts
        │   ├── run_task_retry:    RetryRunner number of attempts to run task
        │   ├── run_reconnect_on_fail:    RetryRunner perform reconnect to host on task failure
        │   ├── run_connect_check:    RetryRunner test TCP connection before opening actual connection
        │   ├── run_connect_timeout:    RetryRunner timeout in seconds to wait for test TCP connection to establish
        │   ├── run_creds_retry:    RetryRunner list of connection credentials and parameters to retry
        │   ├── tf:    File group name to save task results to on worker file system
        │   ├── tf_skip_failed:    Save results to file for failed tasks
        │   ├── diff:    File group name to run the diff for
        │   ├── diff_last:    File version number to diff, default is 1 (last)
        │   ├── progress:    Emit execution progress
        │   ├── FO:    Filter hosts using Filter Object
        │   ├── FB:    Filter hosts by name using Glob Patterns
        │   ├── FH:    Filter hosts by hostname
        │   ├── FC:    Filter hosts containment of pattern in name
        │   ├── FR:    Filter hosts by name using Regular Expressions
        │   ├── FG:    Filter hosts by group
        │   ├── FP:    Filter hosts by hostname using IP Prefix
        │   ├── FL:    Filter hosts by names list
        │   ├── FM:    Filter hosts by platform
        │   ├── FX:    Filter hosts excluding them by name
        │   ├── FN:    Negate the match
        │   ├── hosts:    Filter hosts to target
        │   └── *getters:    Select NAPALM getters, default 'PydanticUndefined'
        └── ttp:    Parse devices output using TTP templates
            ├── timeout:    Job timeout
            ├── workers:    Filter worker to target, default 'all'
            ├── add_details:    Add task details to results
            ├── run_num_workers:    RetryRunner number of threads for tasks execution
            ├── run_num_connectors:    RetryRunner number of threads for device connections
            ├── run_connect_retry:    RetryRunner number of connection attempts
            ├── run_task_retry:    RetryRunner number of attempts to run task
            ├── run_connect_check:    RetryRunner test TCP connection before opening actual connection
            ├── run_connect_timeout:    RetryRunner timeout in seconds to wait for test TCP connection to establish
            ├── run_creds_retry:    RetryRunner list of connection credentials and parameters to retry
            ├── tf:    File group name to save task results to on worker file system
            ├── tf_skip_failed:    Save results to file for failed tasks
            ├── diff:    File group name to run the diff for
            ├── diff_last:    File version number to diff, default is 1 (last)
            ├── progress:    Emit execution progress
            ├── FO:    Filter hosts using Filter Object
            ├── FB:    Filter hosts by name using Glob Patterns
            ├── FH:    Filter hosts by hostname
            ├── FC:    Filter hosts containment of pattern in name
            ├── FR:    Filter hosts by name using Regular Expressions
            ├── FG:    Filter hosts by group
            ├── FP:    Filter hosts by hostname using IP Prefix
            ├── FL:    Filter hosts by names list
            ├── FM:    Filter hosts by platform
            ├── FX:    Filter hosts excluding them by name
            ├── FN:    Negate the match
            ├── hosts:    Filter hosts to target
            ├── *template:    TTP Template to parse commands output, default 'PydanticUndefined'
            └── commands:    Commands to collect form devices
nf#

* - mandatory/required command argument

Python API Reference¤

Parse network device output using specified plugin and options.

Parameters:

Name	Type	Description	Default
`job`	`Job`	NorFab Job object containing relevant metadata	required
`plugin`	`str`	The plugin to use for parsing. Options are: napalm - parse devices output using NAPALM getters ttp - use TTP Templates to parse devices output textfsm - use TextFSM templates to parse devices output	`'napalm'`
`getters`	`str`	The getters to use with the "napalm" plugin.	`'get_facts'`
`template`	`str`	The template to use with the "ttp" or "textfsm" plugin.	`None`
`commands`	`list`	The list of commands to run with the "ttp" or "textfsm" plugin.	`None`
`to_dict`	`bool`	Whether to convert the result to a dictionary.	`True`
`add_details`	`bool`	Whether to add details to the result.	`False`
`**kwargs`		Additional keyword arguments to pass to the plugin.	`{}`

Returns:

Name	Type	Description
`Result`	`Result`	A Result object containing the parsed data.

Raises:

Type	Description
`UnsupportedPluginError`	If the specified plugin is not supported.

Source code in norfab\workers\nornir_worker.py

@Task(fastapi={"methods": ["POST"]})
def parse(
    self,
    job: Job,
    plugin: str = "napalm",
    getters: Union[str, list] = "get_facts",
    template: str = None,
    commands: Union[str, list] = None,
    to_dict: bool = True,
    add_details: bool = False,
    **kwargs,
) -> Result:
    """
    Parse network device output using specified plugin and options.

    Args:
        job: NorFab Job object containing relevant metadata
        plugin (str): The plugin to use for parsing. Options are:

            - napalm - parse devices output using NAPALM getters
            - ttp - use TTP Templates to parse devices output
            - textfsm - use TextFSM templates to parse devices output

        getters (str): The getters to use with the "napalm" plugin.
        template (str): The template to use with the "ttp" or "textfsm" plugin.
        commands (list): The list of commands to run with the "ttp" or "textfsm" plugin.
        to_dict (bool): Whether to convert the result to a dictionary.
        add_details (bool): Whether to add details to the result.
        **kwargs: Additional keyword arguments to pass to the plugin.

    Returns:
        Result: A Result object containing the parsed data.

    Raises:
        UnsupportedPluginError: If the specified plugin is not supported.
    """
    filters = {k: kwargs.pop(k) for k in list(kwargs.keys()) if k in FFun_functions}
    ret = Result(task=f"{self.name}:parse", result={} if to_dict else [])

    self.nr.data.reset_failed_hosts()  # reset failed hosts
    filtered_nornir = FFun(self.nr, **filters)  # filter hosts

    # check if no hosts matched
    if not filtered_nornir.inventory.hosts:
        msg = (
            f"{self.name} - nothing to do, no hosts matched by filters '{filters}'"
        )
        log.debug(msg)
        ret.messages.append(msg)
        ret.status = "no_match"
        return ret

    if plugin == "napalm":
        nr = self._add_processors(filtered_nornir, kwargs, job)  # add processors
        result = nr.run(task=napalm_get, getters=getters, **kwargs)
        ret.result = ResultSerializer(
            result, to_dict=to_dict, add_details=add_details
        )
        ret.failed = result.failed  # failed is true if any of the hosts failed
    elif plugin == "ttp":
        result = self.cli(
            job=job,
            commands=commands or [],
            run_ttp=template,
            **filters,
            **kwargs,
            to_dict=to_dict,
            add_details=add_details,
            plugin="netmiko",
        )
        ret.result = result.result
    elif plugin == "textfsm":
        result = self.cli(
            job=job,
            commands=commands,
            **filters,
            **kwargs,
            to_dict=to_dict,
            add_details=add_details,
            use_textfsm=True,
            textfsm_template=template,
            plugin="netmiko",
        )
        ret.result = result.result
    else:
        raise UnsupportedPluginError(f"Plugin '{plugin}' not supported")

    return ret