Welcome to np_validator’s documentation!

Core API

Awesome np-validator is a Python cli/package created with https://github.com/TezRomacH/python-package-template

class np_validator.Processor(name: str, args: ~typing.Dict[str, ~typing.Any] = <factory>)

Processor abstraction.

Parameters

  • name (str) – Name of the processor to use.

  • args (dict, optional) – Additional arguments supplied to builtin.

name

Name of the processor.

Type

str

args

Additional arguments supplied to builtin.

Type

dict

builtin

BuiltinCallable instance resolved from name.

Type

BuiltinCallable

Note

Refer to Supported Processors for a list of supported name to use.

class np_validator.ValidationStep(path_suffix: str, processor: ~typing.Optional[~np_validator.dataclasses.Processor] = None, validators: ~typing.List[~np_validator.dataclasses.Validator] = <factory>)

Validation step abstraction.

path_suffix

Filepath suffix matched against filepaths. Determines what filepaths this step will be called on.

Type

str

processor

Processors to use on filepath before validation.

Type

Optional[np_validator.dataclasses.Processor]

validators

Validators to apply to a matched filepath, or processor output, if a Processor is supplied.

Type

List[np_validator.dataclasses.Validator]

class np_validator.Validator(name: str, args: ~typing.Dict[str, ~typing.Any] = <factory>)

Validator abstraction.

Parameters

  • name (str) – Name of the validator to use.

  • args (dict, optional) – Additional arguments supplied to builtin.

name

Name of the validator.

Type

str

args

Additional arguments supplied to builtin.

Type

dict

builtin

BuiltinCallable instance resolved from name.

Type

BuiltinCallable

Note

Refer to Supported Validators for a list of supported name to use.

np_validator.load_ValidationStep(data: Dict[str, Any]) → ValidationStep

Loads a ValidationStep from a structured dictionary.

np_validator.run_validation(file_list: List[str], validation_steps: List[ValidationStep] = [], autogen_steps: bool = True, project: str = '', raise_if_errors: bool = False) → List[ValidationResult]

Validates a list of files.

Parameters

  • file_list (list of str) – List of filepaths to validate.

  • validation_steps – List of validation steps to run.

  • autogen_steps – Add additional autogenerated ValidationSteps based on file types in file_list.

  • project – Project id associated with the validation.

  • raise_if_errors – Raise an NpValidatorError if ValidationStep errors occur.

Returns

List of validation results.

Return type

results

Raises

NpValidatorError – One or more validation steps had an unexpected error.

Supported Processors

All processors take a filepath as their first argument, followed by various keyword arguments specific to each processor.

np_validator.processors.load_sync_dataset(filepath: str) → Dataset

Loads a sync Dataset instance from a filepath.

Parameters

filepath (str) – Filepath to load.

Returns

Loaded dataset.

Return type

loaded (Dataset)

np_validator.processors.unpickle(filepath: str, encoding: str = 'latin1') → Any

Unpickles a filepath to an arbitrary object.

Parameters

  • filepath (str) – Filepath to unpickle.

  • encoding (str) – Encoding used to unpickle.

Returns

Unpickled object.

Return type

unpickled (Any)

Supported Validators

All validators take an object (defaults of a filepath), followed by various keyword arguments specific to each validator.

np_validator.validators.has_dict_key(value: Dict[str, Any], path: List[str]) → Tuple[Any, bool]

Validates whether or not a dictionary has a key at given paths.

A single element path will grab from the top level keys. Multiple path elements are used to index into nested dictionaries.

Parameters

  • value – dictionary to validate

  • path – keys to validate

Returns

the value of the key if it exists or None has_key: whether or not the key was found

Return type

value

np_validator.validators.meets_filesize_threshold(value: str, threshold: Union[float, int], operator_name: str = 'ge') → Tuple[int, bool]

Validates whether or not a filepath meets a given filesize threshold.

Parameters

  • value – Filepath to validate.

  • threshold – minimum filesize threshold in bytes

  • operator_name – name of which python operator to use

Returns

A tuple of length 2 wherein, the first element is the filesize of the value and the second element is a boolean whether or not the threshold for this value was met.

np_validator.validators.meets_lick_threshold(value: dict, threshold: int) → Tuple[int, bool]

Validates the number of licks from behavior output.

Parameters

  • value – Behavior pickle data from a session.

  • threshold – Minimum number of calculated licks.

Returns

Calculated number of licks. Passes threshold.

np_validator.validators.meets_value_threshold(value: Union[Dict[str, Any], Dataset], path: List[str], threshold: Union[float, int], tolerance: Optional[Union[float, int]] = None) → Tuple[Any, bool]

Ensures that a value of a dict or Dataset instance at path meets a threshold.

Parameters

  • value – Target dict or Dataset.

  • path – Keys to index into target.

  • threshold – Threshold to compare the indexed value to.

  • tolerance – Tolerance of the threshold.

Returns

The indexed value. Whether or not the indexed value meets the given threshold.

Raises

  • Exception – Dataset supplied as value and length path > 1.

  • TypeError – Value is an unsupported datatype.

Note

To index into nested dict-like objects, use a path with length greater than 1.

Dataset targets only support a path of length 1.

meant to mirror get_data_subtype for synchronization_data in NP_pipeline_validation, an earlier implementation this tool is based off of

np_validator.validators.meets_wheel_rotation_threshold(value: dict, threshold: int) → Tuple[int, bool]

Validates the number of wheel rotations from behavior output.

Parameters

  • value – Behavior pickle data from a session.

  • threshold – Minimum number of calculated wheel rotations.

Returns

Calculated number of wheel rotations. passes: Passes threshold.

Return type

number_of_rotations

Indices and tables