Using the Transpiler

Main workhorse - Transpiler

The main component of Phaistos is the transpiler, which is responsible for converting the YAML schema into a Pydantic model.

This object is used under-the-hood by the Validator class (to be described in a lil' bit) automatically, but there is nothing stopping You from using it declaratively in Your code. The transpiler is devoid of state, so it can be directly used in a functional manner.

Below, the main methods of the transpiler are described.

Schema transpilation

This action is performed by make_schema method and is responsible for converting the YAML schema into a Pydantic model and returns it.

phaistos.transpiler.Transpiler.make_schema

Transpile a schema into a Pydantic model.

Parameters:

Name	Type	Description	Default
schema	SchemaInputFile	A parsed schema, stored as a dictionary with typed fields.	required

Returns:

Type	Description
type[TranspiledSchema]	type[TranspiledSchema]: A Pydantic model class with the schema's properties.

Source code in phaistos/transpiler.py

@classmethod
def make_schema(cls, schema: SchemaInputFile, parent: type[TranspiledSchema] | None = None) -> type[TranspiledSchema]:
    """
    Transpile a schema into a Pydantic model.

    Args:
        schema (SchemaInputFile): A parsed schema, stored as a dictionary with typed fields.

    Returns:
        type[TranspiledSchema]: A Pydantic model class with the schema's properties.
    """
    cls._logger.info(f"Transpiling schema: {schema['name']}")

    class _Schema(TranspiledSchema):
        version: typing.ClassVar[str]
        description: typing.ClassVar[str]

    _Schema.version = schema.get('version', '...')
    _Schema.description = schema.get('description', '')
    _Schema.__name__ = schema['name']
    _Schema.__qualname__ = schema['name']
    _Schema.transpilation_name = schema['name']

    transpilation = cls.make_properties(
        properties=[
            ParsedProperty(
                name=property_name,
                data=property_data
            )
            for property_name, property_data in schema['properties'].items()
        ],
        properties_parent=_Schema
    )

    transpilation['parent'] = parent
    transpilation['context'] = schema.get('context', {})  # type: ignore

    if 'validator' in schema:
        cls._logger.info(f'Compiling {schema["name"]} model validator')
        validator_source = schema['validator'] if isinstance(schema['validator'], str) else schema['validator']['source']
        global_model_validator_function = ValidationFunctionsCompiler._compile_validator({
            'name': f'validate_{schema["name"].lower()}',
            'decorator': '@classmethod',
            'source': validator_source,
            'kind': 'model'
        })
        transpilation['global_validator'] = global_model_validator_function

    transpiled_property_names = ', '.join([
        key
        for key in transpilation['properties'].keys()
        if not key.startswith('_')
    ])
    cls._logger.info(f"Schema {schema['name']} has been transpiled successfully. Transpiled properties: {transpiled_property_names}")

    return _Schema.compile(transpilation)

As for the information returned by this method, it is a Pydantic model, which can be used to validate the data with some fields automatically injected during transpilation:

phaistos.schema.TranspiledSchema

Bases: BaseModel

A Pydantic model that represents a transpiled schema.

Source code in phaistos/schema.py

class TranspiledSchema(pydantic.BaseModel):
    """
    A Pydantic model that represents a transpiled schema.
    """
    transpilation_name: typing.ClassVar[str] = ''
    global_validator: typing.ClassVar[
        typing.Callable[[TranspiledSchema, typing.Any], None] | None
    ] = None
    context: typing.ClassVar[
        dict[str, typing.Any]
    ] = {}
    _validation_errors: typing.ClassVar[
        list[FieldValidationErrorInfo]
    ] = []
    parent: typing.ClassVar[type[TranspiledSchema]]

    # pylint: disable=protected-access
    @property
    def validation_errors(self) -> list[FieldValidationErrorInfo]:
        return self.parent._validation_errors

    # pylint: disable=protected-access
    @classmethod
    def compile(cls, model_data: TranspiledModelData) -> type[TranspiledSchema]:
        if not model_data.get('parent'):
            cls._validation_errors = []
        schema: type[TranspiledSchema] = pydantic.create_model(  # type: ignore
            model_data['name'],
            __base__=TranspiledSchema,
            __validators__={
                validator['name']: validator['method']
                for validator in model_data['validators']
                if validator
            },
            **model_data['properties']
        )
        schema.parent = model_data.get('parent') or copy.deepcopy(cls)
        cls._rename_schema(schema, model_data['name'])

        schema.context = model_data.get('context', {})  # type: ignore
        schema.global_validator = model_data.get('global_validator')
        return schema

    @classmethod
    def _rename_schema(cls, schema: type[TranspiledSchema], name: str) -> None:
        for field in ['__name__', '__qualname__', 'transpilation_name']:
            if hasattr(schema, field):
                setattr(schema, field, name)

    # pylint: disable=no-self-argument, unused-variable, super-init-not-called, not-callable
    def __init__(self, **data: typing.Any) -> None:  # type: ignore
        """
            A modified version of the Pydantic BaseModel __init__ method that
            passed the context to the validator.
        """
        __tracebackhide__ = True
        if self.parent.transpilation_name == self.transpilation_name:
            self.parent._validation_errors = []
        collected_errors: list[FieldValidationErrorInfo] = []
        try:
            if self.global_validator:
                self.global_validator(data)  # type: ignore
        except Exception as validator_exception:  # pylint: disable=broad-except
            collected_errors.append(
                FieldValidationErrorInfo(
                    name=self.__class__.__name__,
                    message=str(validator_exception)
                )
            )
        try:
            self.__pydantic_validator__.validate_python(
                data,
                self_instance=self,
                context=self.context
            )
        except pydantic.ValidationError as validation_error:
            collected_errors.extend([
                FieldValidationErrorInfo(
                    name=str(error['loc'][0]) if error['loc'] else validation_error.title,
                    message=error['msg']
                )
                for error in validation_error.errors()
            ])
        self.parent._validation_errors += self._validation_errors + collected_errors

__init__(**data)

A modified version of the Pydantic BaseModel init method that passed the context to the validator.

Source code in phaistos/schema.py

def __init__(self, **data: typing.Any) -> None:  # type: ignore
    """
        A modified version of the Pydantic BaseModel __init__ method that
        passed the context to the validator.
    """
    __tracebackhide__ = True
    if self.parent.transpilation_name == self.transpilation_name:
        self.parent._validation_errors = []
    collected_errors: list[FieldValidationErrorInfo] = []
    try:
        if self.global_validator:
            self.global_validator(data)  # type: ignore
    except Exception as validator_exception:  # pylint: disable=broad-except
        collected_errors.append(
            FieldValidationErrorInfo(
                name=self.__class__.__name__,
                message=str(validator_exception)
            )
        )
    try:
        self.__pydantic_validator__.validate_python(
            data,
            self_instance=self,
            context=self.context
        )
    except pydantic.ValidationError as validation_error:
        collected_errors.extend([
            FieldValidationErrorInfo(
                name=str(error['loc'][0]) if error['loc'] else validation_error.title,
                message=error['msg']
            )
            for error in validation_error.errors()
        ])
    self.parent._validation_errors += self._validation_errors + collected_errors

model_post_init(__context)

This function is meant to behave like a BaseModel method to initialise private attributes.

It takes context as an argument since that's what pydantic-core passes when calling it.

Parameters:

Name	Type	Description	Default
self	BaseModel	The BaseModel instance.	required
__context		The context.	required

Source code in phaistos/schema.py

_validation_errors: typing.ClassVar[
    list[FieldValidationErrorInfo]
] = []
parent: typing.ClassVar[type[TranspiledSchema]]

# pylint: disable=protected-access
@property
def validation_errors(self) -> list[FieldValidationErrorInfo]:
    return self.parent._validation_errors

If You poke around the source code (or believe me on my word), You will find that the transpiler is a simple class with a single method, which is responsible for compiling the schema data into a Pydantic model.

The schema data is a dictionary, which is a result of parsing the YAML schema file. It is a collection of fields, which are then converted into Pydantic fields and compiled validator functions, which are then used to validate the data.

Bases: TypedDict

A dictionary that represents transpiled model data.

Attributes:

Name	Type	Description
validator	list[TranspiledPropertyValidator]	A list of transpiled property validators.
properties	dict[str, Any]	A dictionary of transpiled properties.
context	dict[str, Any]	A dictionary of the context of the model, used during validation.

Source code in phaistos/typings.py

class TranspiledModelData(typing.TypedDict):
    """
    A dictionary that represents transpiled model data.

    Attributes:
        validator (list[TranspiledPropertyValidator]): A list of transpiled property validators.
        properties (dict[str, typing.Any]): A dictionary of transpiled properties.
        context (dict[str, typing.Any]): A dictionary of the context of the model, used during validation.
    """
    name: str
    validators: list[CompiledValidator]
    properties: dict[str, typing.Any]
    parent: typing.Any
    context: typing.NotRequired[dict[str, typing.Any]]
    global_validator: typing.NotRequired[typing.Any]

Each property is expressed with a 3-tuple, where the first element is the field name, the second is the field type, and the third is the field default value.

As for the validators, the compiled functions are stored as dictionaries of CompiledValidator type:

phaistos.typings.CompiledValidator

Bases: TypedDict

A dictionary that represents a transpiled property validator.

Attributes:

Name	Type	Description
field	str	The field of the property.
name	str	The name of the property.
method	Any	The method of the property.

Source code in phaistos/typings.py

class CompiledValidator(typing.TypedDict):
    """
    A dictionary that represents a transpiled property validator.

    Attributes:
        field (str): The field of the property.
        name (str): The name of the property.
        method (typing.Any): The method of the property.
    """
    field: str
    name: str
    method: typing.Any

So, how can I use it?

The transpiler is a simple object, which can be used in a functional manner. Below is an example of how to use it:

from phaistos import Transpiler
from phaistos.typings import TranspiledSchema

schema_read_from-data = ... # Read the schema from a file or something

transpiled_schema: TranspiledSchema = Transpiler.make_schema(schema_read_from_data)

After that, the TranspiledSchema can be manually invoked via validate method, which takes the data to be validated as an argument:

data = ... # Read the data from somewhere

transpiled_schema.validate(data)

Congratulations! You have performed the thing that the Manager class does in the background. Let's check it out!