Skip to content

Transform Stage

The Transform stage is the final step in the Pipe lifecycle. It is reached only if all Conditions and Matches have passed.

It modifies the value into its final form before it is returned by the pipeline or passed to your function.

Data Integrity

Because Transformation happens last, you are guaranteed to be transforming data that has already been verified as valid, preventing errors like trying to .strip() an Integer or a None value.


Setup vs Transform: When to Use Each

Both setup and transform use transform handlers, but they run at different stages of the Pipe lifecycle:

Feature Setup Transform
When it runs After type validation, before conditions/matches After all validations pass (final stage)
Purpose Data preparation and normalization Data modification and formatting
Validation Only type is validated All conditions and matches have passed
Use case Strip whitespace, normalize input Format output, apply business logic

Example: Setup for Preparation

Use setup when you need to normalize data before validation:

from pipeline import Pipe

# Setup strips whitespace BEFORE checking email format
result = Pipe(
    value="  user@example.com  ",
    type=str,
    setup={Pipe.Transform.Strip: None},  # Runs first
    matches={Pipe.Match.Format.Email: None},  # Validates cleaned value
).run()

print(result.value)  # "user@example.com" (no spaces)

Example: Transform for Formatting

Use transform when you need to modify data after validation:

from pipeline import Pipe

# Transform lowercases AFTER validation passes
result = Pipe(
    value="User@Example.COM",
    type=str,
    matches={Pipe.Match.Format.Email: None},  # Validates first
    transform={Pipe.Transform.Lowercase: None}  # Formats after validation
).run()

print(result.value)  # "user@example.com"

Combining Setup and Transform

You can use both together for a complete data processing pipeline:

from pipeline import Pipe

result = Pipe(
    value="  User@Example.COM  ",
    type=str,
    setup={Pipe.Transform.Strip: None},  # 1. Clean input
    conditions={Pipe.Condition.MaxLength: 50},  # 2. Validate structure
    matches={Pipe.Match.Format.Email: None},  # 3. Validate format
    transform={Pipe.Transform.Lowercase: None}  # 4. Format output
).run()

print(result.value)  # "user@example.com"

Setup vs Pre-Hook

While pre_hook can perform the same data preparation as setup, the setup argument is a developer-friendly simplification. Important: setup is per-pipe (field-specific), while pre_hook is global (applies to all fields in a Pipeline). Use setup for field-specific transformations and pre_hook for global logic or complex custom logic.


Advanced: Custom Transformation Logic

While transformations are powerful, sometimes you need custom logic. Use hooks for this:

from pipeline import Pipeline, Pipe

# Create pipeline
user_pipeline = Pipeline(
    name={"type": str},
    email={"type": str}
)

# Add custom transformation via pre_hook
def custom_transform(hook):
    current_value = hook.value.get
    if hook.field == "name":
        # Custom logic: Title case
        hook.value.set(current_value.title())
    elif hook.field == "email":
        # Custom logic: Lowercase and strip
        hook.value.set(current_value.lower().strip())

user_pipeline.pre_hook = custom_transform

result = user_pipeline.run(data={
    "name": "john doe",
    "email": "  User@Example.COM  "
})

print(result.processed_data)
# {
#     'name': 'John Doe',
#     'email': 'user@example.com'
# }

Best Practices

Order Matters

Transformations are applied in the order they're defined:

transform={
    Pipe.Transform.Strip: None,      # 1. Remove whitespace
    Pipe.Transform.Lowercase: None,  # 2. Convert to lowercase
    Pipe.Transform.Capitalize: None  # 3. Capitalize
}

Combine with Validation

Always validate before transforming:

email={
    "type": str,
    "conditions": {Pipe.Condition.MaxLength: 64},  # Validate
    "matches": {Pipe.Match.Format.Email: None},    # Validate
    "transform": {Pipe.Transform.Lowercase: None}  # Transform
}

Type Safety

Transformations only run if validation passes, ensuring type safety:

# If value is not a string, Lowercase won't run
Pipe(
    value=123,  # Wrong type
    type=str,
    transform={Pipe.Transform.Lowercase: None}
)

Technical Reference

The following section is automatically generated from the source code, detailing transformation handlers like string manipulation, math operations, and unique filtering.

Transform

Central registry for all transform handlers.

Transform handlers modify the value in some way, such as changing case, replacing substrings, or performing arithmetic operations.

Source code in pipeline/handlers/transform_handler/transform.py
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
class Transform:
    """
    Central registry for all transform handlers.

    Transform handlers modify the value in some way, such as changing case,
    replacing substrings, or performing arithmetic operations.
    """
    class Strip(TransformHandler[str, Optional[str]]):
        """Removes leading and trailing whitespace from a string"""
        SUPPORT = (HandlerMode.ROOT, HandlerMode.ITEM)

        def operation(self):
            return self.value.strip()

    class Capitalize(TransformHandler[str, None]):
        """Converts the first character to uppercase and the rest to lowercase"""
        SUPPORT = (HandlerMode.ROOT, HandlerMode.ITEM)

        def operation(self):
            return self.value.capitalize()

    class Lowercase(TransformHandler[str, None]):
        """Converts all characters in the string to lowercase"""
        SUPPORT = (HandlerMode.ROOT, HandlerMode.ITEM)

        def operation(self):
            return self.value.lower()

    class Uppercase(TransformHandler[str, None]):
        """Converts all characters in the string to uppercase"""
        SUPPORT = (HandlerMode.ROOT, HandlerMode.ITEM)

        def operation(self):
            return self.value.upper()

    class Multiply(TransformHandler[str | list | int | float, int | float]):
        """
        Multiplies a string, list, integer or float by the provided argument

        If the value is not numeric, the argument is rounded before multiplication.
        """
        SUPPORT = (HandlerMode.ROOT, HandlerMode.ITEM)

        def operation(self):
            if not isinstance(self.value, (int, float)):
                return self.value * round(self.argument)

            return self.value * self.argument

    class Reverse(TransformHandler[str | list, None]):
        """Reverses the order of characters in a string or items in a list"""
        SUPPORT = (HandlerMode.ROOT, HandlerMode.ITEM)

        def operation(self):
            return self.value[::-1]

    class Title(TransformHandler[str, None]):
        """Converts the first character of every word to uppercase"""
        SUPPORT = (HandlerMode.ROOT, HandlerMode.ITEM)

        def operation(self):
            return self.value.title()

    class SnakeCase(TransformHandler[str, None]):
        """Converts strings to snake_case (e.g., 'HelloWorld' -> 'hello_world')"""
        SUPPORT = (HandlerMode.ROOT, HandlerMode.ITEM)

        def operation(self):
            s = re.sub(r'(?<!^)(?=[A-Z])', '_', self.value).lower()

            return s.replace("-", "_").replace(" ", "_")

    class Unique(TransformHandler[list, None]):
        """Removes duplicate items from a list while preserving order"""
        SUPPORT = (HandlerMode.ROOT, HandlerMode.ITEM)

        def operation(self):
            return list(dict.fromkeys(self.value))

    class Replace(TransformHandler[str, tuple]):
        """Replaces all occurrences of a substring with another (old, new)"""
        SUPPORT = (HandlerMode.ROOT, HandlerMode.ITEM)

        def operation(self):
            old, new = self.argument

            return self.value.replace(old, new)

    class Apply(TransformHandler[Any, Callable]):
        """
        Transforms the value by passing it through the provided callable.

        The callable should take only one argument (the value) and return the 
        transformed value. No checks are performed on the returned value, 
        so ensure it meets your specific requirements.
        """
        SUPPORT = (HandlerMode.ROOT, HandlerMode.ITEM)

        def operation(self):
            return self.argument(self.value)

Apply

Bases: TransformHandler[Any, Callable]

Transforms the value by passing it through the provided callable.

The callable should take only one argument (the value) and return the transformed value. No checks are performed on the returned value, so ensure it meets your specific requirements.

Source code in pipeline/handlers/transform_handler/transform.py
 97
 98
 99
100
101
102
103
104
105
106
107
108
class Apply(TransformHandler[Any, Callable]):
    """
    Transforms the value by passing it through the provided callable.

    The callable should take only one argument (the value) and return the 
    transformed value. No checks are performed on the returned value, 
    so ensure it meets your specific requirements.
    """
    SUPPORT = (HandlerMode.ROOT, HandlerMode.ITEM)

    def operation(self):
        return self.argument(self.value)

Capitalize

Bases: TransformHandler[str, None]

Converts the first character to uppercase and the rest to lowercase

Source code in pipeline/handlers/transform_handler/transform.py
23
24
25
26
27
28
class Capitalize(TransformHandler[str, None]):
    """Converts the first character to uppercase and the rest to lowercase"""
    SUPPORT = (HandlerMode.ROOT, HandlerMode.ITEM)

    def operation(self):
        return self.value.capitalize()

Lowercase

Bases: TransformHandler[str, None]

Converts all characters in the string to lowercase

Source code in pipeline/handlers/transform_handler/transform.py
30
31
32
33
34
35
class Lowercase(TransformHandler[str, None]):
    """Converts all characters in the string to lowercase"""
    SUPPORT = (HandlerMode.ROOT, HandlerMode.ITEM)

    def operation(self):
        return self.value.lower()

Multiply

Bases: TransformHandler[str | list | int | float, int | float]

Multiplies a string, list, integer or float by the provided argument

If the value is not numeric, the argument is rounded before multiplication.

Source code in pipeline/handlers/transform_handler/transform.py
44
45
46
47
48
49
50
51
52
53
54
55
56
class Multiply(TransformHandler[str | list | int | float, int | float]):
    """
    Multiplies a string, list, integer or float by the provided argument

    If the value is not numeric, the argument is rounded before multiplication.
    """
    SUPPORT = (HandlerMode.ROOT, HandlerMode.ITEM)

    def operation(self):
        if not isinstance(self.value, (int, float)):
            return self.value * round(self.argument)

        return self.value * self.argument

Replace

Bases: TransformHandler[str, tuple]

Replaces all occurrences of a substring with another (old, new)

Source code in pipeline/handlers/transform_handler/transform.py
88
89
90
91
92
93
94
95
class Replace(TransformHandler[str, tuple]):
    """Replaces all occurrences of a substring with another (old, new)"""
    SUPPORT = (HandlerMode.ROOT, HandlerMode.ITEM)

    def operation(self):
        old, new = self.argument

        return self.value.replace(old, new)

Reverse

Bases: TransformHandler[str | list, None]

Reverses the order of characters in a string or items in a list

Source code in pipeline/handlers/transform_handler/transform.py
58
59
60
61
62
63
class Reverse(TransformHandler[str | list, None]):
    """Reverses the order of characters in a string or items in a list"""
    SUPPORT = (HandlerMode.ROOT, HandlerMode.ITEM)

    def operation(self):
        return self.value[::-1]

SnakeCase

Bases: TransformHandler[str, None]

Converts strings to snake_case (e.g., 'HelloWorld' -> 'hello_world')

Source code in pipeline/handlers/transform_handler/transform.py
72
73
74
75
76
77
78
79
class SnakeCase(TransformHandler[str, None]):
    """Converts strings to snake_case (e.g., 'HelloWorld' -> 'hello_world')"""
    SUPPORT = (HandlerMode.ROOT, HandlerMode.ITEM)

    def operation(self):
        s = re.sub(r'(?<!^)(?=[A-Z])', '_', self.value).lower()

        return s.replace("-", "_").replace(" ", "_")

Strip

Bases: TransformHandler[str, Optional[str]]

Removes leading and trailing whitespace from a string

Source code in pipeline/handlers/transform_handler/transform.py
16
17
18
19
20
21
class Strip(TransformHandler[str, Optional[str]]):
    """Removes leading and trailing whitespace from a string"""
    SUPPORT = (HandlerMode.ROOT, HandlerMode.ITEM)

    def operation(self):
        return self.value.strip()

Title

Bases: TransformHandler[str, None]

Converts the first character of every word to uppercase

Source code in pipeline/handlers/transform_handler/transform.py
65
66
67
68
69
70
class Title(TransformHandler[str, None]):
    """Converts the first character of every word to uppercase"""
    SUPPORT = (HandlerMode.ROOT, HandlerMode.ITEM)

    def operation(self):
        return self.value.title()

Unique

Bases: TransformHandler[list, None]

Removes duplicate items from a list while preserving order

Source code in pipeline/handlers/transform_handler/transform.py
81
82
83
84
85
86
class Unique(TransformHandler[list, None]):
    """Removes duplicate items from a list while preserving order"""
    SUPPORT = (HandlerMode.ROOT, HandlerMode.ITEM)

    def operation(self):
        return list(dict.fromkeys(self.value))

Uppercase

Bases: TransformHandler[str, None]

Converts all characters in the string to uppercase

Source code in pipeline/handlers/transform_handler/transform.py
37
38
39
40
41
42
class Uppercase(TransformHandler[str, None]):
    """Converts all characters in the string to uppercase"""
    SUPPORT = (HandlerMode.ROOT, HandlerMode.ITEM)

    def operation(self):
        return self.value.upper()