What is Transformer Stage in datstage ?

The Transformer stage is a processing stage used to transform individual input rows into output rows. It allows you to write complex expressions, apply functions, use variables, and route data to different links based on conditions.

---

🔧 Key Features

1. Row-by-row processing

Evaluates each row individually.

2. Derivations / Expressions

You can derive new columns using:

• String functions (Trim, Substring, etc.)

• Date/time functions

• Lookup functions

• Conditional expressions (If…Then…Else)

3. Constraints

Used to filter and control the flow of data to each output link.

Example:

If Amount > 1000 Then 1 Else 0

4. Stage Variables

• Used for complex logic.

• Can store intermediate calculations.

• Can maintain values across rows.

Example: Running total or previous row value.

5. Multiple Output Links

You can split data into multiple outputs based on constraints.
Example:

• Valid Records

• Rejected Records

• Error Records

6. Built-in Lookup

You can reference a reference link directly inside derivations.

---

When to Use Transformer Stage?

Use it when you need:

• Data validation

• Complex business logic

• Conditional filtering

• Field derivations

• Reference lookups (small/moderate datasets)

---

🚫 What Transformer Should NOT Be Used For

• Very large dataset joins (use Join/Merge stage instead)

• Column-level mass transformations (use Modify stage)

• High-performance parallel logic (Transformer is slower)

---

Example Use Case

Source: Customer records
Logic:

• If Age > 18 → mark “Adult”

• If Country = ‘IN’ → output to India link

• Else → output to Others link

All of this is done inside the Transformer stage.

---