Flatten & Explode

Sparkflows provides the Flatten & Explode nodes to extract the columns from complex data types like struct type & array type.

Flatten: Creates the single columns from struct.

Explode: Explode an array columns to rows.

Nodes in fire:

NodeFlatten and NodeExplode

---

Example Data

---

Schema

---

Explode the Topping Column

Explode the array of topping column into rows with new column as explode_topping

---

Flatten the batters and explode_topping

---

Final Output

After dropping the few columns