Scala

In this article, I share my critical view on the current state of data engineering, dominated by heavyweight platforms like Spark and Databricks, and introduce Sail, an open-source engine built on top of Apache Arrow and DataFusion, written in Rust, that offers a new path: lightweight, efficient, and powerful.

We explore in detail what typeclasses are, how to define them in Scala 2, and how they are implemented in other languages such as Haskell and Rust. Through practical examples, we demonstrate how to model behaviors clearly and scalably using this powerful design pattern in functional programming. We also discuss automatic derivation and the improvements Scala 3 brings to this area.

We analyzed different functional pattern designs to address the abstraction of a functionality and its implementation.

My first impressions of Rust from a functional programming background (Scala and Haskell). A mix of excitement, frustration, and a paradigm shift in thinking.

Defining the serializer and transformation from Yaml to Core

Defining the model core from Scala to Spark

Exploring how to create Apache Spark ETL workflows with zero code.

Modify object properties in Scala without losing immutability. Explore the copy pattern to create new instances with specific changes while maintaining safety and correctness. A practical and elegant solution, perfect for concurrent and reactive systems.