Foundry Reference Project | Data Pipeline

Nothing much happens in Palantir Foundry without data. Before you can build applications or answer novel questions, you need to bring data into Foundry and clean, enrich, and structure it for later use.

In this video we review the principle concepts behind a data pipeline through the examples in the Foundry Reference Project. We’ll see a few code snippets, but mostly focus on how our transform logic links together datasets to form a pipeline and how we use schedules to control and monitor its execution.

The Reference Project is shared on every Foundry instance. It includes PySpark and SparkSQL transform implementation, an ontology demonstrating different Object Types and their relations alongside Actions for capturing user writeback, and example Applications built in Workshop and Slate.

Watch the next video in the series to learn about building an ontology data model: Foundry Reference Project | Ontology