Applications & Data · Analytics
Data Lakes
Large repositories of raw data in open formats on object storage, queried in place by analytics engines.
IntermediateUpdated 2026-09-01
Overview
Lakes accept structured, semi structured and unstructured data without upfront modelling, which preserves optionality. Table formats such as Delta, Iceberg and Hudi add transactions, time travel and schema evolution on top of the files.
Without catalogue, ownership and lifecycle rules, a lake becomes an expensive, ungoverned dumping ground, the common failure mode of these projects.
Security considerations
- Enforce access at the catalogue and storage layers, and encrypt with customer managed keys where required.
- Classify data on ingestion; retrofitting classification across petabytes rarely happens.