Jobs, tasks, and DAG dependencies

Lakeflow Jobs orchestrate repeatable workflows made of tasks such as notebooks, pipelines, SQL queries, and scripts. Dependencies form a directed acyclic graph (DAG): a downstream task becomes eligible after its prerequisites complete successfully. Independent branches can run concurrently when resources and design allow. Keep task boundaries meaningful so retries and monitoring isolate failures rather than rerunning unrelated work. Pass outputs or task values through documented mechanisms and make each task's inputs and side effects clear. A job graph should show the actual data dependency, not merely the order someone usually clicks buttons. Practice task: draw a DAG for ingest, validate, transform, publish, and notify tasks, including the failure branch for validation. Applying the pattern: Break work into testable tasks, connect true dependencies, set run identity and resources, and inspect task-level logs after a failed run. Checks and failure modes: A hidden dependency on notebook execution order can make a scheduled job behave differently from manual runs. Before production, test both a representative input and a boundary case. Confirm the expected row grain, permissions, recovery behavior, and observable output for the task. Record the configuration needed to reproduce the run and compare its result with the agreed data contract. Practice task: Why model actual prerequisites as job dependencies?