Lakeflow Jobs and orchestration · Lesson 2 of 3
Job parameters, schedules, and triggers
Pass runtime values and start workflows on a schedule or event.
Lesson
Job parameters make workflows reusable across dates, environments, or source partitions. A job-level default can be overridden by a user with sufficient run permissions, so a parameter is not a security control. Validate parameter values in the task and use permissions to protect sensitive operations. Schedules are appropriate for periodic work; event-based triggers can start work when an upstream event occurs if the source and workspace support them.
Use time zones and date boundaries deliberately, especially around daylight-saving changes and late data. Make backfill behavior explicit so a manual run does not accidentally overwrite a broad date range. Practice task: define a date parameter for a daily job and describe how to safely run a one-day backfill.
Applying the pattern: Define parameters and defaults, configure a supported trigger, validate inputs in tasks, and review effective values for each run.
Checks and failure modes: A default parameter value does not prevent an authorized user from overriding it.
Before production, test both a representative input and a boundary case. Confirm the expected row grain, permissions, recovery behavior, and observable output for the task. Record the configuration needed to reproduce the run and compare its result with the agreed data contract.
Practice task: Are job parameter defaults a security boundary?
Visual guide
A parameterized scheduled run
-
Step 01
Trigger
Schedule or event starts the job.
-
Step 02
Resolve
Set defaults and permitted overrides.
-
Step 03
Validate
Check values in the task.
-
Step 04
Execute
Run the workflow for that input.
Code examples
Use a runtime date parameter
SQL
SELECT *
FROM main.sales.bronze_orders
WHERE order_date = DATE(:run_date);Expected output
The task uses an explicit run date rather than a hidden notebook default.
Key points
- Define parameters and defaults, configure a supported trigger, validate inputs in tasks, and review effective values for each run.
- A default parameter value does not prevent an authorized user from overriding it.
- Re-run the included practice scenario after changing source data or transformation logic.
Check your understanding
Practice
Your learning progress