Skip to content

[flink] Push down lake filters for non-partitioned scans#3169

Open
luoyuxia wants to merge 2 commits intoapache:mainfrom
luoyuxia:flink-fix-lake-filter-pushdown
Open

[flink] Push down lake filters for non-partitioned scans#3169
luoyuxia wants to merge 2 commits intoapache:mainfrom
luoyuxia:flink-fix-lake-filter-pushdown

Conversation

@luoyuxia
Copy link
Copy Markdown
Contributor

Purpose

Linked issue: close #3168

Allow FlinkTableSource to push filters into LakeSource for non-partitioned table scans. Before this change, lake filter pushdown was nested under isPartitioned(), so non-partitioned scans in lake-backed FULL reads always fell back to Flink-side filtering for historical data.

Brief change log

  • extract lake filter pushdown into a dedicated helper and invoke it whenever lakeSource != null
  • preserve the existing fallback to Flink-side filtering when the lake source rejects part of the predicate set
  • add unit coverage for non-partitioned lake pushdown and the fallback path

Tests

  • ./mvnw -pl fluss-flink/fluss-flink-common -DskipITs test

API and Format

No API or storage format changes.

Documentation

No documentation changes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[flink] Lake filters shoud be pushed down for non-partitioned scans

1 participant