From 427b5020eb94f5e32a75ad117f22bd4e1184edc7 Mon Sep 17 00:00:00 2001 From: Yoav Cohen Date: Thu, 9 Apr 2026 09:03:35 +0200 Subject: [PATCH 1/5] Add AGENTS.md to provide guidelines to coding agents --- AGENTS.md | 40 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 40 insertions(+) create mode 100644 AGENTS.md diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 000000000..42b0455c1 --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,40 @@ +# SQL Parser in Rust + +## Agent Workflow +1. You will receive a SQL statement that is not parsed correctly by the project. Typically parsing will fail with an error. +2. You will add a unit test to check that the SQL statement is indeed unparsed. Follow the instructions in the "Unit Tests" section below. +3. You will analyze why the SQL statement is not parsed correctly, pointing to the code section that you think is faulty. See more info in the Analyzing Parsing Issues and General Coding Guidelines below. +4. You will fix the parsing issue and ensure that the new unit test passes successfully by running `cargo test --all-features`. +5. You will remove the first unit test that you added in step 2 and instead create "synthetic" unit tests to cover the code sections that you modified, to ensure that similar parsing issues are caught in the future. +6. Run the commands in the Pre Commit Checks section below to ensure that all unit tests are passing and that the code is ready for a pull request. +7. You will create a PR in the with the changes you made, follow the instructions on Pull Request Guidelines below. + +## General Coding Guidelines +1. Refrain from adding conditions on specific dialects, such as `dialect_id!(self is SnowflakeDialect)` or `dialect_of!(self is SnowflakeDialect | BigQueryDialect | MySqlDialect | HiveDialect)`. Instead, define a new function in the `Dialect` trait that describes the condition, so that dialects can turn this condition on more easily. +2. Make targeted code changes and refrain from refactoring, unless it's absolutely required. + +## Unit Tests +Follow these rules: +- When testing a multi-line SQL statement, use a raw string literal (r#"..."#) to preserve formatting. +- You can use the following template for simple unit tests: +```rust +().parse_sql_statements(r#"..."#).unwrap(); +``` +For example: `snowflake().parse_sql_statements(r#"SELECT * FROM my_table;"#).unwrap();` + +- New unit tests should be added to the `tests` module in the corresponding dialect file (e.g., `src/tests/sqlparser_redshift.rs` for Redshift), and should be placed at the end of the file. + +## Analyzing Parsing Issues +You can try to simplify the SQL statement to identify the root cause of the parsing issue. This may involve removing certain clauses or components of the SQL statement to see if it can be parsed successfully. Additionally, you can compare the problematic SQL statement with similar statements that are parsed correctly to identify any differences that may be causing the issue. + +## Pre Commit Checks +Run the following commands before you commit to ensure the change will pass the CI process: +```bash +cargo test --all-features +cargo fmt --all +cargo clippy --all-targets --all-features -- -D warnings +``` + +## Pull Request Guidelines +1. PR title should follow this format: `: `, For example, `Showflake: Add support for casting to VARIANT`. +2. Make the PR comment short, provide an example of what was not working and a short description of the fix. Be succint. From ee617f02d859f275f2b33964ad36318e32916fd7 Mon Sep 17 00:00:00 2001 From: Yoav Cohen Date: Thu, 9 Apr 2026 09:09:09 +0200 Subject: [PATCH 2/5] Add AGENTS.md to provide guidelines to coding agents --- AGENTS.md | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 42b0455c1..c00836471 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -1,6 +1,13 @@ -# SQL Parser in Rust +# Extensible SQL Lexer and Parser for Rust Agents Guidelines -## Agent Workflow +## General Agent Workflow +1. You will receive a coding task from your operator. +2. You will analyze the task and come up with a code change to complete it. +3. You will write unit tests to ensure your code change is working as expected. +4. You will run the commands in the Pre Commit Checks section below to ensure your change is ready for a pull request. +5. When instructed to open a PR, you will follow the instructions in the Pull Request Guidelines section below. + +## Agent Workflow for Fixing Parsing Issues in SQL Statements 1. You will receive a SQL statement that is not parsed correctly by the project. Typically parsing will fail with an error. 2. You will add a unit test to check that the SQL statement is indeed unparsed. Follow the instructions in the "Unit Tests" section below. 3. You will analyze why the SQL statement is not parsed correctly, pointing to the code section that you think is faulty. See more info in the Analyzing Parsing Issues and General Coding Guidelines below. From 9687a568bb1a139b313338acab0eb443934d1684 Mon Sep 17 00:00:00 2001 From: Yoav Cohen Date: Fri, 10 Apr 2026 15:12:05 +0200 Subject: [PATCH 3/5] Feedback on AGENTS.md --- AGENTS.md | 30 +++++++++--------------------- 1 file changed, 9 insertions(+), 21 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index c00836471..e27164e2b 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -1,35 +1,23 @@ # Extensible SQL Lexer and Parser for Rust Agents Guidelines ## General Agent Workflow -1. You will receive a coding task from your operator. -2. You will analyze the task and come up with a code change to complete it. -3. You will write unit tests to ensure your code change is working as expected. -4. You will run the commands in the Pre Commit Checks section below to ensure your change is ready for a pull request. -5. When instructed to open a PR, you will follow the instructions in the Pull Request Guidelines section below. - -## Agent Workflow for Fixing Parsing Issues in SQL Statements -1. You will receive a SQL statement that is not parsed correctly by the project. Typically parsing will fail with an error. -2. You will add a unit test to check that the SQL statement is indeed unparsed. Follow the instructions in the "Unit Tests" section below. -3. You will analyze why the SQL statement is not parsed correctly, pointing to the code section that you think is faulty. See more info in the Analyzing Parsing Issues and General Coding Guidelines below. -4. You will fix the parsing issue and ensure that the new unit test passes successfully by running `cargo test --all-features`. -5. You will remove the first unit test that you added in step 2 and instead create "synthetic" unit tests to cover the code sections that you modified, to ensure that similar parsing issues are caught in the future. -6. Run the commands in the Pre Commit Checks section below to ensure that all unit tests are passing and that the code is ready for a pull request. -7. You will create a PR in the with the changes you made, follow the instructions on Pull Request Guidelines below. +1. You will write unit tests to ensure your code change is working as expected. +2. You will run the commands in the Pre Commit Checks section below to ensure your change is ready for a pull request. +3. When instructed to open a PR, you will follow the instructions in the Pull Request Guidelines section below. ## General Coding Guidelines -1. Refrain from adding conditions on specific dialects, such as `dialect_id!(self is SnowflakeDialect)` or `dialect_of!(self is SnowflakeDialect | BigQueryDialect | MySqlDialect | HiveDialect)`. Instead, define a new function in the `Dialect` trait that describes the condition, so that dialects can turn this condition on more easily. +1. Refrain from adding conditions on specific dialects, such as `dialect_is!(...)` or `dialect_of!(... | ...)`. Instead, define a new function in the `Dialect` trait that describes the condition, so that dialects can turn this condition on more easily. 2. Make targeted code changes and refrain from refactoring, unless it's absolutely required. -## Unit Tests -Follow these rules: -- When testing a multi-line SQL statement, use a raw string literal (r#"..."#) to preserve formatting. +## Unit Tests Guidelines +- When testing a multi-line SQL statement, use a raw string literal, i.e. `r#"..."#` to preserve formatting. - You can use the following template for simple unit tests: ```rust ().parse_sql_statements(r#"..."#).unwrap(); ``` -For example: `snowflake().parse_sql_statements(r#"SELECT * FROM my_table;"#).unwrap();` - -- New unit tests should be added to the `tests` module in the corresponding dialect file (e.g., `src/tests/sqlparser_redshift.rs` for Redshift), and should be placed at the end of the file. +For example: `snowflake().parse_sql_statements(r#"SELECT * FROM my_table"#).unwrap();` +- New unit tests should be added to the `tests` module in the corresponding dialect file (e.g., `tests/sqlparser_redshift.rs` for Redshift), and should be placed at the end of the file. +- If the new functionality is gated using a dialect function, and the SQL is likely relevant in most dialects, tests should be placed under `tests/sqlparser_common.rs`. ## Analyzing Parsing Issues You can try to simplify the SQL statement to identify the root cause of the parsing issue. This may involve removing certain clauses or components of the SQL statement to see if it can be parsed successfully. Additionally, you can compare the problematic SQL statement with similar statements that are parsed correctly to identify any differences that may be causing the issue. From d36f9a692aeedd9352db61480d36723b65c89c94 Mon Sep 17 00:00:00 2001 From: Yoav Cohen Date: Wed, 15 Apr 2026 05:57:42 -0400 Subject: [PATCH 4/5] Unit test improvements --- AGENTS.md | 16 +++++++++++----- 1 file changed, 11 insertions(+), 5 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index e27164e2b..cb8033c55 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -10,14 +10,20 @@ 2. Make targeted code changes and refrain from refactoring, unless it's absolutely required. ## Unit Tests Guidelines +- New unit tests should be added to the `tests` module in the corresponding dialect file (e.g., `tests/sqlparser_redshift.rs` for Redshift), and should be placed at the end of the file. +- If the new functionality is gated using a dialect function, and the SQL is likely relevant in most dialects, tests should be placed under `tests/sqlparser_common.rs`. - When testing a multi-line SQL statement, use a raw string literal, i.e. `r#"..."#` to preserve formatting. -- You can use the following template for simple unit tests: +- The parser builds an abstract syntax tree (AST) from the SQL statement and has functionality to display the tree as SQL. Use the following template for simple unit tests where you expect the SQL created from the AST to be the same as the input SQL: ```rust -().parse_sql_statements(r#"..."#).unwrap(); +().verified_stmt(r#"..."#); +``` +For example: `snowflake().verified_stmt(r#"SELECT * FROM my_table"#)`. Use `one_statement_parses_to` instead of `verified_stmt` when you expect the SQL created by the AST to differ than the input SQL. For example: +```rust +snowflake().one_statement_parses_to( + "SELECT * FROM my_table t", + "SELECT * FROM my_table AS t", +) ``` -For example: `snowflake().parse_sql_statements(r#"SELECT * FROM my_table"#).unwrap();` -- New unit tests should be added to the `tests` module in the corresponding dialect file (e.g., `tests/sqlparser_redshift.rs` for Redshift), and should be placed at the end of the file. -- If the new functionality is gated using a dialect function, and the SQL is likely relevant in most dialects, tests should be placed under `tests/sqlparser_common.rs`. ## Analyzing Parsing Issues You can try to simplify the SQL statement to identify the root cause of the parsing issue. This may involve removing certain clauses or components of the SQL statement to see if it can be parsed successfully. Additionally, you can compare the problematic SQL statement with similar statements that are parsed correctly to identify any differences that may be causing the issue. From d7334baa53a102a61b231623ce97a1869d9f6c14 Mon Sep 17 00:00:00 2001 From: Yoav Cohen Date: Wed, 15 Apr 2026 06:20:21 -0400 Subject: [PATCH 5/5] Add AGENTS.md to excluded RAT files --- dev/release/rat_exclude_files.txt | 1 + 1 file changed, 1 insertion(+) diff --git a/dev/release/rat_exclude_files.txt b/dev/release/rat_exclude_files.txt index 280b1bce6..b8b7b81d0 100644 --- a/dev/release/rat_exclude_files.txt +++ b/dev/release/rat_exclude_files.txt @@ -6,3 +6,4 @@ dev/release/rat_exclude_files.txt sqlparser_bench/img/flamegraph.svg **Cargo.lock filtered_rat.txt +AGENTS.md \ No newline at end of file