PR Manager — Automated Code Review Tools

This folder contains CLI tools to assist with automated pull request reviews in Azure DevOps. The Test Script Review tool focuses on Playwright and TestNG tests and posts inline comments and a summary directly to the PR.

Main scripts:

• PR_Review_Manager.sh — High-level entry point for PR review workflo(Orchestrator)
• Test_Script_Review.sh — Automated review for Playwright and TestNG test code
• Backend_code_review.sh — Backend-specific review checks
• Frontend_code_review.sh — Frontend-specific review checks

Prerequisites

• OS: Linux or macOS with Bash
• Tools: curl, git, grep, sed, awk
• Python 3 (optional but recommended) for robust JSON escaping
• Azure DevOps Personal Access Token (PAT) with sufficient permissions to read PRs and post comments
• Network access to Azure DevOps API endpoints

Note: The Test Script Review tool does not run tests. It analyzes changed files in a PR and posts suggestions/comments.

Configuration

1. Copy the environment file template and fill in your values: cp pr-manager/.env.example pr-manager/.env

2. Edit pr-manager/.env and set:

• AZURE_ORG: e.g. https://dev.azure.com/
• PROJECT: Azure DevOps project name
• REPO: Repository full name or ID (the scripts also infer it via the local git repo)
• AZURE_TOKEN: your Azure DevOps PAT
• API_VERSION: optional (default 6.0)
• DEBUG_MODE: optional, set to true for verbose logs

3. env_loader.sh will load and validate variables automatically when any script runs.

Running the Test Script Review

From repository root:

• cd pr-manager
• bash Test_Script_Review.sh

You will be prompted for:

1. PR ID or URL — Paste the Azure DevOps PR link or enter the numeric ID.
2. Test framework — Choose 1) Playwright or 2) TestNG.
3. File selection — The tool lists changed files; you can select:
   • Enter: all files
   • Single index: e.g., 3
   • Multiple indices: e.g., 1,4,7
   • Ranges: e.g., 2-5 or combined like 1,3-5,9

The tool posts:

• A single review summary thread on the PR
• Inline comments on specific lines in selected files

How files are categorized and reviewed

The tool categorizes files and applies targeted guidelines per category. Below are the categories and the main checks performed.

Playwright

Categories are determined from filenames and paths:

• functional_test — *.spec.ts or *.spec.js

  • Hard waits: waitForTimeout, setTimeout
  • Test structure: missing test.describe, test.beforeEach
  • Locators: avoid getByText/text=; prefer getByRole/getByTestId
  • Assertions: missing expect()
  • Isolation: remove test.only

• utility — Files under utils/helpers or names starting with util/helper

  • Exports: ensure functions/classes are exported
  • Docs: basic JSDoc for @param/@returns
  • Error handling: try/catch around async work
  • Reusability: avoid hardcoded values/URLs/IPs
  • Type safety (TS): avoid any

• config — playwright.config.ts/js

  • testDir is configured
  • Browser projects include chromium/firefox/webkit
  • Reporter configured
  • Retries/parallel settings
  • Artifacts: screenshot/video/trace
  • baseURL configured

• page_object — Files under pages/ or *Page.ts/js

  • Class structure
  • Locator definitions as properties
  • Action methods
  • Constructor with page
  • Stable locators (avoid text-based)

• general — Fallback (basic hard-wait check)

TestNG

Categorization uses path and content:

• testng_test_class — Files that contain @Test or are test classes under integration_tests_suite

  • Safety: warn on attempts to reference/modify production code paths (e.g., src/main/java)
  • Avoid overriding existing test methods/classes
  • Test isolation: avoid modifying static state or system properties
  • Hard waits: Thread.sleep, TimeUnit.sleep
  • Assertions: missing Assert/assertThat/SoftAssert
  • Setup/cleanup: suggest @BeforeMethod/@AfterMethod and @BeforeClass/@AfterClass
  • Groups: suggest groups = {"integration","smoke","regression"}
  • Naming: encourage descriptive method names
  • Data-driven: prefer @DataProvider over loops in tests
  • Dependencies: caution when using dependsOnMethods/dependsOnGroups

• testng_utility — utils/helpers/Utils/Helper classes or static utility patterns

  • Safety: utilities should not modify production code
  • Avoid overriding existing utilities or extending production classes
  • Global state: avoid System.setProperty/setOut/setErr and mutable statics
  • Docs: JavaDoc with @param/@return
  • Static methods: consider static for utilities
  • Error handling: catch and meaningful exceptions
  • Reusability: avoid hardcoded URLs/IPs/ports
  • Null handling: avoid return null; consider Optional/defaults
  • Logging: add appropriate logging
  • Naming: avoid test prefix in utility methods

• testng_general — Fallback (basic hard-wait check)

Posting behavior:

• If no specific issues are found, the tool posts a single inline fallback tip tailored to the category at the last line of the file.

Troubleshooting

• Connectivity failed: ensure .env values are correct and PAT is valid.
• No changed files found: the tool only analyzes files in the current iteration of the PR; ensure the PR has changes and you have permissions.
• Large PRs: you can select only the target files using indices or ranges for faster runs.
• Debugging: set DEBUG_MODE=true in .env to log API endpoints and payloads.

Notes

• The tool uses the latest PR iteration so comments map to the current diff.
• Comments are JSON-escaped safely; Python 3 is used when available.
• The scripts don’t modify your repository state; they call Azure DevOps APIs to post comments.

Next steps

• To expand automated checks, extend the categorize_* and generate_*_comments functions in Test_Script_Review.sh.
• See PR_Review_Manager.sh, Backend_code_review.sh, and Frontend_code_review.sh for additional review flows.