Migrate integration and provider HTTP types (PR 13)

[prk-Jr]

Summary

• Replace the remaining Fastly request and response boundaries in the integration and auction provider layers so PR13 runs on platform-agnostic HTTP types.
• Move provider dispatch onto RuntimeServices HTTP client primitives with async request_bids, PlatformPendingRequest, and PlatformResponse while preserving orchestrator deadline handling.
• Finish the review hardening for Testlight response normalization and convert the migrated integration tests to HTTP-native fixtures.

Changes

File	Change
crates/trusted-server-adapter-fastly/src/main.rs	Route integration proxy requests through the HTTP-native registry boundary and pass RuntimeServices into the PR13 entrypoint.
crates/trusted-server-core/src/auction/README.md	Update provider guidance and examples to PlatformPendingRequest, PlatformResponse, and the platform HTTP client flow.
crates/trusted-server-core/src/auction/endpoints.rs	Thread RuntimeServices into the auction entrypoint context.
crates/trusted-server-core/src/auction/orchestrator.rs	Await async provider launches, operate on PlatformPendingRequest/select, and update the remaining PR13 migration comments.
crates/trusted-server-core/src/auction/provider.rs	Convert AuctionProvider to async request_bids with PlatformPendingRequest/PlatformResponse and refresh the trait docs.
crates/trusted-server-core/src/auction/types.rs	Add RuntimeServices to AuctionContext.
crates/trusted-server-core/src/integrations/adserver_mock.rs	Migrate the mock provider to platform HTTP pending/response types.
crates/trusted-server-core/src/integrations/aps.rs	Migrate APS provider request dispatch and response parsing to platform HTTP types.
crates/trusted-server-core/src/integrations/datadome.rs	Convert DataDome proxying to HTTP-native request/response handling via the platform client.
crates/trusted-server-core/src/integrations/didomi.rs	Convert Didomi proxying to HTTP-native request/response handling via the platform client.
crates/trusted-server-core/src/integrations/google_tag_manager.rs	Remove Fastly shims, keep bounded POST handling on HTTP bodies, and convert GTM tests to HTTP-native fixtures.
crates/trusted-server-core/src/integrations/gpt.rs	Remove Fastly request/response compat from GPT asset proxying and convert GPT tests to HTTP-native fixtures.
crates/trusted-server-core/src/integrations/lockr.rs	Convert Lockr SDK/API proxy handling to HTTP-native request/response types.
crates/trusted-server-core/src/integrations/permutive.rs	Convert Permutive SDK/API proxy handling to HTTP-native request/response types.
crates/trusted-server-core/src/integrations/prebid.rs	Move Prebid provider dispatch to async platform HTTP requests and parse PlatformResponse results.
crates/trusted-server-core/src/integrations/registry.rs	Migrate IntegrationProxy/routing types to http types and thread RuntimeServices through proxy dispatch.
crates/trusted-server-core/src/integrations/testlight.rs	Convert Testlight to HTTP-native request/response handling, add the stale Content-Length regression test, and drop stale length headers when rebuilding JSON responses.

Closes

Closes #494

Test plan

• cargo test --workspace
• cargo clippy --workspace --all-targets --all-features -- -D warnings
• cargo fmt --all -- --check
• JS tests: cd crates/js/lib && npx vitest run
• JS format: cd crates/js/lib && npm run format
• Docs format: cd docs && npm run format
• WASM build: cargo build --package trusted-server-adapter-fastly --release --target wasm32-wasip1
• Manual testing via fastly compute serve
• Other: cargo doc --no-deps --all-features (passes with pre-existing unrelated rustdoc warnings outside this PR's scope)

Checklist

• Changes follow CLAUDE.md conventions
• No unwrap() in production code — use expect("should ...")
• Uses repo-standard logging macros, not println!
• New code has tests
• No secrets or credentials committed

[ChristianPavilonis]

PR Review — Migrate integration and provider HTTP types (PR 13)

1 blocking finding, 5 non-blocking suggestions.

Blocking

🔧 Missing Content-Type forwarding for Didomi POST/PUT — see inline comment.

Non-blocking (inline)

• 🤔 Prebid still imports from fastly::http — last remaining use fastly in the integration layer
• ♻️ Duplicated body collection helpers (collect_response_body / collect_body_bytes)
• 🌱 Unbounded response body collection (matches pre-migration, but could use a size cap)
• ⛏ Dead "X-" uppercase branch in custom header copy (http crate lowercases names)
• ⛏ Duplicated backend_name_for_url helper across 4 integrations

[ChristianPavilonis]

ignore this

Replacing with full review

[ChristianPavilonis]

Summary

Solid migration of the integration and auction provider layers from Fastly-specific HTTP types to platform-agnostic http crate types. No blocking issues found. CI is passing.

Non-blocking

♻️ refactor

• Duplicated body_as_reader helper: proxy.rs:24 and publisher.rs:23 define identical body_as_reader functions. Consider extracting into a shared location (e.g., an into_reader() method on EdgeBody, or a helper in http_util.rs).

📝 note

• PR description file table incomplete: proxy.rs and publisher.rs have changes in this diff (the body_as_reader extraction and test type alias cleanups) but are not listed in the PR body's change table.

📌 out of scope

• Orchestrator tests disabled pending mock PlatformHttpClient: Several provider integration tests and timeout enforcement paths in orchestrator.rs:751-765 are disabled pending a mock for PlatformHttpClient::select(). This is acknowledged in the PR, but the scope of untested paths has grown — a follow-up to add thin mock support for select() would significantly improve coverage of the deadline enforcement logic.

[aram356]

Summary

The refactor is clean and the ensure_integration_backend / collect_body helpers extracted in the fix commits are a genuine improvement. The main risks are latent panics and missing POST size bounds that were introduced in the base migration commit. None are new in the fix commits, but this PR is the natural owner of the HTTP-type boundary, so they should land before merge.

Blocking

🔧 wrench

• EdgeBody::into_bytes() panics on streaming bodies at auction/endpoints.rs:42 — the inbound auction POST body is user-controlled. Body::into_bytes() panics on Body::Stream(_) (unit test into_bytes_panics_for_stream asserts this). Today Fastly's adapter happens to return Body::Once, so nothing panics — but that invariant is undocumented and the /auction handler is the first publicly-reachable crash if it ever changes. Fix: switch to Body::into_bytes_bounded(max).await with a documented cap (e.g. 256KB).
• EdgeBody::into_bytes() panics across every migrated provider/proxy — same root cause, see inline comments on prebid.rs:1157, permutive.rs:151, lockr.rs:149, datadome.rs:301, aps.rs:554, adserver_mock.rs:373. Make parse_response async and read bodies with into_bytes_bounded.
• Unbounded inbound POST body forwarding on the datadome / didomi / permutive / lockr proxies — only GTM enforces a cap. Promote the GTM size-bounded reader into integrations/mod.rs as collect_body_bounded and apply it uniformly. Inline comments on datadome.rs:367, didomi.rs:245, permutive.rs:211, lockr.rs:203.
• Lockr .expect() on user-controlled Origin header — reachable worker panic at lockr.rs:266.
• GTM stream-read errors mis-classified as PayloadTooLarge — returns 413 for transport errors at google_tag_manager.rs:313. Add a StreamRead variant and map to 502.

Non-blocking

🤔 thinking

• collect_body has no size cap (integrations/mod.rs:101) — silently drains whole bodies to RAM. Same concern applies to the GTM response-rewrite path at google_tag_manager.rs:486.
• Orchestrator select_result.ready error path drops provider identity in run_providers_parallel — when ready is Err, the provider identity is lost so no AuctionResponse::error(...) is pushed and the failing backend vanishes silently from the result set. Track a (backend_name, provider_index) pair alongside fastly_pending so failures are attributable.
• Deadline enforcement relies on every provider honoring backend_name(effective_timeout) — Phase 1 computes remaining_ms.min(provider.timeout_ms()) correctly, but nothing guarantees each provider actually threads that timeout through to the backend config. Not a new regression, but the async migration makes it more load-bearing. Add a unit test that asserts the select loop returns before deadline + epsilon.

♻️ refactor

• aps.rs / adserver_mock.rs still build BackendConfig inline — extend the new ensure_integration_backend helper to cover providers too.

🌱 seedling

• Make parse_response async in the provider trait (auction/provider.rs:48) — needed anyway once the into_bytes_bounded fix lands; zero cost under #[async_trait(?Send)].

📝 note

• Testlight stale Content-Length test is narrow (testlight.rs:425) — good unit-level coverage, but the two handle() rebuild call sites would benefit from an end-to-end assertion.

👍 praise

• Clean helper extraction in integrations/mod.rs — real dedup across six integrations.
• Dropping the dead || name_str.starts_with("X-") branch in lockr.rs / permutive.rs — HeaderName::as_str() is always lowercase, so it was unreachable.
• Adding CONTENT_TYPE to didomi's forwarded headers — POST bodies can now be interpreted by upstream.

CI Status

• browser integration tests: PASS
• integration tests: PASS
• prepare integration artifacts: PASS

[aram356]

Summary

Re-review of the EdgeZero PR13 integration/provider HTTP type migration after the latest "Address review findings" commit. The async parse_response migration and the new collect_body / collect_body_bounded helpers landed cleanly and resolved the prior into_bytes()-on-Stream panic class across all providers. Two blockers remain plus one CI/process question that affects whether the existing test plan can be trusted.

Blocking

🔧 wrench

• cargo fmt --all -- --check fails locally in 7+ files. The PR description's [x] cargo fmt checkbox is incorrect — see the CI question below for why this wasn't caught. Affected files:

  • crates/trusted-server-core/src/auction/endpoints.rs:39-47
  • crates/trusted-server-core/src/auction/orchestrator.rs:12-14, 402-405
  • crates/trusted-server-core/src/integrations/google_tag_manager.rs:16-25, 39-45
  • crates/trusted-server-core/src/integrations/lockr.rs:282-285
  • crates/trusted-server-core/src/integrations/permutive.rs:208-219, 278-281
  • crates/trusted-server-core/src/integrations/prebid.rs:5-13, 1154-1162
  • crates/trusted-server-core/src/integrations/testlight.rs:11-16

  Fix: run cargo fmt --all.

• Testlight POST body is unbounded (inline at integrations/testlight.rs:181).

❓ question

• CI gates aren't enforced for this PR. .github/workflows/format.yml and .github/workflows/test.yml only fire on pull_request: branches: [main]. PR 626's base is feature/edgezero-pr12-handler-layer-types, so on commit 94ec9477 only "Integration Tests" ran — fmt, clippy, and cargo test --workspace did not. Combined with the fmt failure above, every stacked EdgeZero PR ships with broken formatting and an unverified test plan. Was this intentional, or should the workflow triggers be widened (or the gates duplicated under the integration-tests umbrella)? Until that's resolved, the PR description's CI checkboxes can't be relied on as evidence.

  Verified locally on this commit: cargo test --workspace passes (821 tests), cargo clippy --workspace --all-targets --all-features -- -D warnings passes, cargo build --release --target wasm32-wasip1 passes, cargo fmt --all -- --check fails.

Non-blocking

🤔 thinking

• Unbounded response-body reads via collect_body (inline at integrations/mod.rs:93 for stream-read error fidelity; cross-cutting concern listed here). The bounded variant exists for inbound bodies, but every upstream response site still drains an unbounded body to a Vec<u8>:

  • integrations/prebid.rs:1157 — Prebid Server response (highest risk: third-party hop, large payloads)
  • integrations/aps.rs:555 — APS response
  • integrations/adserver_mock.rs:375 — Mediator response
  • integrations/lockr.rs:150 — Lockr SDK fetch
  • integrations/permutive.rs:152 — Permutive SDK fetch
  • integrations/datadome.rs:302 — DataDome SDK fetch
  • integrations/google_tag_manager.rs:491 — GTM script

  Provider responses are the highest risk — a misbehaving Prebid Server / mediator can ship a multi-GB body and OOM the worker. Recommend routing all of these through collect_body_bounded with a per-callsite cap, or removing the unbounded variant in favor of a single bounded helper.

♻️ refactor

• Auction providers still construct BackendConfig inline while integration proxies route through ensure_integration_backend. Reraising the prior-review concern with current locations:

  • integrations/aps.rs:517-528
  • integrations/adserver_mock.rs:338-348
  • integrations/prebid.rs:1131-1135

  The two paths differ on timeout (providers need a configurable per-request timeout; the proxy helper hardcodes 15s). Consider widening ensure_integration_backend to accept a timeout and reusing it from the providers, or at least documenting the divergence.

CI Status

• fmt: FAIL locally (CI did not run for this commit — see ❓ question above)
• clippy: PASS locally (CI did not run)
• rust tests: PASS locally — 821 tests (CI did not run)
• wasm32-wasip1 build: PASS locally
• Integration Tests workflow: PASS (only workflow that ran on 94ec9477)