Skip to content

extmod/zephyr_ble: Add FreeRTOS threading backend.#28

Draft
andrewleech wants to merge 170 commits intopr/zephyr-ble-l2capfrom
pr/zephyr-ble-freertos
Draft

extmod/zephyr_ble: Add FreeRTOS threading backend.#28
andrewleech wants to merge 170 commits intopr/zephyr-ble-l2capfrom
pr/zephyr-ble-freertos

Conversation

@andrewleech
Copy link
Copy Markdown
Owner

Summary

Adds optional FreeRTOS-based threading for the Zephyr BLE HAL shims. When
MICROPY_BLUETOOTH_ZEPHYR_USE_FREERTOS is enabled, semaphores and mutexes
use FreeRTOS primitives instead of polling stubs, and a dedicated HCI RX
task can handle packet reception asynchronously on RP2.

Includes STM32 FreeRTOS integration (hooks, SysTick config, PendSV
priority) and RP2 zephyr_ble_freertos board variants for Pico W and
Pico 2 W.

The work thread is currently disabled (returns early in work_thread_start)
due to GIL issues with SYNC_EVENTS_WITH_INTERLOCK — work is still
processed via cooperative polling. The infrastructure is in place for when
the threading model is resolved.

Testing

Builds verified for RPI_PICO_W zephyr_ble_freertos. Not yet runtime
tested after the branch reorder — needs reflash and multitest run.

Generative AI

I used generative AI tools when creating this PR, but a human has checked the code and is responsible for the description above.

@github-actions
Copy link
Copy Markdown

Code size report:

Reference:  stm32: Use IPCC flag for ACL flow control in rfcore. [88d1c0f]
Comparison: stm32: Add FreeRTOS threading backend for STM32 ports. [merge of 50940e4]
  mpy-cross:    +0 +0.000% 
   bare-arm:    +0 +0.000% 
minimal x86:    +0 +0.000% 
   unix x64:    +0 +0.000% standard
      stm32:    +0 +0.000% PYBV10
      esp32:    +0 +0.000% ESP32_GENERIC
     mimxrt:    +0 +0.000% TEENSY40
        rp2:    +0 +0.000% RPI_PICO_W
       samd:    +0 +0.000% ADAFRUIT_ITSYBITSY_M4_EXPRESS
  qemu rv32:    +0 +0.000% VIRT_RV32

@andrewleech andrewleech force-pushed the pr/zephyr-ble-l2cap branch from 88d1c0f to 4a5a239 Compare March 16, 2026 05:12
@andrewleech andrewleech force-pushed the pr/zephyr-ble-freertos branch from 50940e4 to 53c9de7 Compare March 16, 2026 05:12
@andrewleech andrewleech force-pushed the pr/zephyr-ble-l2cap branch from 4a5a239 to aaa7afd Compare March 16, 2026 05:56
@andrewleech andrewleech force-pushed the pr/zephyr-ble-freertos branch from 53c9de7 to cf52c7e Compare March 16, 2026 05:56
@andrewleech andrewleech force-pushed the pr/zephyr-ble-l2cap branch from aaa7afd to 4bec030 Compare March 16, 2026 06:36
@andrewleech andrewleech force-pushed the pr/zephyr-ble-freertos branch from cf52c7e to 4e4d61c Compare March 16, 2026 06:36
@andrewleech andrewleech force-pushed the pr/zephyr-ble-l2cap branch from 4bec030 to d6fea48 Compare March 16, 2026 08:09
@andrewleech andrewleech force-pushed the pr/zephyr-ble-freertos branch from 4e4d61c to a0c7264 Compare March 16, 2026 08:09
@andrewleech andrewleech force-pushed the pr/zephyr-ble-l2cap branch from d6fea48 to 0da0633 Compare March 16, 2026 09:33
@andrewleech andrewleech force-pushed the pr/zephyr-ble-freertos branch from a0c7264 to 7fbc78a Compare March 16, 2026 09:33
@andrewleech andrewleech force-pushed the pr/zephyr-ble-l2cap branch from 0da0633 to 271e41b Compare March 16, 2026 10:15
@andrewleech andrewleech force-pushed the pr/zephyr-ble-freertos branch from 7fbc78a to 40f0f55 Compare March 16, 2026 10:15
agatti and others added 17 commits March 17, 2026 16:14
This commit updates the documentation for the `re` library, officially
documenting non-capturing grouping rules (ie. "(?:...)").

The documentation mistakenly marked that feature as not supported, but
is is indeed supported in the current iteration of the regex library.

This closes micropython#18900.

Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
This reverts commit 046013a.

Looks like since the latest round of GitHub Actions updates, the
Cache LRU algorithm is working as designed again.

This work was funded through GitHub Sponsors.

Signed-off-by: Angus Gratton <angus@redyak.com.au>
Both the overall IRQ line and the per-channel IRQ, for good measure.

Otherwise, soft reset will remove the handler before the finaliser for the
DMA object(s) run and trigger IRQs if the channel is still active.

Closes micropython#18765

This work was funded through GitHub Sponsors.

Signed-off-by: Angus Gratton <angus@redyak.com.au>
Small tweak to avoid changes in other targets' lockfiles from printing
warnings when building esp32 port.

This work was funded through GitHub Sponsors.

Signed-off-by: Angus Gratton <angus@redyak.com.au>
Not currently building, and too many versions to concurrently support.

This work was funded through GitHub Sponsors.

Signed-off-by: Angus Gratton <angus@redyak.com.au>
This work was funded through GitHub Sponsors.

Signed-off-by: Angus Gratton <angus@redyak.com.au>
Also rename the prefix from can to pyb_can, in anticipation
of machine.CAN tests.

Signed-off-by: Angus Gratton <angus@redyak.com.au>
Closes micropython#18922

Signed-off-by: Angus Gratton <angus@redyak.com.au>
The function arguments mean totally different things for Classic vs FDCAN
hardware, but the argument name wasn't particularly clear for either.

This commit shouldn't really change the binary firmware at all.

This work was funded through GitHub Sponsors.

Signed-off-by: Angus Gratton <angus@redyak.com.au>
This work was funded through GitHub Sponsors.

Signed-off-by: Angus Gratton <angus@redyak.com.au>
Simplifies the pattern of an optional arg which can be a list of at
least a certain length, otherwise one is lazily initialised.

Modify pyb.CAN and ESP-NOW APIs to use the helper. Note this changes
the return type of pyb.CAN.recv() from tuple to list.

This work was funded through GitHub Sponsors.

Signed-off-by: Angus Gratton <angus@redyak.com.au>
This work was funded through GitHub Sponsors.

Signed-off-by: Angus Gratton <angus@redyak.com.au>
API is different to the original machine.CAN proposal, as numerous
shortcomings were found during initial implementation.

This work was funded through GitHub Sponsors.

Signed-off-by: Angus Gratton <angus@redyak.com.au>
These are oddly missing from the STM32G4 HAL, but the
reference manual describes being able to use them and
the implementations seem to work as expected.

Note that unlike STM32H7 it doesn't seem like we must use this approach,
because HAL_FDCAN_AddMessageToTxFifoQ() does seem to not have the issues
with priority inversion seen on the H7. However it's simpler to use the
same API for both...

Signed-off-by: Angus Gratton <angus@redyak.com.au>
Signed-off-by: Angus Gratton <angus@redyak.com.au>
Implemented according to API docs in a parent comment.

Adds new multi_extmod/machine_can_* tests which pass when testing between
NUCLEO_G474RE, NUCLEO_H723ZG and PYBDV11.

This work was mostly funded through GitHub Sponsors.

Signed-off-by: Angus Gratton <angus@redyak.com.au>
The DAR register field is for auto-retransmit, FDCAN doesn't support
automatic restart to clear Bus Off.

This work was funded through GitHub Sponsors.

Signed-off-by: Angus Gratton <angus@redyak.com.au>
Some MCUs (eg N6) have more timers which are 32-bit, and it's best to use
this macro to work that out.

Signed-off-by: Damien George <damien@micropython.org>
pi-anl added 29 commits April 19, 2026 23:48
Add Makefile integration, linker script extensions, and board variant
configurations (PCA10056, PCA10059) for building the nRF port with
Zephyr BLE. Includes PCA10059 DAPLink variant and hci_driver_poll_rx
support for the on-core controller.

Signed-off-by: Andrew Leech <andrew.leech@planetinnovation.com.au>
Add mpzephyrport_nrf.c with on-core BLE controller initialization,
LFXO startup, cooperative HCI polling, and scheduler node callback
handling. Includes NULL callback guard for safe deinit during scheduler
node draining.

Signed-off-by: Andrew Leech <andrew.leech@planetinnovation.com.au>
Extend the bt_disable deinit path to the nRF port's on-core controller.
The controller is shut down via ll_deinit() called from bt_disable(),
with proper LFXO re-start on next init.

Signed-off-by: Andrew Leech <andrew.leech@planetinnovation.com.au>
Add work processing interleaving to the nRF port's controller polling
loop. Each HCI packet is followed by a work_process() call to ensure
connection events are handled before subsequent packets.

Signed-off-by: Andrew Leech <andrew.leech@planetinnovation.com.au>
Move the remaining two Zephyr submodule patches (HCI driver, quirk
reset) into wrapper files, eliminating all custom patches from the
lib/zephyr submodule.

Signed-off-by: Andrew Leech <andrew.leech@planetinnovation.com.au>
Add mp_bluetooth_zephyr_l2cap_flush_recv_notify() call to
port_run_task and re-entrancy guard to prevent recursive
port_run_task execution.

Update lib/zephyr submodule with LLL preempt ticker fix —
ticker_stop(TICKER_ID_LLL_PREEMPT) in init_reset() prevents
ll_deinit assertion on second bt_disable/bt_enable cycle.

Run codeformat.py on nRF port files and hci_driver_wrapper.c.

Signed-off-by: Andrew Leech <andrew.leech@planetinnovation.com.au>
Use ZEPHYR_BLE_POLL_INTERVAL_MS for poll timer values.
Enable CONFIG_BT_AUTO_DATA_LEN_UPDATE for PCA10056 and
PCA10059 on-core controller variants.

Signed-off-by: Andrew Leech <andrew.leech@planetinnovation.com.au>
Signed-off-by: Andrew Leech <andrew.leech@planetinnovation.com.au>
Signed-off-by: Andrew Leech <andrew.leech@planetinnovation.com.au>
New Unix variant `zephyr_ble` that provides:
- Pthread-based soft timer backend for machine.Timer
- Linux HCI_CHANNEL_USER socket transport for USB BT adapters
- Zephyr BLE stack using RP2 cooperative polling pattern

The HCI socket driver uses AF_BLUETOOTH/BTPROTO_HCI with
HCI_CHANNEL_USER for exclusive raw access to a USB Bluetooth
adapter. Device selection via MICROPYBTHCI env var with
auto-detection fallback. Non-blocking socket reads in
port_run_task driven by the soft timer.

Build: make -C ports/unix VARIANT=zephyr_ble

Signed-off-by: Andrew Leech <andrew.leech@planetinnovation.com.au>
- Replace softtimer_unix.c with version that has proper deinit,
  SIGUSR1 main-thread wakeup, and GC marking integration.
- Add soft_timer_init/deinit calls to main.c, GC marking to
  gccollect.c.
- Add MICROPY_SCHED_HOOK_SCHEDULED to wake main thread from
  blocking syscalls when timer callbacks are scheduled.
- Enable L2CAP channels and pairing/bonding by default in
  modbluetooth.h (all stacks support these features).
- Guard le_dbg variable with ZEPHYR_BLE_DEBUG in
  l2cap_disconnected_cb.
- Document HCI TX buffer size bounds.

Signed-off-by: Andrew Leech <andrew.leech@planetinnovation.com.au>
Signed-off-by: Andrew Leech <andrew.leech@planetinnovation.com.au>
Add Zephyr BLE variant configuration for PYBD_SF6 board, with HCI UART
readpacket support for bulk reading from the BT coprocessor.

Signed-off-by: Andrew Leech <andrew.leech@planetinnovation.com.au>
Update mp_handle_pending() calls to use the
mp_handle_pending_behaviour_t enum instead of bool, matching
the current signature in py/scheduler.c.

Add mp_bluetooth_zephyr_l2cap_flush_recv_notify() call to
port_run_task, consistent with RP2 and nRF ports. Without this,
deferred L2CAP recv notifications were never delivered to Python.

Signed-off-by: Andrew Leech <andrew.leech@planetinnovation.com.au>
Use ZEPHYR_BLE_POLL_INTERVAL_MS for poll timer values.
Enable CONFIG_BT_AUTO_DATA_LEN_UPDATE for NUCLEO_WB55.

Signed-off-by: Andrew Leech <andrew.leech@planetinnovation.com.au>
Replace NUM_COMPLETED_PACKETS-based ACL flow control with direct IPCC
channel flag polling.  The IPCC flag clears when M0+ consumes the shared
SRAM2B buffer, which is much faster than waiting for the radio round-trip
needed by NUM_COMPLETED_PACKETS.

Debug instrumentation confirmed the IPCC flag always clears before the
next ACL send attempt, so the timeout path is defensive only.

Signed-off-by: Andrew Leech <andrew.leech@planetinnovation.com.au>
Signed-off-by: Andrew Leech <andrew.leech@planetinnovation.com.au>
Signed-off-by: Andrew Leech <andrew.leech@planetinnovation.com.au>
Signed-off-by: Andrew Leech <andrew.leech@planetinnovation.com.au>
Signed-off-by: Andrew Leech <andrew.leech@planetinnovation.com.au>
Add mp_bluetooth_zephyr_l2cap_flush_recv_notify() call to
port_run_task and re-entrancy guard to prevent recursive
execution from nested mp_handle_pending() calls.

Signed-off-by: Andrew Leech <andrew.leech@planetinnovation.com.au>
Signed-off-by: Andrew Leech <andrew.leech@planetinnovation.com.au>
Replace the native Zephyr port's custom BLE bindings with the shared
extmod/zephyr_ble integration layer. This unifies the BLE API across
all ports using Zephyr BLE, with the native Zephyr port using Zephyr's
own kernel primitives instead of the HAL shim stubs.

Includes machine.idle() fix to yield to Zephyr threads, and test
fixes for nRF52840 DK BLE multitests.

Signed-off-by: Andrew Leech <andrew.leech@planetinnovation.com.au>
Enable synchronous BLE events and increase UART RX buffer to 512 bytes
for reliable raw-paste operation on the nRF52840 DK.

Signed-off-by: Andrew Leech <andrew.leech@planetinnovation.com.au>
The nRF52840 DK's USB CDC ACM console was unreliable — device enumeration
failures and stalls during raw-paste mode. Switch to UART via JLink OB
(uart0 with hw-flow-control) which is always available.

Move USB device stack init from mp_task (after console init) to
zephyr_start.c main() (before console init) so CDC ACM UART is ready
when the console subsystem opens the device. Add DTR wait for CDC ACM
console boards so output isn't lost before a host connects.

Reduce MICROPY_REPL_STDIN_BUFFER_MAX to 64 (raw-paste window=32 bytes)
to avoid overflowing USB-UART bridge buffers at 115200 baud.

Signed-off-by: Andrew Leech <andrew.leech@planetinnovation.com.au>
When MICROPY_PY_THREAD is enabled, use FreeRTOS counting semaphores and
recursive mutexes rather than the cooperative polling fallbacks. The
FreeRTOS sem take path polls with short 10ms timeouts and calls
mp_bluetooth_zephyr_hci_uart_wfi() / work_process() between attempts, so
bt_enable() init traffic is still handled correctly before the HCI RX task
is started.

Signed-off-by: Andrew Leech <andrew.leech@planetinnovation.com.au>
Signed-off-by: Andrew Leech <andrew.leech@planetinnovation.com.au>
Signed-off-by: Andrew Leech <andrew.leech@planetinnovation.com.au>
Signed-off-by: Andrew Leech <andrew.leech@planetinnovation.com.au>
@andrewleech andrewleech force-pushed the pr/zephyr-ble-freertos branch from 40f0f55 to 42e5049 Compare April 20, 2026 21:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.