What Lightning-AI/LitServe shipped
Written by FoxPlug from the repository's own releases; not affiliated with Lightning. Paste your own repository in the box below and FoxPlug writes the same weekly update for your product. This page is an automatic summary of the releases, pull requests and commits in github.com/Lightning-AI/LitServe; Lightning did not write it and does not use or endorse FoxPlug. Every line links to the change it describes.
Get a weekly update like this for your product, free
Week of October 5, 2026
What shipped
- HTTP request headers are now exposed to LitAPI hooks through context["headers"], so auth tokens, trace IDs and user agents can be read without repeating them in the body. Pull request #730
- Queue timeouts in the async loops are now passed by keyword, so the timeout actually applies instead of silently binding to block. Pull request #757
- DefaultContinuousBatchingLoop.add_request now accepts the transport argument, fixing a TypeError on the first request. Pull request #758
- Test_default_continuous_batching_prefill now passes headers in its requests, fixing a failure on main caused by the new five-value prefill unpacking. Pull request #772
- The flaky
test_collate_requestscase on Windows now uses a widerbatch_timeoutof 0.5 instead of 0.1, with unchanged runtime. Pull request #773 - CI workflows are pinned to ubuntu-24.04, macos-26 and windows-2025 instead of floating latest labels. Pull request #774
Why it matters
This week fixes several real breakages: request headers now reach LitAPI hooks, and two loop bugs that caused a TypeError on the first request and a queue timeout that never fired. It also stabilises CI with pinned runner images and two test fixes, so the project builds and behaves more predictably.
Changelog entry
- Expose HTTP request headers to LitAPI hooks via context["headers"]. Pull request #730
- Pass queue timeout by keyword in async loops so the timeout is applied. Pull request #757
- Accept transport in DefaultContinuousBatchingLoop.add_request to prevent a TypeError. Pull request #758
- Pass headers in
test_default_continuous_batching_prefillrequests. Pull request #772 - Widen
batch_timeoutin the flakytest_collate_requestscase on Windows. Pull request #773 - Pin CI runner images to ubuntu-24.04, macos-26 and windows-2025. Pull request #774
This week: request headers are exposed to LitAPI through context["headers"]. Fixed a TypeError in the continuous batching loop and a queue timeout that never fired. Widened a flaky Windows test timeout, fixed another test, and pinned CI runner images to ubuntu-24.04, macos-26 and
This week's changes: HTTP request headers are now passed to LitAPI hooks via context["headers"], so auth tokens, trace IDs and user agents can be read without repeating them in the request body. Two loop fixes landed: DefaultContinuousBatchingLoop.add_request now accepts transport, resolving a TypeError on the first request, and async loops pass the queue timeout by keyword so the timeout works. Two test fixes landed: a flaky Windows collate test got a wider batch_timeout and the continuous batching prefill test now passes headers. CI workflows are pinned to ubuntu-24.04, macos-26 and windows-2025.