The relay authenticated clients itself, with an HMAC handshake and a
background job pulling the access-key list from the target server's management
API. That never worked with a real client and would not have been worth having
if it had.
It could not work: no OSTP client produces those credentials. The TCP path
required an HTTP request (`GET /stream` with `Authorization: Bearer`) and the
UDP path a `timestamp || HMAC` preamble, while the client sends junk frames
followed by length-prefixed frames, and an obfuscated Noise handshake. There
is no HMAC code in ostp-client at all, so every connection was rejected: TCP
answered 404, UDP dropped the datagram.
It was also weak where it applied. The HMAC covered only an 8-byte timestamp,
so a captured signature was a bearer token replayable from any address for the
clock-skew window, with no anti-replay set. And the HTTP handshake put a
literal `GET /stream` on the wire — a greppable signature in a protocol whose
premise is that nothing is recognisable.
Authentication now stays where it is cryptographically meaningful. The target
server already authenticates every session end-to-end via Noise with a PSK
derived from the access key and silently drops what fails; the relay adds
nothing by re-checking, and holding a copy of the key list on a forwarding box
is a liability. The relay makes no security decisions and says so.
What it does need is protection from being a resource sink, so this adds a
session cap, a connection cap, and a token-bucket admission limit on NEW
sessions only. It forwards to one fixed upstream and replies only to the
sender, so it is not a reflector: amplification is one.
Fixes a bug the new end-to-end test caught: upstream sockets were bound as
"[::]:0" and connected to a possibly-IPv4 upstream, which fails wherever
IPV6_V6ONLY defaults on — that is every deployment with an IPv4 target server.
The bind family now follows the resolved upstream.
upstream_api_url/token/sync_interval_secs are accepted and ignored so existing
relay configs keep parsing; the wizard and template no longer emit them, and
`ostp check` flags them as removable.
Reported: with an upstream proxy configured, Google flags the session because
the server's address and the proxy's exit address differ, and YouTube
geolocates to the server rather than the proxy exit. Both follow from the same
defect in three places — an outbound rule that says Proxy was being satisfied
by connecting DIRECTLY whenever the proxy could not be used:
- router.rs, UdpSessionRouter::send_to: action Proxy with no UDP proxy
established fell through to the direct socket.
- outbound.rs, connect_target: any protocol string other than exactly
"socks5"/"http" hit a `_ =>` arm that connected directly. A typo, or just
"SOCKS5", silently un-proxied ALL TCP.
- outbound.rs, connect_udp_target: non-socks5 upstream returned
UdpProxySocket::Direct, with a comment noting HTTP cannot carry UDP —
correct in itself, but the chosen fallback leaks.
The visible symptom is precisely what these produce: TCP goes through the
proxy while UDP (QUIC — which is what YouTube uses) leaves from the server,
so one session presents two exit addresses.
All three now fail closed. A rule asking for the proxy is never honoured by
sending in the clear: a dropped flow is visible and debuggable, a
deanonymising leak is neither. Protocol matching is also case-insensitive
now, and the SOCKS5 UDP ASSOCIATE failure warns unconditionally rather than
only under `debug`, which is why this could go unnoticed.
Behaviour change: with an HTTP upstream, or a broken SOCKS5 UDP ASSOCIATE,
UDP now fails instead of leaking. Where that is genuinely wanted, it must be
stated in the config as an explicit udp rule with action "direct".
Also carries the relay key-sync diagnostics: a 404 there means the URL is
missing the panel's secret webpath (the API is nested under it, not at /api),
which the previous bare "HTTP 404" gave no way to work out.
A relay configured with upstream_api_url = "http://HOST:9090" fails every key
sync with a bare "API returned HTTP 404", which reads like the server is down
or the token is wrong. Neither is true: the management API is nested under the
target server's api.webpath (create_api_router mounts it at
"/{webpath}/api"), because that secret segment is what keeps the panel from
being discoverable by scanners. A bare host:port therefore resolves to a route
that does not exist and the token is never even looked at.
Nothing said so — the config template, the wizard prompt and the shipped
example all suggested exactly the host:port form that cannot work.
- sync_keys now reports the full URL and, for 404 specifically, states that
the webpath must be included and what the URL should look like. 401 is
called out separately as a token mismatch, since the two are otherwise
indistinguishable from the log.
- The relay config template, the shipped example and the wizard prompt now
show the path-bearing form, and the wizard warns when the URL entered has
no path segment rather than letting it fail later.
Docs under docs/ and the wiki are being rewritten concurrently and are left
alone here.
pacing_rate had been computed on every ACK since the controller was written
and never read by anything: admission was decided purely by cwnd. But cwnd
bounds how much may be UNACKNOWLEDGED, not how fast it reaches the wire, so a
full window went out back-to-back. On a bottleneck with a deep buffer that
burst is absorbed rather than dropped, and it lands as standing queue — the
mechanism behind the multi-second RTT this protocol has been showing on
mobile. It is also why BBR could not simply be dropped in: BBR's whole model
is "send at the estimated bottleneck rate", which is meaningless without a
pacer underneath it.
Adds a token bucket to CongestionController, charged in on_send so every byte
that reaches the wire pays exactly once — retransmits included, since those
are precisely what must not bypass the limit and pile into a full queue.
Burst allowance is 10ms-at-rate rather than one packet. Pacing intervals here
are fractions of a millisecond, so strict per-packet release would need a
sub-millisecond timer per packet; sizing the burst to the loop's existing
~10ms wakeup lets the configured rate still be saturated. There is a floor of
4 MTU so a cold or collapsed rate estimate can never wedge sending entirely.
Wired into both directions. The client gates its proxy-event branch on it,
and the server's per-session backpressure snapshot reports zero headroom when
the bucket is empty. The download path matters most here — that is the one
carrying video — and it was also still clamped to the old 16384-packet
ceiling (~20 MB outstanding), now aligned with MAX_CWND_PACKETS.
Tested that the bucket denies once drained and refills over time; the second
is what keeps a stalled bucket from wedging sending permanently.
DnsServer::run_local_udp_listener binds 0.0.0.0 and answers every UDP
datagram by resolving it and replying to the packet's (unverified,
spoofable) source address - a textbook DNS reflection/amplification
primitive. An attacker spoofing a victim's IP as the query source turns
any server with this listener running into a free amplifier against that
victim, with zero authentication gating it (unlike the main OSTP port,
there's no Noise handshake here).
Nothing in the codebase currently calls this function - the live DNS path
is router.route_dns(), reached only through the authenticated OSTP tunnel
relay (relay.rs). But the doc comment describes this as an intended,
not-yet-wired entry point for clients that point their OS resolver
directly at the server, so it's a real latent risk for whoever connects it
without realizing the implication. Added a global (not per-source-IP -
per-IP limiting doesn't help against a reflection attack, since the
attacker never sees the replies and can spread queries across arbitrary
spoofed sources) token bucket capping total replies/sec, so connecting
this later can't silently reintroduce unbounded amplification.
check_token() and handle_login() compared bearer tokens, session tokens,
and the password hash with plain ==, which short-circuits on the first
differing byte - a textbook remote timing side-channel against exactly
the long-lived secrets these gates exist to protect. Added subtle (already
in the dependency tree transitively via chacha20poly1305) as a direct
dependency and route every secret comparison through a small secure_eq()
wrapper over ConstantTimeEq. Username comparison in handle_login is left
as-is: it isn't treated as a secret in this threat model (one fixed admin
username), matching standard practice of only constant-timing the
password/token side of an auth check.
Added tests for secure_eq() itself (equal, different, different-length,
empty) alongside the existing check_token coverage.
GET/POST/DELETE /api/audit were the only three handlers in the whole
Management API that never called check_token() - every other endpoint
(status, users, rules, config) does. Concretely, with the panel's
credentials configured, an unauthenticated request could still:
- read the full audit log (GET)
- inject arbitrary forged entries, e.g. fake "success" events to cover
tracks (POST)
- wipe the entire audit log (DELETE) - the exact mechanism meant to
detect and investigate unauthorized actions, erasable with zero auth
Added the same check_token() gate the rest of the file uses, and fixed
these three handlers' raw .unwrap() on the audit_logs lock to the
poison-recovery pattern (unwrap_or_else(|e| e.into_inner())) used
everywhere else, for consistency.
Added focused unit tests on check_token() itself (missing header, correct/
wrong bearer, raw token, session token, and the documented open-panel
mode when no credentials are configured) - it's the single gate every
sensitive handler depends on, worth pinning down independently of any one
handler.
Matches a real report: traffic counters move (the OSTP tunnel handshakes
fine) but sites don't open, or take very long - on a freshly deployed
DigitalOcean droplet in Amsterdam.
connect_target's fallback path handed the raw "host:port" string straight
to TcpStream::connect, which resolves and tries addresses internally but
shares ONE 10s timeout across the WHOLE attempt (all resolved addresses,
not per-address). Some VPS hosts assign the machine an IPv6 address that
the OS prefers by RFC 6724 ordering but that has no actually-working
outbound route - the connect doesn't get refused, it just hangs. With a
single shared budget, that one dead IPv6 candidate eats the entire 10s and
the working IPv4 candidate is never even attempted: every dual-stack
destination (i.e. most popular sites) times out, while IPv4-only
destinations work fine.
New connect_direct() resolves target itself via lookup_host, sorts IPv4
candidates first, and tries each with its own 3s budget (still bounded
overall by the original 10s outer timeout as a backstop) so a hung IPv6
attempt can't starve the IPv4 fallback of a chance.
Added tests: IPv4-first sort ordering (and stability within a family), a
successful connect against a live local listener, and a refused-port
connect failing well under the timeout (proving failures aren't
needlessly slow). Scoped to connect_target's direct-connect paths; the
SOCKS5/HTTP outbound-proxy paths and the fallback/camouflage TCP proxy
(which targets a fixed admin-configured local address, not arbitrary
dual-stack hostnames) are unaffected.
Matches a real user report after 2 weeks on this version: bandwidth is low,
the client reconnects every 10-20 minutes, sites randomly stop loading or
crawl, and ping visibly jitters. Root cause: the per-target-connection
reader task (handle_relay_message's Connect handler) read from the
upstream target as fast as it would send and forwarded every chunk
straight to send_relay_to_stream -> an immediate UDP datagram, with ZERO
awareness of the client-facing OSTP session's actual congestion window.
The client already gates its own uplink on cwnd (bridge.rs's proxy_ev
select arm); the server's download direction had no equivalent.
On a real (lossy/jittery mobile or Wi-Fi) client path, a fast target (a
CDN, say) gets blasted at the client far beyond what the path can sustain.
That's a self-inflicted loss burst: it wrecks the RTT/RTO estimate (the
"ping jitters" symptom), can push the session into a stall bad enough that
the client's 25s/180s keepalive stall-detection gives up and reconnects
(the "every 10-20 minutes" symptom), and produces exactly the "randomly
stops loading or crawls" experience while it's happening.
Fix: Dispatcher::snapshot_backpressure() computes each session's headroom
(clamped cwnd - in_flight, same clamp(16,16384) the client uses) from the
existing 10ms retransmit tick - no new polling loop. Published to a
lock-free-reader Arc<AtomicI64> per session in a shared map so relay reader
tasks (which don't have access to Dispatcher; it lives on the main loop
task) can check it without touching a lock/mutex on every read. Before
each read, a reader task waits (capped at 2s, so a wedged read can't stall
forever) while headroom is <= 0. New sessions default to a healthy 32
packets until their first snapshot lands (worst case 10ms), so this can't
stall the very first bytes of a fresh connection.
Scoped to the primary TCP CONNECT/proxy path (the one actual web browsing
uses); UDP relay/TURN is unaffected. No load test against real network
jitter was possible in this environment - the mechanism directly targets
the identified cause, but real-world confirmation is still needed.
Brings the crypto security audit into alpha alongside the already-merged
reconnection/UoT/Flutter fixes:
- CRITICAL: transport keys now come from Noise Split() over ck (DH-
inclusive), not the handshake hash — restores forward secrecy.
Wire-breaking, PROTOCOL_VERSION 4->5.
- Rate-limit + cache the O(N_keys) handshake trial path (CPU DoS).
- Quiet hot-path logging; access keys no longer logged verbatim.
- Removed dead 0-RTT resumption module (unsafe XOR ticket crypto).
- Karn's algorithm RTT fix, 32-bit frame-length overflow guard,
replay-cache eviction instead of global reject.
- Updated EN/RU specification docs to match.
No conflicts: protocol-hardening's bridge.rs commit was an independent
duplicate of the same reconnection fix already on alpha, so git merged it
as a no-op on that file.
Two UoT (UDP-over-TCP) correctness issues:
- The accepted UoT stream never had TCP_NODELAY set (the client sets it on
its end, the server didn't). Nagle's algorithm then batched server->client
writes and interacted with the client's delayed ACKs, adding tens-to-
hundreds of ms of stall per burst — throttling the download direction
badly for streaming/video. Every TCP-tunnel proxy disables Nagle; now the
server matches the client.
- handle_tcp_connection join!ed the reader and writer tasks, so a half-open
connection (client's read side gone, no outbound data pending) parked the
writer on rx.recv() forever, leaking the task and a stale tcp_map entry.
Rewrote it with select! so either half closing cancels the other and the
tcp_map entry is always removed. Added duplex-stream tests covering
inbound reassembly across segment boundaries, outbound framing, and
teardown-on-close.
- Karn's algorithm: drop_acked_frames no longer samples RTT from frames
that were retransmitted (last_sent is bumped on each retransmit, so an
ACK for the original transmission would measure a spuriously small RTT
and drag SRTT/RTO down). Added CongestionController::on_ack_no_rtt for
the case where every acked frame was ambiguous, so the window still
advances without polluting the RTT estimator. Refactored the shared
window-growth into grow_window.
- Frame decode: header+payload+pad length now uses checked_add. payload_len
is a u32 from the header and on 32-bit targets (MIPS/ARMv7 routers are
supported) the sum could wrap usize and slip past the truncation check.
- Replay cache: a full cache used to reject ALL new handshakes globally
until the next tick, letting one flooding key-holder deny service to
everyone. Now it reclaims expired entries and, if still full, evicts the
single oldest — new handshakes always get in. Fixed the mislabelled
"100000" log (cap is 50000) and named it REPLAY_CACHE_MAX.
Two classes of issue:
- Hot-path/attacker-triggerable events logged at info/error with internal
detail: a per-handshake info! byte dump (raw_vec[0..6]) and a per-packet
error! on session-id mismatch that dumped expected/got session ids.
Both are log-flood + info-leak surfaces; downgraded to debug and
stripped of the sensitive detail. Close/Resume frame handling likewise
moved from info to debug.
- The access key (a shared secret) was written to logs verbatim in three
places (session drop, key-created UI event, API create-user) and as an
8-char prefix in one. Added key_fp() — a short SHA-256 fingerprint — and
routed all key logging through it so operators can still correlate
events without the secret ever hitting the log.
Every datagram from an unrecognized source ran the full key-trial loop:
for each registered access key, an HKDF (derive_all_secrets) plus two
HMACs (junk markers) plus a Noise read. A garbage flood from spoofed
sources could therefore force unbounded O(N_keys) crypto per packet — a
CPU-amplification DoS with no throttle (the existing token bucket only
guarded the roaming path, not this one).
Two mitigations:
- Memoize the per-key derived secrets (pure function of key+version) and
the per-window junk markers, so the trial loop is now cheap comparisons
plus one Noise read per key instead of HKDF+2*HMAC per key per packet.
Also speeds up every legitimate new connection. Caches are pruned in
on_tick when keys are deleted.
- Gate the trial path behind a global token bucket (TRIAL_RATE=100/s,
same burst). The established-session fast path and roaming are not
gated, so live sessions are unaffected; only unknown-datagram trials
are bounded. Over-budget datagrams are dropped silently.
The junk marker was a per-key CONSTANT sent in plaintext at a fixed offset in
junk frames. Junk is meant to look like random noise (zapret-style), but a
constant prefix is a recognizable per-user structure: an on-path observer
watching one user sees the same 4 bytes on every junk packet, i.e. an OSTP
fingerprint. (The earlier fix only removed the GLOBAL constant.)
Now the marker rotates every 60s window: junk_marker = HKDF(key, ver, 0x04 ||
window). To an observer the prefix changes each window (no fixed signature),
and a captured marker is only valid for ~1 window — the "bit of protection"
against a leaked marker. Only a key holder can compute it, so an outsider still
can't forge a silently-dropped junk packet (and silent-drop is cheaper than
normal processing anyway, so junk spam was never a DoS lever to begin with).
- core: derive_junk_marker(key, window) + current_junk_window() (60s window),
same version-gated HKDF scheme; junk_marker dropped from DerivedSecrets.
- client: stamps junk with the current window's marker.
- server: checks current AND previous window per key (absorbs ~1 window of
clock skew) before falling through to unauthorized-probe handling.
- Not a wire break: only junk framing changes; real handshake/data untouched.
During mixed rollout, unmatched junk merely logs as a probe (cosmetic).
security / protocol:
- Derive a PER-KEY junk marker (obfuscation.rs, info byte 0x04) instead of the
global constant [0x88,0x1A,0x93,0x5D]. A fixed marker was a universal DPI
signature identifying ALL OSTP users at once — exactly what the HKDF version
gate avoids for the handshake. Server drops junk via a new DispatchOutcome::Junk
inside the existing key-trial loop (secrets already derived → zero extra cost);
client stamps its own key's marker.
- §E: configurable junk/fragmentation params (junk_pc / junk_ps / frag_chunk / frag_sleep).
GUI (desktop):
- Light theme + toggle, GUI version footer in Settings.
- Fix mouse-wheel scroll on Settings (flex child needed min-height: 0).
- Drop the false "process exclusions unsupported in TUN mode" warning — they DO
work (native_handler maps port->process via GetExtendedTcpTable).
release / infra:
- build.ps1: add -PreRelease (tag CURRENT version as v<ver>-beta.N, no bump, no
master commit); guard the panel build when ostp-control ships no source; bump
the real ostp-gui/package.json instead of the nonexistent ostp-control one.
- release.yml: mark hyphenated tags as GitHub pre-releases; don't hard-fail the
web-panel step when there is no source (use committed dist/).
- Versions aligned to 0.4.1; README license badge BSL 1.1 -> AGPL v3.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Ported only the fixes that actually apply to the pre-refactor base
(the handshake fixes 6eb7b36/d65af35 fix bugs the 0.3.1 multi-server
refactor introduced into the new outbounds/ostp.rs; the base bridge.rs
already waits for the handshake response with retransmit + NAT64
fallback, so they are intentionally skipped).
- EMFILE (922cf0b): rlimit::increase_nofile_limit at CLI startup.
- Logs (1151726): UoT connect/disconnect → debug; rate-limit the
unauthorized-probe log to one line / ~30s so a junk/probe flood can't
spam the log (and a client running junk-over-UDP can't self-ban).
- Helper lifecycle + bypass routes (b6e78c1):
* ostp-tun-helper forces std::process::exit(0) after run_server so the
WinTun adapter and its metric-0 default route are reclaimed instead
of lingering as a zombie that breaks the next connect.
* windows_route: delete_routes_for_dest() purges stale /32s, dedupe
bypass IPs, and log add failures at warn!.
* windows: retry tun::create through the transient ERROR_INVALID_
PARAMETER window and widen the adapter-index lookup to ~15s.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The core bug: server sent 5 TLS records in server_hello but client only
read the first one (ServerHello), then passed remaining bytes (CCS + fake
records) into RealityStream. RealityStream saw 0x14 (CCS) != 0x17 and
immediately returned an error, killing the connection.
Changes:
- reality.rs: append ChangeCipherSpec after ClientHello (RFC 8446 D.4)
export REALITY_SERVER_HANDSHAKE_RECORDS=5 constant
- xhttp.rs: drain all 5 server handshake records before creating RealityStream
- uot.rs: rebuild server_hello as proper 5-record TLS 1.3 flight:
ServerHello + CCS + fake EE (108B) + fake Cert (812B) + fake Fin (52B)
drain client CCS from raw stream before wrapping in RealityStream
- Flutter: Hide 'DNS Server' field and force '10.1.0.1' if connection link contains owndns=true
- Flutter: Remove 'Use Provider DNS' toggle to eliminate client-side choice
- Server (relay.rs): Intercept DNS queries targeting '10.1.0.1:53' and process them via internal DnsServer if DNS is enabled
- Server (api.rs): Continue appending owndns=true to subscription links to enforce internal DNS logic on clients
- ostp-server/relay.rs: remove DNS port 53 interception — DNS queries
now pass through to the actual DNS server as regular TCP connections
- ostp-client/native_handler.rs (Windows): add explicit gateway/32 route
via real interface BEFORE setting default route via TUN to prevent loop
- ostp-client/native_handler.rs (Linux): properly detect real gateway and
add default route via TUN with metric 10 after server IP exclusion
- Remove redundant extra DNS host routes from Windows setup script
ospab
ospab