<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<title>Phase 3.5 — Cooperative atto Audit and MailSite Daemon
— Arc Plan</title>
<style>
body { font-family: system-ui, sans-serif; max-width: 760px;
margin: 1.5em auto; padding: 0 1em; line-height: 1.5; color: #1a1a1a; }
h1 { font-size: 1.4em; }
h2 { font-size: 1.1em; margin-top: 1.4em; }
ol { margin: 0.3em 0; } li { margin: 0.25em 0; }
ol.slices { list-style: none; padding-left: 0; }
.check { color: #1a7f37; font-weight: bold; }
.next { color: #b35900; font-weight: bold; }
.q { color: #8250df; }
code { background: #f2f2f2; padding: 0 0.25em; border-radius: 3px; }
em { color: #555; }
</style>
</head>
<body>
<h1>Phase 3.5 — Cooperative atto Audit and MailSite Daemon</h1>
<p style="margin:.8rem 2rem;padding:.7rem 1rem;background:#eef2f7">
<strong>Arc Post-Mortem Summary.</strong>
A direct follow-on to Phase 3's cooperative
web server. It audited each atto server for the same single-process blocking
pattern and carried the non-blocking treatment across the rest of atto, and
it made the MailSite daemon's own loop cooperative. Delivered in slices under
the constraints carried in from Phase 3.</p>
<p><em>Arc started 2026-06-20, straight after the Phase 3 cooperative
server work landed. Phase 3 made the atto <strong>web</strong> server
non-blocking: long work (an external IMAP probe, a password hash, a big
media list, an SMTP send, a local-mailbox read) now yields to the event
loop and is re-entered, instead of freezing every other web connection.
This arc carries the same treatment to the rest of atto — it audits
each server for the same single-process blocking pattern, makes the
MailSite <strong>daemon's</strong> own loop cooperative, and writes worked
examples of the fiber primitives so later code follows the pattern rather
than reinventing it.</em></p>
<h2>Audit of the atto servers (verified 2026-06-20)</h2>
<p>Every atto server is a non-blocking reactor: sockets are non-blocking, a
select loop drives reads and writes, partial input is buffered across
passes. So a slow peer never blocks the loop. The freeze instead comes
from a command <em>handler</em> doing blocking work mid-pass — a
store lock, a file read, an upstream lookup. The fix is the web-side one:
run the handler in a fiber so its cooperative blocking op yields.</p>
<ol>
<li><strong><code>WebSite.php</code> — done.</strong> Handlers run
in fibers (Phase 3) that suspend on store reads, IMAP, the TLS
handshake, CPU yields, and worker pipes.</li>
<li><strong><code>MailSite.php</code> — this arc.</strong> Reactor
network path is fine; <code>processOne</code> handlers block on the
file store (<code>flock</code> and disk). Slice 7 made the store read
locks cooperative; this arc runs <code>processOne</code> in a fiber so
they yield, then does the same for the delivery write locks.</li>
<li><strong><code>SshSite</code>, <code>FtpSite</code>,
<code>GopherSite</code>, <code>DnsSite</code> (TCP) — same
pattern, atto repo.</strong> All reactors; handlers block on file reads
(Gopher, FTP retrieve), the FTP data-channel
<code>stream_socket_client</code> connects, or DNS upstream lookups.
Not vendored in Yioop, so they get the same minimal fiber frame in the
atto repo, following this arc's examples.</li>
<li><strong><code>TurnSite</code>, <code>H3Listener</code>,
<code>H3QuicheListener</code>, <code>DnsSite</code> (UDP) — out
of scope.</strong> UDP / QUIC and connectionless: no per-connection
accept or blocking read, so the freeze pattern does not apply. Listed
so the audit is complete.</li>
</ol>
<h2>Slices</h2>
<ol class="slices">
<li><span class="check">✓</span> <strong>Slice 1 — run the
daemon's command handling in a fiber.</strong> <code>readClient</code>
now runs a connection's <code>processOne</code> loop in a fiber; if a
store lock yields, the fiber parks and the loop resumes it each pass
(polling while any are parked), dropping it on finish or shutdown. With
Slice 7's cooperative read locks, the daemon's store reads now yield
instead of freezing every other mail connection. The scheduler question
is settled: each server stays self-contained, so this is a small inline
frame, not a shared class.</li>
<li><span class="check">✓</span> <strong>Slice 2 —
cooperative delivery write locks.</strong> The daemon's exclusive-lock
write paths (<code>allocUid</code>, <code>bumpFolderUidNext</code>),
left blocking in Slice 7, now use the cooperative wait too, so a
delivery waiting on a reader yields rather than stalling the loop.
<code>cooperativeFlock</code> now returns a bool and tells contention
(retry) from a real lock error (fail fast), so the write paths keep
their failure handling. Mirrored to the atto repo's
<code>MailSite</code> (Slice 1 and 2 together).</li>
<li><span class="check">✓</span> <strong>Slice 3 — the
atto-only servers (atto repo).</strong> Of the four, only two have a
blocking handler: <code>FtpSite</code>'s data-channel
<code>stream_socket_client</code> connect, and <code>GopherSite</code>'s
<code>file_get_contents</code> read-all of a served file. Each gets the
command-fiber frame plus a cooperative version of that one op (async
connect; chunked read with a yield). <code>DnsSite</code> (UDP
datagram) and <code>SshSite</code> (reactor reads, one startup
host-key read) have no per-request blocking, so they are left as is.
Done one server per step; not unit-testable in the Yioop sandbox, so
these lean on integration testing. <em>Done:</em>
<code>FtpSite</code> got the command-fiber frame plus cooperative
accept/connect on the data channel (3a) and cooperative transfer
pumps for <code>RETR</code>/<code>STOR</code>/<code>LIST</code> (3b);
<code>GopherSite</code>'s read-all became a chunked cooperative read,
validated byte-identical against a real <code>lynx</code> client;
<code>DnsSite</code> and <code>SshSite</code> were confirmed to have
no per-request blocking and left as is.</li>
<li><span class="check">✓</span> <strong>Slice 4 — worked
examples of the fiber primitives.</strong> A short runnable reference
for the few primitives this work introduced — the
readable/writable suspend, the plain yield, the cooperative lock, the
command-fiber frame, the worker offload — added as a new atto
example, with a defer case in the benchmark example to show the
speed-up. <em>Done:</em> example 26 ships as both a CLI run and a web
page whose per-demo code panels are filled by reflection at request
time (so they cannot drift from the real source); the benchmark
example carries the offload defer case. Closes the arc.</li>
</ol>
<p><strong>Arc closed 2026-06-21.</strong> All four slices landed. The
atto servers that serve real requests (Web, Mail, Ftp, Gopher) now
cooperate inside a fiber, while the daemon run on its own, the
command-line tools, and Apache stay byte-for-byte unchanged. Dns and Ssh
were audited and need no per-request change.</p>
<h2>Constraints carried in</h2>
<ol>
<li>atto files stay self-contained: no new files in
<code>atto_servers/</code>, no reaching into another atto file, and the
daemon defines its own constants rather than reading Yioop
<code>Config</code>.</li>
<li>Every change keeps the non-fiber path (the daemon run on its own, the
command-line tools, Apache) byte-for-byte as it was: cooperation only
engages inside a fiber.</li>
<li>The full suite stays green at each step; new primitives get a focused
test the way the web-side ones did.</li>
</ol>
</body>
</html>
(c) 2026 Seekquarry - Open Source Search Engine Software