Instead of writer pinning, we'll track the last transaction ID of each
write in the session. Then on each read we'll wait for the replica to
report that this transaction is available.
If it doesn't become available within a reasonable timeout, we'll
proceed anyway, and accept the possibility of a stale read.
The hope here is that most of the time, the replica is caught up in the
time between a write request and the following read request. If it's
not, we now have a little tolerance to wait for it, which hopefully
proves enough to stale reads are not encountered in normal use.
We also disable the writer affinity opt-out mechanism that we had
before, since we will no longer be using writer affinity at the load
balancer.
In order to control routing dynamically, we need to expose some Beamer
information to the proxy.
The `beamer_writer` value is used by the proxy to keep track of which
node should receive write requests. When it changes due to a failover,
the proxy will update after seeing a new value in this header. We
provide this to the proxy in the `X-Writer` response header.
The `beamer_last_txn` value will be used to control writer pinning.
Setting it in the cookie here is the first stage of this. The second
stage will be to catch situations where a reader gets a request where
this header value is set to a large value than the reader has seen; when
that happens if means there's a risk of reading stale data, so rather
than serve the request we should request the proxy to reproxy it back to
the writer.
We also set `X-Kamal-Target` in the response to match the
`X-Kamal-Target` that was set in the request; this lets us see which
proxy target served each request, which is useful for diagnostics.
Yes, Platform Agent is overkill for just this but there isn't a one-size JS solution for updating both input placeholders and other HTML text and we're likely to need it later for other things like displaying platform-specific PWA prompts.
Kevin McConnell