Instead of writer pinning, we'll track the last transaction ID of each
write in the session. Then on each read we'll wait for the replica to
report that this transaction is available.
If it doesn't become available within a reasonable timeout, we'll
proceed anyway, and accept the possibility of a stale read.
The hope here is that most of the time, the replica is caught up in the
time between a write request and the following read request. If it's
not, we now have a little tolerance to wait for it, which hopefully
proves enough to stale reads are not encountered in normal use.
We also disable the writer affinity opt-out mechanism that we had
before, since we will no longer be using writer affinity at the load
balancer.
* Bind sessions to identities
* Remove references to the identity token
* Move email changes to identity
* Move account menu into a turbo-frame
* Create tenants from a tenanted route
The new integration test shows the desired user-facing behavior, which
is to make it easy to login without a tenanted URL and to jump between
tenants.
Note that we track two things in the identity_token cookie: a signed
id, and the updated_at for the underlying Identity object. This allows
us to effectively cache on the Identity without having to hit the
database, by using an Identity::Mock object that is compatible with
etag and cache methods.
Up from the previous 5 seconds. It's still short enough that the pages
shouldn't feel stale, but it further reduces the number of requests that
will reach the app.
Also moved this into a shared concern so we can adjust the caching rule
in one place.
In order to control routing dynamically, we need to expose some Beamer
information to the proxy.
The `beamer_writer` value is used by the proxy to keep track of which
node should receive write requests. When it changes due to a failover,
the proxy will update after seeing a new value in this header. We
provide this to the proxy in the `X-Writer` response header.
The `beamer_last_txn` value will be used to control writer pinning.
Setting it in the cookie here is the first stage of this. The second
stage will be to catch situations where a reader gets a request where
this header value is set to a large value than the reader has seen; when
that happens if means there's a risk of reading stale data, so rather
than serve the request we should request the proxy to reproxy it back to
the writer.
We also set `X-Kamal-Target` in the response to match the
`X-Kamal-Target` that was set in the request; this lets us see which
proxy target served each request, which is useful for diagnostics.
Kevin McConnell