ResearchCave.LiveExchangeRates.Client (1.0.0)
Installation
dotnet nuget add source --name ResearchCave --username your_username --password your_token https://git.researchcave.com/api/packages/ResearchCave/nuget/index.jsondotnet add package --source ResearchCave --version 1.0.0 ResearchCave.LiveExchangeRates.ClientAbout this package
Redis client for ResearchCave live exchange rates with timestamp, provenance, and freshness requirements.
ResearchCave.LiveExchangeRates
.NET 10 collector, console host, and independently reusable Redis client. The collector subscribes to selected public spot markets on Binance, BtcTurk, Paribu, Huobi/HTX, OKX, and Kraken simultaneously. No API keys are required.
Projects
| Project | Purpose |
|---|---|
ResearchCave.LiveExchangeRates |
Provider sockets, conversion, aggregation, Redis publishing, disk snapshots, and an IHostedService registration for other .NET hosts. |
ResearchCave.LiveExchangeRates.Client |
DTOs and Redis reader; reference this from the payment service. It does not open exchange connections. |
ResearchCave.LiveExchangeRates.Console |
Configured collector with colored connection, subscription, retry, provider freshness, price, and storage logs. Redirected output contains no ANSI colors. |
ResearchCave.LiveExchangeRates.Tests |
Deterministic pricing/protocol/persistence tests and integration tests using a real loopback Garnet RESP server. Garnet is only a test dependency. |
Run
From this directory:
docker compose up -d
dotnet run --project ResearchCave.LiveExchangeRates.Console
Alternatively point the settings at an existing Redis server. In console mode the collector still runs and saves disk snapshots while Redis is unavailable, logging a warning; Redis clients become available once the connection recovers. Managed service mode requires Redis and stops on failure, as described below. Ctrl+C shuts down connections and flushes the latest state.
Interactive terminals use a live dashboard, refreshed every 250 ms from in-memory WebSocket quotes. It displays each provider's latest received market, price and timestamp, aggregate rates, storage health, and recent connection events. Aggregate values follow RedisPublishIntervalMilliseconds (default 100 ms); disk snapshots follow the independent DiskPersistIntervalSeconds (default 10 seconds). Screen refresh never queries an exchange. Longer rate tables rotate pages every 8 seconds. Ages continue increasing when a feed stalls. Provider rows can show different markets/quote currencies; compare prices using the aggregate table.
Redirected output uses ordinary logs and the periodic summary (StatusIntervalSeconds, default 15). To select ordinary logs in a terminal, pass --Console:LiveDashboard false. Outlier exclusions are shown only when the excluded-provider list is nonempty; they do not indicate rejection of the aggregate itself. FRESH describes quote freshness, not successful Redis persistence; Redis and disk health are displayed separately.
Edit ResearchCave.LiveExchangeRates.Console/appsettings.json before building/running. The console reads settings from its executable directory; the build copies this file there. Environment variables and command-line overrides are also supported:
$env:LiveExchangeRates__Redis__ConnectionString = 'localhost:6379,abortConnect=false'
$env:LiveExchangeRates__SnapshotPath = 'G:\ExchangeRateData\latest-rates.json'
dotnet run --project ResearchCave.LiveExchangeRates.Console
Use an absolute, writable SnapshotPath in production so deployments and working-directory changes do not move the cache. Relative paths are relative to the process working directory. Keep connection credentials in deployment secrets/environment variables. For a remote Redis deployment, use its authenticated/TLS connection string and restrict network access. The supplied development Compose binds to loopback only.
Select currencies and markets
See Run as a Windows Service or systemd service below for automatic service-mode detection and installation.
Symbols: assets consumers need, such asBTC,ETH,SOL,USDT,EUR,TRY.QuoteCurrencies: result currencies to precompute, such asUSD,EUR,TRY. Client calls default toUSD; requested quote currencies must be configured here.BridgeCurrencies: currencies permitted for intermediate conversion, such asUSDT,USD,EUR,TRY. There is no hardcoded stablecoin peg.Providers[].Markets: explicit mapping from the provider's instrument name to canonicalBASE/QUOTE. Only markets whose two assets are in the configured symbol/quote/bridge sets are subscribed. No full-market ticker feed or fetch-everything-and-filter operation is used.Providers[].Enabled: enable/disable each provider independently.
The sample includes mappings for BTC, ETH, SOL, USDT, EUR, TRY and Kraken GBP/USD. To add a new asset, add it to Symbols and add supported instrument mappings under the providers, for example "XRPUSDT": "XRP/USDT" for Binance and "XRP/USD": "XRP/USD" for Kraken. Mappings are intentionally explicit: listing an asset does not invent unsupported exchange markets. Subscriptions are split into groups of 40 and throttled; errors and markets without fresh data are visible in the console. Configuration changes require restarting the host.
Fiat coverage is limited to available exchange markets/conversion routes. EUR/USD can use a traded fiat pair; TRY/USD is an exchange-implied rate via USDT/TRY and USDT/USD. These are not bank settlement or central-bank reference rates. No daily fiat quote is silently treated as live. Unsupported currencies return no result until an appropriate market and conversion route are configured.
Aggregation and freshness
- Keep only the latest valid bid/ask quote per provider and market. Use decimal arithmetic and
(bid + ask) / 2as an indicative midpoint, rather than an old last trade or 24-hour average. - Reject non-positive, crossed, excessively wide-spread, and future-dated quotes. Default spread limit is 2%; maximum permitted clock skew is 5 seconds. Ignore out-of-order source timestamps.
- Only quotes within
FreshnessSeconds(default 30) may participate. Prefer source timestamps; Binance book ticker and BtcTurk ticker lack one, so their freshness measures local receipt time. Receipt time cannot prove exchange-side age. Keep the collector clock synchronized. - For each conversion market, reject prices more than
MaximumDeviationPercent(default 2%) from its median, require a strict majority, and average the retained quotes equally. Two widely disagreeing providers yield no consensus. This fixed median gate is deliberately inspectable and configurable; thin/volatile markets may need a different threshold. - Convert each provider's source price to the requested currency using fresh conversion-market consensus. Prefer shortest routes through the configured bridge currencies, using at most two conversion legs. Routes cannot revisit currencies or use the requested base asset as their own conversion bridge. Prefer direct markets when an exchange supplies multiple alternatives. A provider gets only one vote regardless of tick frequency or number of markets.
- Apply the same median/majority filter to converted provider votes and take their equal-weight arithmetic mean. Report rejected providers, accepted prices, paths, and conversion providers. No exchange receives extra influence from self-reported volume.
AsOfis the oldest timestamp among all contributing prices and conversion legs.CalculatedAtand snapshotPublishedAtare bookkeeping times and must never be used as quote freshness. Rates are recomputed/published to Redis every 100 ms by default. New valid quotes replace older source quotes immediately in memory; there is no rule that retains a one-second-old quote instead of a half-second-old replacement. A client sees the next successful Redis publication, so network/storage latency still matters and 100 ms is a scheduling interval, not a hard end-to-end guarantee.- With no fresh consensus, preserve the last computed price and its original timestamps, setting
HasCurrentConsensus=false. Never re-average yesterday's prices with today's or advance the timestamp of an old rate.IsFresh(now)also checks current consensus. A stopped collector's Redis results age out naturally on the reader side.
One source can produce a latest rate; the payment-oriented GetFreshAsync defaults to at least two contributing source providers. Quorum counts source exchanges, not independent conversion infrastructure: several source exchanges may share one USD conversion provider. The sample's genuine USD bridge relies heavily on Kraken. Each contribution exposes MinimumConversionProviders (zero for a direct price) and ConversionProviders; a payment policy can require stronger conversion quorum as well. Outlier filtering reduces bad ticks but is not a guarantee against correlated venue failures. Midpoints exclude fees, spread/slippage on execution, and payment-provider margins.
Payment service integration
Reference the client project/package and reuse the site's singleton IConnectionMultiplexer:
using ResearchCave.LiveExchangeRates.Client;
using StackExchange.Redis;
// Omit this registration if the site already supplies IConnectionMultiplexer.
builder.Services.AddSingleton<IConnectionMultiplexer>(_ =>
ConnectionMultiplexer.Connect(builder.Configuration.GetConnectionString("Redis")!));
builder.Services.AddLiveExchangeRateClient(new RedisRateOptions
{
Key = "researchcave:live-exchange-rates:v1:snapshot",
Database = 0
});
Inject ILiveExchangeRateClient into the payment component:
// Throws RateUnavailableException if absent, stale, without consensus, or below quorum.
var rate = await rates.GetFreshAsync("BTC", cancellationToken: cancellationToken);
decimal usdValue = btcAmount * rate.Price;
// A different configured output currency:
var lira = await rates.GetFreshAsync("ETH", "TRY", cancellationToken: cancellationToken);
// Explicit fallback for a display or a business policy that allows old prices:
var latest = await rates.GetLatestAsync("BTC", cancellationToken: cancellationToken);
if (latest is not null)
{
var age = latest.Age(DateTimeOffset.UtcNow);
var fresh = latest.IsFresh(DateTimeOffset.UtcNow);
}
// Optional stricter requirements:
var strict = await rates.GetFreshAsync("BTC", "USD", minimumProviders: 3,
maximumAge: TimeSpan.FromSeconds(10), cancellationToken: cancellationToken);
if (strict.Contributions.Any(c => c.Markets.Length > 1 && c.MinimumConversionProviders < 2))
throw new RateUnavailableException("Payment requires two providers per conversion leg.");
Redis outages propagate as Redis exceptions; malformed/unsupported snapshots are rejected. Decide explicitly whether to retry or decline the payment. The client does not silently switch to a local stale cache. GetLatestAsync may return an arbitrarily old last-known price; always inspect its timestamp if that matters to the caller. Client calls read one atomic snapshot, avoiding mixed publication versions. Same-currency pairs such as EUR/EUR and TRY/TRY are omitted from calculation, stored snapshots, restored legacy data, and console output. For these pairs GetLatestAsync returns null and GetFreshAsync throws RateUnavailableException. A payment already denominated in the paid currency needs no exchange-rate lookup.
To embed the collector in another host, reference the collector project and call builder.Services.AddLiveExchangeRateCollector(builder.Configuration). Usually deploy the collector once and reference only the client from websites. No HTTP endpoints, identities, or authorization policies are introduced by these libraries.
Persistence and operations
- Redis stores one versioned JSON snapshot key without TTL; each
SETatomically replaces the previous snapshot. No historical rate series is accumulated. Do not share the key with another application/schema. - The disk snapshot contains only latest provider quotes and latest aggregates. A separate worker saves it every
DiskPersistIntervalSeconds(default 10 seconds), independently of the Redis loop; disk writes do not serialize behind Redis calls or block that loop. It writes a temporary file, flushes it to disk, then replaces the destination on the same filesystem. A failed write retains the previous snapshot and logs an error. Startup and graceful shutdown also flush a snapshot. A crash may lose disk updates since the last disk save (normally up to 10 seconds, longer if storage is delayed/failing); there is no claim of transactional consistency between disk and Redis. Redis AOF persistence is separate. - Startup restores disk state and any newer Redis results, retaining original timestamps. It publishes the restored state before opening provider sockets. Quotes from disabled/unconfigured markets are not restored into the live calculation.
- The supplied Redis Compose configuration enables AOF (
appendfsync everysec) and a persistent volume. Existing Redis installations must enable their own persistence. Collector disk recovery also works after Redis loses its data, provided the collector's snapshot survives. - Run one active collector per Redis key, with a persistent disk location. A local exclusive lock prevents two processes using the same disk path. It is not a distributed lease; multi-host leader election is not implemented.
- Sockets reconnect with exponential backoff and jitter, capped around 60 seconds, and resubscribe after disconnects. WebSocket ping/pong, Huobi gzip/application ping/pong, OKX text ping/pong, fragment assembly, payload limits, connect/send timeouts, and idle-feed reconnects are handled. Quiet symbols age independently even when connection heartbeats continue.
- Public endpoints may be geographically restricted; URLs are configurable. Subscription errors are logged and do not create prices. Fix invalid/removed market mappings and restart. Paribu's current public book-ticker API is documented as beta, so monitor its protocol changes.
- The freshness threshold is shared across fiat and crypto. This intentionally does not pretend that inactive fiat markets remain fresh over a weekend. The last-known value is still readable.
The former shared PublishIntervalSeconds setting has been replaced by:
"RedisPublishIntervalMilliseconds": 100,
"DiskPersistIntervalSeconds": 10
Use the new settings when upgrading existing deployments. Redis interval must be at least 10 ms and disk interval at least 1 second. Neither interval changes the 30-second freshness window or makes old input data fresh.
Run as a Windows Service or systemd service
The console host includes Microsoft.Extensions.Hosting.WindowsServices and Microsoft.Extensions.Hosting.Systemd. It automatically detects native Windows/systemd services. Service wrappers such as NSSM must pass --service because their child process appears to be a console application. This flag enables required Redis, suppresses the dashboard/banner/rate summaries, and writes warnings/errors to redirected console output for wrapper logs. Windows service integration and systemd hosting handle start/stop lifetime signals. The same executable retains its dashboard when launched interactively.
Service mode never starts the dashboard/render timer, prints a banner, or formats periodic rate summaries. Only warnings and errors are logged: Event Log on Windows and journald on systemd. This suppresses routine subscription and price messages without hiding failures. A relative snapshot path in service mode resolves against the executable directory, rather than Windows System32; the installation examples below instead specify a dedicated writable absolute path. Normal command-line runs keep their existing relative-path behavior. Settings are read from the executable directory in both modes.
Service mode automatically enables RequireRedis. After creating the Redis connection, startup waits up to LiveExchangeRates:RedisStartupTimeoutSeconds (default 30 seconds) to read the startup snapshot, allowing Redis to become ready after machine boot. Initial connection negotiation also has its own short timeout. If the startup deadline expires, or a subsequent Redis read/write fails, the collector logs Critical (the .NET equivalent of Fatal), flushes disk state during runtime shutdown, and stops the host. The console host returns process exit code 1 and reports a nonzero Windows service exit code, so failure is visible to the service manager. Redis exception messages and connection strings are not logged.
Normal console mode keeps collecting and saving disk snapshots during Redis outages, logging one warning per outage and republishing the latest state after reconnect. A successful write resets warning suppression so a later outage is reported again. Hosts embedding the collector can opt into strict behavior with LiveExchangeRates:RequireRedis=true; they must stop on background-service exceptions and propagate failure to their own service manager. Explicitly stopping the collector does not turn a failed final Redis flush into a fatal error.
Windows
Publish for the server's architecture (the example requires the .NET 10 runtime on the target):
dotnet publish ResearchCave.LiveExchangeRates.Console -c Release -r win-x64 --self-contained false -o artifacts/windows-service
Copy the published directory to C:\Services\ResearchCave.LiveExchangeRates, configure its appsettings.json, then run an elevated PowerShell:
New-Item -ItemType Directory -Force 'C:\ProgramData\ResearchCave\LiveExchangeRates'
New-Service -Name 'ResearchCave.LiveExchangeRates' `
-DisplayName 'ResearchCave Live Exchange Rates' `
-BinaryPathName '"C:\Services\ResearchCave.LiveExchangeRates\ResearchCave.LiveExchangeRates.Console.exe" --LiveExchangeRates:SnapshotPath "C:\ProgramData\ResearchCave\LiveExchangeRates\latest-rates.json"' `
-StartupType Automatic
Start-Service 'ResearchCave.LiveExchangeRates'
Get-Service 'ResearchCave.LiveExchangeRates'
# Before replacing application files during an upgrade:
Stop-Service 'ResearchCave.LiveExchangeRates'
The service account needs write access to the snapshot directory and network access to Redis and the providers. Inspect warnings/errors under Event Viewer ? Windows Logs ? Application. Do not also run an interactive collector against the same Redis key.
If Redis runs as a Windows service on the same machine, use its actual service name as a dependency. Find the name with Get-Service *redis*, *memurai*. For a new installation add -DependsOn 'Redis' to New-Service; for an existing installation, run in elevated PowerShell (replace Redis with the installed service name):
sc.exe qc 'ResearchCave.LiveExchangeRates'
sc.exe config 'ResearchCave.LiveExchangeRates' depend= 'Redis'
depend= replaces the dependency list; preserve any existing dependencies shown by qc, separating multiple names with /, for example depend= 'Redis/Tcpip'. This makes Windows start the dependency before the collector. It applies only to local services; it cannot order startup against Redis on another server or inside Docker/WSL. Dependency startup does not guarantee that Redis accepts authenticated writes immediately, so the collector still reconnects automatically. See New-Service dependencies and sc.exe config.
Optionally configure recovery to retry after a failure. Because the collector shuts down gracefully with a nonzero service exit code, enable recovery for non-crash failures as well:
sc.exe failure 'ResearchCave.LiveExchangeRates' reset= 86400 actions= restart/10000/restart/30000/restart/60000
sc.exe failureflag 'ResearchCave.LiveExchangeRates' 1
Without recovery actions, a fatal Redis failure leaves the service stopped until it is started again. See Windows service failure actions.
For NSSM, configure the published appsettings.json in a separate staging directory, then run the supplied installer in elevated PowerShell:
.\deploy\install-nssm.ps1 -SourceDirectory 'D:\ServerApps\.staging\RCLiveExchanges\release'
Defaults are service RCLiveExchanges, application directory D:\ServerApps\RCLiveExchanges, NSSM D:\ServerApps\nssm.exe, and Redis service Redis; these are parameters. The installer passes --service, enables automatic startup and the Redis dependency, allows 30 seconds for graceful shutdown, and captures rotating warning/error logs under logs. It configures NSSM to stop when the application exits, preserving visible stopped status after fatal Redis failures. Upgrades stop the service, back up the existing directory under the sibling Backups directory, retain data/log files, and restart it. Check Get-Service RCLiveExchanges and Get-Content D:\ServerApps\RCLiveExchanges\logs\service.log -Tail 30.
Linux / systemd
Publish for the server's architecture (requires the .NET 10 runtime on the target):
dotnet publish ResearchCave.LiveExchangeRates.Console -c Release -r linux-x64 --self-contained false -o artifacts/linux-service
Copy the published directory to /opt/researchcave-live-exchange-rates, configure its settings, and install the supplied deploy/researchcave-live-exchange-rates.service:
sudo useradd --system --no-create-home --shell /usr/sbin/nologin researchcave-rates
sudo chmod +x /opt/researchcave-live-exchange-rates/ResearchCave.LiveExchangeRates.Console
sudo cp deploy/researchcave-live-exchange-rates.service /etc/systemd/system/
sudo systemctl daemon-reload
sudo systemctl enable --now researchcave-live-exchange-rates
sudo systemctl status researchcave-live-exchange-rates
sudo journalctl -u researchcave-live-exchange-rates -f
The unit uses Type=notify, a dedicated account, and StateDirectory to create the writable /var/lib/researchcave-live-exchange-rates snapshot directory. It supports optional environment overrides in /etc/researchcave-live-exchange-rates.env, including LiveExchangeRates__Redis__ConnectionString. Stop with systemctl stop researchcave-live-exchange-rates; stop before replacing application files. The unit allows 45 seconds for shutdown; the host's graceful-shutdown timeout is 30 seconds. Host readiness means the worker has started, not that every market already has fresh prices; payment clients must still enforce freshness.
For Redis managed by systemd on the same machine, run sudo systemctl edit researchcave-live-exchange-rates and add a drop-in using the actual Redis unit name (often redis-server.service or redis.service):
[Unit]
Requires=redis-server.service
After=redis-server.service
Then run sudo systemctl daemon-reload and sudo systemctl restart researchcave-live-exchange-rates. Requires brings Redis into the startup transaction, requires its successful startup, and stops the collector when Redis is explicitly stopped; After orders the collector after it. The supplied base unit supports remote Redis, so it has no dependency on a particular local Redis unit. Its Restart=on-failure retries fatal failures after 5 seconds; to leave the service stopped for manual intervention, set Restart=no in a drop-in. See systemd unit dependencies and ordering.
Browser display notifications
The collector writes authoritative snapshots at its existing Redis cadence and also
publishes the latest snapshot to Redis.UpdatesChannel (default: the snapshot key
plus :updates). RedisNotificationIntervalSeconds defaults to three seconds and
controls only display notifications, independently of fast snapshot writes. Redis
channels are not database-scoped; use distinct keys/channels for each environment.
Applications can consume these events through
ResearchCave.LiveExchangeRates.AspNetCore in ResearchCave.Defaults and their existing
SignalR hub. This uses an initial snapshot read and pushed updates rather than
browser or server price polling. Source timestamps still determine freshness.
Redeploy this collector version to enable push notifications in those applications.
TBT/USD remains unavailable until its real rate source is added; consumers must hide
unavailable equivalents rather than assume a peg.
Build and verification
dotnet build -c Release
dotnet test -c Release
dotnet pack ResearchCave.LiveExchangeRates.Client -c Release -o artifacts/packages
dotnet pack ResearchCave.LiveExchangeRates -c Release -o artifacts/packages
dotnet publish ResearchCave.LiveExchangeRates.Console -c Release -o artifacts/console
The tests do not require external exchange access or an existing Redis server. The loopback RESP integration server uses an ephemeral port. Live smoke testing on 2026-09-20 verified accepted prices from all six providers, six-source BTC/USD consensus, disk restore before the first socket updates, and continued collection with Redis offline. The local Docker daemon was unavailable; Redis protocol integration was exercised with Garnet rather than a Docker Redis instance.