Designing a silver market data pipeline for WSS
By Dimitri Lafleur
Wall Street Silver needs reliable market data: CFTC commitment of traders reports, COMEX inventory levels, LBMA prices, and community sentiment signals from Reddit.
The hard part is joining sources with different schemas, normalizing timestamps, keeping one source failure from stopping the rest, and storing the result in a queryable time-series format.
Pipeline components#
- Reddit API ingestion for sentiment extraction
- CFTC data parsing and normalization
- COMEX/LBMA price feeds
- Time-series database for storage
- Analytics layer for signal extraction
- Visualization dashboard
The pipeline gives Wall Street Silver a single queryable store for market data, sentiment comparisons, and research views.
Related work
The market-data path.
Wall Street Silver's research workflow connects to the market systems and community work below.