60 Seconds to 500 Milliseconds. One Data Model Change. π―
I love data modeling. Creating shapes that map to access patterns as they grow in cardinality β it's a multi-dimensional puzzle I visualize like an Escher drawing in my head. Projections intersecting across planes.
Today that puzzle clicked on a call with a major financial services customer. And it proved something I've been saying for years.
The Problem π
Large advisory platform. Thousands of advisors managing investor portfolios. MongoDB performing well for 90% of queries β entity reads, profile lookups, transaction histories. Classic document strengths. β
But the last 10% was killing them. π
Advisor landing pages needed Top 50 investor lists sorted by book value. Many-to-many relationship. Sort attribute lives on the investor document.
Every NoSQL database does something similar:
β
$lookup across collections (full materialization)
β Runtime
$sort on unindexed attribute
β Fetch thousands of docs to return 50
Result: 60 seconds for a landing page. π
The Vendor Fix π§
Pre-compute and embed Top N book values in the advisor doc. Background process watches book value changes, updates materialized views on advisor documents. Add a sortable index for landing pages on the advisor doc.
That solves for read latency. β
The problem? Book values change constantly throughout the trading day. One investor change triggers hundreds of advisor updates. Write amplification compounds. Views are never consistent. In financial services, "eventuallyΒ consistent" sort queries aren't acceptable. β
The Hybrid Model π‘
@Oracle 26ai. Documents where documents excel. Relations where joins are needed. Same transaction.
Create a relational mapping table between the JSON collections and project book_value from the investor doc as a relational column:
CREATE INDEX idx_adv_book ON advisor_investor_map (advisor_id, book_value DESC);
Pre-sorted composite B-tree index. The Top 50 query becomes an index-only scan. No aggregation pipeline. No runtime sort. O(log N) 50 sequential reads. β‘
The kicker? ACID consistency between the relational column and the document attribute. Same transaction. Same commit.
No CDC pipeline. No change streams. No background processing.
60 seconds β 500ms. 120x improvement. π
Zero pre-computation. Zero eventual consistency. Zero write amplification.
No other database has these primitives. Native JSON collections relational tables ACID spanning both one cost-based optimizer. Not MongoDB. Not Postgres. Not DynamoDB.
That's not philosophy. That's architecture. And today it turned a 60-second nightmare into a sub-second response. On a whiteboard. In one session.
The 90/10 problem is real. So is the solution. π―
#Oracle #Oracle26ai #HybridModeling #DataModeling #JSON #NoSQL #MongoDB #FinancialServices #DatabaseArchitecture
#UnifiedModelTheory #Performance #ACID #DocumentDatabase