Does anyone know of any product or open source project that attempts to strap together Mongo DB and MySQL for instance?
I hear a lot about how relational DBs being ACID compliant makes it nearly impossible to do sharding as easily as in Mongo and this is the achilles heel. The NoSQL fans are surely right. A technology must eventually parallelize or die.
No to NoSchema
However, schemaless DBs are not a sustainable solution to many problems. Schema changes are only hard because they force you to see what you're breaking. Not seeing doesn't mean you're not breaking.
Best of both?
What I need help understanding is why it seems accepted that you can't in theory have the advantages of both. Can't a relational DB be engineered to compromise with a wise strategy of eventual consistency? I understand why it's more difficult than with documents - a single fetch is composed of records stored physically all over the disk. But how about a document store which acts as a front for a relational DB?
Imagine
The DB has a built in ORM to map from relational to document collection and you interface with the document in your app. Essentially, the document collections are like materialised views with write-back capability. By default the relational DB part is ACID compliant and ensures that non-acidic documents are eventually consistent. Cross-record schema constraints may not be performant enough but most other schema controls could be.
When you query a large batch of documents, if a few are out of sync - because another document sharing the same denormalised data has just been written to - then they are silently refreshed live from the underlying DB by default. If blocking is an issue then you can choose to handle those dirty docs with a number of other looser consistency options.
Pros and pros
..and small cons
You don't get the document store advantage of fast writes but
Gets NoSQL advantage: massive and fast reads, easy sharding and no O-R impedance mismatch for front-end developers.
Gets SQL advantage: Organisations can stay nimble, able to work with their data from new angles not thought of initially. Schema helps keep production data from turning into a turgid mess of exceptions before its too late.
Gets hybrid advantage: Rows can be physically re-organised in the RDB to follow the patterns suggested by the most frequently accessed document collections. When the load gets too high for the relational DB on one box this will make sharding more of a possibility (with a possible tradeoff against whole-system analytical queries but that's what you have an OLAP for). The relational DB will also have much fewer reads than if it were directly user facing.
I've misunderstood something. I must have.
I don't imagine this is an easy bit of engineering but it seems within the powers of the those who have brought us the products we already have.
I'm not seeking general discussion here but I wouldn't be surprised if there's a big, concrete gotcha I'm missing in this account and I'd like to know what it is.