Quantcast
Channel: StackExchange Replication Questions
Viewing all articles
Browse latest Browse all 17268

MySQL performing a "No impact" temporary INSERT with replication avoiding Locks

$
0
0

SO, we are trying to run a Report going to screen, which will not change any stored data. However, it is complex, so needs to go through a couple of (TEMPORARY*) tables.

It pulls data from live tables, which are replicated.

The nasty bit when it comes to take the "eligible" records from

temp_PreCalc

and populate them from the live data to create the next (TEMPORARY*) table output

resulting in effectively:

INSERT INTO temp_PostCalc (...) SELECT ... FROM temp_PreCalc JOIN live_Tab1 ON ... JOIN live_Tab2 ON ... JOIN live_Tab3 ON ...

The report is not a "definitive" answer, expectation is that is merely a "snapshot" report and will be out-of-date as soon as it appears on screen.

There is no order or reproducibility issue.

So Ideally, I would turn my TRANSACTION ISOLATION LEVEL down to READ COMMITTED... However, I can't because live_Tab1,2,3 are replicated with BIN_LOG STATEMENT type...

The statement is lovely and quick - it takes hardly any time to run, so the resource load is now less than it used to be (which did separate selects and inserts) but it waits (as I understand it) because of the SELECT that waits for a repeatable/syncable lock on the live_Tab's so that any result could be replicated safely. In fact it now takes more time because of that wait.

I'd like to SEE that performance benefit in response time!

Except the data is written to (TEMPORARY*) tables and then thrown away.

There are no live_ table destinations - only sources...

    • these tables are actually not TEMPORARY TABLES but dynamically created and thrown away InnoDB Tables, as the report Calculation requires Self-join and delete... but they are temporary

I now seem to be going around in circles finding an answer.


I don't have SUPER privilege and don't want it... So can't SET BIN_LOG=0 for this connection session (Why is this a requirement?)

So...

If I have a scratch Database or table wildcard, which excludes all my temp_ "Temporary" tables from replication... (I am awaiting for this change to go through at my host centre)

Will MySQL allow me to

SET SESSION TRANSACTION ISOLATION LEVEL READ COMMITTED;

INSERT INTO temp_PostCalc (...) SELECT ... FROM temp_PreCalc JOIN live_Tab1 ON ... JOIN live_Tab2 ON ... JOIN live_Tab3 ON ... ;

Or will I still get my

"Cannot Execute statement: impossible to write to binary log since BINLOG_FORMAT = STATEMENT and at least one table uses a storage engine limited to row-based logging..."

Even though its not technically true? I am expecting it to, as I presume that the replication will kick in simply because it sees the "INSERT" statement, and will do a simple check on any of the tables involved being replication eligible, even though none of the destinations are actually replication eligible....

or will it pleasantly surprise me?

I really can't face using an unpleasant solution like

SELECT TO OUTFILE LOAD DATA INFILE

In fact I dont think I could even use that - how would I get unique filenames? How would I clean them up? The reports are run on-demand directly by end users, and I only have MySQL interface access to the server.

or streaming it through the PHP client, just to separate the INSERT from the SELECT so that MySQL doesnt get upset about which tables are replication eligible....


Viewing all articles
Browse latest Browse all 17268

Trending Articles



<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>