Quantcast
Channel: StackExchange Replication Questions
Viewing all articles
Browse latest Browse all 17268

Removing the uses of MySQL temporary tables

$
0
0

We have a very huge PHP/MySQL application with a lot of legacy code, this application utilized MySQL for most of it's work. It does most of the processing on the database side using temporary tables, or tables that are used temporarily but are not strictly declared as TEMPORARY.

Also this application uses MySQL replication with 3 replicas for each instance, and we use write on those replicas.

In our current setup we use replication filtering to ignore those tables ,We are moving all of our application instances, which is about 15 deployments, to Amazon's RDS, and Amazon RDS does not allow setting "replicat-ignore-tables" param

So now we want to remove the usage of temporary tables, and we are checking the alternatives.

We mainly use them to process joined data from huge tables (billions of rows) and some matrix processing utilizing the use of indexes to process matrix rows.

What would be the best way to replace temporary tables usages using PHP and MySQL?

EDIT The answer below would solve our first usage, which is direct selects.

Another usage goes as this:

  1. We SELECT a huge amount of data from a huge table into a file using locally located.
  2. We create a TEMPORARY table and load the data in the file into it.
  3. Then we do some calculations and update some rows on that table in every calculation, for example we calculate the height of a building, then we update all rows with that height in the temporary table.
  4. We then run some aggregation calculations on that table, and put the results in a final table.
  5. We drop that table.

Viewing all articles
Browse latest Browse all 17268

Trending Articles



<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>