We operate an empty Galera cluster (without customers):
MariaDB [(none)]> SELECT table_schema "DB Name",
-> Round(Sum(data_length + index_length) / 1024 / 1024, 1) "DB Size in MB"
-> FROM information_schema.tables
-> GROUP BY table_schema;
+--------------------+---------------+
| DB Name | DB Size in MB |
+--------------------+---------------+
| information_schema | 0.1 |
| mysql | 0.6 |
| performance_schema | 0.0 |
+--------------------+---------------+
3 rows in set (0.03 sec)
WSREP is in a good state
MariaDB [(none)]> show global status like '%wsrep%';
+------------------------------+----------------------------------------------------+
| Variable_name | Value |
+------------------------------+----------------------------------------------------+
| wsrep_local_state_uuid | cd8b0a04-baab-11e5-8828-c67d1ead57b7 |
| wsrep_protocol_version | 5 |
| wsrep_last_committed | 393766 |
| wsrep_replicated | 279933 |
| wsrep_replicated_bytes | 74492165 |
| wsrep_repl_keys | 653048 |
| wsrep_repl_keys_bytes | 11662843 |
| wsrep_repl_data_bytes | 44913610 |
| wsrep_repl_other_bytes | 0 |
| wsrep_received | 63367 |
| wsrep_received_bytes | 16198736 |
| wsrep_local_commits | 93279 |
| wsrep_local_cert_failures | 0 |
| wsrep_local_replays | 0 |
| wsrep_local_send_queue | 0 |
| wsrep_local_send_queue_avg | 0.000000 |
| wsrep_local_recv_queue | 0 |
| wsrep_local_recv_queue_avg | 0.001010 |
| wsrep_local_cached_downto | 151395 |
| wsrep_flow_control_paused_ns | 0 |
| wsrep_flow_control_paused | 0.000000 |
| wsrep_flow_control_sent | 0 |
| wsrep_flow_control_recv | 0 |
| wsrep_cert_deps_distance | 1.000000 |
| wsrep_apply_oooe | 0.000026 |
| wsrep_apply_oool | 0.000000 |
| wsrep_apply_window | 1.000026 |
| wsrep_commit_oooe | 0.000000 |
| wsrep_commit_oool | 0.000000 |
| wsrep_commit_window | 1.000000 |
| wsrep_local_state | 4 |
| wsrep_local_state_comment | Synced |
| wsrep_cert_index_size | 3 |
| wsrep_causal_reads | 0 |
| wsrep_cert_interval | 0.000026 |
| wsrep_incoming_addresses | $EDITED |
| wsrep_cluster_conf_id | 36 |
| wsrep_cluster_size | 3 |
| wsrep_cluster_state_uuid | cd8b0a04-baab-11e5-8828-c67d1ead57b7 |
| wsrep_cluster_status | Primary |
| wsrep_connected | ON |
| wsrep_local_bf_aborts | 0 |
| wsrep_local_index | 2 |
| wsrep_provider_name | Galera |
| wsrep_provider_vendor | Codership Oy <info@codership.com> |
| wsrep_provider_version | 3.5(rXXXX) |
| wsrep_ready | ON |
| wsrep_thread_count | 2 |
+------------------------------+----------------------------------------------------+
48 rows in set (0.01 sec)
This the MariaDB logs:
160709 3:23:31 [Note] WSREP: REPL Protocols: 5 (3, 1)
160709 3:23:31 [Note] WSREP: Service thread queue flushed.
160709 3:23:31 [Note] WSREP: Assign initial position for certification: 325800, protocol version: 3
160709 3:23:31 [Note] WSREP: Service thread queue flushed.
160709 3:23:31 [Warning] WSREP: Releasing seqno 325800 before 325801 was assigned.
160709 3:23:31 [Note] WSREP: Synchronized with group, ready for connections
160709 3:23:31 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.
160709 3:23:32 [Note] WSREP: (d65f5c76-e12a-11e5-946f-fe15b3f98aa0, 'tcp://0.0.0.0:4567') address 'tcp://172.30.2.41:4567' pointing to uuid d65f5c76-e12a-11e5-946f-fe15b3f98aa0 is blacklisted, skipping
160709 3:23:32 [Note] WSREP: (d65f5c76-e12a-11e5-946f-fe15b3f98aa0, 'tcp://0.0.0.0:4567') address 'tcp://172.30.2.41:4567' pointing to uuid d65f5c76-e12a-11e5-946f-fe15b3f98aa0 is blacklisted, skipping
160709 3:23:32 [Note] WSREP: (d65f5c76-e12a-11e5-946f-fe15b3f98aa0, 'tcp://0.0.0.0:4567') address 'tcp://172.30.2.41:4567' pointing to uuid d65f5c76-e12a-11e5-946f-fe15b3f98aa0 is blacklisted, skipping
160709 3:23:32 [Note] WSREP: (d65f5c76-e12a-11e5-946f-fe15b3f98aa0, 'tcp://0.0.0.0:4567') turning message relay requesting off
160709 12:05:01 [Note] WSREP: Provider paused at cd8b0a04-baab-11e5-8828-c67d1ead57b7:328916 (278193)
160709 12:05:01 [Note] WSREP: resuming provider at 278193
160709 12:05:01 [Note] WSREP: Provider resumed.
160710 0:05:01 [Note] WSREP: Provider paused at cd8b0a04-baab-11e5-8828-c67d1ead57b7:333206 (282518)
160710 0:05:01 [Note] WSREP: resuming provider at 282518
160710 0:05:01 [Note] WSREP: Provider resumed.
160710 12:05:01 [Note] WSREP: Provider paused at cd8b0a04-baab-11e5-8828-c67d1ead57b7:337496 (286843)
160710 12:05:01 [Note] WSREP: resuming provider at 286843
160710 12:05:01 [Note] WSREP: Provider resumed.
160711 0:05:01 [Note] WSREP: Provider paused at cd8b0a04-baab-11e5-8828-c67d1ead57b7:341786 (291168)
160711 0:05:01 [Note] WSREP: resuming provider at 291168
160711 0:05:01 [Note] WSREP: Provider resumed.
160711 12:05:01 [Note] WSREP: Provider paused at cd8b0a04-baab-11e5-8828-c67d1ead57b7:346076 (295493)
160711 12:05:01 [Note] WSREP: resuming provider at 295493
160711 12:05:01 [Note] WSREP: Provider resumed.
160712 0:05:01 [Note] WSREP: Provider paused at cd8b0a04-baab-11e5-8828-c67d1ead57b7:350366 (299818)
160712 0:05:01 [Note] WSREP: resuming provider at 299818
160712 0:05:01 [Note] WSREP: Provider resumed.
160712 12:05:01 [Note] WSREP: Provider paused at cd8b0a04-baab-11e5-8828-c67d1ead57b7:354656 (304144)
160712 12:05:01 [Note] WSREP: resuming provider at 304144
160712 12:05:01 [Note] WSREP: Provider resumed.
160713 0:05:01 [Note] WSREP: Provider paused at cd8b0a04-baab-11e5-8828-c67d1ead57b7:358946 (308469)
160713 0:05:01 [Note] WSREP: resuming provider at 308469
160713 0:05:01 [Note] WSREP: Provider resumed.
160713 12:05:01 [Note] WSREP: Provider paused at cd8b0a04-baab-11e5-8828-c67d1ead57b7:363236 (312794)
160713 12:05:01 [Note] WSREP: resuming provider at 312794
160713 12:05:01 [Note] WSREP: Provider resumed.
160714 0:05:01 [Note] WSREP: Provider paused at cd8b0a04-baab-11e5-8828-c67d1ead57b7:367526 (317119)
160714 0:05:01 [Note] WSREP: resuming provider at 317119
160714 0:05:01 [Note] WSREP: Provider resumed.
160714 12:05:01 [Note] WSREP: Provider paused at cd8b0a04-baab-11e5-8828-c67d1ead57b7:371816 (321444)
160714 12:05:01 [Note] WSREP: resuming provider at 321444
160714 12:05:01 [Note] WSREP: Provider resumed.
160715 0:05:01 [Note] WSREP: Provider paused at cd8b0a04-baab-11e5-8828-c67d1ead57b7:374482 (324132)
160715 0:05:01 [Note] WSREP: resuming provider at 324132
160715 0:05:01 [Note] WSREP: Provider resumed.
160715 12:05:01 [Note] WSREP: Provider paused at cd8b0a04-baab-11e5-8828-c67d1ead57b7:375202 (324859)
160715 12:05:01 [Note] WSREP: resuming provider at 324859
160715 12:05:01 [Note] WSREP: Provider resumed.
160716 0:05:01 [Note] WSREP: Provider paused at cd8b0a04-baab-11e5-8828-c67d1ead57b7:375922 (325585)
160716 0:05:01 [Note] WSREP: resuming provider at 325585
160716 0:05:01 [Note] WSREP: Provider resumed.
160716 12:05:01 [Note] WSREP: Provider paused at cd8b0a04-baab-11e5-8828-c67d1ead57b7:376639 (326309)
160716 12:05:01 [Note] WSREP: resuming provider at 326309
160716 12:05:01 [Note] WSREP: Provider resumed.
160717 0:05:01 [Note] WSREP: Provider paused at cd8b0a04-baab-11e5-8828-c67d1ead57b7:377359 (327036)
160717 0:05:01 [Note] WSREP: resuming provider at 327036
160717 0:05:01 [Note] WSREP: Provider resumed.
160717 12:05:01 [Note] WSREP: Provider paused at cd8b0a04-baab-11e5-8828-c67d1ead57b7:378079 (327762)
160717 12:05:01 [Note] WSREP: resuming provider at 327762
160717 12:05:01 [Note] WSREP: Provider resumed.
160718 0:05:01 [Note] WSREP: Provider paused at cd8b0a04-baab-11e5-8828-c67d1ead57b7:378799 (328489)
160718 0:05:01 [Note] WSREP: resuming provider at 328489
160718 0:05:01 [Note] WSREP: Provider resumed.
160718 12:05:01 [Note] WSREP: Provider paused at cd8b0a04-baab-11e5-8828-c67d1ead57b7:379516 (329213)
160718 12:05:01 [Note] WSREP: resuming provider at 329213
160718 12:05:01 [Note] WSREP: Provider resumed.
160719 0:05:01 [Note] WSREP: Provider paused at cd8b0a04-baab-11e5-8828-c67d1ead57b7:380236 (329940)
160719 0:05:01 [Note] WSREP: resuming provider at 329940
160719 0:05:01 [Note] WSREP: Provider resumed.
160719 12:05:01 [Note] WSREP: Provider paused at cd8b0a04-baab-11e5-8828-c67d1ead57b7:380956 (330666)
160719 12:05:01 [Note] WSREP: resuming provider at 330666
160719 12:05:01 [Note] WSREP: Provider resumed.
160720 0:05:01 [Note] WSREP: Provider paused at cd8b0a04-baab-11e5-8828-c67d1ead57b7:381673 (331390)
160720 0:05:01 [Note] WSREP: resuming provider at 331390
160720 0:05:01 [Note] WSREP: Provider resumed.
160720 12:05:01 [Note] WSREP: Provider paused at cd8b0a04-baab-11e5-8828-c67d1ead57b7:382393 (332117)
160720 12:05:01 [Note] WSREP: resuming provider at 332117
160720 12:05:01 [Note] WSREP: Provider resumed.
160721 0:05:01 [Note] WSREP: Provider paused at cd8b0a04-baab-11e5-8828-c67d1ead57b7:383113 (332843)
160721 0:05:01 [Note] WSREP: resuming provider at 332843
160721 0:05:01 [Note] WSREP: Provider resumed.
160721 12:05:01 [Note] WSREP: Provider paused at cd8b0a04-baab-11e5-8828-c67d1ead57b7:383833 (333570)
160721 12:05:01 [Note] WSREP: resuming provider at 333570
160721 12:05:01 [Note] WSREP: Provider resumed.
160722 0:05:01 [Note] WSREP: Provider paused at cd8b0a04-baab-11e5-8828-c67d1ead57b7:384550 (334294)
160722 0:05:01 [Note] WSREP: resuming provider at 334294
160722 0:05:01 [Note] WSREP: Provider resumed.
160722 12:05:01 [Note] WSREP: Provider paused at cd8b0a04-baab-11e5-8828-c67d1ead57b7:385270 (335020)
160722 12:05:01 [Note] WSREP: resuming provider at 335020
160722 12:05:01 [Note] WSREP: Provider resumed.
160723 0:05:01 [Note] WSREP: Provider paused at cd8b0a04-baab-11e5-8828-c67d1ead57b7:385990 (335747)
160723 0:05:01 [Note] WSREP: resuming provider at 335747
160723 0:05:01 [Note] WSREP: Provider resumed.
160723 12:05:01 [Note] WSREP: Provider paused at cd8b0a04-baab-11e5-8828-c67d1ead57b7:386707 (336471)
160723 12:05:01 [Note] WSREP: resuming provider at 336471
160723 12:05:01 [Note] WSREP: Provider resumed.
160724 0:05:01 [Note] WSREP: Provider paused at cd8b0a04-baab-11e5-8828-c67d1ead57b7:387427 (337198)
160724 0:05:01 [Note] WSREP: resuming provider at 337198
160724 0:05:01 [Note] WSREP: Provider resumed.
160724 12:05:01 [Note] WSREP: Provider paused at cd8b0a04-baab-11e5-8828-c67d1ead57b7:388147 (337924)
160724 12:05:01 [Note] WSREP: resuming provider at 337924
160724 12:05:01 [Note] WSREP: Provider resumed.
160725 0:05:01 [Note] WSREP: Provider paused at cd8b0a04-baab-11e5-8828-c67d1ead57b7:388864 (338648)
160725 0:05:01 [Note] WSREP: resuming provider at 338648
160725 0:05:01 [Note] WSREP: Provider resumed.
160725 12:05:01 [Note] WSREP: Provider paused at cd8b0a04-baab-11e5-8828-c67d1ead57b7:389584 (339375)
160725 12:05:01 [Note] WSREP: resuming provider at 339375
160725 12:05:01 [Note] WSREP: Provider resumed.
160726 0:05:01 [Note] WSREP: Provider paused at cd8b0a04-baab-11e5-8828-c67d1ead57b7:390304 (340101)
160726 0:05:01 [Note] WSREP: resuming provider at 340101
160726 0:05:01 [Note] WSREP: Provider resumed.
160726 12:05:01 [Note] WSREP: Provider paused at cd8b0a04-baab-11e5-8828-c67d1ead57b7:391024 (340828)
160726 12:05:01 [Note] WSREP: resuming provider at 340828
160726 12:05:01 [Note] WSREP: Provider resumed.
160727 0:05:01 [Note] WSREP: Provider paused at cd8b0a04-baab-11e5-8828-c67d1ead57b7:391741 (341552)
160727 0:05:01 [Note] WSREP: resuming provider at 341552
160727 0:05:01 [Note] WSREP: Provider resumed.
160727 12:05:01 [Note] WSREP: Provider paused at cd8b0a04-baab-11e5-8828-c67d1ead57b7:392461 (342279)
160727 12:05:01 [Note] WSREP: resuming provider at 342279
160727 12:05:01 [Note] WSREP: Provider resumed.
160728 0:05:01 [Note] WSREP: Provider paused at cd8b0a04-baab-11e5-8828-c67d1ead57b7:393181 (343005)
160728 0:05:01 [Note] WSREP: resuming provider at 343005
160728 0:05:01 [Note] WSREP: Provider resumed.
What does WSREP: Provider paused at $GUID and WSREP: Provider resumed exactly mean? Do I need to further investigate?
We use Redhat provided 5.5.42-MariaDB-wsrep
From timestamps it seems backup related:
# crontab -l
# HEADER: This file was autogenerated at 2016-01-14 09:39:42 +0100 by puppet.
# HEADER: While it can still be managed manually, it is definitely not recommended.
# HEADER: Note particularly that the comments starting with 'Puppet Name' should
# HEADER: not be deleted, as doing so could cause duplicate cron jobs.
# Puppet Name: mysql-backup
5 */12 * * * /usr/local/sbin/mysqlbackup.sh
We use mysqldump.