Galera cluster status
> show status like '%wsrep%'; +------------------------------+-------------------------------------------------+
| Variable_name | Value |
+------------------------------+-------------------------------------------------+
| wsrep_local_state_uuid | 09d60a58-c455-11e5-925b-3badcf92ce95 |
| wsrep_protocol_version | 5 |
| wsrep_last_committed | 6991376 |
| wsrep_replicated | 590 |
| wsrep_replicated_bytes | 331990 |
| wsrep_repl_keys | 2158 |
| wsrep_repl_keys_bytes | 30834 |
| wsrep_repl_data_bytes | 263396 |
| wsrep_repl_other_bytes | 0 |
| wsrep_received | 61182 |
| wsrep_received_bytes | 29156107 |
| wsrep_local_commits | 588 |
| wsrep_local_cert_failures | 0 |
| wsrep_local_replays | 0 |
| wsrep_local_send_queue | 0 |
| wsrep_local_send_queue_avg | 0.005540 |
| wsrep_local_recv_queue | 0 |
| wsrep_local_recv_queue_avg | 0.170230 |
| wsrep_local_cached_downto | 6930111 |
| wsrep_flow_control_paused_ns | 93132236 |
| wsrep_flow_control_paused | 0.000044 |
| wsrep_flow_control_sent | 15 |
| wsrep_flow_control_recv | 16 |
| wsrep_cert_deps_distance | 70.278360 |
| wsrep_apply_oooe | 0.002612 |
| wsrep_apply_oool | 0.000000 |
| wsrep_apply_window | 1.013792 |
| wsrep_commit_oooe | 0.000000 |
| wsrep_commit_oool | 0.000000 |
| wsrep_commit_window | 1.011311 |
| wsrep_local_state | 4 |
| wsrep_local_state_comment | Synced |
| wsrep_cert_index_size | 41 |
| wsrep_causal_reads | 0 |
| wsrep_cert_interval | 0.169213 |
| wsrep_incoming_addresses | 192.168.5.40.41:3306,192.168.5.40.42:3306,192.168.5.40.43:3306 |
| wsrep_cluster_conf_id | 3 |
| wsrep_cluster_size | 3 |
| wsrep_cluster_state_uuid | 09d60a58-c455-11e5-925b-3badcf92ce95 |
| wsrep_cluster_status | Primary |
| wsrep_connected | ON |
| wsrep_local_bf_aborts | 0 |
| wsrep_local_index | 0 |
| wsrep_provider_name | Galera |
| wsrep_provider_vendor | Codership Oy <info@codership.com> |
| wsrep_provider_version | 3.5(rXXXX) |
| wsrep_ready | ON |
| wsrep_thread_count | 2 |
+------------------------------+-------------------------------------------------+
48 rows in set (0.00 sec)
version
> select version();
+----------------------+
| version() |
+----------------------+
| 5.5.41-MariaDB-wsrep |
+----------------------+
1 row in set (0.00 sec)
Galera crashed (all 3 nodes). This are the logs from the relevant time stamps before crash.
160224 11:50:34 [ERROR] Slave SQL: Error 'Can't create table 'database324.login' (errno: 150)' on query. Default database: 'database324'. Query: 'CREAT
E TABLE login (
updated DATETIME NOT NULL,
created DATETIME NOT NULL,
deleted BOOL NOT NULL,
id INTEGER NOT NULL AUTO_INCREMENT,
first_name VARCHAR(255) NOT NULL,
second_name VARCHAR(255) NOT NULL,
username VARCHAR(255) NOT NULL,
email VARCHAR(255) NOT NULL,
password VARBINARY(1024),
confirm VARBINARY(1024),
uuid VARCHAR(255),
salt VARCHAR(255),
role_id INTEGER,
group_id INTEGER,
PRIMARY KEY (id),
CHECK (deleted IN (0, 1)),
UNIQUE (username),
FOREIGN KEY(role_id) REFERENCES role (id),
FOREIGN KEY(group_id) REFERENCES `group` (id)
)', Error_code: 1005
160224 11:50:34 [Warning] WSREP: RBR event 1 Query apply warning: 1, 6922968
160224 11:50:34 [Warning] WSREP: Ignoring error for TO isolated action: source: 34b71ef9-c455-11e5-b3f9-2e94e8c36efc version: 3 local: 0 state: APPLYING flags: 65 conn_id: 5837394 trx_id: -1 seqnos (l: 69788
48, g: 6922968, s: 6922967, d: 6922967, ts: 14604556479453079)
160224 11:51:10 [ERROR] Slave SQL: Error 'Can't create table 'database324.login' (errno: 150)' on query. Default database: 'database324'. Query: 'CREAT
E TABLE login (
updated DATETIME NOT NULL,
created DATETIME NOT NULL,
deleted BOOL NOT NULL,
id INTEGER NOT NULL AUTO_INCREMENT,
first_name VARCHAR(255) NOT NULL,
second_name VARCHAR(255) NOT NULL,
username VARCHAR(255) NOT NULL,
email VARCHAR(255) NOT NULL,
password VARBINARY(1024),
confirm VARBINARY(1024),
uuid VARCHAR(255),
salt VARCHAR(255),
role_id INTEGER,
group_id INTEGER,
PRIMARY KEY (id),
CHECK (deleted IN (0, 1)),
UNIQUE (username),
FOREIGN KEY(role_id) REFERENCES role (id),
FOREIGN KEY(group_id) REFERENCES `group` (id)
)', Error_code: 1005
160224 11:51:10 [Warning] WSREP: RBR event 1 Query apply warning: 1, 6923479
160224 11:51:10 [Warning] WSREP: Ignoring error for TO isolated action: source: 34b71ef9-c455-11e5-b3f9-2e94e8c36efc version: 3 local: 0 state: APPLYING flags: 65 conn_id: 5837471 trx_id: -1 seqnos (l: 6979363, g: 6923479, s: 6923478, d: 6923478, ts: 14604592266226083)
160224 11:51:57 [ERROR] Slave SQL: Error 'Can't create table 'database324.login' (errno: 150)' on query. Default database: 'database324'. Query: 'CREATE TABLE login (
updated DATETIME NOT NULL,
created DATETIME NOT NULL,
deleted BOOL NOT NULL,
id INTEGER NOT NULL AUTO_INCREMENT,
first_name VARCHAR(255) NOT NULL,
second_name VARCHAR(255) NOT NULL,
username VARCHAR(255) NOT NULL,
email VARCHAR(255) NOT NULL,
password VARBINARY(1024),
confirm VARBINARY(1024),
uuid VARCHAR(255),
salt VARCHAR(255),
role_id INTEGER,
group_id INTEGER,
PRIMARY KEY (id),
CHECK (deleted IN (0, 1)),
UNIQUE (username),
FOREIGN KEY(role_id) REFERENCES role (id),
FOREIGN KEY(group_id) REFERENCES `group` (id)
)', Error_code: 1005
160224 11:51:57 [Warning] WSREP: RBR event 1 Query apply warning: 1, 6924320
160224 11:51:57 [Warning] WSREP: Ignoring error for TO isolated action: source: 34b71ef9-c455-11e5-b3f9-2e94e8c36efc version: 3 local: 0 state: APPLYING flags: 65 conn_id: 5837556 trx_id: -1 seqnos (l: 6980211, g: 6924320, s: 6924319, d: 6924319, ts: 14604639580444774)
160224 11:53:31 [ERROR] Slave SQL: Error 'Unknown table 'image_capture'' on query. Default database: 'database324'. Query: 'DROP TABLE image_capture', Error_code: 1051
160224 11:53:31 [Warning] WSREP: RBR event 1 Query apply warning: 1, 6926476
160224 11:53:31 [Warning] WSREP: Ignoring error for TO isolated action: source: 34b71ef9-c455-11e5-b3f9-2e94e8c36efc version: 3 local: 0 state: APPLYING flags: 65 conn_id: 5837646 trx_id: -1 seqnos (l: 6982384, g: 6926476, s: 6926475, d: 6926475, ts: 14604733399260795)
160224 11:57:34 [ERROR] mysqld got signal 11 ;
any idea why crash? MariaDB or Galera bug? Link to bug report?
howto prevent same crash in future?
