I had my third node in a gluster replication fail. I rebuilt it from scratch and used the same servername. (app03)
I followed the document located here: http://www.gluster.org/community/documentation/index.php/Archives/Gluster_3.2:_Brick_Restoration_-_Replace_Crashed_Server
gluster volume info displays:
Volume Name: gv0 Type: Replicate Volume ID: 38d3fb52-52d6-42c7-b20f-d6cc0427a595 Status: Started Number of Bricks: 1 x 3 = 3 Transport-type: tcp Bricks: Brick1: app01:/data/brick1/gv0 Brick2: app02:/data/brick1/gv0 Brick3: app03:/data/brick1/gv0
However, the volume on app03 has no data, when I try and run: [root@app-03 ~]# gluster volume heal gv0 Launching Heal operation on volume gv0 has been unsuccessful
When checking the log: /var/log/glusterfs/etc-glusterfs-glusterd.vol.log I see lots of entries for
[2015-07-11 02:47:53.317119] I [glusterd-utils.c:1079:glusterd_volume_brickinfo_get] 0-management: Found brick [2015-07-11 02:47:53.917986] I [glusterd-volume-ops.c:478:__glusterd_handle_cli_heal_volume] 0-management: Received heal vol req for volume gv0 [2015-07-11 02:47:56.317611] E [socket.c:2872:socket_connect] 0-management: connection attempt failed (Connection refused) [2015-07-11 02:47:56.318045] I [glusterd-utils.c:1079:glusterd_volume_brickinfo_get] 0-management: Found brick [2015-07-11 02:47:59.318662] E [socket.c:2872:socket_connect] 0-management: connection attempt failed (Connection refused) [2015-07-11 02:47:59.319067] I [glusterd-utils.c:1079:glusterd_volume_brickinfo_get] 0-management: Found brick
Can someone point me in the right direction? I'm not sure what connection is failing, or where to. app03 has access to app01 and app02 (which are currently replicating just fine) and none of them have a firewall enabled.
Thanks