Our Gerrit replication stopped on July 19th after Bitbucket switched to IPv6
Error from the Gerrit logs /var/gerrit/logs/error_log:[2016-07-29 17:59:51,676]
ERROR com.googlesource.gerrit.plugins.replication.ReplicationQueue : Cannot replicate to git@bitbucket.org:company/product.git org.eclipse.jgit.errors.TransportException: git@bitbucket.org:company/product.git: channel is not opened. at org.eclipse.jgit.transport.JschSession$JschProcess.(JschSession.java:154) at org.eclipse.jgit.transport.JschSession$JschProcess.(JschSession.java:118) at org.eclipse.jgit.transport.JschSession.exec(JschSession.java:91) at org.eclipse.jgit.transport.TransportGitSsh$SshPushConnection.(TransportGitSsh.java:306) at org.eclipse.jgit.transport.TransportGitSsh.openPush(TransportGitSsh.java:152) at org.eclipse.jgit.transport.PushProcess.execute(PushProcess.java:154) at org.eclipse.jgit.transport.Transport.push(Transport.java:1167) at org.eclipse.jgit.transport.Transport.push(Transport.java:1213) at com.googlesource.gerrit.plugins.replication.PushOne.pushVia(PushOne.java:307) at com.googlesource.gerrit.plugins.replication.PushOne.runImpl(PushOne.java:252) at com.googlesource.gerrit.plugins.replication.PushOne.runPushOperation(PushOne.java:207) at com.googlesource.gerrit.plugins.replication.PushOne.access$000(PushOne.java:71) at com.googlesource.gerrit.plugins.replication.PushOne$1.call(PushOne.java:186) at com.googlesource.gerrit.plugins.replication.PushOne$1.call(PushOne.java:183) at com.google.gerrit.server.util.RequestScopePropagator$5.call(RequestScopePropagator.java:222) at com.google.gerrit.server.util.RequestScopePropagator$4.call(RequestScopePropagator.java:201) at com.google.gerrit.server.git.PerThreadRequestScope$Propagator$1.call(PerThreadRequestScope.java:75) at com.googlesource.gerrit.plugins.replication.PushOne.run(PushOne.java:183) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292) at com.google.gerrit.server.git.WorkQueue$Task.run(WorkQueue.java:337) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) Caused by: com.jcraft.jsch.JSchException: channel is not opened. at com.jcraft.jsch.Channel.connect(Channel.java:197) at org.eclipse.jgit.transport.JschSession$JschProcess.(JschSession.java:150) ... 25 more
Our Gerrit version is 2.6.1 installed on CentOS release 6.8 in AWS.
Tried the following to no avail:
- Upgraded the replication plugin
- Upgraded the JSch jar
- Re-added the public key for SSH
My gerrit replication configuration is:
[remote "bitbucket"]
url = git@bitbucket.org:qpidhealth/${name}.git
push = +refs/heads/*:refs/heads/*
push = +refs/tags/*:refs/tags/*
threads = 2
authGroup = Replication
I tried:
-bash-4.1$ ssh-add .ssh/id_rsa Identity added: .ssh/id_rsa (.ssh/id_rsa)
-bash-4.1$ ssh-add -l 2048
e0*:::::::::::::::**:0a .ssh/id_rsa (RSA)
-bash-4.1$ ping bitbucket.org
PING bitbucket.org (104.192.143.2) 56(84) bytes of data. 64 bytes from bitbucket.org (104.192.143.2): icmp_seq=1 ttl=50 time=1.11 ms 64 bytes from bitbucket.org (104.192.143.2): icmp_seq=2 ttl=50 time=1.09 ms 64 bytes from bitbucket.org (104.192.143.2): icmp_seq=3 ttl=50 time=1.11 ms ^C — bitbucket.org ping statistics — 3 packets transmitted, 3 received, 0% packet loss, time 2717ms rtt min/avg/max/mdev =
1.095/1.108/1.119/0.028 ms
-bash-4.1$ ssh -vT git@bitbucket.org
OpenSSH_5.3p1, OpenSSL 1.0.1e-fips 11 Feb 2013 debug1: Reading configuration data /var/gerrit/.ssh/config debug1: Applying options for * debug1: Reading configuration data /etc/ssh/ssh_config debug1: Applying options for * debug1: Connecting to bitbucket.org [104.192.143.3] port 22. debug1: Connection established. debug1: identity file /var/gerrit/.ssh/identity type -1 debug1: identity file /var/gerrit/.ssh/identity-cert type -1 debug1: identity file /var/gerrit/.ssh/id_rsa type 1 debug1: identity file /var/gerrit/.ssh/id_rsa-cert type -1 debug1: identity file /var/gerrit/.ssh/id_dsa type -1 debug1: identity file /var/gerrit/.ssh/id_dsa-cert type -1 debug1: identity file /var/gerrit/.ssh/id_ecdsa type -1 debug1: identity file /var/gerrit/.ssh/id_ecdsa-cert type -1 debug1: Remote protocol version
2.0, remote software version conker_1.0.226.0482edc app-127 debug1: no match: conker_1.0.226.0482edc app-127 debug1: Enabling compatibility mode for protocol 2.0 debug1: Local version string SSH-2.0-OpenSSH_5.3 debug1: SSH2_MSG_KEXINIT sent debug1: SSH2_MSG_KEXINIT received debug1: kex: server->client aes128-ctr hmac-sha1 none debug1: kex: client->server aes128-ctr hmac-sha1 none debug1: sending SSH2_MSG_KEXDH_INIT debug1: expecting SSH2_MSG_KEXDH_REPLY debug1: Host 'bitbucket.org' is known and matches the RSA host key. debug1: Found key in /var/gerrit/.ssh/known_hosts:8 debug1: ssh_rsa_verify: signature correct debug1: SSH2_MSG_NEWKEYS sent debug1: expecting SSH2_MSG_NEWKEYS debug1: SSH2_MSG_NEWKEYS received debug1: SSH2_MSG_SERVICE_REQUEST sent debug1: SSH2_MSG_SERVICE_ACCEPT received debug1: Authentications that can continue: publickey debug1: Next authentication method: publickey debug1: Offering public key: /var/gerrit/.ssh/id_rsa debug1: Server accepts key: pkalg ssh-rsa blen 277 debug1: Authentication succeeded (publickey). debug1: channel 0: new [client-session] debug1: Entering interactive session. debug1: Sending environment. debug1: Sending env LANG = en_US.UTF-8 logged in as gerrit-qpid. You can use git or hg to connect to Bitbucket. Shell access is disabled. debug1: client_input_channel_req: channel 0 rtype exit-status reply 0 debug1: channel 0: free: client-session, nchannels 1 Transferred: sent 2496, received 1952 bytes, in 0.2 seconds Bytes per second: sent 12380.4, received 9682.1 debug1: Exit status 0
-bash-4.1$ git clone git@bitbucket.org:company/repo.git
Cloning into 'checker-framework-config'... remote: Counting objects: 21, done. remote: Compressing objects: 100% (16/16), done. remote: Total 21 (delta 5), reused 0 (delta 0) Receiving objects: 100% (21/21), done. Resolving deltas: 100% (5/5), done.
Any ideas? Thanks in advance!