Cannot find "a server that matches PrimaryServerSelector" in a mongo replication group

I got the following error while trying to write in a sharded/replicated configuration of mongodb.

com.mongodb.MongoTimeoutException: Timed out after 30000 ms while waiting for a server that matches PrimaryServerSelector. Client view of cluster state is {type=REPLICA_SET, servers=[{address=XX.XXX.109.38:26016, type=REPLICA_SET_SECONDARY, roundTripTime=0.5 ms, state=CONNECTED}]
    at com.mongodb.connection.BaseCluster.createTimeoutException(BaseCluster.java:370)
    at com.mongodb.connection.BaseCluster.selectServer(BaseCluster.java:101)
    at com.mongodb.binding.ClusterBinding$ClusterBindingConnectionSource.<init>(ClusterBinding.java:75)
    at com.mongodb.binding.ClusterBinding$ClusterBindingConnectionSource.<init>(ClusterBinding.java:71)
    at com.mongodb.binding.ClusterBinding.getWriteConnectionSource(ClusterBinding.java:68)
    at com.mongodb.operation.OperationHelper.withConnection(OperationHelper.java:175)
    at com.mongodb.operation.MixedBulkWriteOperation.execute(MixedBulkWriteOperation.java:141)
    at com.mongodb.operation.MixedBulkWriteOperation.execute(MixedBulkWriteOperation.java:72)
    at com.mongodb.Mongo.execute(Mongo.java:747)
    at com.mongodb.Mongo$2.execute(Mongo.java:730)

My java client is connected to the router mongos which seems to be configured correctly with all the shards replica being up and running:

mongos> sh.status()
--- Sharding Status --- 
  sharding version: {    
    "_id" : 1,
    "minCompatibleVersion" : 5,
    "currentVersion" : 6,
    "clusterId" : ObjectId("588227e6bc4b87a39fbbe9ed")
}
  shards:
    {  "_id" : "torii",  "host" : "torii/XX.XXX.109.38:26016,XX.XXX.109.5:26016" }
    {  "_id" : "torii_2",  "host" : "torii_2/XX.XXX.109.38:22012,XX.XXX.109.5:22012" }
  active mongoses:
    "3.2.10" : 1
  balancer:
    Currently enabled:  yes
    Currently running:  no
    Failed balancer rounds in last 5 attempts:  5
    Last reported error:  could not find host matching read preference { mode: "primary" } for set torii
    Time of Reported error:  Thu Jan 26 2017 10:37:25 GMT+0100 (CET)
    Migration Results for the last 24 hours: 
        No recent migrations
  databases:
    {  "_id" : "Torii",  "primary" : "torii",  "partitioned" : true }
        Torii.mountpoints
            shard key: { "server.id" : 1 }
            unique: false
            balancing: true
            chunks:
                torii   1
            { "server.id" : { "$minKey" : 1 } } -->> { "server.id" : { "$maxKey" : 1 } } on : torii Timestamp(1, 0) 
        Torii.tests
            shard key: { "_id" : 1 }
            unique: false
            balancing: true
            chunks:
                torii   5
                torii_2 5
            { "_id" : { "$minKey" : 1 } } -->> { "_id" : ObjectId("5881dfad6edc3a2af33c2dbf") } on : torii_2 Timestamp(5, 1) 
            { "_id" : ObjectId("5881dfad6edc3a2af33c2dbf") } -->> { "_id" : ObjectId("588234f3e795dfd19715b96d") } on : torii_2 Timestamp(3, 0) 
            { "_id" : ObjectId("588234f3e795dfd19715b96d") } -->> { "_id" : ObjectId("588234f3e795dfd19715bb67") } on : torii Timestamp(6, 1) 
            { "_id" : ObjectId("588234f3e795dfd19715bb67") } -->> { "_id" : ObjectId("588234fae795dfd197178f84") } on : torii Timestamp(3, 2) 
            { "_id" : ObjectId("588234fae795dfd197178f84") } -->> { "_id" : ObjectId("58823500e795dfd1971a745a") } on : torii Timestamp(3, 3) 
            { "_id" : ObjectId("58823500e795dfd1971a745a") } -->> { "_id" : ObjectId("58823508e795dfd1971c4877") } on : torii_2 Timestamp(4, 2) 
            { "_id" : ObjectId("58823508e795dfd1971c4877") } -->> { "_id" : ObjectId("5882350fe795dfd1971e6fe2") } on : torii_2 Timestamp(4, 3) 
            { "_id" : ObjectId("5882350fe795dfd1971e6fe2") } -->> { "_id" : ObjectId("58823514e795dfd1972043ff") } on : torii Timestamp(5, 2) 
            { "_id" : ObjectId("58823514e795dfd1972043ff") } -->> { "_id" : ObjectId("58823519e795dfd197226b6a") } on : torii Timestamp(5, 3) 
            { "_id" : ObjectId("58823519e795dfd197226b6a") } -->> { "_id" : { "$maxKey" : 1 } } on : torii_2 Timestamp(6, 0) 

I can read/write to my collections without any problem when I connect directly to mongos with the client mongo or with robimongo.

I'm using the following versions : - mongo 3.2 - mongo-java-driver 3.4.1

Any idea would be welcome.