We have a MEAN application with a 3-member MongoDB replica set. Somehow the total connection count keeps increasing until the primary is not able to take any new connections and goes down, making the application painfully slow. I have set the ulimit to 30000 and restarted the MongoDB and in no time it got 28000+ connection opened and primary went down.
Any idea why this may be happening? Most of the connections are between the different ports on primary with port 27017 on primary (for instance, primary_server:45002 to primary_server:27017).