missing connection timeout defaults can make amqp bunnies cranky
recent ubuntu system updates seem to have changed something in our rabbit amqp server. Suddenly, we this in our (Ruby bunny) logs:
ERROR -- #<Bunny::Session:70050536054900 me@oam1:5672, vhost=/, hosts=[myhost]>: Got an exception when receiving data: IO timeout when reading 7 bytes (Timeout::Error) W, [2014-11-26T20:35:26.171279 #31449] WARN -- #<Bunny::Session:70050536054900 me@myhost:5672, vhost=/, hosts=[myhost]>: Recovering from a network failure... W, [2014-11-26T20:35:36.171637 #31449] WARN -- #<Bunny::Session:70050536054900 me@myhost:5672, vhost=/, hosts=[myhost]>: Retrying connection on next host in line: myhost:5672
turns out that Bunny, the ruby amqp client handle, now assumes a 0 second heartbeat. Maybe deep down in the amqp release notes, there is something about having to change the server config, but it's also easily fixed on the client on connect.
Instead of just specifying the connection URL
b = Bunny.new( "amqp://me:secret@myhost" )
you can just specify a hash that also includes the heartbeat value. Apparently, just about any number larger than 0 is infinitely better than 0, and 10 seems to work great even on a fairly loaded box.
b = Bunny.new( user: 'me', pass: 'secret', host: 'myhost', heartbeat: 10 )
Now our bunnies are happy again... until next time.