Thread deadlock in BalancedMetricResolver

[billoley]

1. The ServiceCacheListener cacheChanged handler grabs the balancerLock.writeLock and then calls writeAssignmentsToHdfs
2. The RebalanceTimer periodically calls writeAssignmentsToHdfs
3. In writeAssignmentsToHdfs, we first acquire the assignmentsIPRWLock.writeLock
4. In writeAssignmentsToHdfs, we then acquire the balancerLock.readLock

if (1) ServiceCacheListener cacheChanged acquires the balancerLock.writeLock at the same time that the (2) RebalanceTimer acquires the assignmentsIPRWLock.writeLock then (1) will be stuck trying to acquire the assignmentsIPRWLock.writeLock while (1) will be stuck trying to acquire the balancerLock.readLock.

Solution is in ServiceCacheListener cacheChanged to move the call to writeAssignmentsToHdfs outside the scope of the balancerLock.writeLock. It's unnecessary and causes this issue.