Tenants of the luxe building are not thrilled about sharing their lifts with post-workout crowds of fitness fanatics.