Agreed. But if it's not slavery, it could be a function of inequality - and our history of slavery and Jim Crow are responsible for at least some of that.
IMO, it's particularly bad in the US as we have inequality on top of caste. (make Brahmins in India think they are going to slide into being 'untouchables' and see what happens)