What did Burakumin look like?

In my novel, I have a character who is one of the “untouchables” of Japan, an Eta, or Burakumin. These were the leatherworkers who were deemed untouchable by Buddhist vegetarians.

Eta were discriminated against though it was outlawed. Apparently even today,some people look into family backgrounds to make sure there are no untouchables in families before marriages take place.

Wanting to know what they actually looked like, I found this image on Flickr from the Tom Burnett collection.

1873 Japanese Leather Workers (burakumin)
1873 Japanese Leather Workers (burakumin)

I’m wondering, like the folks who commented in the link’s comments, why this outcast status did not apply to fishermen or, in particular, whalers since whales are mammals. Discrimination, it seems, is always hypocritical.