About a white school teacher shunned and ran out of town after having a ual relationship with a black man?

It's about a white school teacher shunned and ran out of town after having a ual relationship with a black man who worked at the school as a janitor. I saw it once in the early 80s.