We’re not sure of how much time passed before Joseph was resold to Potiphar. Probably just the time it took to get to Egypt, which could be anywhere from a couple of weeks to a few years (hold the Moses lost in the desert jokes until we get there).
But once Joseph belonged to Potiphar, he was the best man he could be, earning Potiphar a lot of wealth and becoming the head of the household. Again, slavery allowed for such things. It was common for slaves to gain such high esteem and positions of trust from their masters. So Joseph is awesome at what he does.
He also happens to be good looking, and draws the attention of Potiphar’s wife. And it isn’t long until the question is hanging in the air, “Are you trying to seduce me, Mrs. Potiphar?” Okay, well, no, it wasn’t like that. Potiphar’s actually wasn’t coy about it. She “cast her eyes on Joseph and said, ‘Lie with me’” (Gen 39:7).
Joseph, though, is moral to a fault, and refuses. Now, in ancient times, sex with slaves was not necessarily seen as cheating. The Romans and Greeks especially engaged in this, and the Egyptians were not that different for the time period. It is quite possible that this something quite ordinary as far as a request (command?) and Joseph disobeyed.
At least until we get the explanation from Joseph that Potiphar has not “kept back anything from [Joseph] except [her]” (Gen 39:9). While the practice of sleeping with slaves was not uncommon, and possibly very widespread, Potiphar did not do this. The fact that Joseph points out that she was kept back is a likely indication that the practice was widespread. So even if it was culturally accepted, Potiphar had the right and chose to draw the line that his slaves, at least, would not sleep with his wife.
However, doing the right thing is not without its consequences.