The high priest was never told to put the sins of the people on the goat's head and let him go. He was told to put the sins on the goat and give him to someone who would take him out of the camp and let him go.
It's a small difference, but an important one. I've heard a lot of sermons either about this chapter or mentioning it's, and I've never heard about the second man. Let's stick to what's actually in theBible.