Frazier was not threatened with physical violence by Fritz or anyone else.
"and in comes Captain Will Fritz, who I have never seen in my life, never talked to. He brought in a typed statement, and he had a pen. He says here, sign this. He gave me a pen. I started reading it. Well, they wanted me to confess to being part of or having knowledge of the assassination of John F. Kennedy. I read about two sentences of that, and I looked at him and I said that's ludicrous. I said I'm not signing that.
So he drew his hand back to hit me, and I did my arm up like this because he was over here like you are. I told him -- I said -- he got very red-faced. He wasn't a real big man for his physical stature, but I hear he had a temper. Anyway, he -- I told him, I said, you know, I know there's policemen outside of the door. I said when they get in, you and I are going to have a hell of a fight."
https://www.c-span.org/video/?313792-1/lee-harvey-oswald-kennedy-assassination&start=2757