Oswald said that Norman and Jarman walked "through" the lunch room, so clearly after heard Oswald heard the two men appear directly below his sniper's nest, Oswald made made a guess, an incorrect guess.
Bookhout: "Oswald stated that on November 22, 1963, he had eaten lunch in the lunch room of the Texas School Book Depository, alone, but recalled possibly two negro employees walking through the room in this period. He stated possibly one of these employees was called 'Junior' and the other was a short individual whose name he could not recall, but whom he would be able to recognize."
Also Oswald told Fritz;
Mr. BALL. He mentioned who he was having lunch with, did he not?
Mr. FRITZ. Yes, sir; he told me he was having lunch when the President was shot.
Mr. BALL. With whom?
Mr. FRITZ. With someone called Junior, someone he worked with down there, but he didn't remember the other boy's name.
But Junior(Jarman) told the WC that he didn't!

Oswald tried to cover his tracks by lying and guessing.
JohnM
"Oswald said that Norman and Jarman walked "through" the lunch room..."Just to be accurate, Bookhout said that Oswald said they walked "through" the lunchroom.
Fritz's notes just said they "came in".
Kelley said they all had lunch together.
Oswald's interrogators recall that when he was having his lunch in the domino room he saw Jarman and Norman. That's all we need to know because the testimonies of Jarman and Norman tell us exactly what happened.
Approximately 5 minutes before the assassination occurred, Jarman and Norman entered the Houston Street loading dock door and it was at this time,
and at this time only, that Oswald could have seen them from the domino room.
This places Oswald on the first floor about 5 minutes before the shooting.
Rather than deny Oswald ever mentioned it or just disappear when a topic like this comes up (as true Nutters tend to do), you have tried to deal with this thorny issue for the LNer scenario. I don't accept your explanation as I find it very weak and I'll explain why.
You have Oswald basically hanging out of the SN window making himself as obvious to the world as possible only minutes before the motorcade enters Dealey Plaza. I'm not sure why he would purposely do that.
You have his attention focused on the street directly below at the hundreds of people milling about and, for some inexplicable reason, he makes a special note of Norman and Jarman walking around the side of the building. I'm not sure why he would be so preoccupied with the people in the street when he has Bonnie Ray Williams to deal with, not to mention the little matter of an assassination.
He hears movement below and somehow knows it's Norman and Jarman, maybe they've got loud voices, and when he is being questioned he sees the opportunity to place himself elsewhere in the building - when he saw Norman and Jarman walk around the side of the building they must have been heading for the back door and if they entered through there he could say he was sat in the domino room and saw them, this would put him on the first floor minutes before the shooting. Sneaky Oswald.
As weak as this proposed scenario already is, there is a factor that further undermines it.
When Norman and Jarman enter the loading dock door their intention is to take an elevator to the 5th floor, however, when they get in the building they notice the east elevator isn't down on the 1st floor. This forces them to walk around the elevator shaft to take the west elevator and it is this movement that makes them visible to someone in the domino room. If the east elevator was available and they took that it would be impossible for someone in the domino room to see them. It is the movement around the elevator shaft that makes this possible.
The diagram below represents this situation:

Now, you have Oswald making truly miraculous guesses about the movements of Norman and Jarman. Tucked away in the southeast corner of the 6th floor he has to correctly guess the positions of the elevators in the northwest corner and then remember to incorporate it into to his already elaborate guess??
Obviously you are free to believe whatever you want but as a rational person I find your elaborate 'guessing' hypothesis to be incredibly far-fetched.
It seems far more reasonable to assume that Oswald was in the domino room where he said he was and that he actually saw Norman and Jarman as they made their way around the elevator shaft to the west elevator. This was five minutes before the assassination and ten minutes after Rowland had already observed the man on the 6th floor with the rifle.
I have a feeling we will agree to disagree on this.