Throughout the discussions found in the Gemara (Rosh Hashana 20a-25a), Rashi continuously states that the New Moon is visible after six hours from the lunar conjunction (molad) and that it was visible six hours prior to the molad. The purpose of the Beis Din knowing this fact is to reject any witnesses that testify about the sighting of a New Moon within this time period. Beis Din will not declare that day to be the first of the month and will automatically assume the witnesses were mistaken or lying.
The Rambam and Baal HaMaor clearly state that the Moon disappears for a couple days because it takes approximately twenty-four hours for the Moon to separate from the Sun to reach a distance that we can see.
Currently, astronomers use a tool called the Danjon Limit. This limit is defined as the distance, in degrees, that the Moon needs to travel from the Sun in order for us to be able to see it. If the Moon is too close to the Sun, the sunlight will outshine the Moon. The Danjon Limit is set at approximately 7 degrees; however, practically most people (under most conditions) need more like 10.5 degrees of separation. The average should come to approximately twenty-four hours from the lunar conjunction which is consistent with the Rambam and Baal HaMaor's opinion. The record (without optical aid) is set at about 15 hours and with optical aid (telescope), the record is close to 12 hours. This is still a far cry from Rashi's six hours. This is something that is easily seen; Rashi was able to witness this phenomenon every month, why would he state such a hard to understand opinion?
(There is a sefer that is cited in the back of the Gemara that mentions that Kepler and Reinhold witnessed a New Moon in Seville on March 13, 1553 and that that sighting proved Rashi's opinion. There seems to be a typo, because the conjunction would have been on March 14, but I believe the reputable author of this sefer was misled by misinformation. Reinhold died in February of 1553 and Kepler was not born until 1571; also, the Moon was far below the Danjon Limit and, if this were true, we would have records still available to us from Kepler's writings verifying this sighting.)
Perhaps, the answer is that Rashi agrees that they could not have seen the Moon. Rashi cites that in many of these cases, the ramifications are only according to the opinion that, when necessary, we trick witnesses into thinking they saw a New Moon even if they did not. This, and other nuances found in Rashi, seem to show that Rashi's overall opinion (even when discussing the opinion that we do not trick witnesses in this fashion) is that we want our calendar to be consistent with the lunar cycles (while making some necessary exceptions when we will have holidays falling on days we do not want) and as long as someone testifies about an event that will keep this consistency, we will admit it into the court. This is even if we know that the witnesses did not really see that which they claim to have seen. This opinion is very similar to that of Rabbeinu Chananel and Rav Saadiah Gaon who maintain that the main focus is the lunar calculations, and the witnesses just verify the sighting for us to be able to apply our calculations.
There are two reasons why the Moon is invisible to us when it is too close to the Sun. One is that the surface of the Moon has mountains and bumps, and therefore, until the Sun can shine above these bumps the sunlight is not reflected to Earth. The second reason is that the human eye cannot detect the faint light reflected from the Moon until there is sufficient darkness around the surrounding area. When the Sun is too close it is still daytime and we cannot see the Moon. In other words, one is a factor because of the Moon's inability to reflect the light; the other is the inability of the human to see the reflection. If we only focus on the constraint caused by the Moon, we can lessen the Danjon Limit to somewhere close to 2 degrees. This is much more consistent with Rashi's opinion. If witnesses would testify that they saw the Moon, we would know that they did not, but we would still accept the testimony because the Moon is at a point where it is new and it is clear from Rashi that the main focus of the process is that the Moon and calendar be in sync and not that we care whether the witnesses actually saw the Moon. This would be another example of us not being so careful with testimony of the Moon which is a concept that even the Rambam records.
I would also suggest one more point. It is clear from Rav Saadiah Gaon and Rabbeinu Chananel (and I would suggest, Rashi) that the commandment upon Beis Din to regulate a calendar is to do so via calculation and not eyewitness accounts. Therefore, the machlokes in R"H 24b - 25a between Rabban Gamliel and other Tanaim as to whether to trust certain suspect witnesses is clearly rooted in a machlokes of whether or not to follow calculation or witnesses. It is interesting to see that Rabban Gamliel invokes a tradition from his family to support his claim. A few lines later Rebbi seems to be expressing a similar claim. Rebbi was a descendant of Rabban Gamliel. It would seem that the household of the Nasi (Rabban Gamliel's family) was in charge with regulating the calendar.
Now, when looking at the famous Mishna later when witnesses came late in the day on Rosh Hashana and the Levi'im sang a weekday shir instead of a yom tov shir, we can see something fantastic! One could have asked why they were bothering with witnesses and not using calculations (according to Rav Saadiah Gaon, Rabbeinu Chananel and Rashi). The answer may be that the Nasi at the time (and posek for these matters, as mentioned in the Mishna) was Rabban Yochanan ben Zakai. He was a temporary Nasi. Rabban Shimon ben Gamliel I was horrifically murdered and his son Rabban Gamliel II was in hiding. The position of Nasi was therefore given to Rabban Yochanan ben Zakai until it could go back to Rabban Gamliel II. Clearly, Rabban Yochanan ben Zakai, not being from the family of the Nasi, did not have their tradition. He was of the opinion of those that argued with Rabban Gamliel (II) on 24b - 25a.
