Even very young children know that sometimes, as you move, nearer objects will seem to move relative to more distant objects. Children wouldn't know the name of this phenomenon; it is called parallax. Scientists in the time of Copernicus and Galileo also knew about parallax. This knowledge led to the most serious and valid challenge to the theory that the earth moved; stellar parallax. They argued that if the earth moved around the sun as Copernicus and Galileo suggested, then there must be some evidence of parallax effects. A moving earth would mean that the earth would be on opposite sides of the sun six months apart. And if the earth was on opposite sides of the sun six months apart there should be some visible change in the relative position of the stars during the year. Nearer stars should seem to move relative to more distant stars. The relative positions of nearer and farther stars seemed to remain the same.
No one in Galileo's time or for almost 200 years after his death was able to demonstrate this necessary effect of earth's motion around the sun. Stellar parallax was finally observed in 1838 by Friedrich Bessel, a German scientist. But it is not Bessel that is credited with finally proving that the earth moved around the earth. In 1729, James Bradley, while searching for the elusive stellar parallax, detected motion of the stars over the course of the year which did not fit the pattern of stellar parallax. He had discovered stellar aberration, which is also related to the motion of the earth. Regardless, proof of the earth's motion was not available in the seventeenth century and those arguing for it's motion had no answer for why stellar parallax could not be observed. It is a huge problem when a consequence of a theory is expected and it cannot be observed. It is enough to keep a hypothesis from being accepted as a proven theory, regardless of the number of positive arguments in its favor. This applies as much today as it did in the seventeeth century.
These scientists were correct that a moving earth required that there must be stellar parallax. They were also correct that this parallax could not be observed. However, their rejection of the hypothesis was based on more than this; it was also based on the assumption that the stars were millions of miles away. These stars were billions of miles away. Given the instrumentation of the day, one could expect to detect stellar parallax if the stars were millions of miles away but not if they were billions of miles away.
The difficulty of eliminating assumptions in testing theories exists as much today as it did four centuries ago. It even has a name; the Duhem-Quine thesis. It was originally proposed by Pierre Duhem. Pierre Duhem was an important scientist, philosopher and historian whose work was censored because of his controversial views on history (see The Real DaVinci Code). Simply put, this thesis states that in practice it is difficult to ever test a theory independently of other theories or assumptions. This means that when an experiment 'proves' a theory false it is really just proving the collection of theories and assumptions false, not necessarily the theory itself.