MAJUSCULE and miniscule letters are the actual terms for them.
So why do we call them "upper case" and "lower case"?

In the early days of the printing press, a block preparer would sit down and set all of the letters in a block about to be printed. He would sit at a desk with drawers of cast metal letters, and individually place them. As you can imagine, that is a whole lot of letters to have in a drawer, so they were considerably heavy. I can't find the exact ratio, but for this purpose, we can ballpark it. For every one majusule F, you would probably use about 50 miniscule f's. For typesetting, the two cases are taken out of the storage rack and placed on a rack on the desk. By convention, the case containing the capitals (and small capitals) stands at a steeper angle at the back of the desk, with the case for the small letters, punctuation and spaces, at a shallower angle below it to the front of the desk, hence upper and lower case.

How 'bout that.

*paraphrased from Wikipedia, verified by Dr. Eric Casteel