The Century Dictionary says there is no difference between O and oh except that of their present spelling. The New Standard and Webster’s New International do not go so far, but they point out the difference observed by most good writers.
O is generally used only in direct address; and, as the name of the person or thing addressed immediately follows it, it takes no mark of punctuation after it. An exclamation-point may follow the group of words introduced by O. Its vocative character is not lost when the person or thing addressed is not named, for it may be understood.
O is used more in poetry than in prose.
O is used in an ejaculatory expression when followed by for or that. It does not here seem to lose its vocative character, although the name of the thing or person addressed may not readily be supplied.
O is sometimes used colloquially in expressions like “O my!” “O dear!” etc.
Oh is purely ejaculatory, and takes a comma or an exclamation-point immediately after it; but the latter mark may follow the group of words beginning with oh, with a comma before oh.
O is always written with a capital, but oh takes a capital only when beginning a sentence. Some writers prefer always to write oh with a capital.
EXAMPLES
1. Yesterday was my last bad day, but I remember the preceding bad days.