Nos. 17 and 18 formed old Halifax House. Many political intrigues and meetings must have taken place here, for Lord Halifax gained the name of always being on the winning side. In 1725 Halifax House was demolished and the present buildings erected. In 1820 Queen Caroline stayed in No. 17 during her trial. The house was afterwards used by the Colonial Club.

No. 18 boasts such names among its tenants as the fourth Earl of Chesterfield, the first Lord Thurlow, and Viscount Castlereagh, afterwards second Marquis of Londonderry. It was used by the Oxford and Cambridge Club and the Army and Navy Club.

At the south-east corner of King Street, in the square, was Cleveland House, which has been demolished and replaced by "mansions."

Apsley and Winchester Houses follow. The former was rebuilt by Robert Adam in 1772-74, and follows the well-known lines of his work, with fluted pilasters rising from above the basement to an entablature. The entrance has the fan-shaped glass above the door so characteristic of Adam's work.

Winchester House was from 1826 to 1875 occupied by the Bishops of that see, and was later a branch of the War Office, several departments of which are still here. The next magnificent building, which really faces George Street, but was formerly considered to be in the square, is one of the palatial clubs evolved by the demands of modern luxury. The house which formerly stood here was used by the Parthenon Club from 1837-41, and was subsequently pulled down to make way for the present clubhouse, opened 1851, and built from designs by Parnell and Smith. The exterior is a combination of Sansovino's Palazzo Cornaro and the Library of St. Mark at Venice. The lower part follows Sansovino's beautiful work very closely. On the site of this stood formerly a house belonging to Nell Gwynne, of which Pennant writes: "The back-room of the ground-floor was (within memory) entirely of looking-glass, as was said to have been the ceiling; over the chimney was her picture, and that of her sister in a third room." He describes this house as the "first good one on the left hand of St. James's Square entered from Pall Mall."

The south side of the square has never been held in such esteem as the remaining three-fourths. But the Junior Carlton Club, facing Pall Mall, has removed this stigma; it is a fine specimen of architecture. Demolition, previous to reconstruction, has already begun next to it. After this as far as John Street is a row of comparatively insignificant narrow houses of various heights and styles. Some of the houses on the north side of Pall Mall were built before the completion of the square, so that there was no room for large mansions here. At the corner of John Street and Pall Mall is what is called "Ye Olde Bull Tavern," a square box-like stuccoed house. This is probably contemporary with the first building of Pall Mall, and may have been the substitute of the seventeenth century wits and men of letters for the magnificent clubs of the present day.

Charles Street was built about 1671, and was, of course, named after the King. Burke and Canning are numbered among the former residents.

York Street was named in compliment to the Duke of York, afterwards James II. It may be noted that the four streets surrounding the square form the names King Charles and Duke of York.

Bury Street was named after a Mr. Berry, who was landlord of many of the houses; the spelling is a corruption. Sir Richard Steele lodged here, also Thomas Moore and Crabbe, the poet, during one of his later visits to London, when contact with cultured men had rubbed off his early boorishness.

"St. James's Street is much more remarkable for the natural advantages and beauty of the ground, than from any addition it has received from art," so says Ralph ("Critical Review of Public Buildings," 1783 edition). In the very earliest maps of the parish a road is marked on this site, leading northward from the palace. The street was built about 1670, and was first known as Long Street. In the time of the Stuarts it shared the aristocratic tendency of the square, and had a list of noble occupiers. It was levelled and made uniform in 1764, having previously descended from Piccadilly by steps.