In the course of a few years the 'Company's Garden' was spoiled. Black Town had been springing up close by; and, when a wall was built round old Black Town, the Company's Garden was unpleasantly included therein, and the Garden was now in the north-west corner of the Indian city. Moreover, a part of the Garden had begun to be utilized as a European burial-ground, and huge funeral monstrosities of the bygone style had begun to dominate the enclosure.

The Company's agents in Madras felt that a new recreation ground was a necessity; and they were agreed that there ought to be not merely a 'Company's Garden,' but a 'Company's Garden-House.' They wrote to the Directors saying that there were occasions on which the Company in Madras had to entertain 'the King (Golconda) and persons of quality,' and that they had no building that was suitable for any such ceremonial proceedings. True there was the Council Chamber in the Fort, but the Council Chamber was the place where the Company's mercantile transactions were discussed; and the Chamber, as well as all the other buildings in the Fort, was closely identified with the 'Factory;' and the Company's chief officials in Madras declared—not, we may suppose, without regard for their own convenience—that a stately 'Garden House,' unassociated with ledgers and bills of sale, ought to be built, in due accord with the stateliness of the Company itself. Their application for permission to put the work in hand was met by the Directors in London with the typically frugal reply that the work might be done but care was to be taken that the Company should be put to 'no great charge.' Possibly the representatives in Madras were able to provide additional supplies on the spot, but, however that may have been, the house was 'handsomely built,' yet 'with little expense to the Company.' The new garden seems to have comprised the area within which the Medical College and the General Hospital are now situated. The grounds, which stretched down, even as now, to the bank of the river, were well laid out, and the Company's first 'Garden House' was a fine possession.

In 1686 Master William Gyfford, Governor of Fort St. George, had a fancy for using the Garden House as a private residence for himself. It is not to be wondered at that he did so; for Master Gyfford, after twenty-seven years' residence in Madras and more than twenty-seven years in the East, was in poor health, and lately he had been taken ill with a 'a violent fitt of the Stone and Wind Collick.' The gardenless 'Factory' in the Fort was a gloomy apology for a 'Governor's House,' and the crowd of employees that were accommodated there must have been a serious infliction upon the invalid Governor; and he found the Garden House an agreeable retreat. In his new quarters he got better of his illness; and he dwelt there a considerable time, till in the following year he left Madras for England for good. The story is interesting, for it records the first occasion on which a Governor of Madras lived in a separate house outside the Fort.

On various occasions the Company's 'Garden House,' with its extensive grounds, was used for public purposes, justifying the plea for its construction. For example, when the Company received the news of the accession of King James II, the event was celebrated with brilliant proceedings at the Garden House. Similarly, at the accession of Queen Anne 'all Europeans of fashion in the City' were invited to the Garden House, where they 'drank the Queen's Health, and Prosperity to old England.' In an earlier chapter we have related how a young Nawab of Arcot who had just succeeded to his murdered father's throne was entertained at the Garden House with great doings. Governor Pitt made great developments in the Gardens, and was another Governor who liked the Garden House as a residence. An Englishman who was living in Madras in 1704, when Pitt was Governor, has left an interesting account of the Garden House as he saw it:—

'The Governor, during the hot Winds, retires to the Company's new Garden for refreshment, which he has made a very delightful Place of a barren one. Its costly Gates, lovely Bowling-Green, spacious Walks, Teal-pond, and Curiosities preserved in several Divisions are worthy to be Admired. Lemons and Grapes grow there, but five Shillings worth of Water and attendance will scarcely mature one of them.'

Before long it had come to be an unwritten regulation that Governors at Fort St. George might reside at their choice either in the Fort or at the Garden House. There came a time, however, when the Governor had of necessity to betake himself to the Fort; it was the time when the French were besieging Madras. During the siege the enemy used the Garden House as a vantage-ground for their big guns; and afterwards, when they had captured Fort St. George and were in occupation of the city, they pulled the Garden House down, lest the English, trying perhaps to recapture the Fort, should be able to use it as a vantage-ground in their turn.

Thus, when Madras was restored to the English, the Garden House had disappeared, and the only house for Governor Saunders was the original residence in the middle of the Fort. Governor Saunders, however, was not content with the walled-in accommodation that the Fort provided and was unwilling to forgo the residential privileges that his predecessors had enjoyed; so a private 'garden-house' in Chepauk was rented in his behalf. It belonged to a Mrs. Madeiros, a rich Portuguese widow, whose husband, lately deceased, had been a leading merchant in White Town.

Mrs. Madeiros's house was 'Government House, Madras,' of the present day. The house, however, has been enlarged and the grounds have been extended since Governor Saunders lived there as a tenant.