The land that is now called Castle Hill was originally home to the Bidjigal people, who are believed to be a clan of the Dharuk people, who occupied all the land to the immediate west of Sydney. The best-known Aboriginal person from that time is Pemulwuy, a Bidjigal leader who led the resistance movement against settlers, including sacking farms in Castle Hill, before his eventual capture and dispatch by local law enforcement.

The Bidjigal people are today commemorated by Bidjigal Reserve which straddles the suburbs of Castle Hill, Baulkham Hills, North Rocks and West Pennant Hills.

The first European visitors to the district were led by Governor Phillip in April 1791 accompanied by an entourage. Travelling from Parramatta reaching the 'hills' following the Aboriginal trails, which today are overlaid by the Windsor and Old Northern Roads. As Governor he needed to find new country for settlement and farming land for crops so as to feed a struggling infant colony.[4]

Governor King began Government Farm 3 there on 8 July 1801, referring to it as "Castle Hill" on 1 March 1802. The majority of the convicts who worked the prison farm were Irish Catholics, many having been transported for seditious activity in 1798. The most notorious incident being the Battle of Vinegar Hill where around 39 were slaughtered. They were branded "politicals" and exiled for life, never to return.  (Wikipedia)