James Hilterbrand was born and raised in Adelaide, Australia, and played for South Australia at junior rugby level. He represented them at Under-16 level in 2004 and 2005, at Under-18 level in the Australian Schools Championship Division II in 2005 and 2006, and at Under-21 level in 2007.
Hilterbrand moved to Sydney in 2007, and joined Manly of the Shute Shield for the 2008 season. Starting for their Colts side in 2008, Hilterbrand worked his way up until he broke into their Grade I side in 2011. He became the regular starter at Hooker for Manly in 2012 and helped them to second position on the ladder before they were eliminated from the competition by Southern Districts in the preliminary final. He was also selected to represent a Sydney Anchors team against NSW Country.
In 2013, Hilterbrand joined Western Force on an Extended Playing Squad contract for the 2013 Super Rugby season and made his Super Rugby Debut one week later. At season's end, Hilterbrand returned to Sydney to make four more appearances for Manly before signing a two-year contract with Edinburgh of the Guinness Pro12. At the end of the 2014-15 season Hilterbrand returned to Manly with 23 appearances in Guinness Pro12 under his belt.
In 2015, Hilterbrand joined the Kings in the South African Currie Cup before returning to Australia to rejoin Manly.
It was in 2016 that Hilterbrand made his Men's Eagles debut, coming in as a substitute in a Americas Rugby Championship win against Canada at Dell Diamond.
Hilterbrand signed a contract with the Waratahs of Super Rugby in 2016.