Cattle are naturally prey animals.  They are predators of no other living animal and their first instinct is "flight" from any threat if that is an option.  Having not been previously acclimated to humans at a younger age, they are most likely beyond ever fully trusting people. 

When around this particular group of calves try and think of them viewing you as a predator and act accordingly.  Act as if they will never trust you, but act as if you are trying to earn their trust every time you are around them.  Try not to ever act like a predator, (no sneaking up, no sudden moves, announce your presence before they see you with a cough or some other sound, etc.)