A sergeant and two NCOs are reported to have killed a dog and several cats, and their commanding officer is thought to have known what they were doing and turned a blind eye.
The incidents occurred last week in the Nothing Hill military camp in northern Kosovo, where the animals are reported to have been strays.
A press spokesman for the Austrian army told The Local that as soon as the allegations were made disciplinary proceedings were initiated.
The sergeant and the two NCOs have already been discharged from service and will return to Austria on Friday.
Around 525 Austrian soldiers are currently stationed in Kosovo - making up approximately three percent of the overall strength of KFOR.
KFOR's mission is to ensure a safe environment in Kosovo and to enable proper conditions for the development of a future state.