The church was originally circular- on the model of Solomon's temple- but was recast as a rectangle by the Knights Hospitaller who were heirs to the Templars. The tower is also a keep, capable of withstanding siege, because- well- this is the Welsh Marches. As a character remarks in Phil Rickman's The Fabric of Sin (from which much of the information in this post has been extracted) "There's England, there's Wales and then there's Garway."
The odd horned head- which forms the capital of one of the pillars of the dog-toothed chancel arch- is commonly described as a green man- but patently isn't because those aren't branches coming out of his mouth but decorated cords with tassels on the end. Could this be a representation of the bearded head- known as Baphomet- that the Templars are alleged to have worshipped?