It was the 'most annoying thing' Amelia Warner said of Jamie Dornan's s-e-x-y sc