"It was still quite challenging for a couple of days," Luke Bryan told PEOPLE, adding that he did not have a "long set of symptoms"
