Swapping driving duties and recording travelling time

As a business, we operate a number of vehicles and regularly have drivers swapping over driving duties, midway through journeys. We are now being told by VOSA that the drivers have to record their travelling time to or from the vehicle, when the changeover takes place, as part of their duty. Is this correct?