It will make a difference but it should hopefully be low enough to not be noticed. A big bore backbox will slow the exhaust gases and create a wall of slow air at the end, which the faster gases have to push out, its possible there will be a reduction of mid range torque. An ideal system keeps the gas speed constant through to the tailpipe which is why race systems tend to have the same diameter tailpipe as the bore of the exhaust system.