I can't remember whether the fuel pump relay by-passes the fuel pump resistor during the start up sequence; but, it is definitely de-energized during idle and low engine load operation. You definitely should not be hearing the fuel pump relay click on - off if you are just driving around and not entering V Tec mode (high engine load). If you want to confirm that the problem is the fuel pump relay, just remove it. This will leave the fuel pump resistor continuously in the fuel pump circuit (low voltage mode for the pump). You can drive around without any problems as long as you don't go wide open throttle or do anything that would cause the engine to enter V tec mode.
If the noise goes away with the relay removed, then the relay may be the source of the noise. However, I suspect that the relay is not the cause of the problem. As long as the engine is in non V tec mode the relay should stay open (be de-energized) and you should not be hearing it go click - click if you are doing relaxed driving. If the relay is going click - click when you are just driving around in non V tec mode, then you have one of two problems. The ECU is sending erroneous on-off signals to the relay or you have an intermittent short to ground on the red wire between the fuel pump relay and the ECU that is causing the relay to open and close.
View attachment 169943
The ECU monitors the pump voltage (black/red wire); but, I don't know what control action it takes with that signal. Check to make sure the black/red wire is OK.