dnyhof, nope, no need to remove the speaker/amp unless you want to access the front window rail. Pulling back the liner back halfway, you expose an access hole for everything else. You do have to reach inside to get at the rear window rail.
How long.... I am embarrased to say it took me 6 hours from start to finish. But that included preparation and a pile of time to go and buy a pair of multigrips, rent an angle grinder, go home and grind the slots into the tool, return the grinder, go home, do more work, then later go out to a couple of different places looking for material to reattach the liner to the door. On that last note, the DIY says to use puckey but nobody at the local hardware stores had anything like it (I took some to show them). I ended up using a roll-up strip of outdoor weatherproofing stuff, similar to silly putty. It's not as gummy as the OEM stuff, but it appears to have done the job just fine. Time will tell if it holds.
It appears my passenger side door had already been opened up at some point as the liner was poorly reattached and ripped in places. The original owner mentioned he had problems with the keyless entry & alarm so that may have been why. There was gooey grease all over one of the electrical connectors and I spent some time trying to figure out why that was and cleaning it up.
The passenger side took me so long that my helper had to take off (never to be seen again for an NSX DIY job to be sure! ha ha), so I still need to do the drivers' door. I am quite sure I can do it in 1.5 hrs max, less if I can get the cables into the the fixit-thingy without many re-attempts. The door was actually very easy to disassemble thanks to a few other threads here about needing brute force to pop the door panel straight up and out (ie over the door lock stem). One key piece of advice (thanks Larry B): the chrome strip does not come out, so you have to pull up and towards you to get the thing off. Pop the back end out first.
Good luck!