My fuel pressure gauge arrived today. Pressure after priming is about 56 psi. When idling also about 56. When I give it some throttle it jumps up to 60 for a moment and then down to about 52 until I let go. Not sure if that's normal but it doesn't affect starting. Returns to 56 on idle again.
I've let it sit (with ignition off) for about 30 mins now and pressure dropped only a couple psi, which from what I've read is normal. If there was a leak worth fixing pressure should be 0 within minutes.

I checked underneath the throttle body and everything is "nice and sooty" there, no clean spots from what I can see without removing more stuff.

I have yet to reproduce the actual starting problem I've been having. It's been remarkably reliable the past couple days.
In my opinion leaks in injectors or the pressure regulator don't just go away like that, so for now I'm still assuming the problem is in the pump.

I'll keep trying to reproduce it so I can check pressure when it's acting up.

 

Update, it just didn't start again. I still had the gauge attached and it measured around 43 psi, never got higher than that until the sound in the pump changed and it was all back to the values in the previous post. The pressure didn't drop from that 43 psi either, so still no leak, it's just sometimes not giving enough pressure to begin with.

After removing the fuel pressure gauge my pressure was 0 because I bled off the pressure before disconnecting the gauge, and it started instantly after that after 1 prime. That probably rules out some other potential problems.

The new pump has also arrived so now I just gotta wait for better weather so I can replace it.

 

According to GM, 1996-2001 Vortec 4.3L V6 engines (VIN "W") must be 55psi to 64psi at idle. You are in spec for that.

The "Key on Engine Off" spec is 60-66PSI. Most techs look for that value during the priming cycle (when the fuel pump is running). It's not unusual for it to drop a few PSI when the pump shuts off after the prime.

The movement of the fuel pressure needle as throttle position changes sounds normal. The pressure regulator is "vacuum sensitive" and increases pressure as vacuum drops, decreases pressure when vacuum increases. That's the way it is designed to work.

Leaking (stuck open) poppet nozzles can actually clear up with an aggressive use of "fuel injector cleaner" additives in the fuel. A bottle of Techron or a can of BG 44K in 5 gallons of fuel can actually clean up some of the dirt and "sticky stuff" in the poppets.

If the vehicle sat for a long time (undriven) before you got it, the fuel could have evaporated/dried in the system and "gummed up" the poppet nozzles, and even new fuel could clear it to some degree.

RESPONSE to your update: That does sound more like a bad pump now. Something leaked pressure for it to drop to 43psi.

While you have the tank out, be sure to clean it out. I often have them steam cleaned and use a fuel tank coating/paint kit on vehicles I plan to keep long term.

 

I suspect your pump may be fine.

You say the pump was replaced already, so that means
it would have the new wiring harness. These connections
on the new harness may be corroded or just poorly installed.
Could also be a poor ground. A voltage drop test would confirm.
I just went through this hell myself.

See link:Electric fuel pump diagnosis: Houston, we have a problem - Automotive Service Professional

 

Well the new pump is in place, and so far so good! I've tested a couple dozen times and it pumps properly every time now. Pressure with key on and engine off is about 57 psi now, otherwise all the same as before. This pump seems to run quieter too but that might just be my imagination.

We'll see how it holds up but for now I'm pretty positive that the problem has been solved

 

yje pump has to be at least 55psi to open poppets