Bad fuel cap or is it the fuel pump?
 

Okay, so I have replaced the plugs, wires, rotor, cap, intake gaskets, upgraded the fuel spider and replaced the fuel filter. This thing still has hard starts unless I prime it by cycling the key a couple times. Now its worse in the morning and not so bad at the end of the day.

So now im wondering is it the fuel pump or the fuel cap?
At stop lights I do seem to feel a rough idle only after the thing is warmed up though.
Im thinking it is hopefully the fuel cap because I do not get that pressurized sound when I undo the cap. Could this be my problem? I hope its not the pump because im trying to sell this thing and I dont want to put anymore money into it.

I doubt it would have anything to do with the fuel cap. You need to do a fuel pressure test and report the figures. Let's see, what else...oh, yeah...what are you driving?

Sure sounds like a leaking check valve in the fuel pump assembly.

Its a 1996 4dr 4x4 4.3l; Its not throwing any codes or anything. The funny thing is that when it starts up cold it runs good until its warm then its got a little bit of a shaky idle. Which makes me think its a maybe a leaking fuel cap.

A leaking fuel cap will cause no driveability concerns at all, not with out other problems. Your starting problem and running problems sound like two different things. Do a fuel pressure test and let us know the results. Without that information, you are just spinning your wheels.