Would changing from D3100 to D80 considered an upgrade?

I'm using Nikon D3100 which I bought it earlier in April, but I was thinking to swap it for a used D80.Is this a fair trade? Because mainly I'm not using the video recording and live view at all. Please advice.

Not really.

As you can see by this link, the D3100 sensor out performs the D80 sensor

http://www.dxomark.com/...nd2)/Nikon

An upgrade would be to the newer D90 or D7000 both of which can be purchased used.

Here is how the sensors compare.

http://www.dxomark.com/...nd3)/Nikon

The D80 is rather old technology replaced by the D90, D7000 and new D7100.

Just because a camera has a video feature, does not mean you need to replace it. If the still images you produce are excellent, there's really no need to change it just because it has a video feature. As time goes on, All dSLR cameras will have video features, there's just no getting around it.