Do FX lenses that are used on FX cameras greatly increase the quality of the pictures compared to FX cameras using DX lenses?

I'm a beginner photographer who purchased a Nikon D850. It's the second dslr camera I've purchased since I upgraded from a Nikon D3200. I purchased a D850 and use a DX lens instead of an FX lens because my previous camera used that lens so I decided to give the DX lens a try on my D850. So far I've taken a couple of pictures in auto settings but when I zoom in to see the focused area of the pictures, the pictures look blurry. I'm wondering if they look blurry because of the DX lens or if it may be because some other reason. I'm planning on getting an FX lens to see if it fixes the blur issue. I hope that switching to that lens will greatly improve the quality of the pictures. Has anyone seen a huge difference in quality with full frame FX cameras when taking pictures with FX lenses compared to DX lenses?

It's hard to answer your question without seeing the actual image(s) that you don't like.

The D850 has a 46MP sensor vs the 24MP in the D3200. Because of the dramatic increase in resolution, you will be able see more of the optical defects when using any lens, and not just a kit lens like the 18-55mm that you're using.

Going to a 46MP camera will force you to improve the way you take photos. The resolution is so high that you can easily see what is and what's not in focus. Therefore you will notice a decrease in the depth of field since DOF is just what appears to be in focus. The higher MP in the D850 will make it significantly easier to determine what's in and what's not in focus. So you will need to close the lens down a stop or two to make up for the diminished apparent DOF. Camera shake will also be much easier to detect, too.

I noticed this when I started using a 40MP camera. My old way of taking shots proved inadequate with a higher MP camera. I found that I had to use faster shutter speeds than typical to avoid camera shake.

For this and other reasons, high-resolution cameras like the D850 are often described as tripod cameras. I would strongly suggest that if you don't use a tripod that you invest in lenses that have image stabilization. And yeah, get rid of that kit lens and go with FX lenses.

To see the resolution of various lenses for the Nikon system, go to photozone.de which has excellent resolution charts that use real numbers. Knowing that a specific lens as a resolution of, say, 2400 at a given f/stop (resolution will change when using various f/stops) will allow you to find the best lens within your budget. For example, the Nikon 24-70 f/2.8 lens is about 10% sharper than the Tamron equivalent. But the Nikon costs more than $1,000 more than the equivalent Tamron. Do you need that extra 10%? Probably not, so the Tamron would be the better choice even though it is not technically sharper.

When you use a DX lens on an FX camera, the image projected onto the sensor doesn't actually match the full frame. So the camera hides this from you by cropping down to the DX size. If the image is viewed at a higher resolution than it was taken at, the difference is like watching an SD show on an HD TV, i.e. There's going to be some blur.

If you use a DX lens on a FX body it will not cover the whole frame, thus you will get only a file effectively of DX size, and the edges of the photo will fade to black. That applies especially to wide-angle lenses.

The other way round it works - you can use FX lenses on DX bodies without problems.