Trouble shooting up close video with Nikon d5000?

I don't know what happened but every time I try shooting a video and I zoom in the image gets blurry and the same thing has happened when I tried taking pictures. I'm using the 18-55 mm lenses. I'll appreciate the help.

You will probably have to refocus each time you "zoom" in.

I hope you are not using the zoom lens to zoom in on subjects during each video take. Think of the zoom in video or film as an exclamation point in a piece of writing. How many of those can you stand until you just put the magazine or book down?

To assure good focus, extend the zoom to its maximum focal length and manually focus the lens. Then zoom back until your scene is properly framed. This should assure that all your video is in focus.

If you are really shooting macro shots, then you need a macro lens.

Is your camera (or probably on the lens) set to auto focus or manual focus (A/M)

Unless you're using a macro lens, which you aren't, "up close" means about 2.5 feet from your subject.
If you're closer than that, back up,. If you zoom you'll need to re-focus for every shot. For video I recommend you learn to do manual focus, the only way to shoot video.

As seen at
http://www.cameralabs.com/reviews/Nikon_D5000/movie_mode.shtml
"you can autofocus before you start filming, but once the D5000 starts 'rolling', it becomes manual focus only."
Most zoom lenses are not parfocal. That means that changing the focal length shifts the focus. The D5000 will not autofocus during video, it will not correct that focus for you. You can manually refocus, but doing that accurately tends to not happen very often.