Vuescan isn't your problem.
The Nikon 8000ED is a Firewire scanner, and new computers don't come with Firewire. A lot of people are trying to use USB to Firewire adapters for these scanners and most of the time they give the kinds of issues you're experiencing.
The 8000ED came with a Firewire PCI card to put in the computer, but it probably won't work on modern systems.
What you'll need to do is try to find a Firewire card that works with modern PCIe slots found in new computers, or get an old computer to run your scanner from.