Those rubber spacers were a poor idea from the beginning. For one, the engine is an integral part of the frame, and without it clamped securely, the frame can flex much more than it should. Handling will suffer.
Secondly, without the solid spacers, the frame can be drawn in tighter and possibly break at the mounts. Not uncommon.
Thirdly, they don't really eliminate vibration - the best solution is a good tuneup. Attack the problem at the source, don't mask it!
I'd caution not going there, and either return the bike to stock with the solid mounts, or don't ever install them in the first place!