For the favorites/desktop item:
Click the Show Applications icon on the bottom-left. Right-click on the “Install Ubuntu” app, and select Show Details. Next, click on the Remove button. If after removing the app, there is still an icon on your desktop for the install Ubuntu, right-click and remove it.
For the grub (boot up) menu:
Mount the windows partition…
sudo mkdir -p /media/USBDRIVE
sudo mount /dev/sdb3 /media/USBDRIVE
Edit the grub file…
sudo nano /media/USBDRIVE/boot/grub/grub.conf
Eliminate in the grub file any blocks not relative to the persistent boot.
Save the file and unmount the partition…
sudo umount /media/USBDRIVE
Reboot the machine with the USB option and now you should see only the options you left in the grub menu.
Note: If you happen to put your USB stick in the machine while running Windows, you will likely get a few notices stating that the USB needs to be formatted. Do not format the USB as this will damage the files created under the Linux system.