Other controllers controlling my PC

I am having a strange issue, to me at least, regarding my Xbox controllers downstairs controlling my PC when my device they are connected to is turned off. So I have an Nvidia Shield that I use a wireless Xbox dongle to simulate it being connected to my pc via VirtualHere. But all devices, mouse, and keyboard included will begin to control my PC even when I don't have the shield running, which I thought meant the VH part wasn't running and even with the server not open or running on my PC. Now I know this might be as simple as just unplugging the dongle, but there are headaches with reconnecting everything that happens when I do this on many occasions so I'd rather find another solution. Can someone help me understand why this is happening or through experience give me a solution? Thank you in advance!

#2

My guess is that the devices are connecting directly to the bluetooth inside your pc when virtualhere is not running. Which is expected as they might have enough range/reception to do that if your pc is not too far away