Games : Skyrim - Would switching all my mods from the Steam Workshop to Nexus affect my current saves?

I currently use workshop for some mods but recently installed nexus. Both systems work fine side by side. However, I would like to switch to using nexus only. I'd prefer just using one modding app for simplicity sake. Can anyone tell me, if I unsubscribe from the workshop version and install the nexus version of a mod (particularly follower mods), will I lose things like my followers, the items they are carrying, etc?