Games : Unable to leave random steam groups

I have a huge list of steam groups I've never joined in my "Groups" tab of the "Friends" section of Steam. It looks like this:

Some of the games I own, some I had on my wishlist, some I've never heard of. The top two are the only ones I actually want to be in. When I go to the Arrow -> Leave this group, I only see this:

Those are the two groups I remember joining and want to be a part of. I tried to join and leave the group, but when I click Arrow -> View Group Profile, all I see on the side is this:

I don't see any option to join or leave the group.

When I go to my public profile, it says I'm only in two groups and doesn't mention these groups anywhere:

How do I get rid of these groups?