Toggling a calendar coloring between Only Reclaim events and All events is a "heavy" operation because it may need to color code hundreds of events. Toggling it more than twice within a minute can cause issues.
To reset your calendar colors
Set all colors to Calendar default.
Wait a few minutes.
Set the colors back to their original values.