Maintain exceptions separately
Keep it Smart: If there are exceptions (that is, a "denylist") of devices to exclude from a smart group, maintain a separate denylist device group containing these devices rather than abandoning the rules and manually removing the devices from the group. For example, when troubleshooting, or as bad actors emerge in the network, add them to the denylist device group, and then add a rule to the end of your device match rules to exclude that device group from all of your Smart device groups.