brcm47xx: make it possible to unregister the adm switch