Revert "swconfig: fix Broadcom b53 support"

Revert "swconfig: fix Broadcom b53 support"

This reverts commit 8f9cd1af0f9c325a902dbd0e79e12015372e6bb0.

That commit was meant to add a single EXPORT_SYMBOL_GPL() but it
actually also added few .of_match_table-s. One commit should handle one
thing and should not introduce unrelated changes.

Regarding actual changes:
1. EXPORT_SYMBOL_GPL is not required as we don't build swconfig drivers
   as modules.
2. PHY drivers must not have .of_match_table. That is allowed for MDIO
   drivers. This could work for some time (although is didn't for me on
   bcm53xx) but does not with kernel 5.10. It causes a soft lockup and
   upstream developers confirmed it's an unsupported design.

Link: https://lore.kernel.org/netdev/2b1dc053-8c9a-e3e4-b450-eecdfca3fe16@gmail.com/t/#mf80e472f35ee23f7a75cbf5b1e101a17ab3a64a3
Cc: Tobias Schramm <tobleminer@gmail.com>
Signed-off-by: Rafał Miłecki <rafal@milecki.pl>