No. Firstly, SSH is not used at the scale where any such misuse might be a problem. Secondly, SSH is used for connecting to specific hosts, which is what the A/AAAA records actually are (being by definition the host address).
Are you somehow thinking that if you have a web server serving HTTP traffic on a server, you need a different A record to connect to it with SSH? Because that is very much not how it works.
