Opening up the ssh port isnt a big security risk. But be sure to keep your SSH server/client updated as they are a very good point of attack for a cracker.
True. If you are paranoid, you can try setting up port knocking or similar schemes that operate over HTTPS to close and open SSH port whenever required.
Regards,
ah