This might not be the best thing to do. The other option is to get a host
key from the user and check based off of that. That is just more process
though.