I don't want to hurt him, but I want to be fair to him, help?

Tell him. That's all you can do. Sit with him alone. Talk about it. Say the truth. Maybe exclude the part of you never feeling anything, but the rest you have to. He's a senior moving to college. If you do it now maybe he'll be over you by the time he starts college and he can concentrate on his career.