Do you think he fancies me?

Well theres this guy in my cl at college. We get on brilliantly, and after college we sometimes go into town, or even just hang around the train station for an hour or so. The other week we were having a right laugh when we decided it was time to head off home. Anyway at the last minute he asked if I wanted to go for a drink, which I did, and we fell out of this pub about 3 hours later. When we got to the train station, my train was due, so that was it , we both went home. At one point while we were out asked him why he not married or anything and all he would say was Ive had my heart broken three times I'm not going down that road, . That was it, he wouldn't talk about it any more. In cl like during lesson time, he sits at one side of the room, and I'm at the other, we just do our work, but as soon as he has chance he comes over to me, always taking the mick and joking around. He has never attempted to like kiss me or anything although he had loads of opportunities, shall I just ask him out, or are we only friends.