First of all... I make no claim to having any idea what i'm talking about, I can only say what's worked for me...  

Well, research your time period, unless you're making it up. If you're making it all up then really i'd go for impressive with the castles, but think of the personality of the ruler. Like... one time i had a castle in a children's book i never really went anywhere with, anyway it was always run by girls. Yes, and entire kingdom of girls... think praying mantis, the husbands always disappeared, the only boys were sons of the royal family, who became knights, and townsfolk laborers... anyway, their castle was made of limestone spirals and towers and the roofing either had silver powder of was made from rose quarts and such...

 It was very decorative... i wanted the entire town to look like... well, okay this may sound stupid, but, like the top of a ballroom dress i had seen... i can explain that more if the craziness hasn't scared you off.

Really, it works. Set stuff up, ordinary or extraordinary things you see and... be like a kid, build a castle, a fiefdom, a kingdom, whatever you wish. If, try as you might, you can't seem to make a castle, then just keep certain things in mind, like a spring, or a q-tip or even a skeleton- skull castle! Can you see it?

Anyway, from there you print out pictures of castles, find floor plans to castles so you know what is in them, and draw a castle, adding in things, like... maybe you want your castle to look something like a volcano? Just draw a base castle, put trace paper over that and hold up certain objects and think how the would fit in.

Like... okay, this Dasani bottle. It has a cup- like thing above the label. A rim too...  and above those,  the plastic forms  an upside-down cup with lightly curves designs... or a tower with curved windows. Perhaps that design appears throughout the castle? Anyway, that's what i do, just take something that works, find out what is in it and what its needs are. Understand that castles were made for protection, so what type of protection does your castle have?

It takes a lot of thought, but in the end you have a castle that looks cool, can function  and is battle ready,

well when i draw castles i get a picture of a real one and visualise the towers and the arcs and how its constructed and use it in my art

What I do is just visualize a castle or town in my head and sketch the picture out on paper.

I did have a go at drawing a classic fantasy castle, although I may do variations of this theme to get some more inspirations for it!

You can check it out here - Drawing Classic Fantasy Castles