Just like a human child can be a girl or a boy, a horse foal can either be a filly (female) or a colt (male). See diagram below for an example.
HUMANS
HORSES
BOY
COLT
GIRL
FILLY
When a mare (an adult female horse) has a baby, she is said to have "foaled", when the foal stands and nurses, it can be called suckling, but is usually just refered to as a foal. In some parts of the western United States, local dialect makes it okay for any foal to be called a "colt" regardless of their gender.
What are older baby horses called?
At 4 to 6 months, a baby horse is seperated from it's mother. After this seperation process, called weaning, the baby horse is called a weanling. "Weanling" is the proper term to call the foal until the baby horse turns 1 year old. After one year old the baby horse is called a "yearling" and yearlings over 18 months old are sometimes refered to as "long yearlings". When the horse turns two years old they are usually refered to by their adult names, mares or stallions.