Language is one form of communication and, by definition, involves thinking. However, not all communication is language. For example, "body language" is a misnomer and may not involve thinking at all. The signals your body gives off are not necessarily abstract and do not necessarily have to follow any form of syntax.
Symbols
A symbol is anything that is a referent for something else.
However, the appearance of an icon or a photo gives clues
about what it represents, while an abstract symbol does not.
Consider this abstract symbol: "Apple". This English word represents the fruit to us. It conjures up a mental image. However, its appearance gives us no clue about its meaning. This symbol is made up of 'a', two 'p's, 'I' and V. Each of these is meaningless alone; their correct arrangement is what gives the symbol meaning. If the users are able to speak, these also allow for a phonetic element of speech.
Words in other languages function the same way. For instance, the Spanish word 'manzana' is an equivalent abstract symbol for apple.
Syntax
Syntax is the agreed upon set of rules for how symbols are
arranged to express meaning. For example: "Mary loves
John." defines a clear scenario. A simple change in syntax
(rearranging the symbol order) defines a different scenario:
"John loves Mary." The symbols are exactly the same.
The syntax however, greatly affects the meaning.
The Power of Language
Humans are the only species able to record their thoughts.
Recorded messages, as well as spoken language, allow us to
transcend time and space. A woman who records her thoughts
in a letter in 1885 can transcend time. Over 100 years later,
we have an exact mental image of what she was trying to express
in words. Even though she has been dead for decades, we have
access to what she was thinking and feeling, assuming that
deception was not involved. Today we can transcend the limits
that distance imposes every time we use a telephone or send
an e?mail. We can immediately know the thoughts of a friend
thousands of miles away using these two methods to transmit
language.