Language barriers are tricky to portray in written stories. There's a danger of excluding the reader if you use too much of an unfamiliar or created language. A lot of stories in the past have used handwavium to get past the problem of a language barrier - "well, they use telepathy!" "Babelfishes!" etc. I'm not even going to go into the reasons why I think telepathy is problematic, but it's the kind of solution to a language barrier that doesn't hold up well to close scrutiny. On the other hand, it's fully possible to get a reader to accept it if you just put it out there and say "Yeah, so trust me, there's telepathy." Many readers will say "Okay." I will say "Hmm, okay... but couldn't you have done better?" Another way of dealing with an unfamiliar language is having the point of view character not understand the foreign language either, and show him/her dealing with that lack of comprehension (which lets the reader empathize).
Brian was quick to note that language barriers are a huge real-life problem as well, and played a huge part in early European interactions when they arrived in unfamiliar locations and tried to do reconnaissance. Sometimes it worked (surprisingly) well. Other times not so well, as when the locals brought out an array of barter items and those were interpreted by the Europeans as "gifts." As Brian said, why should they have been surprised when they got attacked for thieves later? He shared a story idea with us, where humans are "shipwrecked" in a location with an intelligent creature that doesn't use auditory language, and thus has no way to communicate with them. What do you do then?
I mentioned one story which stood out to me as having zero linguistic communication between an alien and a human - "Spar" by Kij Johnson (where they were having concourse of an entirely different variety). In my own work I looked at the problem of using a non-auditory channel of communication in "The Liars," where the humans were confused because the Poik were using both auditory and non-auditory channels at the same time, but the humans could only perceive the auditory one.
I'm sure many of you have seen the scenario (and all the discussants had) of two creatures comparing objects and assigning names to them. This is a classic way to approach what I call the "code-breaking" language barrier scenario, where one entity has to learn the other's language. However, Star Trek once showed how the object-comparison scenario has problems with it (remember Troi and the cup of coffee?). Also, there is a fundamental assumption involved that both of the entities involved discern objects in their environment (I'll return to that in a minute). Another common code-breaking scenario is hard-SF-radio-waves-across-the-cosmos, where the main goal is to show the interlocutor that one understands math. It's exciting, but I have a really hard time imagining what next steps might be taken. (I'd love to see a story that takes that question on!)
Language acquisition at the most basic level is all about associating language cues with surrounding context. This makes it very difficult when we do not share context, or share only very basic aspects of context. Given the complexity of social context and the fact that it coexists with circumstantial context, this can be quite challenging.
Different languages tend to categorize things differently, and this is one of the most interesting aspects of bilingualism (to my mind). Sheila Finch had a very interesting way of approaching the language barrier problem in her Xenolinguist stories, because she directly addressed the categorization problem. She gave her xenolinguists drugs to take that were specifically geared to dissociate the mind from its traditional categories. It's an important point, because categories limit us in our thinking. Cultural viewpoints already make huge differences in categories, and species viewpoint could take that even further. What if we were to encounter a species that did not recognize objects as objects, but spoke about the world in terms of the salience of differences in texture? I'm not sure we humans would know what to do - and the old point-and-name technique might lead to unexpected results.
Lykara researches conlangs (constructed languages) and was telling us that she has seen conlangs based on a synaesthetic individual's perception of how words taste. Based on this, she speculates that understanding actual aliens is unlikely. I tend to agree with her, and so does Sheila Finch (as Sheila and I have argued this point before on convention panels).
Jaleh mentioned seeing some interesting Bollywood movies where one character only understood a few words, which led to much misunderstanding and trouble. There's certainly a lot of potential for that with language barriers, which is one reason they make for such good stories!
Another approach that people sometimes take to bridging a language barrier is exposing children to the alien language and taking advantage of their natural language-learning skill. This can work, so long as the language is auditory and perceivable by the children! But it is not without its challenges.
We also spoke about translators. They can make an interaction work, but they can also manipulate it. I talked about a friend of my family's who works as a translator in Japan, and she is appreciated by the governor because she translates his jokes. (Apparently Koko the gorilla also translates jokes...) The point about joke-translation is it relates directly to a politician's purpose in speaking, which is more than just basic message-sending. This relates back to what we spoke about earlier, with social context. A politician has social cues he/she wants to send, and those are culturally based, which means that translating them can be a big challenge. Depending on the amount of cultural information implied by the words being used, a translator may be able to turn a long speech into just a few words... or may have to take a few words and explain them at length in order to make them meaningful. This is related to the question of shared metaphors and shared stories, which always brings me back to the Star Trek: TNG episode "Darmok" and my own story using the same language principle, "Let the Word Take Me." The extreme situation of a people speaking only in literary allusions, i.e. references to a set of shared stories, is only an intensification of what we already do all the time in language. A great many of us love to play that game where we start a movie quote and let someone else finish it. It's hard to say "I'll be back," or "Inconceivable," without evoking an additional layer of emotion - and solidarity on the basis of shared experience. As Lykara mentioned, one can also deliberately misquote for humorous purposes, and subvert the basic context.
The problem of language barriers has been around for a long time. Shakespeare dealt with it in several different ways in Henry V, having the French speakers alternately speak French or English as the surrounding scene required for audience comprehension.
In my story, "Cold Words," I used a minimum of alien language but used the style of my English prose to imply that everyone in the story was actually speaking Aurrel (and readers were simply seeing a translation).
Lykara noted that Jean Auel dealt with language barriers in Earth's Children, often with great humor. Sometimes she had two different language groups use two different English words for the same plant to imply the language difference.
Jaleh mentioned The Blue Sword, in which the reader is asked to identify with the protagonist, who doesn't understand the new language at first. Later, when she understands it, it comes to be rendered in English while other characters comment on their lack of understanding.
I mentioned that I thought there had been a lost opportunity in James Cameron's Avatar, because the design of Na'vi was so brilliant but there were no substantive misunderstandings - i.e. misunderstandings that weren't just quaint and charming, but actually influenced the plot proceedings.
If you are creating languages, it's good to remember that languages are not monoliths. They always have variation and dialect within them - different vocabulary, different sayings, etc. You can also make use of shared cognates between languages that grew out of the same parent language, to ease understanding. Word borrowing can lead to some ease of understanding, but totally unrelated languages (like English and Japanese) are much harder. We are also up against the problem that adult brains will often re-interpret what they hear as being a language sound from their own language, even when it most definitely is not. Jaleh mentioned that there were five people on Babylon 5 with the name "Zathras" which sounded exactly the same to all the viewers (and English-speaking characters) but were treated as different by the five people in question.
Be aware that using spelling changes to indicate an accent can be problematic, and that changing meter and modes of expression can often be more effective. Putting a glossary in a story definitely changes the reading experience.
This was as much as we were able to cover during this discussion - there is always so much more we could say, and we only tend to realize it right at the end. :) Thank you to everyone who participated.