Multi Language Support on Website

[sketchgal]

Hi I'm looking for some advice please.
I'm about to start building a site and I want it to have multi language support. Mainly, French, Spanish, Italian and English being the main/ default language.
I have heard that there might be a way to get the browser to automatically select the language based on the ip address of the user. This sounds quite handy but I will also require a selection option from the home page of the site to allow for any users to choose the different languages themselves.

Can you point me in the right direction for where to start with this please.

Any help much appreciated as always.

[UFTimmy]

Google IP 2 location for info on how to make an educated guess where the user is coming from.

[Jake Arkinstall]

It's quite a painstaking tast to be honest.

Have a database table called languages, with just an ID and a name (English etc)

Have another table called 'Translations' or something like that, and have the columns ID (pk), SectionID, Language, and Text.

for every block of text (including menu text, titles, paragraphs, form labels, etc), there will be a different SectionID.

You can then have a row for every different language (using their IDs) with each block of text. So say you have 5 languages and 200 text blocks, there'll be 1000 rows.

Then, have a few functions like so:

function GetText($ID, $Language = 1){ //1 = english, 2 = french etc - by their IDs
    $Query = MySQL_Query("SELECT Text FROM Translations WHERE Language = {$Language} AND ID = {$ID}");
    if(MySQL_Num_Rows($Query) < 1 && $Language == 1) return false;
    if(MySQL_Num_Rows($Query) < 1) return GetText($ID, 1); //returns english if there is no translation
    return MySQL_Result($Query, 0);
}
function LanguageExists($ID){
    $Query = MySQL_Query("SELECT ID FROM Languages WHERE ID = {$ID}");
    return (MySQL_Num_Rows($Query) == 1);
} 

And when you need an element of text, just put (for example on the menu):

<li><a href="/SomePage/"><?php echo GetText(1, $_SESSION['Language']); ?>

At the top of the file, you'll want to make sure they have a language, so for example:

<?php
Session_Start();
if(array_key_exists('Language', $_GET)){
    $_SESSION['Language'] = (int)$_GET['Language'];
}
if(!array_key_exists('Language', $_SESSION) || !LanguageExists($_SESSION['Language']){
    $_SESSION['Language'] = 1;
}

And to change the language, simply have a link putting '?Language=x', where x is the ID of the language.

A slight variation could use the actual names of the languages and/or the english text not the block ID, but that's upto you.

[khuramyz]

Why not use a framework like CodeIngiter that supports multilanguage support.

[sketchgal]

Thanks for the help so far with this.
Does anyone know if there is something out there that will automatically convert dynamic text to another language on a website if a users clicks on a button say? I have a datafeed which pulls text and images onto my website and I want the user to be able to view the text in a number of languages, is there a way this can be done without me have a databse with the description in several different languages?

[Cups]

You can use google translate I suppose, but from experience this can have an awful effect on the perception of your professionalism (and therefore credibility).

If you want to know what I mean take some ordinary text, translate it to one language using google translate (was babelfish) then translate it straight back into english, the results can be at once laughable, tragic or downright misleading and dangerous.

[oddz]

Yeah, seems like the best way to do this is translate the info and store it it inside a database. Then pull the translated section as Cups has suggested. That is probably the way I would also handle it assuming someone else translated everything.

[sketchgal]

Thanks for the responses on this. I'm working on this at present. I've got a few problems though. When I display the text from the db's on the front of the site some characters are showing as strange symbols. How do I get around this for the different languages?

[sketchgal]

I've used htmlentities() to fix the above problem with the characters not displaying correctly just incase anyone else gets this problem

[c55]

go get a google translate API

[Jake Arkinstall]

Have you even read the thread, c55?

[sketchgal]

I'm trying to use Arkinsall's code from further up this thread but I'm getting the following errors:

Fatal error: Cannot redeclare gettext()

Can anyone help please?

[darkdrgn2k]

Ive started going down the path of transalting my own site.

Its not easy ill tell you that much, especialy if you have dynamic content.

I have no idea how im gonna do tables yet at this point, righ now im aiming for "alternative" content.

No idea what the best way to SEO the site is w/o breaking the SSL url or relative paths

:-S

[Vali]

It's quite a painstaking tast to be honest.

Have a database table called languages, with just an ID and a name (English etc)

Have another table called 'Translations' or something like that, and have the columns ID (pk), SectionID, Language, and Text.

for every block of text (including menu text, titles, paragraphs, form labels, etc), there will be a different SectionID.

You can then have a row for every different language (using their IDs) with each block of text. So say you have 5 languages and 200 text blocks, there'll be 1000 rows.

Then, have a few functions like so:

function GetText($ID, $Language = 1){ //1 = english, 2 = french etc - by their IDs
    $Query = MySQL_Query("SELECT Text FROM Translations WHERE Language = {$Language} AND ID = {$ID}");
    if(MySQL_Num_Rows($Query) < 1 && $Language == 1) return false;
    if(MySQL_Num_Rows($Query) < 1) return GetText($ID, 1); //returns english if there is no translation
    return MySQL_Result($Query, 0);
}
function LanguageExists($ID){
    $Query = MySQL_Query("SELECT ID FROM Languages WHERE ID = {$ID}");
    return (MySQL_Num_Rows($Query) == 1);
} 

And when you need an element of text, just put (for example on the menu):

<li><a href="/SomePage/"><?php echo GetText(1, $_SESSION['Language']); ?>

At the top of the file, you'll want to make sure they have a language, so for example:

<?php
Session_Start();
if(array_key_exists('Language', $_GET)){
    $_SESSION['Language'] = (int)$_GET['Language'];
}
if(!array_key_exists('Language', $_SESSION) || !LanguageExists($_SESSION['Language']){
    $_SESSION['Language'] = 1;
}

And to change the language, simply have a link putting '?Language=x', where x is the ID of the language.

A slight variation could use the actual names of the languages and/or the english text not the block ID, but that's upto you.

That will never work on a site with some traffic...

Say you have a page with 200 text blocks on it, that means 200 selects + your normal DB selects per page load.

So, say you have 10 users hitting that page at a time, that's 2000+selects, could work... but what about 100 users? 1000 users?

Even if you cache those queries using APC or memcache it will be slow...

I suggest you have different language files, an array with the KEY being your label and VALUE the text to show.
Wrap that in a function that does your language name detection (first time IP/user selection, then session), and your done.

As your site grows, you can split that language file into more files, and load them in a smart way (since they tend to get big )

Hope that helps.

[sketchgal]

Can you give me an example of what you mean please vali because the site is going to end up being really large so I want to get this right from the start to save me having to go back and redo everything.
Thanks for the help

[Jake Arkinstall]

Well, that does make sense. It was a rather crude example and the intention would be to preload all of the language text in one go, but a file method would be even faster.

@Sketchgal - the use of a function there would mean that going back and redoing everything is avoided. Just change how the function works.

Ok, so what I'd suggest in terms of file storage is either constants or an XML file, for example:

<translations lang="en">
    <translation name="Greeting">Welcome to SomeSite.com</translation>
    <translation name="MenuHeader">Menu</translation>
</translations>
<translations lang="cy">
    <translation name="Greeting">Croeso i SomeSite.com</translation>
    <translation name="MenuHeader">Dewislen</translation>
</translations>

Or maybe a translation language per file, for speed. You'd then use an XML parsing library to grab the text.

A maybe more efficient method would be a file full of constants, such as:
languages/en.php:

define('TEXT_GREETING', 'Welcome to SomeSite.com');
define('TEXT_MENU_HEADER', 'Menu'); 

and have a different file for different languages.

Then, simply load the file (languages/{$lang}.php) and you have it loaded.

[sketchgal]

I'm using the .php language file solution suggested by arkinstall above. the only problem I'm having is that some of the foreign language characters are not returning correctly on the page. I've only noticed it with the French characters just now but I'd imagine that there will be the smae issues with the other languages. I thought htmlentities() would work but it return <br> tags as they are and doesn't see them as actual line breaks.
Is there a quick way for me to fix this?

[Jake Arkinstall]

That's to do with character encoding. HTMLEntities replaces characters with their HTML-friendly equivalents, but that includes the tags themselves (i.e. <br /> => &lt;br /&gt which is why you see the actual < and >

Character encodings really isn't a strong point with me at all. In fact, when I have some time I'll look more into it, but what you need to do now is find out how to fix this issue by fixing your encodings.

I think UTF-8 is a suitable encoding for you to use. How you can tell the PHP engine or browser to read it like that is variable, but I think:

header('Content-Type: Text/HTML;Charset=UTF-8'); 

should do the trick.

[sketchgal]

I'm using the following but its still not working.
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">

any other suggestions?

[Jake Arkinstall]

As I posted above.

The HTML meta tags aren't generally taken seriously. HTTP Headers take priority.

[sketchgal]

I've pasted exactly what you posted at the very top of my file and its not making any difference. Have I put it in the wrong please or something?

[Vali]

You need the character set set on the HTML file.


Here is my example:
lang/en.php

$lang['label'] = 'this is my label'; 

lang/fr.php

$lang['label'] = 'c\'est mon etiquette'; 

autoloader.php

$lang ='en';
if ((isset($_GET['lang'])) && (isset($validLanguages[$_GET['lang']]))) {
  $lang = $_GET['lang'];
  $_SESSION['lang'] = $lang;
}else if (isset($_SESSION['lang'])) {
  $lang = $_SESSION['lang'];
}

require_once('lang/' . $lang . '.php');

function l($key) {
  if(isset($lang[$key])) {
    return $lang[$key];
  }
  return '<!-- NOT IN LIST -->' . $key;
} 

index.php

<html>
...
<body>
<label for="a"><?php echo l('label'); ?></label>
<input type="text" id="a" name="foo" value="" />
</body>
</html>

Then, as you site grows, you can divide the "lang" files into multiple files.
The most common used KEYs in 'en.php', and less common in: 'en_level1.php', 'en_level2.php' etc.

And in your 'l' function, you load up the next level as each requested KEY is not found in your language files. (so by default you load level1, then level2, etc till you find your key)

There are better ways to split them out, and I'm sure you can come up with a better solution than one I posted here.

This solution will be way faster than storing your data in XML files or database. And also has the advantage of having just one code base, in English, that it's easy to maintain.

Ps: I ended up using plain English text for the Keys, or short hand versions of it, so when I look at the HTML, it doesn't look confusing at all.

Hope that helps.
(Code is not tested )

[crmalibu]

php has the gettext extension.

[sketchgal]

Thanks for the suggestions with this but I've still got the problem that some of the characters won't show as they are supposed to. The site is set to UTF-8 charset which I thought would work. Does anyone know how I can fix this?

[Cups]

Join the utf-8 club. Here's some interesting reading others have given me;

php charset encoding FAQ
php utf-8 cheatsheet
Charsets

If you are going to get serious about utf-8 then you have to make sure you are doing it from start to end.

Start in your code editor, end on the html page - and everything in and out of your scripts, xml, files and databases must match up. Assume nothing, then it will work for you - well it did for me.