The fact that MS-word and some other sources use CP-1252, and that it is so close to Latin1 ('ISO-8859-1') causes a lot of confusion. What confused me the most was finding that mySQL uses CP-1252 by default.
You may run into trouble if you find yourself tempted to do something like this:
<?php
$trans[chr(149)] = '•'; // Bullet
$trans[chr(150)] = '–'; // En Dash
$trans[chr(151)] = '—'; // Em Dash
$trans[chr(152)] = '˜'; // Small Tilde
$trans[chr(153)] = '™'; // Trade Mark Sign
?>
Don't do it. DON'T DO IT!
You can use:
<?php
$translationTable = get_html_translation_table(HTML_ENTITIES, ENT_NOQUOTES, 'WINDOWS-1252');
?>
or just convert directly:
<?php
$output = htmlentities($input, ENT_NOQUOTES, 'WINDOWS-1252');
?>
But your web page is probably encoded UTF-8, and you probably don't really want CP-1252 text flying around, so fix the character encoding first:
<?php
$output = mb_convert_encoding($input, 'UTF-8', 'WINDOWS-1252');
$ouput = htmlentities($output);
?>
get_html_translation_table
(PHP 4, PHP 5)
get_html_translation_table — htmlspecialchars() および htmlentities() で使用される変換テーブルを返す
説明
$table = HTML_SPECIALCHARS
[, int $flags = ENT_COMPAT | ENT_HTML401
[, string $encoding = 'UTF-8'
]]] )get_html_translation_table() は、 htmlspecialchars() および htmlentities() において内部的に使用される変換テーブルを返します。
注意:
特殊文字はいくつかの方法でエンコードすることができます。 例えば、" は ", " もしくは " としてエンコードすることができます。 get_html_translation_table() の返す値は、 htmlspecialchars() や htmlentities() で使っている形式だけです。
パラメータ
-
table -
どちらのテーブルを返すか。
HTML_ENTITIESあるいはHTML_SPECIALCHARS。 -
flags -
以下のフラグのビットマスクによる組み合わせで、 どのクォートをテーブルに含めるのか、そしてどのドキュメント形式用のテーブルにするのかを指定します。 デフォルトは ENT_COMPAT | ENT_HTML401 です。
使用可能な flags定数定数名 説明 ENT_COMPATテーブルにダブルクォート用のエンティティを含めますが、シングルクォート用は含めません。 ENT_QUOTESテーブルにダブルクォート用、シングルクォート用の両方のエンティティを含めます。 ENT_NOQUOTESテーブルにダブルクォート用、シングルクォート用のどちらのエンティティも含めません。 ENT_HTML401HTML 4.01 用のテーブル。 ENT_XML1XML 1 用のテーブル。 ENT_XHTMLXHTML 用のテーブル。 ENT_HTML5HTML 5 用のテーブル。 -
encoding -
使用するエンコーディング。省略した場合のデフォルト値は、 PHP 5.4.0 より前のバージョンでは ISO-8859-1、そして PHP 5.4.0 以降では UTF-8 となります。
以下の文字セットをサポートします。
サポートする文字セット 文字セット エイリアス 説明 ISO-8859-1 ISO8859-1 西欧、Latin-1 ISO-8859-5 ISO8859-5 ほとんど使われないキリル文字セット (Latin/Cyrillic)。 ISO-8859-15 ISO8859-15 西欧、Latin-9 。Latin-1(ISO-8859-1) に欠けている ユーロ記号やフランス・フィンランドの文字を追加したもの。 UTF-8 ASCII 互換のマルチバイト 8 ビット Unicode 。 cp866 ibm866, 866 DOS 固有のキリル文字セット。 cp1251 Windows-1251, win-1251, 1251 Windows 固有のキリル文字セット。 cp1252 Windows-1252, 1252 西欧のための Windows 固有の文字セット。 KOI8-R koi8-ru, koi8r ロシア語。 BIG5 950 繁体字中国語。主に台湾で使用されます。 GB2312 936 簡体字中国語。国の標準文字セットです。 BIG5-HKSCS Big5 に香港の拡張を含めたもの。繁体字中国語。 Shift_JIS SJIS, SJIS-win, cp932, 932 日本語。 EUC-JP EUCJP, eucJP-win 日本語。 MacRoman Mac OS で使われる文字セット。 '' 空文字列を指定すると、 スクリプトのエンコーディング (Zend multibyte)、 default_charset、 そして現在のロケール ( nl_langinfo() および setlocale() を参照ください) の順でエンコーディングを検出します。 この方法はおすすめしません。 注意: これら以外の文字セットは理解できません。 かわりにデフォルトのエンコーディングを使用し、警告を発生させます。
返り値
変換テーブルを配列で返します。元の文字がキー、そしてエンティティが値となります。
変更履歴
| バージョン | 説明 |
|---|---|
| 5.4.0 |
encoding パラメータのデフォルトが UTF-8 に変わりました。
|
| 5.4.0 |
定数 ENT_HTML401、ENT_XML1、
ENT_XHTML および ENT_HTML5 が追加されました。
|
| 5.3.4 |
encoding パラメータが追加されました。
|
例
例1 変換テーブルの例
<?php
var_dump(get_html_translation_table(HTML_ENTITIES, ENT_QUOTES | ENT_HTML5));
?>
上の例の出力は、 たとえば以下のようになります。
array(1510) {
[" "]=>
string(5) "	"
["
"]=>
string(9) "
"
["!"]=>
string(6) "!"
["""]=>
string(6) """
["#"]=>
string(5) "#"
["$"]=>
string(8) "$"
["%"]=>
string(8) "%"
["&"]=>
string(5) "&"
["'"]=>
string(6) "'"
// ...
}
参考
- htmlspecialchars() - 特殊文字を HTML エンティティに変換する
- htmlentities() - 適用可能な文字を全て HTML エンティティに変換する
- html_entity_decode() - HTML エンティティを適切な文字に変換する
Be careful using get_html_translation_table() in a loop, as it's very slow.
I wrote a quick little function for converting something like '·' into '·':
$to_convert = '·';
$table = get_html_translation_table(HTML_ENTITIES);
$equiv = '&#'.ord(array_search($to_convert,$table)).';';
If you have troubles (like me) getting data from ISO-8859-1 encoded forms where user copy and paste from word, this routine could be useful.
It adds to the standard get_html_translation_table the codes of the characters usually M$ Word replacs into typed text.
Otherwise those characters would never be displayed correctly in html output.
function get_html_translation_table_CP1252() {
$trans = get_html_translation_table(HTML_ENTITIES);
$trans[chr(130)] = '‚'; // Single Low-9 Quotation Mark
$trans[chr(131)] = 'ƒ'; // Latin Small Letter F With Hook
$trans[chr(132)] = '„'; // Double Low-9 Quotation Mark
$trans[chr(133)] = '…'; // Horizontal Ellipsis
$trans[chr(134)] = '†'; // Dagger
$trans[chr(135)] = '‡'; // Double Dagger
$trans[chr(136)] = 'ˆ'; // Modifier Letter Circumflex Accent
$trans[chr(137)] = '‰'; // Per Mille Sign
$trans[chr(138)] = 'Š'; // Latin Capital Letter S With Caron
$trans[chr(139)] = '‹'; // Single Left-Pointing Angle Quotation Mark
$trans[chr(140)] = 'Œ '; // Latin Capital Ligature OE
$trans[chr(145)] = '‘'; // Left Single Quotation Mark
$trans[chr(146)] = '’'; // Right Single Quotation Mark
$trans[chr(147)] = '“'; // Left Double Quotation Mark
$trans[chr(148)] = '”'; // Right Double Quotation Mark
$trans[chr(149)] = '•'; // Bullet
$trans[chr(150)] = '–'; // En Dash
$trans[chr(151)] = '—'; // Em Dash
$trans[chr(152)] = '˜'; // Small Tilde
$trans[chr(153)] = '™'; // Trade Mark Sign
$trans[chr(154)] = 'š'; // Latin Small Letter S With Caron
$trans[chr(155)] = '›'; // Single Right-Pointing Angle Quotation Mark
$trans[chr(156)] = 'œ'; // Latin Small Ligature OE
$trans[chr(159)] = 'Ÿ'; // Latin Capital Letter Y With Diaeresis
ksort($trans);
return $trans;
}
htmlentities includes htmlspecialchars, so here's how to convert an UTF-8 string :
htmlentities($string, ENT_QUOTES, 'UTF-8');
There have been issues when hispanic websites or other websites dont use the corrent collision in mysql.
Some problems result that the accents (éä ... ) result in weird characters when a backup is done and restored later on. Or when database is changed to another one.
To fix this try something like this
function accents($text){
foreach(get_html_translation_table(HTML_ENTITIES) as $a=>$b){
$text = str_replace($a,$b,$text);
}
return $text;
}
and use as accents("Hello ....... WITH ACCENTS") and it will return the escaped string.
Alans version didn't seem to work right. If you're having the same problem consider using this slightly modified version instead:
function unhtmlentities ($string) {
$trans_tbl = get_html_translation_table (HTML_ENTITIES);
$trans_tbl = array_flip ($trans_tbl);
$ret = strtr ($string, $trans_tbl);
return preg_replace('/&#(\d+);/me',
"chr('\\1')",$ret);
}
get_html_translation_table
It works only with the first 256 Codepositions.
For Higher Positions, for Example ф
(a kyrillic Letter) it shows the same.
A lot of quite common characters (or at least not rare, like oelig, euro or minus) are missing from the table unfortunately.
Here are some, if you want to make your translation table more complete and your xml data less error-prone. Not sure why some characters have 2 codes, just use one. Here goes: '''=>''', '−'=>'-', 'ˆ'=>'^', '˜'=>'~', 'Š'=>'Š', '‹'=>'‹', 'Œ'=>'Œ', '‘'=>'‘', '’'=>'’', '“'=>'“', '”'=>'”', '•'=>'•', '–'=>'–', '—'=>'—', '˜'=>'˜', '™'=>'™', 'š'=>'š', '›'=>'›', 'œ'=>'œ', 'Ÿ'=>'Ÿ', 'ÿ'=>'ÿ', 'Œ'=>'Œ', 'œ'=>'œ', 'Š'=>'Š', 'š'=>'š', 'Ÿ'=>'Ÿ', 'ƒ'=>'ƒ', 'ˆ'=>'ˆ', '˜'=>'˜', 'Α'=>'Α', 'Β'=>'Β', 'Γ'=>'Γ', 'Δ'=>'Δ', 'Ε'=>'Ε', 'Ζ'=>'Ζ', 'Η'=>'Η', 'Θ'=>'Θ', 'Ι'=>'Ι', 'Κ'=>'Κ', 'Λ'=>'Λ', 'Μ'=>'Μ', 'Ν'=>'Ν', 'Ξ'=>'Ξ', 'Ο'=>'Ο', 'Π'=>'Π', 'Ρ'=>'Ρ', 'Σ'=>'Σ', 'Τ'=>'Τ', 'Υ'=>'Υ', 'Φ'=>'Φ', 'Χ'=>'Χ', 'Ψ'=>'Ψ', 'Ω'=>'Ω', 'α'=>'α', 'β'=>'β', 'γ'=>'γ', 'δ'=>'δ', 'ε'=>'ε', 'ζ'=>'ζ', 'η'=>'η', 'θ'=>'θ', 'ι'=>'ι', 'κ'=>'κ', 'λ'=>'λ', 'μ'=>'μ', 'ν'=>'ν', 'ξ'=>'ξ', 'ο'=>'ο', 'π'=>'π', 'ρ'=>'ρ', 'ς'=>'ς', 'σ'=>'σ', 'τ'=>'τ', 'υ'=>'υ', 'φ'=>'φ', 'χ'=>'χ', 'ψ'=>'ψ', 'ω'=>'ω', 'ϑ'=>'ϑ', 'ϒ'=>'ϒ', 'ϖ'=>'ϖ', ' '=>' ', ' '=>' ', ' '=>' ', '‌'=>'‌', '‍'=>'‍', '‎'=>'‎', '‏'=>'‏', '–'=>'–', '—'=>'—', '‘'=>'‘', '’'=>'’', '‚'=>'‚', '“'=>'“', '”'=>'”', '„'=>'„', '†'=>'†', '‡'=>'‡', '•'=>'•', '…'=>'…', '‰'=>'‰', '′'=>'′', '″'=>'″', '‹'=>'‹', '›'=>'›', '‾'=>'‾', '⁄'=>'⁄', '€'=>'€'
Not sure what's going on here but I've run into a problem that others might face as well...
<?php
$translations = array_flip(get_html_translation_table(HTML_ENTITIES,ENT_QUOTES));
?>
returns the single quote ' as being equal to ' while
<?php
$translatedString = htmlentities($string,ENT_QUOTES);
?>
returns it as being equal to '
I've had to do a specific string replacement for the time being... Not sure if it's an issue with the function or the array manipulation.
-Pat
In case you want a 'htmlentities' function which prevents 'double' encoding of the ampersands of already present entities (> => &gt;), use this:
<?php
function htmlentities2($myHTML) {
$translation_table=get_html_translation_table (HTML_ENTITIES,ENT_QUOTES);
$translation_table[chr(38)] = '&';
return preg_replace("/&(?![A-Za-z]{0,4}\w{2,3};|#[0-9]{2,3};)/","&" , strtr($myHTML, $translation_table));
}
?>
This function will convert get_html_translation_table from a ISO-8859-1 string to UTF-8 string.
<?php
function translation_table_to_utf8($arTranslationtable)
{
//loop through the array and convert everything both keys and values
foreach($arTranslationtable as $charkey => $char)
{
$charkey = utf8_encode($charkey);
$arUTFchars[$charkey]= utf8_encode($char);
}
return $arUTFchars;
}
//get the translation table
$arSpecialchar = get_html_translation_table(HTML_ENTITIES);
//call the function to convert to utf-8
$arUTFchars = translation_table_to_utf8($arSpecialchar);
print_r($arUTFchars);
?>
If you want to display special HTML entities in a web browser, you can use the following code:
<?
$entities = get_html_translation_table(HTML_ENTITIES);
foreach ($entities as $entity) {
$new_entities[$entity] = htmlspecialchars($entity);
}
echo "<pre>";
print_r($new_entities);
echo "</pre>";
?>
If you don't, the key name of each element will appear to be the same as the element content itself, making it look mighty stupid. ;)
If you want to decode all those { symbols as well....
function unhtmlentities ($string) {
$trans_tbl = get_html_translation_table (HTML_ENTITIES);
$trans_tbl = array_flip ($trans_tbl);
$ret = strtr ($string, $trans_tbl);
return preg_replace('/\&\#([0-9]+)\;/me',
"chr('\\1')",$ret);
}
"rafael at phpit dot com dot br" your solution only works for the ISO-8859-1 encoding, I mean, it works but only for that encoding and that's because get_html_translation_table won't let you specify the charset... it uses the default one, that is ISO-8859-1
The solution from "olito24 at gmx dot de" does work for UTF-8, I just modified it a bit specifying the UTF-8 charset, also the $str parameter wasn't being used at all, I just renamed it to $string
Note:
Change ENT_NOQUOTES to ENT_QUOTES to convert both double and single quotes
These are the functions to encode html but tags using UTF-8 and ISO-8859-1
<?php
class Html
{
/*by olito24 at gmx dot de*/
function htmlButTags($string) {
$pattern = '<([a-zA-Z0-9\. "\'_\/-=;\(\)?&#%]+)>';
preg_match_all ('/' . $pattern . '/', $string, $tagMatches, PREG_SET_ORDER);
$textMatches = preg_split ('/' . $pattern . '/', $string);
foreach ($textMatches as $key => $value) {
$textMatches [$key] = htmlentities ($value, ENT_NOQUOTES, 'UTF-8');
}
for ($i = 0; $i < count ($textMatches); $i ++) {
$textMatches [$i] = $textMatches [$i] . $tagMatches [$i] [0];
}
return implode ($textMatches);
}
/*by "rafael at phpit dot com dot br" */
function htmlButTags_iso($str){
// Take all the html entities
$caracteres = get_html_translation_table(HTML_ENTITIES,ENT_NOQUOTES);
// Find out the "tags" entities
$remover = get_html_translation_table(HTML_SPECIALCHARS,ENT_NOQUOTES);
// Spit out the tags entities from the original table
$caracteres = array_diff($caracteres, $remover);
// Translate the string....
$str = strtr($str, $caracteres);
// And that's it!
return $str;
}
}
?>
Searching for a fast replacement of the MS WORD special characters which are not covered by get_html_translation_table() , I think the following function might help someone
<?php
function clean_up($str){
$str = stripslashes($str);
$str = strtr($str, get_html_translation_table(HTML_ENTITIES));
$str = str_replace( array("\x82", "\x84", "\x85", "\x91", "\x92", "\x93", "\x94", "\x95", "\x96", "\x97"), array("‚", "„", "…", "‘", "’", "“", "”", "•", "–", "—"),$str);
return $str;
}
?>
It replaces all types of quotes (single and double), horizontal ellipsis (...), bullet, en dash and em dash.
and a few more :
'ℑ'=>'ℑ', '℘'=>'℘', 'ℜ'=>'ℜ', '™'=>'™', 'ℵ'=>'ℵ', '←'=>'←', '↑'=>'↑', '→'=>'→', '↓'=>'↓', '↔'=>'↔', '↵'=>'↵', '⇐'=>'⇐', '⇑'=>'⇑', '⇒'=>'⇒', '⇓'=>'⇓', '⇔'=>'⇔', '∀'=>'∀', '∂'=>'∂', '∃'=>'∃', '∅'=>'∅', '∇'=>'∇', '∈'=>'∈', '∉'=>'∉', '∋'=>'∋', '∏'=>'∏', '∑'=>'∑', '−'=>'−', '∗'=>'∗', '√'=>'√', '∝'=>'∝', '∞'=>'∞', '∠'=>'∠', '∧'=>'∧', '∨'=>'∨', '∩'=>'∩', '∪'=>'∪', '∫'=>'∫', '∴'=>'∴', '∼'=>'∼', '≅'=>'≅', '≈'=>'≈', '≠'=>'≠', '≡'=>'≡', '≤'=>'≤', '≥'=>'≥', '⊂'=>'⊂', '⊃'=>'⊃', '⊄'=>'⊄', '⊆'=>'⊆', '⊇'=>'⊇', '⊕'=>'⊕', '⊗'=>'⊗', '⊥'=>'⊥', '⋅'=>'⋅', '⌈'=>'⌈', '⌉'=>'⌉', '⌊'=>'⌊', '⌋'=>'⌋', '⟨'=>'〈', '⟩'=>'〉', '◊'=>'◊', '♠'=>'♠', '♣'=>'♣', '♥'=>'♥', '♦'=>'♦'
Another way of converting HTML entities into numeric entities to please XML parsers is using two arrays as conversion tables in a preg_replace function. The conversion table mechanism is based on Ryan's examples above.
<?php
function xmlEntities($s){
//build first an assoc. array with the entities we want to match
$table1 = get_html_translation_table(HTML_ENTITIES, ENT_QUOTES);
//now build another assoc. array with the entities we want to replace (numeric entities)
foreach ($table1 as $k=>$v){
$table1[$k] = "/$v/";
$c = htmlentities($k,ENT_QUOTES,"UTF-8");
$table2[$c] = "&#".ord($k).";";
}
//now perform a replacement using preg_replace
//each matched value in array 1 will be replaced with the corresponding value in array 2
$s = preg_replace($table1,$table2,$s);
return $s;
}
?>
The existance of html entities such as " inside an xml node causes most xml parsers to throw an error. The following function cleans an input string by converting html entities to valid unicode entities.
<?php
function htmlentities2unicodeentities ($input) {
$htmlEntities = array_values (get_html_translation_table (HTML_ENTITIES, ENT_QUOTES));
$entitiesDecoded = array_keys (get_html_translation_table (HTML_ENTITIES, ENT_QUOTES));
$num = count ($entitiesDecoded);
for ($u = 0; $u < $num; $u++) {
$utf8Entities[$u] = '&#'.ord($entitiesDecoded[$u]).';';
}
return str_replace ($htmlEntities, $utf8Entities, $input);
}
?>
So, an input of
Copyrights © make "me" grin ®
outputs
Copyrights © make "me" grin ®
In XML, you can't assume that the doctype will include the same character entity definitions as HTML. XML authors may require character references instead. The following two functions use get_html_translation_table() to encode data in numeric references. The second, optional argument can be used to substitute a different translation table.
function xmlcharacters($string, $trans='') {
$trans=(is_array($trans))? $trans:get_html_translation_table(HTML_ENTITIES, ENT_QUOTES);
foreach ($trans as $k=>$v)
$trans[$k]= "&#".ord($k).";";
return strtr($string, $trans);
}
function xml_character_decode($string, $trans='') {
$trans=(is_array($trans))? $trans:get_html_translation_table(HTML_ENTITIES, ENT_QUOTES);
foreach ($trans as $k=>$v)
$trans[$k]= "&#".ord($k).";";
$trans=array_flip($trans);
return strtr($string, $trans);
}
I found this useful in converting latin characters
<?php
function convertLatin1ToHtml($str) {
$allEntities = get_html_translation_table(HTML_ENTITIES, ENT_NOQUOTES);
$specialEntities = get_html_translation_table(HTML_SPECIALCHARS, ENT_NOQUOTES);
$noTags = array_diff($allEntities, $specialEntities);
$str = strtr($str, $noTags);
return $str;
}
?>
Here is a simple way to convert named character entities to numeric character entities:
<?php
function numeric_entities($string){
$mapping = array();
foreach (get_html_translation_table(HTML_ENTITIES, ENT_QUOTES) as $char => $entity){
$mapping[$entity] = '&#' . ord($char) . ';';
}
return str_replace(array_keys($mapping), $mapping, $string);
}
?>
Quite disappointingly, get_html_translation_table() only gives the characters for ISO-8859-1, making it quite useless for UTF-8 or anything else like that (as a previous commenter noticed).
to display the mapping on a webpage no matter what the server encoding is, this can be used
echo "<pre>\n";
echo htmlentities(print_r((get_html_translation_table(HTML_SPECIALCHARS)), true));
echo htmlentities(print_r((get_html_translation_table(HTML_ENTITIES)), true));
since get_html_translation_table() actually gives the special chars in iso-8859-1 (Latin-1) encoding, so to see the tables correctly using
print_r(get_html_translation_table(HTML_ENTITIES));
your server needs to give a HTTP header as iso-8859-1, unless you use header() or manually set the browser's encoding setting to iso-8859-1. And you need to view the source of the page to see the mapping. (except English version of IE 7 outputs the page source as iso-8859-1 anyway).
without heavy scientific analysis, this seems to work as a quick fix to making text originating from a Microsoft Word document display as HTML:
<?php
function DoHTMLEntities ($string)
{
$trans_tbl = get_html_translation_table (HTML_ENTITIES);
// MS Word strangeness..
// smart single/ double quotes:
$trans_tbl[chr(145)] = '\'';
$trans_tbl[chr(146)] = '\'';
$trans_tbl[chr(147)] = '"';
$trans_tbl[chr(148)] = '"';
// Acute 'e'
$trans_tbl[chr(142)] = 'é';
return strtr ($string, $trans_tbl);
}
?>
