看到有前辈写了一个UTF-8与UNICODE相互转换的代码,顺便提一下,希望可以给大家提供一点帮助.
下面是一些编码格式的bit长
Examples of fixed-width encoding forms:
Type |
Each character encoded as |
Notes |
7-bit |
a single 7-bit quantity |
example: ISO 646 |
8-bit G0/G1 |
a single 8-bit quantity |
with constraints on use of C0 and C1 spaces |
8-bit |
a single 8-bit quantity |
with no constraints on use of C1 space |
8-bit EBCDIC |
a single 8-bit quantity |
with the EBCDIC conventions rather than ASCII conventions |
16-bit (UCS-2) |
a single 16-bit quantity |
within a code space of 0..FFFF |
32-bit (UCS-4) |
a single 32-bit quantity |
within a code space 0..7FFFFFFF |
32-bit (UTF-32) |
a single 32-bit quantity |
within a code space of 0..10FFFF |
16-bit DBCS process code |
a single 16-bit quantity |
example: UNIX widechar implementations of Asian CCS's |
32-bit DBCS process code |
a single 32-bit quantity |
example: UNIX widechar implementations of Asian CCS's |
DBCS Host |
two 8-bit quantities |
following IBM host conventions |
Examples of variable-width encoding forms:
Name |
Characters are encoded as |
Notes |
UTF-8 |
a mix of one to four 8-bit code units in Unicode and one to six code units in 10646 |
used only with Unicode/10646 |
UTF-16 |
a mix of one to two 16 bit code units |
used only with Unicode/10646 |
Boost中提供了一个UTF-8 Codecvt Facet,可以在utf8和UCS-4(Unicode-32)之间转换.
使用方式如下
//...
// My encoding type
typedef wchar_t ucs4_t;
std::locale old_locale;
std::locale utf8_locale(old_locale,new utf8_codecvt_facet<ucs4_t>);
// Set a New global locale
std::locale::global(utf8_locale);
// UCS-4 转换为 UTF-8
{
std::wofstream ofs("data.ucd");
ofs.imbue(utf8_locale);
std::copy(ucs4_data.begin(),ucs4_data.end(),
std::ostream_iterator<ucs4_t,ucs4_t>(ofs));
}
// 读入 UTF-8 ,转换为 UCS-4
std::vector<ucs4_t> from_file;
{
std::wifstream ifs("data.ucd");
ifs.imbue(utf8_locale);
ucs4_t item = 0;
while (ifs >> item) from_file.push_back(item);
}
//...
UTF-8 Codecvt Facet详见
http://www.boost.org/libs/serialization/doc/codecvt.html