1

Trying to find out why emoji caused even short messages to being sent as multiple chunks I came across this question, explaining this happens because they are encoded UTF-16 rather than ascii.

My question, now is: is there a way to have them sent in UTF-8, instead? Most of the messages I send contain many latin letters and very few other characters, so they all should be much shorter if encoded in UTF-8: is it possible?

I'm running Cyanogenmod 11-M12 (Android 4.4.4) and, in case it matters, I use the SwiftKey keyboard.

In case this is not possible, is it a matter of the SMS protocol, or is it due to the OS, or even to the keyboard?

Thanks a lot!

1
  • It's a matter of the codepages the OS uses. The 8-bit page (0-255) is already filled with common letters, symbols and control characters. Non-latin languages also fall back to either UTF-16 or their custom codepage. The problem is that UTF-16 is usually used to overcome any conversion issues so that everyone gets the same message and not some garbage out of random characters in a row.
    – GiantTree
    Feb 19, 2015 at 17:08

1 Answer 1

3

Sadly this is a limitation in the SMS protocol.

The "ascii" charset is not the normal we are used to in the computer world, instead it is a 7bit (or GSM 03.38 https://en.wikipedia.org/wiki/GSM_03.38) and very limited.

If one single character is used outside of this 7 bit charset, it is switched to UCS2 (See more information about UCS2 not exactly being UTF-16 here https://en.wikipedia.org/wiki/UTF-16).

With 7 bit, each character takes 7 bits (doh), and with UCS2 each character takes 16 bits.

While coding some SMPP (the SMS protocol) integrations, I've seen sporadic support for other charsets, but I do not think this is at all adopted in the wild.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .