1

Trying to find out why emoji caused even short messages to being sent as multiple chunks I came across this question, explaining this happens because they are encoded UTF-16 rather than ascii.

My question, now is: is there a way to have them sent in UTF-8, instead? Most of the messages I send contain many latin letters and very few other characters, so they all should be much shorter if encoded in UTF-8: is it possible?

I'm running Cyanogenmod 11-M12 (Android 4.4.4) and, in case it matters, I use the SwiftKey keyboard.

In case this is not possible, is it a matter of the SMS protocol, or is it due to the OS, or even to the keyboard?

Thanks a lot!

|improve this question
  • It's a matter of the codepages the OS uses. The 8-bit page (0-255) is already filled with common letters, symbols and control characters. Non-latin languages also fall back to either UTF-16 or their custom codepage. The problem is that UTF-16 is usually used to overcome any conversion issues so that everyone gets the same message and not some garbage out of random characters in a row. – GiantTree Feb 19 '15 at 17:08
3

Sadly this is a limitation in the SMS protocol.

The "ascii" charset is not the normal we are used to in the computer world, instead it is a 7bit (or GSM 03.38 https://en.wikipedia.org/wiki/GSM_03.38) and very limited.

If one single character is used outside of this 7 bit charset, it is switched to UCS2 (See more information about UCS2 not exactly being UTF-16 here https://en.wikipedia.org/wiki/UTF-16).

With 7 bit, each character takes 7 bits (doh), and with UCS2 each character takes 16 bits.

While coding some SMPP (the SMS protocol) integrations, I've seen sporadic support for other charsets, but I do not think this is at all adopted in the wild.

|improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.