Prior to these plugins the applications we’ve developed rely on the sender of the text message to input the mobile phone number, just as you do when you’re sending a text message from your mobile phone. You, as the sender, type in the recipients number.

This is generally foolproof because the sender has an inherent understanding of the number format required to send the text message.

With an eCommerce plugin we have another party to consider. The purchaser or “receiver” of the text message is the one who inputs their mobile number.

And they do so without any knowledge of the format the store owner needs to actually send the text message.

e.g. ask someone to fill in a form with their UK mobile number and you’ll get things like:

07741234567

07741 234567

07741 234 567

447741234567

447741 234567

+447741234567

44(0)7741234567

and so on, I think you get the picture.

Couple this with the international reach of most eCommerce applications and you’ve got people from all over the world inputting mobile numbers in all sorts of formats, and both the purchaser and their store owners expecting to receive text messages.

The answer is relatively simple, we need to convert any mobile number into a valid international format. That way we’ll have consistency in the data and our SMS API can route the messages to the appropriate country.

Here’s how we currently do it:

To do this you’ll need a table of international dialling codes. We use one hacked together from Wikipedia. Here’s the PHP code.

1. Strip out any leading zeros.
2. Remove any numbers within parentheses
3. Strip any spaces or non numeric characters.
4. Take the country of residence of the purchaser. If the purchaser didn’t input a country then use the store default, and look this up on the table of dialling codes.
5. If the mobile number starts with the dialling code then great, you’re good to go.
6. If not then add the dialling code to the start of the number, and then your good to go.

We are aware that this could be improved.

For instance, if the number starts with a ‘+’ then we could take it that the user has already supplied the country code.

We could also do this if the number starts with ’00′. That is for the UK though, other countries have other codes to dial an international number.

We will be looking at improving the process as we go on.

In theory you could concatenate 255 SMS to produce one long SMS, but in practice this is likely to crash a mobile phone, incur significant costs, and annoy the recipient.

Our statistics show that most long SMS contain less than 306 characters (the limit of 2 concatenated SMS), and very few exceed 459 (3 concatenated SMS). For this reason both our web application textburst and our SMS API allow you to send up to 3 SMS that can make up a single long SMS.

If you’re looking for a more in depth technical discussion then refer to our article on Concatenating SMS. It covers the technical requirements and use of the UDH (user data header) to form the long SMS.

For sending messages, our SMS API will automatically take care of the settings need to do this as long as the ‘Concat’ parameter is set when sending the message. The Concat parameter can take the value of 1, 2 or 3, and indicates how many SMS you are willing to use to attempt to send a message.

However, if you have messages which are sent from phones to your application (MO Messages), an understanding of how concatenation works can be useful if users are likely to send messages over 160 characters.

User Data Header (UDH)

The SMS that make up a concatenated SMS are related together by using the User Data Header (UDH) of an SMS. The UDH is a collection of bytes which can be put at the start of the SMS content. It can be used to control what happens to the rest of the content. For instance, it was used to send the older style of Nokia picture messages.

To indicate that the content contains a UDH, a flag on the SMS called the UDH Indicator (UDHI) must be turned on. This tells the phone that it must separate the UDH from the rest of the content.

Format of an SMS with a UDH

The phone separates the UDH by reading the first byte of the content. The number in this byte is the length of the rest of UDH and is called the User Data Header Length (UDHL). The phone then knows how many bytes make up the UDH and can separate it from the rest of the message.

As already stated, a UDH can control various things and so can contain various commands. These commands are called Information Elements (IE’s). These IE’s always take the following format: an Identity Element Identifier (IEI) followed by the Length of the IE Data (IEDL) followed by the IE Data (IED). A UDH can contain 1 or more of these IE’s.

UDH’s for Concatenated SMS

For concatenated messages the IE’s that we are interested in are the ones with Identifiers (IEI) of 00 or 08. These indicate that the rest of the message is a concatenated message.

Let’s take an example of using the 00 Information Element Identifier.

The 00 IEI always has an IEDL of 03. That is, there are 3 bytes in the IE Data section.

The first byte is a reference number. This reference number must be the same for all SMS that make up a particular concatenated message.

The second byte is the total number of SMS that go to make up the concatenated message.

The third byte is the part number that this particular SMS is.

So if we have a message that is split across two SMS the bytes in the SMS may be:

First SMS: 05 00 03 A6 02 01 .. bytes that make up the first part text ..
Second SMS: 05 00 03 A6 02 02 .. bytes that make up the second part text ..

The 08 identifier works the same way as the 00, but instead of a 1 byte reference number, it uses a 2 byte (16 bit) reference number. Therefore its IEDL is 04, there are 4 bytes in the IE Data section. These are as follows:

The first and second bytes make up the 16-bit reference number.

The third byte is the total number of SMS that go to make up the concatenated message.

The forth byte is the part number that this particular SMS is.

So another two SMS examples could be:

First SMS: 06 08 04 F4 2E 02 01 .. bytes that make up the first part text ..
Second SMS: 06 08 04 F4 2E 02 02 .. bytes that make up the second part text ..

The API

As mentioned at the start of this article, our API will take care of all of this for sending messages. All you do is set the ‘Concat’ field to maximum SMS that you are willing to use to send a message.

For receiving messages, our API will forward through the individual SMS that make up a concatenated message. The UDH is already removed from the content and is present in the UDH parameter as the hex encoded bytes. The rest of the content is placed into the Payload parameter.

For most single SMS, a UDH parameter will not be present and so the parameter will be blank. But for concatenated SMS, the receiving script can check the UDH for the 00 or 08 IEI’s to allow it to piece together the original message.

The UK mobile networks all use the GSM standard, and as such, a standard text message is limited to the GSM character set.

An SMS can contain up to 140 bytes. The GSM character set is encoded using 7-bits, rather than the usual 8-bits that make a byte. This means there can be 160 characters in an SMS.

This 7-bit limitation means only 128 standard characters can be encoded. The GSM standard gets round this by also having the Extended GSM character set. These are another 10 characters which are actually sent by sending two 7-bit characters, an escape (ESC) character followed by another character. This means that 160 ‘£’ symbols can fit in a single SMS, but only 80 ‘{‘ symbols.

If you need to send other characters than those in the GSM character set, then take a look at UCS-2 messaging which allows sending most unicode characters, but as each of these takes 2 bytes it means only 70 characters can be sent per SMS.

So our customers don’t need to bother converting characters into the GSM encoding or escaping the extended characters, our SMS API accepts messages in UTF-8 encoding.

Update: Since posting this we’ve released an article on how to do GSM character encoding in .NET

The tables below show a full list of the GSM characters, standard and extended. They also show the equivalent UTF-8 encoding need to send into our API.

Standard GSM Characters

Extended GSM Characters

In an SMS these are prefixed with the escape character (1B) and therefore take up 2 of the 160 characters of an SMS. They do not need escaping when sending into our API.

When a message delivery fails and the network returns extra information to us regarding the failure, we can now send this information back to you in a standardised way.

The error code parameter is already defined in section 3.5 of the our API. Please contact us if you want this enabling on your default delivery receipt script. Alternatively, if you set this information when you send a message, you just need to use the ERR_CODE parameter. For instance, if delivery receipts are currently set up to be a GET request using the following parameters:

client_id=#CLIENT_ID#&status=#DELIVERY_STATUS#&msg_id=#MSG_ID#

Then the error code can be also passed by changing this to:

client_id=#CLIENT_ID#&status=#DELIVERY_STATUS#&msg_id=#MSG_ID#&err_code=#ERR_CODE#

By default, the encoding is set to UTF-8, but it also supports Latin-1, UCS2 (Unicode), or GSM encodings.

Give us a phone if you want to know more.

We have been investigating the various build/packaging options for perl modules.

Up to now we have been using the standard method of ‘h2xs -AXn Module::Name‘. This generates a template module with a ExtUtils::MakeMaker build file.

We can then use cpan2rpm to create rpm files that we can put on our internal yum repositories.

This allows us to easily install/upgrade the same module across our various servers.

The problem has come when we wanted to have a slightly modified rpm e.g. have the rpm create a standard user, install a binary and/or init script, or create a config file.

The ExtUtils / cpan2rpm combination does not appear to have this functionality. We started looking at putting it in, but it was becoming increasingly difficult to do so.

So we started looking at Module::Build instead of ExtUtls. With this it is easier to make custom builds, and because of this seems to becoming increasingly popular as the Perl module builder.

For instance the Build.PL to enable a config file install into /etc and a logrotate script can be as follows:

use 5.008005;
use Module::Build;

my $build = Module::Build->new (
module_name => 'ModuleName',
license => 'perl',
dist_author => 'A.N. Other <a.n.other@example.com>',
dist_version_from => 'lib/ModuleName.pm',
etc_files => {
'etc/logrotate.d/modulename' => 'etc/logrotate.d/modulename',
'etc/modulename.conf' => 'etc/modulename.conf'
},
install_path => { 'etc' => '/etc' },
);
$build->add_build_element('etc');
$build->create_build_script;

We still need to use our modified cpan2rpm if we need any pre or post install scripts in the rpm. But so far this appears to be working.

Premium SMS codes have been set up in more than 20 countries around the world, including the US, UK, South Africa, Australia, Spain, Germany and many African nations. People will be able to text their own message to their country-specific short code and will then receive a thank you message with a unique pin number, allowing them to view the message securely online at a specially created birthday site. The site will also contain a raft of celebrity messages both written and filmed by Mandela supporters around the world. In the UK texts cost £1.50 plus the standard network rate.

All profits received from text charges will go to the Nelson Mandela Legacy Trust and other charitable organisations supported by Mandela. Tim Massey, 46664 international director, said: “A text-based service allows more people to feel part of the celebrations in a very simple way and we hope to raise significant revenues to support the ongoing work of the charities that bear Mr Mandela’s name and 46664.” We wish 46664 every success with this ground breaking initiative.