We first noticed that textburst pages would occasionally freeze a few days before before Christmas and released a temporary fix that solved it for 99% of users.

Unfortunately as with most temporary fixes it wasn’t 100% effective so we’ve been investigating the root cause since getting back from the holidays.

After many, many hours digging through tens of thousands of lines of log files (it’s amazing how much logging a debug mode can create) and tweaking code here and there we at last found the cause yesterday afternoon. And two lines of code later it’s fixed.

For the technically inclined here’s a little more information:

Textburst runs on four web servers, these are set up as two load balanced pairs, one in Derby and one in Manchester. To ensure users stay logged in when switching between servers we store our session data in a MySQL database.

Our MySQL based session store is a slightly tweaked version of the MySQL session state provider which is in turn based on the Microsoft sample session state provider.

Unfortunately it turns out both of these have a bug when handling concurrent connections.

How things are supposed to work

When a page is requested it should should check to see if the session is currently locked by another page, wait until it’s free and then obtain the session data, along with a lock if the page wants write access and then increment an internal Lock ID by one which is used to make sure the same request is locking and unlocking the session.

What was actually happening

Requests that wanted read/write access to the session worked exactly as expected, however read-only sessions missed the all important step of waiting until the session was free and simply incremented the Lock ID. This meant that if a read-only request was processed in between the lock and unlock phase of a read/write request it would never be unlocked causing the next read/write request to freeze.

Luckily the ASP.NET session code is designed for this eventuality and after 110 seconds the stuck lock is forcibly removed, unfortunately our users were stuck waiting for this to happen while everything appeared to be frozen.

Our fix

We’ve updated the session provider to always wait until the session is free. To help out others with similar problems we’ve added this as a comment on the Microsoft sample and will be filing a bug report with MySQL shortly.

As we were re-writing it we took the chance to make a few other changes too, don’t worry though, if you’re already using the API we haven’t changed anything, all the old URLs and parameters will still work.

Url Changes

From now on all API addresses are on api.mediaburst.co.uk rather than sms.message-platform.com, we feel it makes more sense as you can instantly tell it’s the mediaburst API.

All examples now use SSL for added security, you’re still able to turn it off though.
We have no plans to disable the old sms.message-platform.com.

So the URL for sending through our simple HTTP API becomes https://api.mediaburst.co.uk/http/send.aspx

Delivery Receipts

We’ve renamed a few of the merge parameters for consistency, when you send a text message you use the To parameter for the mobile number, and now the delivery receipt will too. Don’t worry though all the old names still work.

Simple HTTP API

We’ve clarified which features are available through the HTTP API. If you want advanced features they’re all available through the XML interface.

Our new simple parameter list now consists of:

• Username
• Password
• To
• Content
• From (Optional – Message originator)
• Long (Optional – Allow up to 459 characters)

Email to SMS address changes

Again to tidy things up, we’ve changed all email to SMS email addresses to use mediaburst.co.uk rather than message-platform.com.

The new addresses are:

Upload contacts from Excel spreadsheets

We’ve listened to your feedback about uploading CSV files, now you can upload Excel files directly into textburst without having to convert them.

As with most things there are a few caveats:

• You must be using Excel 2007 or Excel 2010, older versions save in a different format which we can’t process.
• All your contacts must be on the first worksheet.
• Numbers need to start with 44 or 0, if you can’t see either of these at the start in Excel it won’t work in textburst.

Delete sent messages

We want to make sure you’re always in control of your data, so on this note, we’ve added delete buttons to the sent message page.  Simply click this button and it’s gone forever.
Just to make this clear though: The delete button deletes the message, there isn’t a way to get it back.

Works in Internet Explorer 9

Microsoft are well on their way to releasing the new version of Internet Explorer.  Surprisingly, from a web development point of view, it seems like a reasonable browser.   We’ve made a few tweaks to our styling and scripting to remove work arounds previous versions needed, making sure textburst always looks its best.

This first demo was a simple affair, we manually copied the page HTML, output by textburst, into a few new files and then ran them through a shell script which called sed a dozen or so times to replace anything that didn’t work without the full application behind it.

Since putting the new design live we felt we needed to show off all the features to prospective customers. What really surprised us was just how much work is involved to create a fully featured demo.

Our requirements seem simple at a glance:

• Look and feel like textburst
• Include all major textburst features (except payments)
• Be easy to update
• Don’t really send text messages

For various internal reasons the demo needed to run on the same server as this website, this left us needing to run a C#, ASP.NET web application on a server running PHP and Linux. To make this work we came up with the following steps:

1. Log in to textburst on our development platform
2. Grab a copy of all the relevant pages
3. Run some standard find and replaces to remove ASP.NET specific content
4. Add PHP include files at the top of each page
5. Add in page specific PHP includes
6. Remove any development data
7. Copy this altered page and any includes in to a directory ready for release
8. Create some demo contacts and sends using PHP Sessions for display to users

Just over 800 lines of perl later we have a wonderful jumble of regular expressions and a dozen or so php files that make textburst into the simple demo you can see today.

By automating this step we can just re-run the perl script each time we do a release keeping the demo perfectly up to date and preventing any surprises once you’ve signed up.

We do still have to create page specific include files for some of the pages to show how the product actually works. For example, we add an include to the send message page which displays either a sent successfully or failure message depending upon whether all the form fields were completed. It also adds the sent message to your session so it shows on the sent messages page.

One of our SMS suppliers recently changed the encoding on MO messages from Latin1 to GSM as it supports a wider range of characters. After quite a bit of searching the web we couldn’t find any examples of how to work with GSM encoded text in C# so we wrote our own.

To keep things simple we’ve inherited from System.Text.Encoding creating a Mediaburst.Text.GSMEncoding class.  Characters are encoded using a simple one to one mapping of GSM bytes to Unicode character.

We’ve just released it on github under the ISC license, one of the few open-source licenses written in simple clear English, so feel free to grab a copy and use it in your projects.

How to use

We use the following to convert GSM encoded text to UTF-8 on a web server.

The utf8Enc.GetBytes(body) that looks a little out of place is needed as the server assumes all GET and POST parameters are UTF-8 encoded.

string body = "Some GSM encoded text";
Encoding gsmEnc = new Mediaburst.Text.GSMEncoding();
Encoding utf8Enc = new System.Text.UTF8Encoding();
byte[] gsmBytes = utf8Enc.GetBytes(body);
byte[] utf8Bytes = Encoding.Convert(gsmEnc, utf8Enc, gsmBytes);
body = utf8Enc.GetString(utf8Bytes);

Caveats

The encoding functions don’t currently support the EncoderFallback and DecoderFallback error handlers for invalid characters or follow the Microsoft convention of replacing an invalid character with a question marks. Invalid characters will be ignored and missing from the converted string.

Get the code

Download from github

What’s in this release?

This milestone release doesn’t contain any major new features for you, our users, it’s all about making our lives easier at mediaburst and allowing for future changes.

We’ve changed to .NET 4

Microsoft released version 4 of their .NET framework a couple of months ago, along with it came Visual Studio 2010. Although there are no groundbreaking changes in this new Visual Studio it solves various day to day niggles.

Code rewriting

We’ve re-written some internal code that passes messages around between systems. It’s just housekeeping really, making use of the improvements between versions 2 and 4 of the .NET framework. The biggest change is the switch from ArrayLists of messages to strongly typed Lists using the new support of covariance and contravariance in generic types. This change allows the compiler to check what we’re adding to the list at compile time, rather than run time, hopefully avoiding any mistakes.

No more image messages

We’ve removed support for Nokia and EMS image messages. They’re small black and white bitmaps which look awful on modern phones, a much better choice, with wider handset support, is MMS. Processing these took large chunks of bespoke code, without them our sending scripts are so much easier to maintain as there’s only one set of logic.

Updated logging

And finally we’ve tweaked our sent message logging to make a number of internal reports run off a single database table. We currently have to link two databases on different servers if we want the same level of detail.

Big sends, instant response.

Rather than making you wait around while we process large bulk SMS sends, the send page now responds instantly. 

We now process the send in the background. It means you can keep working while the send processes.

Manage scheduled messages

In the past if you wanted to cancel scheduled message you had to call us and we’d do it for you. 

You’ll now find a list of all scheduled messages within the application and can cancel them up to a few minutes before they’re sent.

Up to date groups

If you schedule a message to a group we’ll now pick up the list of group members at the time of send rather than schedule.

This means you can continue to manage your groups and the send will be up to date with your latest contact list.

Additionally, to make this change work we now deduct message credit when we process the send.

Faster uploads

We’ve made a few behind the scenes tweaks to how uploads are processed.  In our tests an upload of 50,000 contacts now completes 10 times quicker than before.

Bigger uploads, bigger sends

There used to be a limit of 10,000 people in a single upload or send. 

With the speed improvements above we’ve removed this limit and so far have tested with groups of 100,000 contacts with no speed issues what so ever.

Summary

So there you have it hopefully you’ll enjoy the benefits, but as always contact us if there  is something we can improve.

Shared Groups

Sub users can view and send to groups shared by their admin user.

Quicker Uploads

Contact uploads now only require the mobile column, all others are optional.

Outlook 2010 Compatibility

oms can now be used from Microsoft Outlook 2010.

Clearer Pricing

To make pricing clearer message prices are now displayed both with and without VAT.

Enhanced Merging

Merged messages correctly count the number of characters in fields, in a small number of situations a send failed if one contact had a much longer name than the others.

IE6 tweaks

No one loves Internet Explorer 6 but we can’t ignore it completely, O2 branded products now work much better in this (legacy) browser.

Summary Pie Charts

Sent messages now shows a little pie chart rather than a bar graph, this gave us space for the next feature…

Quick Resend

You can resend messages from the sent page.  Just click the icon next to the pie chart and it will fill in the contacts and message text for a new send.

Clearer Buttons

Disabled buttons are greyed out making it very clear they can’t be clicked – You’ll see this on the received page, the add to phonebook button is greyed out if they’re already a contact.

See Who you are sending to

When you reply to a contact we now show their name in the To box.

Improved Statistics

The send date and originator are now shown on the Send statistics page.

If you do have any feedback or suggestions then leave a comment or head over to our customer forum.

This work follows on from our network upgrades in Derby last week and will bring our network fully up-to-date.We’ll be starting mid morning (around 10) and should be finished by early afternoon.

What we’re doing

We’re upgrading our remaining Cisco PIX firewalls to shiny new Cisco ASAs.

We’re setting up a backup link between our sites to keep your data safe.

And finally we’ll upgrade our Manchester backup database servers to make sure they can always keep up with the load.

Technical details

Our Manchester site is on the 217.20.43.96/27 network (217.20.43.96 through to 217.20.43.127), whilst we do the work you won’t be able to reach any servers running on these addresses.   Our Derby site will be up and working fine on the 89.248.48.192/27 network (IP addresses 89.248.48.192 to 89.248.48.223).

This afternoon we’ll take the Manchester servers out of service by removing the DNS records, we’ll then change them back once everything is tested and finished.

If you login to any of our products you can see which site you’re currently using, our Manchester servers always have a number 3 near the start such as textburst3.mediaburst.co.uk, Derby uses a 3 instead.

If you have any questions give us a call on 0845 050 0000 or drop us an email.

While Derby is turned off  we’ll also be upgrading our network and servers to make sure we offer you the best service possible.

We’ll be starting mid morning (around 10) and should be finished by mid afternoon.

What we’re doing

We’re upgrading our Cisco PIX firewalls to shiny new Cisco ASAs, they’re considerably more powerful and should make our life much easier.

We’ll also install new servers for our mobile marketing services, the old ones currently work fine but we don’t want to take any chances.

And finally we’ll upgrade our backup database servers to make sure they can always keep up with the load.

Technical details

Our Derby site is on the 89.248.48.192/27 network (IP addresses 89.248.48.192 through to 89.248.48.223), you won’t be able to reach any servers running on these addresses while we do the work.  Our Manchester site is on 217.20.43.96/27 (217.20.43.96 through to 217.20.43.127) these addresses will all work fine.

On Monday afternoon we’ll take the Derby servers out of our DNS so that our service addresses only point to Manchester, we’ll then change them back once everything is tested and finished.

If you login to any of our products you can see which site you’re currently using, our Derby servers always have a number 5 near the start such as textburst5.mediaburst.co.uk, Manchester uses a 3 instead.

If you have any questions give us a call on 0845 050 0000 or drop us an email.

Having tried a Juniper SRX for a fortnight we were left with the impression that a few features aren’t quite finished yet. The routing side, which Juniper are traditionally associated with, was really powerful and easy to use, but for a firewall it seemed to be lacking a few useful features found on the majority of the competition. These included authenticating VPN users locally if your Radius server is down and assigning IP addresses without an additional DHCP server. From talking to a couple of Juniper experts it sounds like these and many other features are coming soon, unfortunately for Juniper we needed them now.

The Juniper SSG range were almost identical to the Cisco products both in features and price, in the end we rejected these because we had major trouble obtaining some competitive quotes. One major international IT reseller were that frustrating to deal with that we’ve vowed never to use them again.

One of our core requirements was dynamic VPN access so we can reach our servers when on-call and people can work from home. Given that Microsoft have just released Windows 7 and that 64bit operating systems are now being pre-installed on laptops it had to work with both of these. The Juniper VPN client only works on a 32bit OS, we’d have to purchase some third party software to connect from a 64bit OS, Cisco on the other hand have added full 64bit support to their AnyConnect client.

Our final reason is that Cisco were willing to accept our current PIX firewalls as trade-in. Of all the reasons we considered this was least important but it became a small factor once we realised the features were essentially the same. The same international IT supplier strung us along for 3 weeks while trying to sort out this trade-in with Cisco, they then gave up. Our normal, much smaller, UK supplier had it all sorted in an afternoon and managed a much better price too.

These new firewalls should be with us shortly and I’ll post an update once we’re ready to put them live.

A large number of our servers run CentOS Linux, and like all Redhat based distributions software is installed through RPM. Unfortunately installing perl modules using CPAN has nasty habit of clashing with the base perl install and causing no end of problems.

To work around this we’ve taken to installing all perl modules through RPM. Some modules are easy to find in third party repositories, but our software uses some less common modules such as Net::SMPP.

The best way we’ve found to convert these elusive perl modules into RPMs is through cpan2rpm by Erick Calder, it’s also a very convenient way to make our own software available as an RPM.

Creating an RPM from a CPAN perl module is extremely simple, just pass the module name as the only command line parameter, it will download from CPAN and build the RPM.

[rpmbuild@dev-linux ~]$cpan2rpm Net::SMPP
...
RPM: /home/rpmbuild/rpm/RPMS/noarch/perl-Net-SMPP-1.03-1.noarch.rpm
SRPM: /home/rpmbuild/rpm/SRPMS/perl-Net-SMPP-1.03-1.src.rpm
-- Done --
[rpmbuild@dev-linux ~]$

cpan2rpm however does have it’s limitations so we’ve added a few extra features to our version.

RPM Pre install commands

Commands that need to be run before you install the package, such as creating users. Defined in Build.PL using rpm_pre or passed on the command line as –pre=”my pre-install command”

RPM Post install commands

Commands to run once the package is successfully installed (These don’t run if it fails). Defined in Build.PL using rpm_post or passed on the command line as --post="my post-install command"

Disable auto-generation of the provides list

By default rpmbuild will automatically create a list of all packages it provides, on occasion this is wrong, so we added a flag to disable this automatic list creation. Passed to cpan2rpm on the command line as –no-auto-prov

Support for Module::Build dependencies

cpan2rpm couldn’t wouldn’t correctly create dependency lists for modules build using Module::Build rather than ExtUtils::MakeMaker, we tweaked the script so these are now listed on the RPM. These come from the requires section of Build.PL

Support for the RPM config flag on files

RPM allows files to be flagged as configuration files within the RPM. This flag prevents RPM from overwriting your configuration files when changed by creating the new version as file.rpmnew. By default we’ll flag anything from the etc_files section of Build.PL

Using John’s earlier Build.PL example we can add in these extra features so that it creates it’s own user and then runs a postinstall script once the RPM is installed (this could setup a database or download updated configuration files for example). These rpm_ tag’s aren’t standard Module::Build features but it will ignore unrecognised tags when building.

use 5.008005;
use Module::Build;

my $build = Module::Build->new (
    module_name => 'ModuleName',
    license  => 'perl',
    dist_author => 'A.N. Other <a.n.other@example.com>',
    dist_version_from => 'lib/ModuleName.pm',
    etc_files => {
        'etc/logrotate.d/modulename' => 'etc/logrotate.d/modulename',
        'etc/modulename.conf' => 'etc/modulename.conf'
    },
    install_path => { 'etc' => '/etc', 'script' => '/usr/local/bin/modulename' },
    rpm_pre => 'if ! id module_user > /dev/null 2>&1 ; then useradd -r -c "Module User" -s /sbin/nologin -d /usr/local/bin/modulename module_user; fi;'
    rpm_post => '/usr/local/bin/modulename/postinstall_script'
);
$build->add_build_element('etc');
$build->create_build_script;

If you use cpan2rpm regularly I’d recommend creating a .rpmmacros file in your home directory to save you having to pass a few of the options. This also allows you to enable the optional GPG signing of packages. My .rpmmacros file is:

%_topdir /home/rpmbuild/rpm
%packager Mediaburst Ltd <rpmbuild@mediaburst.co.uk>
%_signature gpg
%_gpgbin /usr/bin/gpg
%_gpg_path /home/rpmbuild/.gnupg
%_gpg_name my_key_name

If you plan to use cpan2rpm with standard Perl modules from CPAN I recommend you download it directly from the authors site, however I’m also making available our updated version in case you want to use any of these update features.

• Official cpan2rpm release
• Mediaburst cpan2rpm (we’ve added an mb suffix to the executable)
• Patch containing our changes

I’m not the world’s greatest Perl programmer as I normally write C# so if you’ve got any feedback please leave a comment below.

The obvious choice seemed to be the Cisco ASA range, probably the ASA5510, but in some areas the changes between the PIX and ASA platforms seem quite significant, so we thought we’d look at the competition too.

A little research narrowed us down to two suppliers that could provide all the features we need within a sensible budget, Cisco and Juniper. Cisco are a safe bet, we’ve used them before and you don’t get any surprises, Juniper on the other hand have only recently started targeting smaller networks, most of their products are used in core networks.

The main contenders were:

Cisco ASA5510

The ASA line is a new improved version of the PIX. They run the same OS and have a number of new features such as SSL VPN tunnels.

Juniper SSG140

The Juniper SSG range is based on the products previously made by Netscreen (acquired by Juniper a few years back). They run a custom OS called ScreenOS and very similar to the ASA5510 in all features we need.

Juniper SRX210

Juniper have only recently introduced the smaller models in their SRX range, these run Junos, the same OS as Junipers core routers. From the figures we obtained they appear to have considerably higher throughput and performance then either the ASA or SSG ranges and a number of additional features.

At the moment Juniper seem to be really pushing the SRX line, they’ve got really competitive pricing and free online training including certification, and given that they have more features than either of the other products we’ve going to give them a try.

A friendly Juniper reseller has agreed to lend us one for a couple of weeks while we see if they’re as good as they look on paper.

Once we’ve got our hands on one I’ll add another post letting you know what we think.

PwC partner David Lancefield, outlined that the media landscape is changing at a previously unseen rate of knots.

Mobile is again grabbing market share as more established digital media is in decline – UK 2009/10 online ad spend is projected to dip 3.2%

PwC also highlights just how many small media businesses have fallen prey to the recession: the downturn has claimed some high-profile victims as we know but PwC reports that a frightening 910 smaller advertising and media businesses have gone bust in the past two years, 525 in the last 12 months-

• Advertising: 211 in 12 months to June 31 (50 percent up annually). 352 in the last 24 months.
• Broadcast: 81 in last 12 months (40 percent rise).
• Publishing: 260 in last two years.
• Music: 18 in last two years

The future may be uncertain, but it definitely is mobile

However a more interesting side show also occurred with a demonstration of an iPhone being hacked via a set of binary codes sent to the phone via SMS.

The conclusion was that as users we trust the communications to our phones more than we do e mails to our computers.

Once more this underlines the high value of mobile marketing, but also underlines the responsibility that comes with it.

If you’ve got any questions about any of the changes we’ve made give us a call or drop us an email.

We’re now happy to say everything appears to be working fine. If you have any questions or queries about the move give us a call on 0845 050 0000.

Here’s a few pictures to show off our tastefully coloured network cables.

We no longer have any services running on the 213.218.217.160/27 range of IP addresses.

Update

Everything is now running fine in our Derby site, we’re not going to transfer over customer traffic until Tomorrow (Tuesday) so we can keep an eye on the log files.

A few pictures of where we’ll be hosting the services

IP Address Range Changes

The development requires clients that restrict access to/from Mediaburst services based on IP addresses to make the following changes:

From 27th June 2009:

Services will start running on the 89.248.48.192/27 range. (89.248.48.192 through to 89.248.48.223)

From 29th June 2009:

Services will no longer be running on the 213.218.217.160/27 range. (213.218.217.160 through to 213.218.217.191)

There will be no change to our Manchester, 217.20.43.96/27, range of IP addresses (217.20.43.96 through to 217.20.43.127)

If you have any questions give us a call.