November 2004

To encode, or not to encode: Whether ’tis nobler in the mind to suffer, The zips and rars of hacked distributions. Or to take arms against a sea of pirates, And by opposing upset customers?

Bad lampooning aside, there is a real question here: To encode our source code and possibly upset customers or to continue to allow readable source code and continue to battle piracy?

Pros and Cons

Readable source code is the default method of choice for downloadable code written for the likes of PHP, Perl and ASP. This is largely because readable source code can be run without the need to install server level decoders and is how scripted files were designed to be used. You write the code, save it and run it. No need to compile it or fuss around with it in any other way.
The big downside with readable source code is that it’s, well, readable. You can’t add any real restrictions in the code to prevent it from being abused as the code can simply be found and deleted. Another negative is that your code is available for everyone to see and steal.

Encoded source does away with those problems. The code is compiled and encoded and then available for download. The encoding is pretty solid and is hard to "crack". The downside is that the customer has to be running the decoder on their server for it to run. Fortunately, Zend offer a free optimizer/decoder which is very simple to install and IonCube offers a server level version and local file based versions. The IonCube approach is excellent as it allows (virtually) everyone to use encoded software without the need for root level access to the server, which is a real problem on shared and virtual server accounts.
The big downside with encoded source code is that you can’t modify it to suit your needs - and that’s the crux of the problem.

For years, PHP and perl applications have been available as readable source and this means that, with a little knowledge, you can tailor the application to suit your own needs. This side benefit has now boomed into its own little industry with modification communities springing up for popular products. In a competitive industry, readable source code is another selling point and may be a customer’s deciding factor.

Software Piracy

Piracy is a problem in every industry. Even compiled applications get cracked and put on peer-to-peer networks for free downloading so encoding PHP source code isn’t going to remove it totally but it will reduce it. As the industry stands now, it takes no skill to pay for one copy of a piece of software and make it available for everyone. Most commercial software has license keys and other data to make it a little harder, but any such code can be removed quickly making it useless.

This brings us back to the original question. There is an argument to say that when you purchase an application, you purchase the application - not the right to have access to the source code to tweak it for your own use. This is certainly true for desktop software. There is an argument to say that encoding files will kill the modifications community and market, and this is where I think we can find a compromise between allowing code modification and encoding source code. 

Necessity is the mother of invention

If we can rethink the concept of a modification and how the
modification is applied then we can create a new way of allowing
additional code to be run alongside our existing code. Indeed, an often
asked request is a way to automatically "add" code modifications into
the product to avoid having to search for x and replace with y.

If we can find a way to allow people to write and add modifications
without touching a line of the original code then we can kill two birds
with one stone. We, at IPS have already made a start with our modules
system and synchronization system and we plan to extend that. With
custom settings, HTML logic and a modules system, most modifications
can be rewritten to use these systems.

I definitely think the future is with encoded source. Many applications already encode their source, Modernbill is a good example. If modifications don’t need to alter the existing code and it makes it easy to install modifications, then there really can’t be any complaints, can there?

Oh Lord, give me the strength to get through this ticket.
Give the will to complete the reply,
Without getting the urge to curl up and die.
No matter how ridiculous this request, I must be strong,
Even though it’s "him" again and he’s never wrong.
It’s not his fault he doesn’t know what he’s doing,
Even though he ends up angry and wants to start suing.
Take a deep breath and try and be polite.
With a swift reply, it’ll be all right.
Then onto the next, you can boldly go,
For more of the same, just don’t let it show,
That you’re slowly and quietly going mad,
Not too long to go now, so don’t feel sad.
Another one down, 99 to go.
There’s the link.
Go on.
Take a breath and click it.

Writing internet software is always a roller coaster ride. I often look at ‘desktop’ application developers with envy. They have the luxury of a relatively stable operating system which can go many years without any major developments which could make their code redundant. They also can write for a specific operating system, whether it’s Windows, OS X or Linux, they know they only have to target one specific operating system. I’m sure that application developers view internet developers with envy, but you’ll have to visit their blogs for that insight.

Internet scripters don’t have those luxuries. We have to write portable code that has to run on a handful of different operating systems that all bring their own development problems. (Side note: Who at Microsoft thought that using the escape character, the backslash, as a path separator was a good idea? I’d like to meet him and see if I can get some money back on therapy required after running the stripslashes mine field). We also have to use a series of ever-changing scripting tools such as PHP or perl tied into databases with frequent and code-breaking changes. This creates a very unstable development environment.

This brings us to the title of this prose: twelve months developing a product for the internet is too long. Many things change on a daily basis and you can never assume that the development platform you’re using to write your code is still going to be valid by the time you’ve finished the initial code base or rewrite. I learned this the hard way.

Learning the hard way

In a previous life, I developed and maintained a perl project. This project got to the point where it was obvious that a rewrite was required. The architecture was struggling under the weight of bug fixes, hacked in features and code limitations. I had taken a basic and badly designed code base as far as I could. I was conversant in perl and didn’t really want to switch to another language. This was back in those dark days when perl 6 was lauded as "right around the corner" and PHP had triumphantly released PHP 3 and development of PHP 4 started. PHP 3 was a weak language with too many restrictions and work arounds needed to manage a project the size of the one I was managing. It wasn’t a hard decision to continue writing perl. Perl 6 was on the horizon so it made sense to ditch procedural code and work entirely in OOP (object orientated programming). I created a lavish frame work with a modules and class system. I figured that when perl 6 was picked up and used on our customers’ servers I could simply rewrite these modules independently without affecting the framework. I also wrote an intricate database abstraction class which could load sub-classes to process a query set I developed which meant that if a database engine changed, I could rewrite the sub-class without needing to touch a single other file.

The theory was sound, no doubt about it. Unfortunately this took nearly 18 months to complete. I went over deadline by a spectacular 6 months and was forced into releasing a product which I knew wasn’t complete or anywhere near bug free. To complete the project properly could have taken well over two years. We upset a lot of our customers with repeatedly missed deadlines and while we were rewriting our code, the web hosting industry moved from perl to PHP 4 and perl 6 became more of a pipe dream than a reality. After 18 months of hard development all I was left with was a brilliant theory, outdated code and unhappy customers.

Lesson learned

This mistake was still very raw when I began the foundations of a new product. It was obvious that it couldn’t take too long to release the first version and the experience helped me form a development roadmap. This time around, I’d develop the framework with the same ideals but introduce a smaller feature set. These features could be introduced over time with quicker version releases.

OOP was an obvious choice for this new project. A lighter, more manageable database abstraction engine was created to handle changes to relational database engines and a class structure was employed to cover the basics such as emailing, printing output, template handling, etc. Modules were introduced to handle different information requests. All of these objects can quickly be rewritten without changing any other code.

Procedural code is simply too hard to manage when a project reaches a certain size. The need to bug fix, maintain and introduce new features creates a mess of code which runs out of control. The architecture is hardcoded into each file which makes the system inflexible. A rewrite is inevitable and just a matter of time and it’ll take a good deal longer to rewrite procedural code, which is time badly invested.

I recently released a major version increment which introduced many new features and a fresh interface. I’m still using the framework I devised nearly three years ago. Nearly every module and class has undergone some surgery to introduce new features and to maintain the project but I was able to approach each module as an independent part of the greater program. I was able to turn around a major release in a little over 6 months and without needing to rewrite every line of code. I couldn’t do this with the procedural code I had previously developed.

The moral of the story

The moral of the story is very simple. Don’t ever bite off more than you can chew and the first release doesn’t have to be the last release. Plan your development wisely and aim to release frequent updates to achieve the feature set you originally had in mind. Use OOP where you can and always keep in mind that the code you’re writing will need to be maintained and eventually rewritten. Invest your time well and you’ll never have to apologise for missing your own deadline by six months.

Oh, and if you’re writing an operating system, never use the backslash character for anything important. It’s for escaping characters and nothing else.

Now, I’ve often been accused of taking my time to fix the odd bug. Sometimes it’s hard to reproduce something and relies on certain circumstances for the gremlin to show itself. These reports often get shunted to the back of the queue in favour of the ones that can be fixed with a minutes of investigation and a quick code change.

Recently I finally fixed a rather annoying and hard to locate bug in the bulk mailing system. It took a few months of testing, head scratching and a minor nervous breakdown before the cause of the problem became obvious.

Microsoft get a lot of bad press from the internet community for their products which contain the odd bug or three. It’s easy to assume they don’t care about the quality of the product or that their software engineers are inept. Indeed, the most famous bug in Microsoft’s history was in the Mac version of Word. This bug raised its ugly head now and again when working on a document when a quick save attempt resulted in a "disk full" error. The only way to continue working on this document was to hard save, close the file, rename the file and open it again. This bug took ten years to fix and the anatomy of this bug makes for very interesting reading. I recommend you take a look at Rick Shaut’s blog over at Microsoft.com and spend a few minutes reading his entry on this bug. It’ll definitely change your mind about their developers.

I’ll finish this blog entry with a quote from Chris Mason, another Microsoft developer:  “Since human
beings themselves are not fully debugged yet, there will be bugs in your code
no matter what you do.”

I sometimes come across the odd post or blog entry from a programmer who proudly declares that they use nothing more than NotePad (or a basic equivalent) for all their coding. This apparently promotes them to "super programmer" status. A true hero amongst code hackers. A real man who has no need for syntax highlighting and auto tab balancing. A cape wearing super programming rock star, indeed.

The next time I’m in need of a doctor’s services I’m going to make sure that the successful candidate only uses a kitchen knife and nothing else for his surgical operations because being able to deftly weild a blunt bread knife and shunning easily available time saving tools makes him the one for me.

Take off the cape and buy an IDE. You can prove how manly you are by wrestling with bears at the weekend if you feel emasculated.

So, Monday rolled around again.

I love my job. I really do. I’m lucky to be doing something I really enjoy. However my "irritation-o-meter" has been running in the red zone today. From very rude people, to know-it-alls looking so hard between the lines they’ve gone all cross eyed.  It can be a little irritating when something you’ve said is misquoted and made into something more sinister but that’s the way it goes. As a fellow forum developer noted in his blog recently: It gets tiring when people claim to "know better" than we do.

Today was more about catching up with support, billing and general enquiries but I did manage to squeeze out a maintenance release of IPB which fixes many common problems. It was nice to finally nail the oddity in the bulk mailer that caused multiple emails to be sent out. Creating new features is what’s really exciting but bug fixing can be extremely satisfying. You get a real feeling of acheivement after knocking down a dozen or so bugs in a mornings work.

Still, I’m really looking forward to getting back to work on IPDynamic…

I’m more of a right brain thinker. I do colours, shapes and pretty pictures but every now and again the odd statistic or set of numbers come along that amuse and delight in equal measure.

For example: the recent presidential election (I’ll leave the political and social commentary to others). After the big event this week, there were literally millions of statistics available for general consumption and I mentally compiled a profile of the type of person who voted for Bush. This person is a rich, white, middle-aged male with an IQ of 100 or under. Now - this interests me because how did a such an individual manage to become rich with an IQ of the average 8 year old? I’m guessing they didn’t do it the old fashioned way and work their way up the corporate ladder to be on top of their game.

I’ll let you assume what that says about the average George W. Bush fan.

Everyone now and again some leading (and often self proclaimed) expert will announce that something is good for you. Recently, it was discovered that a small amount of chocolate was actually good for you. Apparently a glass of red wine a day is also good for you. Red meat, white meat, potatoes, fish, 2 litres of water and 5 portions of fruit and/or vegetable are also good for you. That’s great news. However, another expert claims you shouldn’t have more than 1500 calories a day. This is confusing, because after stuffing your face with all that chocolate, red wine and assorted other food-stuffs you’re more likely to be eating 3000 calories a day which means that it can’t all be good for you. Someone is lying somewhere.

I guess the moral of this story is don’t believe experts, mathematicians or anyone who’s rich and stupid.