(?:Dis)?Organi(?:z|s)ed Chaos

by Matt Mecham on April 22, 2005

in Uncategorized

So, it’s the day after the afternoon before. In that time, we upgraded our corporate forums, suffered a DDoS attack, send out 12,000 emails, fixed 50 odd bugs and even found time to write in a legacy mode to enable the blog and gallery to work without modification.

The upgrade to 2.1 seems to have gone fairly well. It was a bumpy ride for the first few hours and the odd (hundred) bugs here and there did their bit to add to the overall chaos. Ever wondered what it’s like to upgrade a relatively busy forum to a largely untested release? It goes something like this:

1:00pm: Turn company forums offline. Add in a redirect to a basic index.html file.
1:30pm: New file uploads complete
1:50pm: MySQL updates done surprisingly quickly. Note to self: MySQL 4 is so much better than MySQL 3.
2:00pm: Hack index.php to allow access for myself and the US office IP address.
2:10pm: Various staff members AIM me with bugs they’ve found.
2:14pm: It’s apparently turned into a competition.
2:22pm: Fixed the first round of bugs (parse errors, the biggies)
2:34pm: AIM is lit up like a christmas tree
2:38pm: Fixed some more bugs. Start roughing out an announcement
2:43pm: AIM is lit up like the Empire State Building
2:44pm: Few more bugs are fixed.
3:00pm: Finished announcement. Load Safari to post it (saves logging out of Mozilla)
3:01pm: Notice the standard editor doesn’t work at all with Safari.
3:20pm: It does now.
3:22pm: MySQL error when posting from Safari
3:30pm: Get a message from CTerry via Stewart ‘Stop slacking’
3:31pm: Motivated, I submit the announcement again
3:32pm: Remove IP address filter and turn board back online
3:33pm: Charles starts sending emails to our subscribers
3:45pm: Can’t load a forum. This can’t be good. ssh admin@invisionpower.com….
3:50pm: MySQL locks up, 156 locked processes. Lindy?
3:51pm: MySQL restarted.
4:10pm: Ah, that topic table index? Yeah. I’ll go add that now.
4:15pm: MySQL locks up again.
4:20pm: Oh yes, LOW PRIORITY was removed from the topic counter update
4:25pm: Server restarted
4:26pm: Things are much improved. Nearly 400 users active and load @ 0.3 per CPU
4:30pm: Initial feedback comes rolling in. Bug reports are posted everywhere.
4:30pm – 6:00pm: Crazy bug fixing period.

Or something like that, anyway.

I’ve spent the better part of today bug fixing and fiddling with bits of javascript that didn’t work on Opera. Overall feedback has been good. Apparently some people have taken offence to the topic rating images and the rich text editor buttons. I was thinking that the blue posting form would have caused the most upset.

{ 26 comments… read them below or add one }

1 Zain April 22, 2005 at 9:22 pm

Way to go matt….!

2 Tim Dorr April 22, 2005 at 9:36 pm

Now, we’re just going to have to do that with 5x the active users over at Neowin. At least by then the source will be a bit more stable :D

3 Don Wilson April 22, 2005 at 10:20 pm

It seemed that when I clicked “New Posts” the first couple of times it would show perfectly, then it would show the entire IPS Forum page in that little bit, then it was later fixed. Was this one of the bug fixes?

4 Oblivion. April 22, 2005 at 10:34 pm

IPB 2.1 PWNS!

IT’s good to see all the long-awaited features faling into place, the WYSIWYG editor has to be me favourite ‘new thing’. I’m not too sure on the topic rating, but that’s just me :) .

5 Matt April 22, 2005 at 10:40 pm

Tim,

Got rid of that ugly memberlist query so you might be able to get 5 x the members on the site. ;)

6 Surferdude April 22, 2005 at 10:47 pm

Sounds like fun!

7 Cybertimber2005 and Cooldude7273 April 22, 2005 at 11:51 pm

Mega thumbsup Matt! You ultimatley surprised me with the update soo quickly… “We plan to update the company forums in the coming weeks…” That was last weekend!
Btw… the blue posting screen could take some getting used to, but maybe make it match the Editor… which I hope is skinable O:)

LOL! Aim like a christmas tree :)
Try a Chat next time… that way each other knows who’s reported what… not [D] :)

8 Louis April 23, 2005 at 12:45 am

Poor Matt,

With al the feedback in the customer forums (be it contructive or other) you guys did an OUTSTANDING job. There are bugs, there will be them but you guys stepped up and worked very fast on them. I was rather impressed to see what you have done. 2.1 is looking to be a very good upgrade! KEEP UP THE WORK!

We love you Matt (well when you give us blogs and those wonderfull pics of ID LOL).

9 Anonymous April 23, 2005 at 1:24 am

Wait… the ugly memberlist query?

The one that grabs EVERY single user? and crashes mySQL and we’ve had to sdiable at IZE?

W00t!

Er – good job – I love the blue posating form (adds a bit more cxlass really) and well – while the topic rating images may not be the best choice of image, I’m not going to take offence (just change them :P )

Anyways :D …. er… nice job :D cant wait to try it out myself *itches to see the new ACP*

10 akurashy April 23, 2005 at 3:59 am

sure you had some fun matt ^_^

11 JPT62089 April 23, 2005 at 8:25 am

ditch IPB and get vBulletin :P jk IPB is a good piece of software… just too many bugs atm… Matt you could’ve done a better job at getting those bugs fixed before you upgraded the comp. forums… blogs dont work, chat dont work, posting text area resize thing dont work in FF and users cannot change profile settings…. WAY too many bugs… Well I guess you just gotta learn from your mistakes…. keep up the good work matt and same goes to the rest of the IPS staff!

12 Bob April 23, 2005 at 10:39 am

What do you expect from an alpha release? it’s not as if it’s been released to the world and every upgraded IPB is suffering from these bugs!

Can’t wait for the final, i know it’ll be a while yet…

13 Louis April 23, 2005 at 11:18 am

Read the blogs and comments here. Read the posts in Invision’s forums. They all say pretty much the same thing “When are we getting it?” We are like abunch of rabid dogs staring at food that is just 2 inches in front of our faces. I am sure that our pressure had some cause on this.

Relax and enjoy the process.

14 Matt April 23, 2005 at 11:31 am

“blogs dont work, chat dont work, posting text area resize thing dont work in FF and users cannot change profile settings…. WAY too many bugs…”

All those bugs were fixed in under 20 hours of the board going live. Most of them were fixed within a few hours.

15 Anonymous April 23, 2005 at 3:17 pm

LOL, i laughed a few times reading that entry

Good job on IPB v2.1, its shaping up to be a excellent new release

16 Franklin April 23, 2005 at 3:39 pm

Hey Matt,

It’s working great! (I won’t start about the bugz and that the RTE crashed FF103)

Goodluck and enjoy your weekend!

17 Jaak April 23, 2005 at 9:29 pm

Sounds cool!

I should start reading this more often..

18 William April 23, 2005 at 10:14 pm

Sorry that so many of those christmas tree lights were mine. :-”

Wonderful thing, 2.1 – Great stuff Matt! :)

19 Dave Sims April 24, 2005 at 2:59 am

Looks pretty nice, I would suspect that if you change things in such a huge PHP coding enviroment you’d get at least 100 bugs.
I noticed for some reason The download for the IPB board is down.

20 Will April 24, 2005 at 9:44 am

nice work Matt well worth the trouble

21 JPT62089 April 25, 2005 at 5:04 am

But Matt you could’ve fixed those bugs before you upgraded the support forums (The blog and stuff I understand but the edit profile, no working +/- things in the post screen, etc.) But now that you fixed em very nice release! just gotta change that UGLY mew PM notification…. very nice though lol

22 Wilko April 25, 2005 at 10:37 am

You have to remember that most bugs don’t show themselves until you test the board with 400-500 members online in a live environment.

23 Madob April 25, 2005 at 3:10 pm

I haven’t had any PMs to see the new notificatio nin action :(

24 Madbob April 25, 2005 at 3:17 pm

I haven’t had any PMs to see the new notification in action :(

25 Madbob April 25, 2005 at 3:18 pm

I haven’t had any PMs to see the new notificatio nin action :(

26 Madbob April 25, 2005 at 5:59 pm

I must apologise for posting that last comment 3 times… sorry!

The new PM notification isn’t very pretty, but hopefully it’ll be a lot nicer when IPB2.1 Final comes our way

Leave a Comment

Connect with Facebook

Previous post:

Next post: