As I'm sure everyone has noticed, we have had a number of outages since we deployed the new server. I've reached the end of my rope with trying to diagnose and fix the problem while the server is in the field and have moved the major services back to our previous server for the time being. I figure that slow but functional is an a improvement over the state of affairs that we've had lately.

Mere words can't do justice to the degree of frustration that I feel about the whole fiasco. I'm going to go pick the server up and bring it back for some more extensive testing and to get to the bottom of the issue once and for all. Thanks to everyone for the amazing level of patience that has been shown throughout the entire ordeal.

There are bound to be some things that are broken during this time -- please let us know if you see anything behaving strangely and we'll get it resolved as quickly as we can.

We've lost a few hours worth of posts: that permanent?

Thanks for the effort, by the by.

~J

I think that any missing posts should be minimal -- most of the time gap was actual downtime anyway so no posts were being made. There is some missing data though, and I apologize for that.

Also note that since we're back on the old server, the same funky time issues that we had in the past are likely to crop up again. Nothing to be done for it unfortunately. I'll try to make it out to the colocation facility tomorrow to pick up the server and bring it back to put it through the ringer. It's not going to get redeployed until I've torture tested it and am satisfied that it'll be stable.

So it is permanent ... ?????????

(A few kind of seriously crucial posts lost in the three major LitS threads -- you know, things like the missile finally hitting after a month of set-up? things like the post that followed the only surprise roll one character had since the beginning of the campaign -- and it was a hugely major one? Things like some major character work from people who had just returned from major RL work today? Several major posts in each? Is there absolutely no way to recover these?)

Is there an ack/head hits wall/feeling sick/really really really depressed smiley?

OOC thread: at least 6 posts.
48-page IC thread -- we'd been a good way down p.48. (I'd originally had the first post on p.48 -- which was where the missile finally landed.)
11-page IC thread: at least 5-6 posts.

On what there is of a plus side, I'm grateful that at least Alleycat's last post on p.47 -- only post I was able to give her in a week -- didn't get swallowed. And I'm grateful that the Infiltration Challenge people hadn't gotten to the point of finalising the exit strategy.

Thats why I keep a text file with backups of my recent important posts when the server is unstable. I just copy and paste them back if anything gets lost.

I lost a few posts of mine today... any chance they can be retrieved?

SR13

There should only be an hour or two worth of posts that may have gone missing and the effort that would be required to try to find and merge them at this point would be pretty extreme. If there are one or two very long posts that someone needs, and you know the forum it was posted in, and nearly exactly what time it was posted, send me a PM and I might be able to at least get the raw text out and send it to you. I can't promise anything, but I'll give it a shot at least.

I'd be very happy with the raw text, if you can find it, of any of the posts in those three threads. Since there are multiple posts involved, I can't be more specific about time beyond that they were written yesterday between 10 am and 4 pm. In a couple of hours I'll be able to get onto the computer on which I wrote mine, but it's a public one and the odds that the cache was erased are very high. (I know for a fact the history was.)

I have a script that I wrote that can dump an entire thread from the DB on the new server to a text file. The text isn't that pretty, but it should at least be usable to get the contents of your post back anyhow. I emailed the contents to the email address in your profile, Sedna.

It's a lot prettier than most of what I normally deal with. (I cold-code.) But the most recent date in that text is April 29/30? (Sent an e-mail.)

Edit: whoops, just got your revision, going through it now :)
Re-edit: YES! That looks like what we need! Thank you! :)

DNS just did something strange, but I've got a workaround up and running now.