More Server Crashes

Forum News, Announcements and any issues you're having with the forum

Moderator: myglaren

User avatar
CitroJim
A very naughty boy
Posts: 39633
Joined: 30 Apr 2005, 23:33
Location: North Bucks
My Cars: An orange Skoda 120, a pretty blue Pug 207, Sally the white Saxo, a Giant TCR Bike and a Giant TCR Pro Bike.
x 747

More Server Crashes

Post by CitroJim » 09 Jan 2010, 18:51

I'm aware that the forum server is now throwing it's toys with a bit of worrying regularity.

I'm busy investigating why. It looks like a memory leak in IIS7 stemming from a known issue between it and PHP.

As a temporary "fix" I'll be setting the IIS worker threads to recycle on a regular basis to clear down the memory they are holding.

I have made some other configuration changes to IIS and I'm now monitoring memory usage and event logs.

Grateful if all apparent crashes can be reported here just in case I miss any.
0x
Jim

Please note I do not accept PMs...

User avatar
spider
Posts: 3951
Joined: 05 Jan 2010, 15:28
Location: Derby.
My Cars: Soon, I hope...
x 63

Post by spider » 10 Jan 2010, 19:27

It was down for me about 6pm GMT for a short time. Just getting the usual index.php download after it timed out.

Anyway about 5 mins ago, there was a problem with the sessions table (duplicate key) although that appears to have settled down now, at least that's the best one to go if any are going to act up. ;)

[off topic]Am surprised to read you are using IIS though rather than using a 'nix box with apache :D[/off topic]
0x

User avatar
CitroJim
A very naughty boy
Posts: 39633
Joined: 30 Apr 2005, 23:33
Location: North Bucks
My Cars: An orange Skoda 120, a pretty blue Pug 207, Sally the white Saxo, a Giant TCR Bike and a Giant TCR Pro Bike.
x 747

Post by CitroJim » 10 Jan 2010, 19:37

Thanks Spider,

I saw it go down :twisted: Webfusion had a report of connectivity issues but this was a crash in the good old style. Strangely, nothing in the NT Event logs as there never is :roll:

It came back up of it's own accord but the phpbb_sessions tables on both the FCF and BXC were corrupt. They repaired OK.

I'm a dyed in the wool Windows person and know diddly-all about 'nix in comparason :roll: We were on a W2K3/IIS6 platform until not so long ago and that was relatively stable, now on a W2008/IIs7 platform and I don't like it. Vista for servers. need I say more :evil: :evil: :evil: About as stable as quicksand at present.

I was watching the IIS memory leak build. I need to recycle the IIS application pools for the forums a bit more frequently to see if that helps.

If this continues, I may try a 'nix box....
0x
Jim

Please note I do not accept PMs...

HDI
Posts: 1476
Joined: 22 Sep 2009, 15:46
Location: West Mids
My Cars:
x 7

Post by HDI » 10 Jan 2010, 19:49

Yeah , I've had a couple of google , "ooops , this link appears to be broken" events recently.
0x
Now using '00 Xantia LX HDI, pov spec :(
My past Citroens :-
'00 Xantia SX HDI, now dead due to accident :(
'99 Xantia HDI 110 Exclusive, RIP :(
'97 Xantia TD SX
'96 Xantia TD LX
'96 ZX TD
'89 BX TD
'88 AX GT
'79 CX2400 Pallas (scrapped :( )
& a couple of Peugeots !

User avatar
CitroJim
A very naughty boy
Posts: 39633
Joined: 30 Apr 2005, 23:33
Location: North Bucks
My Cars: An orange Skoda 120, a pretty blue Pug 207, Sally the white Saxo, a Giant TCR Bike and a Giant TCR Pro Bike.
x 747

Post by CitroJim » 10 Jan 2010, 19:59

Yes, I'm sorry about all this. I shall be having carefully considered words with the hosting company in the not too distant.

This level of service is not good enough...
0x
Jim

Please note I do not accept PMs...

User avatar
spider
Posts: 3951
Joined: 05 Jan 2010, 15:28
Location: Derby.
My Cars: Soon, I hope...
x 63

Post by spider » 12 Jan 2010, 14:33

As requested:

Was down (usual timeout) about 12.30 for a while

Came back up (I got to view a forum) a short time after that.

Session table crashed (now fixed I see)


I don't want to stick my nose in (but having seen this happen before on other software as the session table is the most used, given how it works) , you could probably change it to memory (assuming mysql) which would stop that in most cases, server load may go up though...

/ Random thought, mainly just posting the times for you as requested anyway...
0x

User avatar
CitroJim
A very naughty boy
Posts: 39633
Joined: 30 Apr 2005, 23:33
Location: North Bucks
My Cars: An orange Skoda 120, a pretty blue Pug 207, Sally the white Saxo, a Giant TCR Bike and a Giant TCR Pro Bike.
x 747

Post by CitroJim » 12 Jan 2010, 14:40

Thanks Spider :D

I've raised a support ticket with the hosting company to look into this issue as it should not happen as frequently as it does. Luckily they're taking it seriously...

I've just repaired the sessions table for the nth time :twisted: I can do it blindfold now from the mysql command pronpt. Biggest problem was finding an internet machine where I presently am that would let me do an RDP session into the server....

Please bear with me on this, hopefully a proper resolution very soon. I have (to keep the) faith...
0x
Jim

Please note I do not accept PMs...

User avatar
spider
Posts: 3951
Joined: 05 Jan 2010, 15:28
Location: Derby.
My Cars: Soon, I hope...
x 63

Post by spider » 12 Jan 2010, 14:45

Yes, understand the problems :)

Changing it to memory (just that table!) will stop it crashing, obviously the contents of it would be lost on a restart, not the structure but as it does not contain anything critical as far as I know etc. Not used phpbb for ages. In memory it can't crash (much!) as its not having to keep mysql up with the constant disc writes. Only problem is memory usage goes up :idea:

But I agree, should see what the host says really. I'd personally expect that to only crash about once a year to be honest. If its any help, mines crashed once in 3 years but I have been lucky...
0x

User avatar
KennyW
Donor 2016
Posts: 1577
Joined: 17 Jan 2006, 22:22
Location: NEWTONGRANGE,MIDLOTHIAN
My Cars:
x 13

Post by KennyW » 13 Jan 2010, 22:17

Jim,

The program is throwing another wobbly for me at thuis time and not allowing me access to the other parts of the site.
Will log off and log back on again

Kenny
0x
C5 x7 2.0 HDi 160 hp Estate exclusive, the newbie
Xantia 2.0 Hdi 90hp 1999 only 189,000 and rising!!!!!!! Moved on to a new home
C3 1.4 petrol (SWMBO)
Xantia TD 1.9 Mk 2 Estate LX 1998 model over 210,000 miles now and still rising!!!!!!!!!!!!! now deceased 17/12/2010.

User avatar
spider
Posts: 3951
Joined: 05 Jan 2010, 15:28
Location: Derby.
My Cars: Soon, I hope...
x 63

Post by spider » 13 Jan 2010, 22:21

Couple of issues today, once the members table was crashed (earlier today)

Couple more timeouts during the day, although I'll be honest I've been that busy I did not take note of the times...

EDIT... I'm getting odd intermittant cannot modify header errors along with warnings about unsaved queries, I think your tables are in need of a repair :( , error specifies table "u"
0x

User avatar
CitroJim
A very naughty boy
Posts: 39633
Joined: 30 Apr 2005, 23:33
Location: North Bucks
My Cars: An orange Skoda 120, a pretty blue Pug 207, Sally the white Saxo, a Giant TCR Bike and a Giant TCR Pro Bike.
x 747

Post by CitroJim » 13 Jan 2010, 22:29

I'm doing my best to sort it. Host being no help at all. Getting very frustrating....

Please bear with me...

Stress levels are building somewhat...
0x
Jim

Please note I do not accept PMs...

User avatar
Xaccers
Posts: 7655
Joined: 08 Feb 2007, 00:46
Location: Milling around Milton Keynes
My Cars:
x 2

Post by Xaccers » 13 Jan 2010, 22:37

CitroJim wrote:I'm doing my best to sort it. Host being no help at all. Getting very frustrating....

Please bear with me...

Stress levels are building somewhat...


Jim I hope you've got a cup of tea next to you.
There is no rush to get things up and running, we know you work your socks off repairing the forum when it goes down, we can wait as long as it takes.
Deep breath, count to ten.

I'm currently fighting with a sharepoint install at home so you have my sympathy.
0x
1.9TD+ SX Xantia Estate (Cassy) running on 100% veg
1.9TD SX Xantia Hatchback (Jenny) running on 100% veg for sale
Laguna II 2.0dCi Privilege (Monty)

DIY sphere tool

addo
Sara Watson's Stalker
Posts: 7104
Joined: 19 Aug 2008, 12:38
Location: NEW South Wales, Australia. I'll show you "Far, far away" ;-)
My Cars: Peugeot 605
Citroën Berlingo
Alfa 147
x 1

Post by addo » 14 Jan 2010, 04:13

I don't find these crashlets a problem. What we still get, is far ahead of resources available to likeminded people fifteen years ago. It's the efforts in maintaining an overall good website, that I appreciate.
0x

User avatar
Xaccers
Posts: 7655
Joined: 08 Feb 2007, 00:46
Location: Milling around Milton Keynes
My Cars:
x 2

Post by Xaccers » 14 Jan 2010, 10:03

Jim, when a company I hosted a forum with decided to let it's service plummet, I used sitescope to record whenever the forum went down, and used that to show them I knew very well how bad their server was.
They moved me to a different server and gave me a month free hosting.
It's a handy tool anyway.
Hmm I might check it out again myself, I know it can run through web sites automatically, but I can't remember off the top of my head if it could be used to run a script (to repair the tables) if certain text strings appear on the website.
0x
1.9TD+ SX Xantia Estate (Cassy) running on 100% veg
1.9TD SX Xantia Hatchback (Jenny) running on 100% veg for sale
Laguna II 2.0dCi Privilege (Monty)

DIY sphere tool

User avatar
CitroJim
A very naughty boy
Posts: 39633
Joined: 30 Apr 2005, 23:33
Location: North Bucks
My Cars: An orange Skoda 120, a pretty blue Pug 207, Sally the white Saxo, a Giant TCR Bike and a Giant TCR Pro Bike.
x 747

Post by CitroJim » 14 Jan 2010, 16:50

Xac wrote:Jim, when a company I hosted a forum with decided to let it's service plummet, I used sitescope to record whenever the forum went down, and used that to show them I knew very well how bad their server was.


Thanks Xac :D I've been doing same by hand and never fear I've not told the host! I'll have a look at sitescope as it looks promising.

I've also told them in no uncertain terms that their level of service is not acceptable and that if the server remains unstable. I want a new one. End of.

I have a support ticket open and last evening I asked for an update. fair enough. The 1st line desk informed me that yes, the ticket had been looked at by the sysadmins but they'd not updated the progress log so there was no update to give :evil: :evil:

Now, I work on a help desk, I action support tickets. If I had handled a ticket and NOT updated the progress log I'd be into big trouble. Makes me wonder just what sort of operation they run there. I did thell them my thoughts.

Strangely, and maybe not co-incidentially, the server has now been stable for 18 hours...

Again, I apologise for the crashlets (lovely term Adam!) and my absence from here whilst I have been investigating every angle. On the plus-side, my knowledge of W2008 Server is now very much enhanced :D
0x
Jim

Please note I do not accept PMs...