Author Topic: November 27-28 DB failure  (Read 103 times)

0 Members and 1 Guest are viewing this topic.

Offline Jim

  • TM Collaborator
  • Hero Member
  • *****
  • Posts: 1923
  • Last Login:Yesterday at 11:48:26 pm
November 27-28 DB failure
« on: November 28, 2020, 09:34:58 am »
Something in the November 27 site update caused the mysql server to crash, disabling both TM services and this forum.  It wasn't the old problem with a "User Sleep" query.  I've restarted the server process, but I would like to investigate a bit before I try to relaunch the site update.

Offline Jim

  • TM Collaborator
  • Hero Member
  • *****
  • Posts: 1923
  • Last Login:Yesterday at 11:48:26 pm
Re: November 27-28 DB failure
« Reply #1 on: November 28, 2020, 10:07:13 am »
Good news.  It was just a full partition.  I have moved some files and successfully re-run last night's update.

I will be looking into extra storage, as this will continue to happen if I don't pay attention to space available on noreaster's /var partition.  Fortunately, disks are cheap, and I am confident that I will be able to acquire and install a second disk before the year is out.

Offline yakra

  • TM Collaborator
  • Hero Member
  • *****
  • Posts: 2945
  • Last Login:Yesterday at 07:55:50 pm
Re: November 27-28 DB failure
« Reply #2 on: November 28, 2020, 12:26:20 pm »
Yikes, At least there's the option of compressing /home/tmp/tm to buy some time.

Offline Jim

  • TM Collaborator
  • Hero Member
  • *****
  • Posts: 1923
  • Last Login:Yesterday at 11:48:26 pm
Re: November 27-28 DB failure
« Reply #3 on: November 28, 2020, 01:44:23 pm »
I made a very poor decision when setting up to keep the /var partition relatively small.  As our databases have grown, so has the usage for their storage in /var/db .

My plan is to get a new drive and give /var a ridiculously large amount of space on either the old or new drive.