DB hang 2/28/20

[Jim]

I wasn't paying attention to the start of the site update when it reports "User sleep" DB processes and by letting the update go without fixing those, the DB got into a bad state and had to be restarted.  This caused all of TM and the Forum to fail for a short time.  Things look fine now, and tonight's site update is being re-run.

[yakra]

What do the results of
echo "show processlist;" | mysql --defaults-group-suffix=travmap -u travmap
look like?

Are the results something you have to kill manually while siteupdate.py works away?

[Jim]

Yes, if there are any processes shown in "User Sleep" it means they're hanging, and the DB ingestion will grind things to a halt.  I kill them manually by pid while the update proceeds.  They seem to come in bunches.  A few days in a row there will be those kinds of processes and then a month or two with none.

[yakra]

Thinking about automated updates. Is it possible to isolate the info we need using grep, cut, sed, etc., and kill them automatically? Are there risks involved?

[Jim]

Probably could.  I'd rather figure out why the processes hang in the first place.  Some earlier discussion:  https://github.com/TravelMapping/DataProcessing/issues/111